iiinova — where your digital self keeps evolving

Arijit

@arijitnoobstar.bsky.social

PhD Student - MIT ProbComp / CoCoSci

Arijit @arijitnoobstar.bsky.social · 30/07/2025

A huge thanks to my fantastic mentors, collaborators and advisors: Andrew Bolton, Vikash Mansinghka, @joshtenenbaum.bsky.social and @thiskevinsmith.bsky.social. Do check out our project page (arijit-dasgupta.github.io/jtap/) for more details and the code to run this experiment.

010

Arijit @arijitnoobstar.bsky.social · 30/07/2025

This can be explained with an intuitive example. If the object does not come out the left side, it makes sense to think it might have bounced and will eventually hit red. Just like humans, JTAP can predict this, showing how the lack of changing visual evidence is evidence itself.

100

Arijit @arijitnoobstar.bsky.social · 30/07/2025

To test whether humans update their beliefs during occlusion, we compared against two ablative baselines that either decay or freeze beliefs when the object is hidden. A targeted analysis of occluded time-steps shows that the full JTAP model captures human behavior much better.

100

Arijit @arijitnoobstar.bsky.social · 30/07/2025

And when we do this, we find that we do really well at predicting overall time-aligned human behavior, both on when to make a decision, and which decision to make. Not only that, we also capture those moments of graded uncertainty, when humans don’t have full agreement.

100

Arijit @arijitnoobstar.bsky.social · 30/07/2025

A single JTAP run acts as a proxy for one pseudo-participant’s belief states over time. By feeding multiple randomly-seeded runs into a decision model accounting for time delay and hysteresis, we predict both individual-level human decisions and aggregate behavior.

100

Arijit @arijitnoobstar.bsky.social · 30/07/2025

JTAP produces current and future beliefs over a single object’s position, speed, & direction. In this example, it predicts that future beliefs over red vs green outcomes spread under occlusion, then sharpen to red once visible, while in the visible case they stay confidently red.

100

Arijit @arijitnoobstar.bsky.social · 30/07/2025

We approach this with the Joint Tracking and Prediction (JTAP) model: a structured probabilistic program that combines 2D physical dynamics with an observation model. Posterior beliefs are inferred via a particle-based approximation using a Sequential Monte Carlo algorithm.

100

Arijit @arijitnoobstar.bsky.social · 30/07/2025

Computationally, we frame this as a Bayesian inference problem: jointly inferring an observer’s beliefs about an object’s current and future state from a stream of images.

100

Arijit @arijitnoobstar.bsky.social · 30/07/2025

We started by asking participants to continuously predict whether a ball moving in a 2.5D scene with gray occluders and black barriers would hit the red or green region first. People showed varying uncertainty in their responses, while giving reliable responses (ICC1k = 0.952).

110

Arijit @arijitnoobstar.bsky.social · 30/07/2025

Although often studied separately, we explore whether people jointly integrate state uncertainty from occlusion and uncertainty about future physical dynamics to form intuitive beliefs about what will happen next. Can machines capture this kind of physically grounded reasoning?

100

Arijit @arijitnoobstar.bsky.social · 30/07/2025

One remarkable aspect of human physical reasoning and perception is our ability to track and predict the motion of hidden objects. Come see my poster at #CogSci2025 (P1-D-44), where I show how people “see through occlusion”, and how we model it using Bayesian inference. @cogscisociety.bsky.social

292