iiinova — where your digital self keeps evolving

Mark Ibrahim

@markibrahim.bsky.social

Researching the dark arts of deep learning at Meta's FAIR (Fundamental AI Research) Lab markibrahim.me

Mark Ibrahim @markibrahim.bsky.social · 10/12/2025

Want to teach AI agents to use apps like humans? Get started with digital agents research using OpenApps, our new Python-based environment.

133

Mark Ibrahim @markibrahim.bsky.social · 07/11/2025

Despite saturating single image perception, Common-O establishes a new challenging multimodal benchmark. The best performing model only achieves 35% on Common-O and on Common-O Complex, consisting of more complex scenes, the best model achieves only 1%. 🧵2/3

100

Mark Ibrahim @markibrahim.bsky.social · 07/11/2025

We introduce, Common-O, a new multimodal benchmark for hallucination when reasoning across scenes. We find leading multimodal LLMs can reliably identify objects, yet hallucinate when reasoning across scenes. 🧵1/3

100

Mark Ibrahim @markibrahim.bsky.social · 09/10/2025

One can manipulate LLM rankings to put any model in the lead—only by modifying the single character separating demonstration examples. Learn more in our new paper arxiv.org/abs/2510.05152 w/ Jingtong Su, Jianyu Zhang, @karen-ullrich.bsky.social , and Léon Bottou. 🧵

1313

Mark Ibrahim @markibrahim.bsky.social · 17/06/2025

A good language model should say “I don’t know” by reasoning about the limits of its knowledge. Our new work AbstentionBench carefully measures this overlooked skill in an open-codebase others can build on! We find frontier reasoning degrades models’ ability to know when NOT to answer. 🧵1/2

110

Mark Ibrahim @markibrahim.bsky.social · 11/12/2024

Recently, we also applied the same MLM-U objective to maze navigation. We find when training parameter-matched transformers on identical data, MLM-U without any tweaks outperforms standard next token training across all maze grid sizes (up to 30x30).

110

Mark Ibrahim @markibrahim.bsky.social · 11/12/2024

Can we boost transformers’ ability to retrieve knowledge and plan in maze navigation by only tweaking the learning objective? We emphatically say YES in our #NeurIPS 2024 study! 🧵 w/ Ouail Kitouni, Niklas Nolte, Diane Bouchacourt, Adina Williams, and Mike Rabbat Paper arxiv.org/abs/2406.05183

240