iiinova — where your digital self keeps evolving

Daniel Khashabi @danielkhashabi.bsky.social · 06/04/2026

Very honored and excited to receive the NSF CAREER Award! HUGE thank you to my amazing students, collaborators, mentors, and advisors, who helped make this happen. And to my family who are the real heroes in my story! ♥️

120

Daniel Khashabi @danielkhashabi.bsky.social · 11/02/2026

LLMs continue to struggle with long-context tasks—such as needle-in-a-haystack problems—because of “positional bias.” What can we do if we only have 𝘣𝘭𝘢𝘤𝘬-𝘣𝘰𝘹 access to the model? (i.e., we can’t modify the model weights or attention patterns, as is often the case with API models.)

100

Daniel Khashabi @danielkhashabi.bsky.social · 22/01/2026

We have been busy building our science co-pilot for Genomics AI Agent at @DataTecnica which is specialized in Alzheimer’s and neurodegenerative disease research. This system: * Synthesizes complex biomedical data across literature and genomics databases

110

Daniel Khashabi @danielkhashabi.bsky.social · 18/11/2025

For years since the GPT-2 paper, emergent in-context learning (ICL) from 'next-token' training has been treated as something deeply tied to 𝐡𝐮𝐦𝐚𝐧 𝐥𝐚𝐧𝐠𝐮𝐚𝐠𝐞. But … is it?

122

Daniel Khashabi @danielkhashabi.bsky.social · 19/09/2025

Imagine this: excited about the recent progress, you’ve built an agentic system that uses 🔧tools (API calls) to solve complex problems. What could go wrong? We studied agentic tool recovery—when your LLM selects a set of tools to execute, but one turns out to be unavailable or incorrect.

110

Daniel Khashabi @danielkhashabi.bsky.social · 26/08/2025

But how do we measure benchmark effectiveness? A key premise is that, effectiveness of attack prompts on dev models can predict their effectiveness on unseen eval models. Jack verifies that indeed this is the case: his resulting benchmark JBDistill-Bench is effective on a *unseen* models.

100

Daniel Khashabi @danielkhashabi.bsky.social · 26/08/2025

100

Daniel Khashabi @danielkhashabi.bsky.social · 07/07/2025

Highlighting a key result in the figure: when we inspect intermediate layers, we see that models often solve the task in the wrong (off-target_ language; that is, high off-target accuracy early on. Only in the later layers does the answer get translated into the intended language.

100

Daniel Khashabi @danielkhashabi.bsky.social · 29/05/2025

🚨New LLM benchmark🚨 We're releasing BiomedSQL🔬 for tabular reasoning over large-scale biomedical databases. This includes questions based on implicit scientific conventions—like statistical thresholds, effect direction, and drug approval status. 📄 Preprint: arxiv.org/pdf/2505.20321

100

Daniel Khashabi @danielkhashabi.bsky.social · 28/05/2025

In our latest study, we look into how the size of these gold contexts impacts LLM performance in needle-in-a-haystack scenarios. The verdict? **Smaller gold contexts severely amplify positional bias.**

100

Daniel Khashabi @danielkhashabi.bsky.social · 22/05/2025

There have been various efforts on disentangling "task learning" vs "task recall" in LLMs. We've recently explored a fresh angle by borrowing from cryptography: with substitution ciphers, we transform a given task into an equivalent, but cryptic (no pun intended!!) forms.

100

Daniel Khashabi @danielkhashabi.bsky.social · 16/05/2025

What is a university without "freedom of speech"? Apparently, ChatGPT has a better grasp than @nyuniversity. x.com/nebedaay/st...

000

Daniel Khashabi @danielkhashabi.bsky.social · 12/05/2025

Our approach: We propose BloomScrub 🧽, a framework that certifiably mitigates worst-case infringement risks while maintaining output utility. * It's simple: Rewrite content by targeting and transforming the few longest quotes.

100

Daniel Khashabi @danielkhashabi.bsky.social · 30/04/2025

Can LLMs can be co-pilots for peer review? Answering this requires evaluating *evaluate* whether LLMs can provide critiques that are *grounded* in the context of science papers. See @JiefuOu's dataset which has a collection of paper claims and their critiques: arxiv.org/pdf/2503.21717

000

Daniel Khashabi @danielkhashabi.bsky.social · 23/04/2025

People rely on search engines/chatbots to access science. But what if you want a bird’s-eye view of science, or to identify over- and under-explored areas? We introduce 🔺Science Hierarchography🔺, the goal of organizing science papers into conceptual hierarchies. arxiv.org/abs/2504.13834

100

Daniel Khashabi @danielkhashabi.bsky.social · 14/04/2025

Research is important, but so is recharging! Took the team out for some badminton fun today—amazing energy, lots of laughs, and a reminder of how lucky I am to work with this crew!

010