iiinova — where your digital self keeps evolving

iiinova is part of the Atmosphere — an open, public network where you own your identity.

Deqing Fu

@deqing.bsky.social

179 followers 469 following 14 posts

CS PhD Student @USC. deqingfu.github.io

Posts Replies Media

Deqing Fu @handle.invalid · 08/02/2025

Finally, token-level annotations given by TLDR model could speedup human annotators to fix image captions that are slightly off. In fact, it can speed up human annotation by 3 times!

100 1

Deqing Fu @handle.invalid · 08/02/2025

Next, there is something interesting. After finishing training the TLDR model, one can simply remove the reward model head and re-attach the original language model head, to, obviously, become a new vision-language model. It's shown that these new models become better.

100 1

Deqing Fu @handle.invalid · 08/02/2025

TLDR has rich usefulness. First, it can serve as a hallucination rate evaluation metric. As shown in the table, GPT-4o is still the best vision language model in the token level while open-weight models such as Llama-3.2-90B is catching up in the sentence and response level.

100 1

Deqing Fu @handle.invalid · 08/02/2025

TLDR is trained on synthetic hard negatives generated via a perturbation-based method. The architecture is very simple. Instead of applying the reward model head to the last token, as many RMs are doing, TLDR applies the reward model head to every token.

100 1

Deqing Fu @handle.invalid · 08/02/2025

Excited to share that my intern work at Meta GenAI is accepted to @iclr-conf.bsky.social #ICLR2025 Introducing TLDR: Token-Level Detective Reward Model For Large Vision Language Models. TLDR provides fine-grained annotations to each text token. 🔗arXiv: arxiv.org/abs/2410.04734

151 5