iiinova — where your digital self keeps evolving

Florian Schneider

@floschne-nlp.bsky.social

he/him 3rd and final year PhD Student Researching on the applications and limitations of multimodal transformer encoder and decoder models.

Posts Replies Media

Reposted by Florian Schneider

KnitTogether @knittogethernlp.bsky.social · 18/02/2025

😩Tired of the latest ARR cycle? @aclmeeting.bsky.social Join us for #KnitTogether25 ⛰️✨ – a week to focus on “Bias and Social Factors in NLP”! Lets explore and exchange research - together. But we also cook and explore nature - together. 🚀💡 🔗 knittogether.github.io/kt25 #NLPProc

knittogether.github.io

Knit Togehter 25

081

Reposted by Florian Schneider

Fabian David Schmidt @fdschmidt.bsky.social · 21/02/2025

Introducing MVL-SIB, a massively multilingual vision-language benchmark for cross-modal topic matching in 205 languages! 🤔Tasks: Given images (sentences), select topically matching sentence (image). Arxiv: arxiv.org/abs/2502.12852 HF: huggingface.co/datasets/Wue... Details👇

145

Reposted by Florian Schneider

Fabian David Schmidt @fdschmidt.bsky.social · 21/02/2025

Strong vision-language models (VLMs) like GPT-4o-mini maintain good performance for top-150 languages, only to drop to performing no better than chance for the lowest resource languages!

111

Reposted by Florian Schneider

Fabian David Schmidt @fdschmidt.bsky.social · 21/02/2025

X-modal to text-only perf. *gap* shows that VL support decreases from high to low-resource language tiers: Images/Topic→Sentence (for I/T, pick S): narrows with less textual support (left) Sentences→Image/Topic (for S, pick I/T): increases with less VL support worse (right)

111

Reposted by Florian Schneider

Fabian David Schmidt @fdschmidt.bsky.social · 21/02/2025

Cross-modal topic matching correlates well with other multilingual vision-language tasks! 🤗Images-To-Sentence (given Images, select topically fitting sentence) & Sentences-To-Image (given Sentences, pick topically matching image) probe complementary aspects in VLU

121