Sign in

Sara Rosenthal

@seirasto.bsky.social
1.3K followers 339 following 19 posts

NLP Research Scientist at IBM Research

PostsRepliesMedia
Sara Rosenthal @seirasto.bsky.social · 05/07/2026
I’m at #ACLNLP2026! I’ll be presenting: MTRAG-UN: A Benchmark for Open Challenges in Multi-Turn RAG Conversations, Session 9: Oral/Posters/Demos E Mon. July 6 11:00-12:30 Paper: aclanthology.org/2026.finding... GitHub: github.com/IBM/mt-rag-b...
aclanthology.org
MTRAG-UN: A Benchmark for Open Challenges in Multi-Turn RAG Conversations
Sara Rosenthal, Yannis Katsis, Vraj Shah, Lihong He, Lucian Popa, Marina Danilevsky. Findings of the Association for Computational Linguistics: ACL 2026. 2026.
000 0
Sara Rosenthal @seirasto.bsky.social · 28/08/2025
📣📣Presenting our platform used to build MTRAG!! RAGAPHENE: A RAG Annotation Platform with Human ENhancements and Edits Arxiv: arxiv.org/abs/2508.19272 MTRAG GitHub: github.com/IBM/mt-rag-b... Join our MTRAGEval Task: ibm.github.io/mt-rag-bench...
arxiv.org
RAGAPHENE: A RAG Annotation Platform with Human Enhancements and Edits
Retrieval Augmented Generation (RAG) is an important aspect of conversing with Large Language Models (LLMs) when factually correct information is important. LLMs may provide answers that appear correc...
000 0
Sara Rosenthal @seirasto.bsky.social · 04/08/2025
🚀Excited to announce our MTRAGEval task at SemEval 2026! Arxiv: arxiv.org/abs/2501.03468 Github: github.com/IBM/mt-rag-b... (please 🌟!) MTRAGEval: ibm.github.io/mt-rag-bench...
arxiv.org
MTRAG: A Multi-Turn Conversational Benchmark for Evaluating Retrieval-Augmented Generation Systems
Retrieval-augmented generation (RAG) has recently become a very popular task for Large Language Models (LLMs). Evaluating them on multi-turn RAG conversations, where the system is asked to generate a ...
000 0
Sara Rosenthal @seirasto.bsky.social · 01/05/2025
Working on RAG? Come check out our InspectorRAGet DEMO presented by Siva Sankalp Patel May 2 (Friday), 11-12:30 at Demo Session 8 in Hall 3! Looking forward to attending ACL in a few months! #NAACL2025 @naaclmeeting.bsky.social paper: arxiv.org/abs/2404.17347 github: github.com/IBM/Inspecto...
arxiv.org
InspectorRAGet: An Introspection Platform for RAG Evaluation
Large Language Models (LLM) have become a popular approach for implementing Retrieval Augmented Generation (RAG) systems, and a significant amount of effort has been spent on building good models and ...
020 0
Sara Rosenthal @seirasto.bsky.social · 26/03/2025
Excited about this collab! Come check out FeeL and help advance multilingual generation in your language! huggingface.co/spaces/feel-...
huggingface.co
Feel - a Hugging Face Space by feel-fl
Discover amazing ML apps made by the community
021 0
Sara Rosenthal @seirasto.bsky.social · 04/02/2025
🌟Want to know more about our MTRAG benchmark? Check out the IBM blog highlighting our work! research.ibm.com/blog/convers...
research.ibm.com
How well can your RAG agent carry out a conversation?
IBM’s new benchmark evaluates LLMs on interactive question-answering tasks using
030 0
Sara Rosenthal @seirasto.bsky.social · 08/01/2025
Retrievers (Elser shown here) struggle with later turns and non-standalone questions:
000 0
Sara Rosenthal @seirasto.bsky.social · 08/01/2025
SOTA LLMs struggle with later turns and unanswerable questions:
100 0
Sara Rosenthal @seirasto.bsky.social · 08/01/2025
Sample Conversation:
100 0
Sara Rosenthal @seirasto.bsky.social · 08/01/2025
MTRAG is a challenging benchmark for SOTA LLMs and a great way to evaluate across multiple domains for Retrieval and Generation! MTRAG contains 110 conversations averaging 7.7 turns each across four domains for a total of 842 tasks. We also explore synthetic data and LLM-as-a-judge.
100 0
Sara Rosenthal @seirasto.bsky.social · 08/01/2025
🌟 New Benchmark! 🌟 Do you work on RAG? Are you interested in Multi-Turn conversations? Very excited to share the new MTRAG benchmark we have released! Data: github.com/ibm/mt-rag-b... Paper: arxiv.org/abs/2501.03468
github.com
GitHub - IBM/mt-rag-benchmark: Multi-Turn RAG Benchmark
Multi-Turn RAG Benchmark. Contribute to IBM/mt-rag-benchmark development by creating an account on GitHub.
164 0
Sara Rosenthal @seirasto.bsky.social · 27/11/2024
Anyone else feel like Google scholar is missing citations lately? I have a recent paper that has 8 citations on semantic scholar and only 3 on Google scholar…. and I have two papers that are cited in one paper but only one has the citation 🤔
030 0
Sara Rosenthal @seirasto.bsky.social · 25/11/2024
Please just message me on slack
010 0
Sara Rosenthal @seirasto.bsky.social · 24/11/2024
Please add me. Thanks!
010 0
Reposted by Sara Rosenthal
Ramon Astudillo @ramon-astudillo.bsky.social · 19/11/2024
I did a starter pack of people in New York (City) working on ML/AI. Please distribute and feel free to self nominate! go.bsky.app/BoEtagz
428919 0
Sara Rosenthal @seirasto.bsky.social · 22/11/2024
If you work on RAG check out InspectorRAGet - an awesome RAG tool for evaluation. Available on HuggingFace! We provide the interface, you provide the experiments and metrics. Want to know more? Just reach out! github.com/IBM/Inspecto... huggingface.co/spaces/kpfad... arxiv.org/abs/2404.17347
github.com
GitHub - IBM/InspectorRAGet: The repository contains generative AI analytics platform application code.
The repository contains generative AI analytics platform application code. - IBM/InspectorRAGet
050 0
Sara Rosenthal @seirasto.bsky.social · 19/11/2024
Starter pack for IBM Research! Follow awesome IBM researchers! IBMers, let me know and I will add you! go.bsky.app/2SXcRmA
3216 0
Sara Rosenthal @seirasto.bsky.social · 19/11/2024
Working on RAG? Check out our ClapNQ benchmark (accepted to TACL) to test the full RAG pipeline! github.com/primeqa/clapnq arxiv.org/abs/2404.02103
github.com
GitHub - primeqa/clapnq
Contribute to primeqa/clapnq development by creating an account on GitHub.
1122 0
Sara Rosenthal @seirasto.bsky.social · 19/11/2024
Please add me!
010 0
Sara Rosenthal @seirasto.bsky.social · 19/11/2024
This is great! Please add me as well!
010 0