iiinova — where your digital self keeps evolving

Reposted by Fateme Hashemi Chaleshtori

Ana Marasović @anamarasovic.bsky.social · 07/11/2025

𝙒𝙚'𝙧𝙚 𝙝𝙞𝙧𝙞𝙣𝙜 𝙣𝙚𝙬 𝙛𝙖𝙘𝙪𝙡𝙩𝙮 𝙢𝙚𝙢𝙗𝙚𝙧𝙨! KSoC: utah.peopleadmin.com/postings/190... (AI broadly) Education + AI: - utah.peopleadmin.com/postings/189... - utah.peopleadmin.com/postings/190... Computer Vision: - utah.peopleadmin.com/postings/183...

11610

Reposted by Fateme Hashemi Chaleshtori

Martin Tutek @mtutek.bsky.social · 07/11/2025

Very honored to be one out of seven outstanding papers at this years' EMNLP :) Huge thanks to my amazing collaborators @fatemehc.bsky.social @anamarasovic.bsky.social @boknilev.bsky.social , this would not have been possible without them!

2236

Reposted by Fateme Hashemi Chaleshtori

Martin Tutek @mtutek.bsky.social · 21/08/2025

Thrilled that FUR was accepted to @emnlpmeeting.bsky.social Main🎉 In case you can’t wait so long to hear about it in person, it will also be presented as an oral at @interplay-workshop.bsky.social @colmweb.org 🥳 FUR is a parametric test assessing whether CoTs faithfully verbalize latent reasoning.

1133

Fateme Hashemi Chaleshtori @fatemehc.bsky.social · 20/06/2025

9/ We hope BriefMe encourages more Legal NLP development that directly aids legal professionals! Check out our paper for the full methodology, human evaluation details, and comprehensive benchmarks. What other legal NLP applications can we design using BriefMe? 🤔

010

Fateme Hashemi Chaleshtori @fatemehc.bsky.social · 20/06/2025

8/ ⚖️ BriefMe extends Legal NLP by introducing a dataset of legal briefs, a type of legal document that hasn't been overlooked before. We've designed tasks that attorneys actually need in their daily work, opening up new research directions to be explored to assist professionals.

110

Fateme Hashemi Chaleshtori @fatemehc.bsky.social · 20/06/2025

7/ However, LLMs struggle with these complex tasks: - Realistic argument completion: Llama-3.1-70B finds missing arguments only 18% of the time - Case retrieval: Best method finds correct precedents in top-5 results just 31.4% of the time Lots of room for improvement! 📈

110

Fateme Hashemi Chaleshtori @fatemehc.bsky.social · 20/06/2025

6/ Surprising finding: GPT-4o outperforms human-written headings! 🤖 GPT-4o: 4.3/5 avg. LLM-as-judge rating for both arg. summ. & comp. 🤵 Lawyers: 4.0/5 (summ.) and 3.9/5 (comp.) avg. rating LLMs excel at summarization and guided completion tasks, requiring only minor edits.

110

Fateme Hashemi Chaleshtori @fatemehc.bsky.social · 20/06/2025

5/ Evaluating generated text is challenging: traditional metrics (BLEU/ROUGE/...) are not aligned with human preferences. Instead, we built an LLM-as-judge using o3-mini, instructed with expert-written guidelines for brief headings, proving more reliable than human raters!

110

Fateme Hashemi Chaleshtori @fatemehc.bsky.social · 20/06/2025

4/ Our novel argument completion task tests if LLMs can identify WHERE exactly a missing argument should go in a brief's logical flow and WHAT that argument should be. 🧩 This realistic version is especially challenging: models must spot gaps in the ToCs with no guidance.

110

Fateme Hashemi Chaleshtori @fatemehc.bsky.social · 20/06/2025

3/ We built BriefMe from Supreme Court briefs with 3 key tasks: - Argument summarization - Realistic/Guided Argument completion: filling in missing arguments within the Table of Contents (ToC) - Case retrieval Each assesses different practical aspects of legal reasoning.

110

Fateme Hashemi Chaleshtori @fatemehc.bsky.social · 20/06/2025

2/ Legal briefs are documents where attorneys present their arguments to judges, making the case for their client's position by interpreting the law and citing relevant precedents. Most legal NLP work focuses on judicial opinions, but we target the attorney's perspective instead 🏛️

110

Fateme Hashemi Chaleshtori @fatemehc.bsky.social · 20/06/2025

1/ 🚨NEW PAPER: "BriefMe: A Legal NLP Benchmark for Assisting with Legal Briefs", accepted to ACL Findings 2025! We introduce the first benchmark specifically designed to help LLMs assist lawyers in writing legal briefs 🧑‍⚖️ 📄 arxiv.org/abs/2506.06619 🗂️ huggingface.co/datasets/jw4...

174

Reposted by Fateme Hashemi Chaleshtori

Martin Tutek @mtutek.bsky.social · 21/02/2025

It has been amazing to work with @fatemehc.bsky.social, @anamarasovic.bsky.social and Yonatan Belinkov on this incredibly important topic. I look forward to further works on the parametric faithfulness route! Codebase (& data): github.com/technion-cs-...

github.com

GitHub - technion-cs-nlp/parametric-faithfulness

Contribute to technion-cs-nlp/parametric-faithfulness development by creating an account on GitHub.

062