iiinova — where your digital self keeps evolving

Try it out at civicchats.org Read our full blog post: cichicago.substack.com/p/civicchats...

000

A platform for exploring, debating, and thinking through upcoming ballot measures on your local ballot or across the U.S.

CivicChats (civicchats.org) aims to address these shortcomings. Starting with ballot measures, it helps people understand political issues through three modes: Q&A to understand what's at stake, an argumentative mode presenting competing views, and a reflective mode to examine your own thinking.

civicchats.org

CivicChats

100

LLM chat interfaces may help people stay informed, but mainstream chatbots tend to match user beliefs rather than challenging them. Without the kind of pushback you'd encounter between disagreeing friends, chatbots are ill-suited for helping people think through political issues in a balanced way.

100

Technology offers ways to help, but not without problems of its own. The Internet and social media promised new ways to connect and learn from each other. In practice, both often inflame passions, solidify echo chambers, and spread misinformation.

100

chenhaotan.bsky.social @chenhaotan.bsky.social · 02/07/2025

Democracy depends on an informed electorate. But political issues and ballot measures can be confusing, obscuring the effects of one outcome versus another. And politics is personal. Once we make an initial decision, it can be hard to see things from "the other side."

242

Reposted by Mourad Heddaya

It predicts pretty well—not just shifts in the last week, but also: 1. Who’s working an overnight shift (in our data + external validation in MIMIC) 2. Who’s working on a disruptive circadian schedule 3. How many patients has the doc seen *on the current shift*

153

I'll be presenting this work at 2pm and will be around until Sunday. Please reach out if you're interested in this line of work - would love to connect in person or virtually! Thank you to my great collaborators @kyle-macmillan.bsky.social , Anup Malani, Hongyuan Mei, and @chenhaotan.bsky.social

131

ChicagoHAI/CaseSumm · Datasets at Hugging Face

CaseSumm is publicly available on HuggingFace! We hope this dataset enables: - Better evaluation of long-context summarization - Research on legal language understanding - Development of more accurate & reliable legal AI tools Dataset: huggingface.co/datasets/Chi...

huggingface.co

We’re on a journey to advance and democratize artificial intelligence through open source and open science.

100

Analysis reveals different types of hallucinations: - Simple factual errors - Incorrect legal citations - Misrepresentation of procedural history - Mischaracterization of Court's reasoning Fine-tuned smaller models tend to make more egregious errors than GPT-4.

111

CaseSumm is a useful resource for long-context reasoning and legal research: - Largest legal case summarization dataset - 200+ years of Supreme Court cases - "Ground truth" summaries written by Court attorneys and approved by Justices - Variation in summary styles and compression rates over time

Key findings: 1. A smaller fine-tuned LLM scores well on metrics but has more factual errors. 2. Experts prefer GPT-4 summaries—even over the “ground-truth” syllabuses. 3. ROUGE and similar metrics poorly reflect human preferences. 4. Even LLM-based evaluations still misalign with human judgment.

ChicagoHAI/CaseSumm · Datasets at Hugging Face

Dataset: huggingface.co/datasets/Chi... Paper: arxiv.org/abs/2501.00097 When evaluating LLM-generated and human-written summaries, we find interesting discrepancies between automatic metrics, LLM-based evaluation, and human expert judgements.

huggingface.co

We’re on a journey to advance and democratize artificial intelligence through open source and open science.

120

We develop CaseSumm, a comprehensive dataset comprising 25K U.S. Supreme Court opinions and their official syllabuses spanning over 200 years, and conduct a rigorous evaluation of long-document summarization using CaseSumm.

120

chenhaotan.bsky.social @chenhaotan.bsky.social · 30/04/2025

🧑‍⚖️How well can LLMs summarize complex legal documents? And can we use LLMs to evaluate? Excited to be in Albuquerque presenting our paper this afternoon at @naaclmeeting 2025!

22313

Reposted by Mourad Heddaya

Although I cannot make #NAACL2025, @chicagohai.bsky.social will be there. Please say hi! @chachachen.bsky.social GPT ❌ x-rays (Friday 9-10:30) @mheddaya.bsky.social CaseSumm and LLM 🧑‍⚖️ (Thursday 2-3:30) @haokunliu.bsky.social @qiaoyu-rosa.bsky.social hypothesis generation 🔬 (Saturday at 4pm)

0167

Reposted by Mourad Heddaya

Dang Nguyen @divingwithorcas.bsky.social · 14/04/2025

1/n You may know that large language models (LLMs) can be biased in their decision-making, but ever wondered how those biases are encoded internally and whether we can surgically remove them?

11712

Reposted by Mourad Heddaya

chenhaotan.bsky.social @chenhaotan.bsky.social · 24/01/2025

Spent a great day at Boulder meeting new students and old colleagues. I used to take this view every day. Here are the slides for my talk titled "Alignment Beyond Human Preferences: Use Human Goals to Guide AI towards Complementary AI": chenhaot.com/talks/alignm...

0165

Thank you to my excellent collaborators Qingcheng Zeng, @chenhaotan.bsky.social, @robvoigt.bsky.social, and Alexander Zentefis!

010

Mourad Heddaya, Qingcheng Zeng, Alexander Zentefis, Rob Voigt, Chenhao Tan. Proceedings of the The 6th Workshop on Narrative Understanding. 2024.

I’ll be presenting this work at the EMNLP 2024 Workshop on Narrative Understanding. If you are in Miami, the presentation will be at 3:30pm! Paper: aclanthology.org/2024.wnu-1.12/ Dataset (soon): mheddaya.com/research/nar...

aclanthology.org

Causal Micro-Narratives

120

Please reach you if you are interested in this line of work, I’d love to connect in-person or virtually!

100

Our ongoing work aims to discover narratives automatically, investigate their geographic and temporal trends, understand their potential spread, and assess their influence on economic indicators.

We're scaling up! Using our fine-tuned models, we're identifying narratives in millions of news articles. Techniques like Design-Based Supervised Learning ensure validity in downstream analyses.

100

Even human annotators sometimes disagree on narrative presence, but fine-tuned LLMs mirror these natural disagreements more closely than larger models. Our error analysis shows some mistakes arise from genuine interpretative ambiguity. Check out the last three examples here:

100

Fine-tuning shines in teaching models to spot narratives, unlike in-context learning. GPT-4o struggles, often misclassifying non-narratives as narratives.

100

This is a difficult hierarchical classification task, with many, somestimes semantically similar, classes. We find that smaller fine-tuned LLMs outperform larger models like GPT-4o, while also offering better scalability and cost efficiency. But they also err differently.

100

We define a causal micro-narrative as a sentence-level explanation of a target subject's cause(s) and/or effect(s). As an application, we propose an ontology for inflation's causes/effects and create a large-scale dataset classifying sentences from U.S. news articles.

While the importance of narratives has become well recognized, formulating an operational definition remains challenging. Particularly one that is flexible to informal and ambiguous language. In our work, we address both the conceptual and technical challenges.

121

Mourad Heddaya, Qingcheng Zeng, Alexander Zentefis, Rob Voigt, Chenhao Tan. Proceedings of the The 6th Workshop on Narrative Understanding. 2024.

We focus on sentences but do not make any further assumptions about the syntax or structure of language used to convey a narrative. 📄 aclanthology.org/2024.wnu-1.12/

aclanthology.org

Causal Micro-Narratives

131