iiinova — where your digital self keeps evolving

Patty Liu

@pattliu.bsky.social

7 followers 10 following 7 posts

PhD student in CS @princetoncitp.bsky.social researching AI for public good

Posts Replies Media

Patty Liu @pattliu.bsky.social · 09/07/2026

Paper: arxiv.org/abs/2606.211... Blogpost: blog.citp.princeton.edu/2026/05/27/c... (7/7)

000

Patty Liu @pattliu.bsky.social · 09/07/2026

We argue for expanding free public access to legal opinions, giving courts better screening tools, and updating pro se guidance on AI usage. AI will ultimately be a false promise for pro se litigation if structural limitations around citation verification are not addressed. (6/7)

100

Patty Liu @pattliu.bsky.social · 09/07/2026

However, it catches only 18% of wrong pincites. This is due to official page numbers being behind paywalls. So free case citation verification is capped by what's publicly available, and pro se litigants pay the price. (5/7)

100

Patty Liu @pattliu.bsky.social · 09/07/2026

LLM agents catch certain hallucination types with high recalls. GPT-5 hits 82.8% recall catching fake cases (100%) and name mismatches (97%). It even flagged 10+ citation typos in real briefs that human authors had missed. (4/7)

100

Patty Liu @pattliu.bsky.social · 09/07/2026

To assess verification capabilities, we built LePhantomCite: 1,300 legal brief excerpts with injected hallucinations across 5 types (fake cases, name mismatches, wrong pincites, misquotes, misrepresented holdings). Each needs different verification strategies. (3/7)

100

Patty Liu @pattliu.bsky.social · 09/07/2026

First, free-tier models aren't hallucinating less over time. Across 8 ChatGPT generations, GPT-5.1 fabricates legal cites at at least 6.57%, which is a significant jump from early GPT-4o. And newer models cite more cases, increasing the verification time. (2/7)

100

Patty Liu @pattliu.bsky.social · 09/07/2026

AI-fabricated case citations continue to show up in courts, but can AI also automatically flag hallucinated legal citations to reduce the verification burden? In our new paper (accepted at COLM 2026), we built a benchmark and evaluated several agentic models to find out (1/7)

131