iiinova — where your digital self keeps evolving

Reposted by Neha Srikanth

Maharshi Gor @maharshigor.bsky.social · 01/05/2025

🏆ADVSCORE won an Outstanding Paper Award at #NAACL2025 🚨 Don't miss out on our poster presentation *today at 2 pm* by Yoo Yeon (first author). 📍Poster Session 5 - HC: Human-centered NLP 💼 Highly recommend talking to her if you are hiring and/or interested in Human-focused Al dev and evals!

171

Reposted by Neha Srikanth

Rupak @rupak-s.bsky.social · 01/05/2025

Are you tired of using traditional stance detection to measure the polarity of text? Our #NAACL25 paper proposes an approach that uses pairwise comparisons to order texts on a continuous scale, capturing both implicit and explicit evidence in language. 📍Today in Hall 3 from 4-5:30pm Come say hi!

A screenshot of a paper showing the title - "Pairscale: Analyzing Attitude Change in Online Communities" by Rupak Sarkar, Patrick Wu, Kristina Miler, Alexander Hoyle and Philip Resnik.

061

Neha Srikanth @nehasrikanth.bsky.social · 29/04/2025

This helps us build groups of examples that evaluate the same pieces of knowledge, allowing us to measure under what *contexts* an LLM can correctly draw a particular inference ("inferential consistency"). We find that LLMs still exhibit room for improvement on this front. (5/n)

000

Neha Srikanth @nehasrikanth.bsky.social · 29/04/2025

We propose a method to pinpoint the particular pieces of knowledge a defeasible reasoning example aims to evaluate by identifying the atom(s) that are most critical in determining the overall label of a defeasible NLI example. (4/n)

000

Neha Srikanth @nehasrikanth.bsky.social · 29/04/2025

We also explore how atomic hypothesis decomposition can help us better understand the complexities of defeasible reasoning, a softer inference task that requires models to weigh the effects of multiple, sometimes competing, pieces of evidence on a hypothesis. (3/n)

000

Neha Srikanth @nehasrikanth.bsky.social · 29/04/2025

For example, after decomposing hypothesis from an NLI premise-hypothesis pair into atoms, we can measure whether its judgment on the overall pair is consistent with its set of judgments on each premise-atom sub-problem in a logical way. (2/n)

000

Neha Srikanth @nehasrikanth.bsky.social · 29/04/2025

I'll be presenting this work with @rachelrudinger at #NAACL2025 tomorrow (Wednesday 4/30) in Albuquerque during Session C (Oral/Poster 2) at 2pm! 🔬 Decomposing hypotheses in traditional NLI and defeasible NLI helps us measure various forms of consistency of LLMs. Come join us!

582

Neha Srikanth @nehasrikanth.bsky.social · 19/02/2025

Ah, thanks Joe!! :) And a huge thank you to you for all your early feedback -- it definitely helped the way we framed the concept of atomic inference.

020