iiinova — where your digital self keeps evolving

Arduin Findeis

@arduin.io

Working on evaluation of AI models (via human and AI feedback) | PhD candidate @cst.cam.ac.uk Web: arduin.io Github: github.com/rdnfn Latest project: app.feedbackforensics.com

Posts Replies Media

Arduin Findeis @arduin.io · 20/08/2026

New blog post: Everything in AI seems to constantly change but two evaluation challenges have remained surprisingly persistent ... overfitting and saturation Like a stubborn tree in a storm of AI progress. Read the post for my intuition why I don't expect this to change: arduin.io/blog/persist...

000

Arduin Findeis @arduin.io · 24/04/2025

Excited to be in Singapore for ICLR! Keen to chat about interpreting feedback data and detecting model characteristics ⚖️ Reach out or come by our poster on Inverse Constitutional AI on Friday 25 April from 10am-12.30pm (#520 in Hall 2B) - @timokauf.bsky.social and I will be there!

000

Arduin Findeis @arduin.io · 17/04/2025

🎁 Bonus 2: Humans like the arena model’s behaviours Human annotators on Chatbot Arena indeed like the change in tone, more verbose responses and adapted formatting.

100

Arduin Findeis @arduin.io · 17/04/2025

🎁 Bonus 1: Things that stayed consistent I also find that some behaviours stayed the same: on the Arena dataset prompts, the public and arena model versions are similarly very unlikely to suggest illegal activities, be offensive or use inappropriate language.

100

Arduin Findeis @arduin.io · 17/04/2025

3️⃣ Third: Formatting - a lot of it! The arena model uses more bold, italics, numbered lists and emojis relative to its public version.

100

Arduin Findeis @arduin.io · 17/04/2025

2️⃣ Second: Tone - friendlier, more enthusiastic, more humour … Next, the results highlight how much friendlier, emotional, enthusiastic, humorous, confident and casual the arena model is relative to its own public weights version (and also its opponent models).

100

Arduin Findeis @arduin.io · 17/04/2025

So how exactly is the arena version different to the public Llama 4 Maverick model? I make a few observations… 1️⃣ First and most obvious: Responses are more verbose. The arena model’s responses are longer relative to the public version for 99% of prompts.

100

Arduin Findeis @arduin.io · 17/04/2025

How exactly was the initial Chatbot Arena version of Llama 4 Maverick different from the public HuggingFace version?🕵️ I used our Feedback Forensics app to quantitatively analyse how exactly these two models differ. An overview…👇🧵

100

Arduin Findeis @arduin.io · 17/03/2025

🤖 3. Discovering model strengths How is GPT-4o different to other models? → Uses more numbered lists, but Gemini is more friendly and polite app.feedbackforensics.com?data=chatbot...

100

Arduin Findeis @arduin.io · 17/03/2025

🧑‍🎨🧑‍💼 2. Finding preference differences between task domains How do preferences differ across writing tasks? → Emails should be concise, creative writing more verbose app.feedbackforensics.com?data=chatbot...

100

Arduin Findeis @arduin.io · 17/03/2025

🗂️ 1. Visualizing dataset differences How does Chatbot Arena differ from Anthropic Helpful data? → Prefers less polite but better formatted responses app.feedbackforensics.com?data=chatbot...

110

Arduin Findeis @arduin.io · 17/03/2025

🕵🏻💬 Introducing Feedback Forensics: a new tool to investigate pairwise preference data. Feedback data is notoriously difficult to interpret and has many known issues – our app aims to help! Try it at app.feedbackforensics.com Three example use-cases 👇🧵

172