iiinova — where your digital self keeps evolving

Parameter Lab @parameterlab.bsky.social · 23/03/2026

If you care about rigorous evaluation of agentic systems, give it a look at MASEval! The harness is an important element of agents. MASEval makes it straightforward to change its components and evaluate their impact. MASEval is our first software! parameterlab.github.io/MASEval/ ⬇️

parameterlab.github.io

MASEval — Multi-Agentic System Evaluation

MASEval is a unified, agent-agnostic evaluation framework and benchmark library for multi-agent systems. Compare frameworks, not just models.

020

Parameter Lab @parameterlab.bsky.social · 03/02/2026

‼️New paper from Parameter Lab! ⛓️‍💥 We identify privacy collapse, a silent failure mode of LLMs: LLMs fine-tuned on seemingly benign data can lose their ability to respect contextual privacy norms. Done by @anmolgoel.bsky.social during his internship! Check-out 👇

051

Reposted by Parameter Lab

Martin Gubri @mgubri.bsky.social · 04/11/2025

Our #EMNLP2025 paper Leaky Thoughts 🫗 shows that Large Reasoning Models (LRMs) can unintentionally leak sensitive information hidden in their internal thoughts. 📍 Come chat with Tommaso at our poster on Friday 7th, 10:30–12:00 in Hall C3 📄 aclanthology.org/2025.emnlp-m...

aclanthology.org

Leaky Thoughts: Large Reasoning Models Are Not Private Thinkers

Tommaso Green, Martin Gubri, Haritz Puerto, Sangdoo Yun, Seong Joon Oh. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. 2025.

021

Parameter Lab @parameterlab.bsky.social · 21/08/2025

🫗 An LLM's "private" reasoning may leak your sensitive data! 🎉 Excited to share our paper "Leaky Thoughts: Large Reasoning Models Are Not Private Thinkers" was accepted at #EMNLP main! 1/2

Overall diagram about contextual privacy & LRMs

151

Parameter Lab @parameterlab.bsky.social · 23/06/2025

🔎Does Conversational SEO actually work? Our new benchmark has an answer! Excited to announce our new paper: C-SEO Bench: Does Conversational SEO Work? 🌐 RTAI: researchtrend.ai/papers/2506.... 📄 Paper: arxiv.org/abs/2506.11097 💻 Code: github.com/parameterlab... 📊 Data: huggingface.co/datasets/par...

121

Parameter Lab @parameterlab.bsky.social · 26/04/2025

Excited to share that our paper "Scaling Up Membership Inference: When and How Attacks Succeed on LLMs" will be presented next week at #NAACL2025! 🖼️ Catch us at Poster Session 8 - APP: NLP Applications 🗓️ May 2, 11:00 AM - 12:30 PM 🗺️ Hall 3 Hope to see you there!

021

Parameter Lab @parameterlab.bsky.social · 14/02/2025

👥 We're Hiring: Senior/Junior Data Engineer! 📍 Remote or Local | Full-Time or Part-Time At ResearchTrend.AI, we’re building a platform that connects researchers and AI engineers worldwide—helping them stay ahead with daily digests, insightful summaries, and interactive events.

researchtrend.ai

ResearchTrend.AI

Explore the most trending research topics in AI

120

Parameter Lab @parameterlab.bsky.social · 06/02/2025

🔎 Wonder how to prove an LLM was trained on a specific text? The camera ready of our Findings of #NAACL 2025 paper is available! 📌 TLDR: longs texts are needed to gather enough evidence to determine whether specific data points were included in training of LLMs: arxiv.org/abs/2411.00154

arxiv.org

Scaling Up Membership Inference: When and How Attacks Succeed on Large Language Models

Membership inference attacks (MIA) attempt to verify the membership of a given data sample in the training set for a model. MIA has become relevant in recent years, following the rapid development of ...

051

Parameter Lab @parameterlab.bsky.social · 23/01/2025

We are delighted to announce that our research paper on the scale of LLM membership inference has been accepted for publication in the Findings of #NAACL2025! 🎉

040

Parameter Lab @parameterlab.bsky.social · 20/11/2024

🎉We’re pleased to share the release of the models from our Apricot🍑 paper, accepted at ACL 2024! At Parameter Lab, we believe openness and reproducibility are essential for advancing science, and we've put in our best effort to ensure it. 🤗 huggingface.co/collections/... 🧵 bsky.app/profile/dnns...

huggingface.co

🍑 Apricot Models - a parameterlab Collection

Fine-tuned models for black-box LLM calibration, trained for "Apricot: Calibrating Large Language Models Using Their Generations Only" (ACL 2024)

093

Parameter Lab @parameterlab.bsky.social · 19/11/2024

🚨📄 Exciting new research! Discover when and at what scale we can detect if specific data was used in training LLMs — a method known as Membership Inference (MIA)! Our findings open new doors for using MIA as potential legal evidence in AI. 🧵 arxiv.org/abs/2411.00154

151

Parameter Lab @parameterlab.bsky.social · 18/11/2024

📄 We’ve been working on the calibration of confidence score for black-box LLMs. See the thread below for an overview of the 🍑 Apricot paper, proudly accepted at #ACL24! #uncertainty

031

Parameter Lab @parameterlab.bsky.social · 18/11/2024

Check out one of our latest papers about LLM fingerprinting!

000

Parameter Lab @parameterlab.bsky.social · 18/11/2024

We are excited to join Bluesky! At Parameter Lab, we're committed to enhancing AI safety and trustworthiness. Our research addresses privacy, copyright, and security challenges in foundational models. Follow us for insights and updates on trustworthy AI research!

000