Sign in

Parameter Lab

@parameterlab.bsky.social
35 followers 8 following 37 posts

Empowering individuals and organisations to safely use foundational AI models. parameterlab.de

PostsRepliesMedia
Parameter Lab @parameterlab.bsky.social · 03/02/2026
‼️New paper from Parameter Lab! ⛓️‍💥 We identify privacy collapse, a silent failure mode of LLMs: LLMs fine-tuned on seemingly benign data can lose their ability to respect contextual privacy norms. Done by @anmolgoel.bsky.social during his internship! Check-out 👇
051
Parameter Lab @parameterlab.bsky.social · 21/08/2025
🫗 An LLM's "private" reasoning may leak your sensitive data! 🎉 Excited to share our paper "Leaky Thoughts: Large Reasoning Models Are Not Private Thinkers" was accepted at #EMNLP main! 1/2
Overall diagram about contextual privacy & LRMs
151
Parameter Lab @parameterlab.bsky.social · 23/06/2025
Work done with: Haritz Puerto, Martin Gubri ‪‪@mgubri.bsky.social‬ , Tommaso Green, Sangdoo Yun and Seong Joon Oh @coallaoh.bsky.social‬ #SEO #AI #LLM #GenerativeAI #Marketing #DigitalMarketing #Perplexity #NLProc
010
Parameter Lab @parameterlab.bsky.social · 23/06/2025
🔎 The results are clear: current C-SEO strategies don’t work. This challenges the recent hype and suggests that creators don’t need to game LLMs and create even more clickbaits. Just focus on producing genuinely good content and let traditional SEO do its work.
100
Parameter Lab @parameterlab.bsky.social · 23/06/2025
💥 With the rise of conversational search, a new technique of "Conversational SEO" (C-SEO) emerged, claiming it can boost content inclusion in AI-generated answers. We put these claims to the test by building C-SEO Bench, the first comprehensive benchmark to rigorously evaluate these new strategies.
 Illustration of a conversational search engine for product recommendation. After applying a C-SEO method on the third document, its ranking gets boosted by +2 positions.
100
Parameter Lab @parameterlab.bsky.social · 23/06/2025
🔎Does Conversational SEO actually work? Our new benchmark has an answer! Excited to announce our new paper: C-SEO Bench: Does Conversational SEO Work? 🌐 RTAI: researchtrend.ai/papers/2506.... 📄 Paper: arxiv.org/abs/2506.11097 💻 Code: github.com/parameterlab... 📊 Data: huggingface.co/datasets/par...
Paper thumbnail.
121
Parameter Lab @parameterlab.bsky.social · 19/11/2024
🔎 Better Results in Fine-Tuning: Fine-tuned models show even stronger MIA results. The table shows the performance at sentence level and for collections of 20 sentences, evaluated on Phi-2 fine-tuned for QA (huggingface.co/haritzpuerto/phi-2-d… ).
100
Parameter Lab @parameterlab.bsky.social · 19/11/2024
🚀 The Key? Number of tokens & Aggregation: MIA’s accuracy improves as we aggregate MIA scores across multiple paragraphs. Longer documents or larger document collections significantly boost MIA effectiveness.
100
Parameter Lab @parameterlab.bsky.social · 19/11/2024
🛠️ First Success on Pre-Trained LLMs: By adapting recent work on Dataset Inference (@pratyushmaini.bsky.social ), we successfully applied MIA on pre-trained LLMs. Check out our figure below: MIA achieves an AUROC of 0.75 on documents of up to 20k tokens!
100
Parameter Lab @parameterlab.bsky.social · 19/11/2024
🔍 New Benchmark, New Insights: We developed a new benchmark to assess MIA effectiveness across data scales, from single sentences to document collections. This lets us identify precisely when and how MIA succeeds on LLMs.
100
Parameter Lab @parameterlab.bsky.social · 19/11/2024
🚨 What’s MIA? It’s a method to detect if a specific data sample was used in model training. We show that MIA works effectively on long documents (~20k tokens) and in collections of documents (>100 docs) —just the scale relevant for legal applications! 👮
100
Parameter Lab @parameterlab.bsky.social · 19/11/2024
🚨📄 Exciting new research! Discover when and at what scale we can detect if specific data was used in training LLMs — a method known as Membership Inference (MIA)! Our findings open new doors for using MIA as potential legal evidence in AI. 🧵 arxiv.org/abs/2411.00154
151