EvalEval Coalition @eval-eval.bsky.social · 17/02/2026What we built: 📋 Metadata schema for cross-framework comparison 🔧 Validation via Hugging Face Jobs 🔌 Converters (Inspect AI, HELM, lm-eval-harness) 📊 Community repo organized by benchmark/model/run ✨ Captures scores AND context: settings, prompts, example-level data 110
EvalEval Coalition @eval-eval.bsky.social · 17/02/2026How can you help? We are launching a shared task alongside our workshop at @aclmeeting.bsky.social → Two tracks: public + proprietary eval data → Co-authorship for qualifying contributors → Workshop at ACL 2026 (San Diego) → Deadline: May 1, 2026 📅 110
EvalEval Coalition @eval-eval.bsky.social · 17/02/2026Thankful to our partners for the feedback: CAISI, AIEleuther, Huggingface, NomaSecurity, TrustibleAI, InspectAI, Meridian, AVERI, CIP, Stanford HELM, Weizenbaum, Evidence Prime, MIT, TUM, IBM Research 🤝 120