Samidh @samidh.bsky.social · 03/06/2026When we released CoPE-B last week-- our newest language model for content classification-- the standard metrics showed only modest gains. But every time we used the model it *felt* like a huge step up from our prior model, responding to policy subtlety and changes with much greater fidelity. 110
Samidh @samidh.bsky.social · 03/06/2026So we decided to study this phenomenon in a lot more rigor and devised this new policy steerability benchmark. As more AI-powered systems start using LLMs behind the scenes for classification, steerability is crucial. When you write a rule for your platform, your system should follow it. 110
Samidh @samidh.bsky.social · 03/06/2026We hope this new methodology on measuring policy compliance is helpful to the trust & safety ecosystem and so we're sharing our work in the open. Critique and questions are welcome! Full details here: blog.zentropi.ai/introducing-...blog.zentropi.aiBeyond Static Accuracy: Introducing the Policy Steerability BenchmarkA new way to measure how likely a model is to accurately follow your rules 000