iiinova — where your digital self keeps evolving

Replies

Samidh @samidh.bsky.social · 03/06/2026

Can your LLM actually follow your content policies, or will it just revert to its own static training? Today we're introducing a new benchmark that we call policy steerability that tries to measure this concept: blog.zentropi.ai/introducing-...

blog.zentropi.ai

Beyond Static Accuracy: Introducing the Policy Steerability Benchmark

A new way to measure how likely a model is to accurately follow your rules

120

Samidh @samidh.bsky.social · 03/06/2026

When we released CoPE-B last week-- our newest language model for content classification-- the standard metrics showed only modest gains. But every time we used the model it *felt* like a huge step up from our prior model, responding to policy subtlety and changes with much greater fidelity.

110

Samidh @samidh.bsky.social · 03/06/2026

So we decided to study this phenomenon in a lot more rigor and devised this new policy steerability benchmark. As more AI-powered systems start using LLMs behind the scenes for classification, steerability is crucial. When you write a rule for your platform, your system should follow it.

110