Finding and controlling the low-dimensional persona structure in models — from emergent misalignment and subliminal learning through to superintelligence.

A magazine of weird, forward-gazing writing.

We steer Qwen on the "automated grader" vs "human evaluator" dimension. This influences the model's persona in surprising ways, e.g. changing how Machiavellian/violent it is. We can't fully explain that. LW post in a comment.

Deliberative alignment: reasoning enables safer language models Introducing our new alignment strategy for o1 models, which are directly taught safety specifications and how to reason over them.