If you turn something into a benchmark with quantifiable rewards and progress metrics, don’t be surprised when the machines achieve dominance there. At that point it will be too late to gesture at intrinsic non-economic values, you’ve made them illegible (in Scott’s sense) to any outside observer.
