iiinova — where your digital self keeps evolving

Reposted by John Sandall

PyData London @pydatalondon.bsky.social · 01/04/2025

Kicking off tonight’s PyData London with @johnsandall.bsky.social telling us about Death by RMSE: Cautionary Tales of Metrics Gone Wild Choosing the wrong eval metrics can look great but give terrible real world performance—it’s always a trade off!

132

John Sandall @johnsandall.bsky.social · 31/03/2025

2️⃣ Are they effectively using techniques like cohorting, backtesting to simulate past futures, and aligning eval metrics to downstream company KPIs? 3️⃣ Would it help to workshop it together? Check out coefficient.ai and we can book in a session, this is exactly what we do at Coefficient!

coefficient.ai

Coefficient

Coefficient is a data consultancy offering data science, engineering, machine learning and other AI-related services as well as bespoke training courses.

000

John Sandall @johnsandall.bsky.social · 31/03/2025

Some questions to ask your data team: 1️⃣ Are they using the right eval metrics as their "north star"? The wrong model evaluation metric can quietly sabotage your ML team's impact. Choosing between MAE, F1, RMSE and others isn't academic, it's business critical.

coefficient.ai

Coefficient

Coefficient is a data consultancy offering data science, engineering, machine learning and other AI-related services as well as bespoke training courses.

100

John Sandall @johnsandall.bsky.social · 31/03/2025

The model is trained, eval metrics are great, nothing changes — the KPIs are static, business impact isn’t there, did you optimise for the wrong thing? I'm speaking at @pydatalondon.bsky.social meetup tomorrow on "Death by RMSE: Cautionary Tales of Metrics Gone Wild" www.meetup.com/pydata-londo...

meetup.com

PyData London - 95th Meetup, Tue, Apr 1, 2025, 7:00 PM | Meetup

## Details **Venue**: Riverbank House, 2 Swan Ln, London EC4R 3AD **Please note**: 1. 🚨🚨🚨A valid photo ID is required by building security. 🚨🚨🚨 2. This event follo

131