Field Notes
Case studies, essays, and guides in data thinking — the mistakes working analysts, engineers, and researchers make, and the intuition that would have caught them.
When Dashboard Averages Hide a Churn Cliff
The dashboard says average 30-day retention is 42%. Leadership is reassured. The number describes a user who doesn't exist — half the cohort churned in week one, and a small group of power users carries the rest. Here is what the average is hiding.
Why Your A/B Test Peeks Lie
Stopping a test the moment p drops below 0.05 doesn't make you fast — it inflates your false positive rate from 5% to over 20%. Here is what peeking actually does to your numbers, and what to do instead.
Reading Research After the Replication Crisis
A single p < 0.05 study is weaker evidence than it looks — here is why the replication crisis should change the discount rate every data practitioner applies to published findings.
How to Read a Study Without Being Fooled
When a vendor's whitepaper or an industry report lands in your inbox, seven questions will tell you whether the evidence is worth acting on.
Why Averages Lie
Your API dashboard shows a healthy 200 ms average response time. Real users are hitting 4-second loads. The mean was never lying to you — it just wasn't looking where the pain was.
The Alert at 11 PM
A fraud alert fires on a Friday night. You have ninety seconds to decide whether to escalate, page the on-call engineer, and potentially lock out a real customer — or ignore it and move on. This is what good probabilistic reasoning looks like under pressure.