The World’s Safest Driver Isn’t Human. Can Waymo Stop Traffic Deaths? | Chinmay Jain, Dir. Product
From YouTube to Waymo, Chinmay Jain explains how building a product that bets lives on AI forces you to rethink evaluation, unlearn misleading metrics, and make trust your real north star.
40,000 people a year die from traffic accidents in the US. Our guest today is Chinmay Jain, Director of Product Management on Waymo's Driving Behavior team, who is working to make that number 90% smaller.
In this episode, Chinmay shares:
How he thought through leaving YouTube at its peak to join a moonshot company that could have civilization-level impact
Waymo’s actual AI eval process, using massive simulations based on millions of real-world driving miles to maximize edge cases, ultimately turning trust into their real product
And the misleading, but common, metrics Chinmay and his team learned to spot that could have seriously derailed Waymo’s progress
1. Leaving a sure thing for a startup with the power to change the world (3:06)
When Chinmay joined Waymo in 2018, the outcome was genuinely uncertain. This wasn’t a calculated bet on an obvious winner — it was a leap into the unknown.
“I always have tried to go back and work on something going from zero to one and be more present with that great opportunity to take it from zero to one.”
For Chinmay, Waymo’s mission tipped the scales: 40,000 people die in US traffic accidents every year. Waymo’s goal is to reduce that by 90%.
The lesson for PMs in any vertical: don’t just optimize for stability. The products that change industries, whether in healthcare, fintech, logistics, or consumer tech, are usually built by people who were willing to bet on something before it was obvious.
2. What to do when AI evals are high-stakes — or even life-or-death (5:50)
When Chinmay was at YouTube, a bad A/B test meant a feature didn’t ship. At Waymo, a bad eval could mean someone gets hurt. That difference fundamentally changes how you think about testing.
This is increasingly relevant across all of product management, not just autonomous vehicles. As AI becomes embedded in medical diagnostics, financial decision-making, and construction infrastructure, the stakes of evaluation are rising everywhere. The question isn’t just “did the metric go up?” — it’s “do we actually understand why, and are we measuring the right thing?”
For this reason, Chinmay’s team runs massive simulations — built on millions of real-world driving miles — to stress test edge cases before anything touches the road.
For PMs building on top of ML — whether in consumer apps, B2B SaaS, or physical AI — this is the core discipline. You can’t rely on traditional A/B testing intuitions when your system is probabilistic. You need to define what “good” looks like before you can measure it.
3. The misleading metrics that could have derailed Waymo (14:52)
One of the most underappreciated PM skills is knowing which metrics to stop trusting. Vanity metrics are a well-known problem in consumer apps — DAUs that don’t reflect real engagement, NPS scores that mask churn risk. But in complex, high-stakes systems, the danger is more subtle.
The broader PM lesson: metric selection isn’t a setup task you do once at launch. It requires ongoing interrogation, especially as your product scales and user behavior evolves. Whether you’re running a marketplace, a fintech platform, or an enterprise SaaS tool, the metrics that got you to product-market fit may not be the ones that keep you successful as you scale.
4. What are the hardest things to teach a self-driving car? (24:41)
Two answers, both surprising:
Unprotected left turns: Massive negotiation happening in real time between cars, pedestrians, and intent. There's no one rule that resolves it cleanly. It's negotiation in real time
Pulling over: Looks simple, but requires the kind of human intuition that drivers must learn over years. Experienced Uber drivers learn pickup nuance over years (think the person hovering at the corner, the building entrance that's technically on the side street, etc.) It's tacit knowledge, built from thousands of micro-observations humans don't even consciously register
“It’s the same reason why can’t a robot can’t just fold a shirt — there are some aspects which are very easy for humans that machine learning systems have to really learn well.”
What seems obvious to your team — "just click here to get started" — may require years of learned context for your ML systems. The gap between your mental model and theirs is almost always larger than you think, which is why taking the time to comprehensively train your models is crucial.
Links
Resources
Linear’s Secret to Building Powerful AI Products | Nan Yu, Head of Product (Linear)
Designing for Attention: How CrossFit Builds Product for Community-Led Growth | Ben McAllister, CPTO
Chapters
00:00 Introduction
02:01 Chinmay’s career journey
03:06 Chinmay’s decision to leave YouTube for Waymo
05:50 How does Waymo test its AI in the physical world?
07:59 Waymo’s layered evaluation system
12:07 Simulations and ML gains at Waymo
19:23 Waymo’s metrics for safety
21:59 Can Waymo make roads 90% safer?
24:36 What driving choices make training AI drivers the hardest?
27:10 Conclusion
What does LogRocket do?
LogRocket’s Galileo AI watches user sessions for you and surfaces the technical and usability issues holding back your web and mobile apps. Understand where your users are struggling by trying it for free at LogRocket.com.