Chip Huyen

As I'm getting back to building tools, I'd like to formally apologize to every tool developer whose tool I've complained about before. It's easy to comment on how overwhelming, fragmented, and slow the ML tooling space is. It's 1000x harder to build something to make it better.

The more I work in prod, the more I realize how hard it is to do prob & stats right. Splitting data is easy. Sampling data correctly is hard. Writing metrics is easy. Understanding what metrics measure is hard. Monitoring numbers is easy. Interpreting them is very, very hard.

It amazes me how tool developers underestimate the onboarding process. Every time I see a new tool, I want to try it out, but 90% of the time, I give up because it takes over 2 hours to just set up the right environment and get the tool to even import.

The challenge for ML in production is to generalize to constantly changing edge cases. 2 main approaches: 1. Use massive data because more data can lead to better generalization 2. Build infra that allows models to learn to adapt in real-time Hmu if you're excited about #2!

Takeaway from @karpathy's CVPR talk: The most successful ML projects in prod (Tesla, iPhone, Amazon drones, Zipline) are where you own the entire stack. They iterate not just ML algorithms but also: - how to collect/label data - infrastructure - hardware ML models run on

Why no/low-code platforms are exciting In most ML projects, subject matter expertise is only used for labels. In most successful ML projects, SME is also used for feature engineering, error analysis, reranking predictions, etc. No/low-code allows SMEs to get directly involved!