Metaflow was started at Netflix to answer a pressing business need: How to enable an organization of data scientists, who are not software engineers by training, build and deploy end-to-end machine learning workflows and applications independently. We wanted to provide the best possible user experience for data scientists, allowing them to focus on parts they like (modeling using their favorite off-the-shelf libraries) while providing robust built-in solutions for the foundational infrastructure: data, compute, orchestration, and versioning.
Today, the open-source Metaflow powers hundreds of business-critical ML projects at Netflix and other companies from bioinformatics to real estate.
In this talk, you will learn about:
- What to expect from a modern ML infrastructure stack.
- Using Metaflow to boost the productivity of your data science organization, based on lessons learned from Netflix.
- Deployment strategies for a full stack of ML infrastructure that plays nicely with your existing systems and policies.
Ville has been developing infrastructure for machine learning for more than two decades. He is a co-founder and CEO of Outerbounds, a startup focusing on human-centric ML infrastructure. Prior to Outerbounds, he led the machine learning infrastructure team at Netflix where he started Metaflow, an open-source framework to support the full lifecycle of data science projects. He is also the author of an upcoming book, Effective Data Science Infrastructure, published by Manning. .