Netflix uses data science and machine learning across all facets of the company, powering a wide range of business applications. The Metaflow machine learning platform at Netflix provides an entire user-centric ecosystem of abstractions and integrations that allow practitioners to tackle a diverse set of business problems.
In this talk, we will first introduce Metaflow, an Open Source Software, and how it accelerates the work of ML practitioners by providing simple and consistent abstractions over core properties of ML pipelines. We will show how components such as data, orchestration, hosting and others can be simply combined to iterate quickly from design to production over a wide range of complex data science and ML workflows.
Through the talk we will highlight use cases to demonstrate the breadth of ML applications developed on Metaflow, and we will point out the user-design, technical and operational principles that helped us scale to hundreds of users and several hundreds of workflows while maintaining a small team with a focus on customer service and interactions.
The presentation will cover:
- How to build foundational components that can be combined to create novel ML applications across diverse use cases.
- How to design user-centric systems that cater to a wide range of ML practitioners.
- Technical and operational lessons about scaling and maintaining a large platform with a small team.
Speaker
David Berg
Senior Software Engineer @Netflix, Previously @IBM Almaden Research Center, Ph.D in Computational Neuroscience
David Berg is a member of the Metaflow machine learning platform at Netflix where he is responsible for increasing the velocity of ML research and reliability of applications. David has a PhD in computational neuroscience and applies his knowledge to both bridge the gap between research and engineering and also bring human factors into our software interaction paradigms. Prior to Netflix, he was a staff member in the Brain Inspired Computing group at IBM research.
Find David Berg at:
Speaker
Romain Cledat
Senior Software Engineer @Netflix, Metaflow Core Contributor, Previously @Facebook and @Intel
Romain Cledat is an engineer at Netflix, where he has played a pivotal role in the development and open-sourcing of Metaflow since 2019. As the technical lead of the project at Netflix, Romain has continued making significant contributions to the open-source community, driving innovation and excellence in a platform that is used by many companies. Before joining Netflix, Romain was part of the Applied Machine Learning group at Facebook (now Meta), where he specialized in building foundational ML infrastructure for the company's training platform. His career began in the high-performance computing (HPC) domain and compiler technology, working on exascale computing initiatives at Intel. Romain holds a Ph.D. in Computer Science from the Georgia Institute of Technology, awarded in 2011, with a dissertation focused on compilers and operating systems.
Find Romain Cledat at:
Date
Monday Nov 18 / 10:35AM PST ( 50 minutes )
Location
Ballroom A
