In this talk, I will introduce and deep dive the open source project: Kubeflow Pipelines (https://github.com/kubeflow/pipelines)

Kubeflow pipelines are reusable end-to-end ML workflows built using the Kubeflow Pipelines SDK. The Kubeflow pipelines service has the following goals:

End to end orchestration: enabling and simplifying the orchestration of end to end machine learning pipelines

Easy experimentation: making it easy for you to try numerous ideas and techniques, and manage your various trials/experiments.

Easy re-use: enabling you to re-use components and pipelines to quickly cobble together end to end solutions, without having to re-build each time.

Kubeflow pipelines are implemented on top of Kubernetes so it is highly portable and also supports hybrid solutions. Although it is on K8s, users of pipelines does not need to understand anything about Kubernetes or docker. They are given the familiar tools: Python + Jupyter notebooks to create their pipelines and pipeline components.