Kubeflow operations guide : managing cloud and on-premise deployment

When deploying machine learning applications, building models is only a small part of the story. The entire process involves developing, orchestrating, deploying, and running scalable and portable machine learning workloads—a process Kubeflow makes much easier. With this practical guide, data scientists, data engineers, and platform architects will learn how to plan and execute a Kubeflow project that can support workflows from on-premises to the cloud. Kubeflow is an open source Kubernetes-native platform based on Google’s internal machine learning pipelines, and yet major cloud vendors including AWS and Azure advocate the use of Kubernetes and Kubeflow to manage containers and machine learning infrastructure. In today’s cloud-based world, this book is ideal for any team planning to build machine learning applications. With this book, you will: • Get a concise overview of Kubernetes and Kubeflow • Learn how to plan and build a Kubeflow installation • Operate, monitor, and automate your installation • Provide your Kubeflow installation with adequate security • Serve machine learning models on Kubeflow Copyright 8 Table of Contents 11 Preface 17 What Is in This Book? 17 Who Is This Book For? 18 Conventions Used in This Book 18 Using Code Examples 19 O’Reilly Online Learning 20 How to Contact Us 20 Acknowledgments 21 Josh 21 Michael 21 Austin 22 Chapter 1. Introduction to Kubeflow 23 Machine Learning on Kubernetes 23 The Evolution of Machine Learning in Enterprise 24 It’s Harder Than Ever to Run Enterprise Infrastructure 26 Identifying Next-Generation Infrastructure (NGI) Core Principles 28 Kubernetes for Production Application Deployment 30 Enter: Kubeflow 34 What Problems Does Kubeflow Solve? 36 Origin of Kubeflow 38 Who Uses Kubeflow? 39 Common Kubeflow Use Cases 40 Running Notebooks on GPUs 40 Shared Multitenant Machine Learning Environment 43 Building a Transfer Learning Pipeline 43 Deploying Models to Production for Application Integration 45 Components of Kubeflow 46 Machine Learning Tools 48 Applications and Scaffolding 50 Machine Learning Model Inference Serving with KFServing 57 Platforms and Clouds 59 Summary 61 Chapter 2. Kubeflow Architecture and Best Practices 63 Kubeflow Architecture Overview 63 Kubeflow and Kubernetes 65 Ways to Run a Job on Kubeflow 66 Machine Learning Metadata Service 66 Artifact Storage 67 Istio Operations in Kubeflow 67 Kubeflow Multitenancy Architecture 70 Multitenancy and Isolation 70 Multiuser Architecture 71 Multiuser Authorization Flow 71 Kubeflow Profiles 72 Multiuser Isolation 74 Notebook Architecture 75 Notebook Server Launcher UI 75 Notebook Controller 77 Pipelines Architecture 78 Kubeflow Best Practices 79 Managing Job Dependencies 79 Using GPUs 82 Experiment Management 84 Summary 85 Chapter 3. Planning a Kubeflow Installation 87 Security Planning 87 Components That Extend the Kubernetes API 88 Components Running Atop Kubernetes 88 Background and Motivation 89 Kubeflow and Deployed Applications 90 Integration 91 Users 92 Profiling Users 92 Varying Skillsets 94 Workloads 95 Cluster Utilization 95 Data Patterns 97 GPU Planning 97 Planning for GPUs 98 Models that Benefit from GPUs 99 Infrastructure Planning 101 Kubernetes Considerations 101 On-Premise 102 Cloud 103 Placement 104 Container Management 105 Serverless Container Operations with Knative 105 Sizing and Growing 106 Forecasting 106 Storage 107 Scaling 108 Summary 109 Chapter 4. Installing Kubeflow On-Premise 111 Kubernetes Operations from the Command Line 111 Installing kubectl 111 Using kubectl 115 Using Docker 117 Basic Install Process 119 Installing On-Premise 119 Considerations for Building Kubernetes Clusters 119 Gateway Host Access to Kubernetes Cluster 121 Active Directory Integration and User Management 121 Kerberos Integration 122 Storage Integration 122 Container Management and Artifact Repositories 125 Accessing and Interacting with Kubeflow 126 Common Command-Line Operations 126 Accessible Web UIs 126 Installing Kubeflow 127 System Requirements 127 Set Up and Deploy 127 Summary 129 Chapter 5. Running Kubeflow on Google Cloud 131 Overview of the Google Cloud Platform 132 Storage 133 Google Cloud Identity-Aware Proxy 134 Google Cloud Security and the Cloud Identity-Aware Proxy 136 GCP Projects for Application Deployments 140 GCP Service Accounts 141 Signing Up for Google Cloud Platform 142 Installing the Google Cloud SDK 142 Update Python 143 Download and Install Google Cloud SDK 143 Installing Kubeflow on Google Cloud Platform 143 Create a Project in the GCP Console 144 Enabling APIs for a Project 145 Set Up OAuth for GCP Cloud IAP 147 Deploy Kubeflow Using the Command-Line Interface 153 Accessing the Kubeflow UI Post-Installation 163 Summary 164 Chapter 6. Running Kubeflow on Amazon Web Services 165 Overview of Amazon Web Services 165 Storage 166 Amazon Storage Pricing 167 Amazon Cloud Security 167 AWS Compute Services 167 Managed Kubernetes on EKS 168 Signing Up for Amazon Web Services 168 Installing the AWS CLI 169 Update Python 169 Install the AWS CLI 169 Kubeflow on Amazon Web Services 172 Installing kubectl 173 Install the eksctl CLI for Amazon EKS 173 Install AWS IAM Authenticator 173 Install jq 173 Using Managed Kubernetes on Amazon EKS 174 Create an EKS Service Role 174 Create an AWS VPC 176 Creating EKS Clusters 179 Deploying an EKS Cluster with eksctl 180 Understanding the Deployment Process 180 Kubeflow Configuration and Deployment 181 Customize the Kubeflow Deployment 183 Customize Authentication 183 Resizing EKS Clusters 183 Deleting EKS Clusters 183 Adding Logging 184 Troubleshooting Deployments 186 Summary 186 Chapter 7. Running Kubeflow on Azure 187 Overview of the Azure Cloud Platform 187 Key Azure Components 188 Storage on Azure 189 The Azure Security Model 192 Service Accounts 194 Resources and Resource Groups 194 Azure Virtual Machines 195 Containers and Managed Azure Kubernetes Services 196 The Azure CLI 197 Installing the Azure CLI 197 Installing Kubeflow on Azure Kubernetes 197 Azure Login and Configuration 198 Create an AKS Cluster for Kubeflow 199 Kubeflow Installation 202 Authorizing Network Access to Deployment 209 Summary 209 Chapter 8. Model Serving and Integration 211 Basic Concepts of Model Management 211 Understanding Training Models Versus Model Inference 212 Building an Intuition for Model Integration 214 Scaling Model Inference Throughput 217 Model Management 220 Introduction to KFServing 221 Advantages of Using KFServing 223 Core Concepts in KFServing 224 Supported Pre-Built Model Servers 232 KFServing Security Model 236 Managing Models with KFServing 237 Installing KFServing on a Kubernetes Cluster 237 Deploying a Model on KFServing 240 Managing Model Traffic with Canarying 246 Deploying a Custom Transformer 248 Roll Back a Deployed Model 250 Removing a Deployed Model 251 Summary 251 Appendix A. Infrastructure Concepts 253 Public Key Infrastructure 253 Authentication 254 Kubeflow and Authentication 254 Authorization 254 Authorization and Role-Based Access Control 254 Lightweight Directory Access Protocol 255 Kerberos 255 Transport Layer Security 256 X.509 Cert 256 Webhook 256 Active Directory 257 Identity Providers 257 Identity-Aware Proxy (IAP) 258 IAP and Google Cloud Platform 259 OAuth 260 OpenID Connect 260 End-User Authentication with JWT 260 Simple and Protected GSS_API Negotiation Mechanism 261 Dex: A Federated OpenID Connect Provider 261 Dex and Kerberos 263 Service Accounts 264 The Control Plane 264 Options for Securing the Control Plane 266 Appendix B. An Overview of Kubernetes 267 Core Kubernetes Concepts 267 Pod 269 Object Spec and Status 270 Describing a Kubernetes Object 271 Submitting Containers to Kubernetes 271 Kubernetes Resource Model 272 Custom Resources, Controllers, and Operators 272 Custom Controllers 273 Custom Resource Definition 273 Appendix C. Istio Operations and Kubeflow 275 Service Mesh Management with Istio 275 Istio Architecture 278 Traffic Management 280 Istio Security Architecture 282 Istio Authorization and Role-Based Access Control 285 Index 289 About the Authors 300 Colophon 300 Building models is a small part of the story when it comes to deploying machine learning applications. The entire process involves developing, orchestrating, deploying, and running scalable and portable machine learning workloads--a process Kubeflow makes much easier. This practical book shows data scientists, data engineers, and platform architects how to plan and execute a Kubeflow project to make their Kubernetes workflows portable and scalable.Authors Josh Patterson, Michael Katzenellenbogen, and Austin Harris demonstrate how this open source platform orchestrates workflows by managing machine learning pipelines. You'll learn how to plan and execute a Kubeflow platform that can support workflows from on-premises to cloud providers including Google, Amazon, and Microsoft.Dive into Kubeflow architecture and learn best practices for using the platformUnderstand the process of planning your Kubeflow deploymentInstall Kubeflow on an existing on-premises Kubernetes clusterDeploy Kubeflow on Google Cloud Platform step-by-step from the command lineUse the managed Amazon Elastic Kubernetes Service (EKS) to deploy Kubeflow on AWSDeploy and manage Kubeflow across a network of Azure cloud data centers around the worldUse KFServing to develop and deploy machine learning models Building models is a small part of the story when it comes to deploying machine learning applications. The entire process involves developing, orchestrating, deploying, and running scalable and portable machine learning workloads--a process Kubeflow makes much easier. This practical book shows data scientists, data engineers, and platform architects how to plan and execute a Kubeflow project to make their Kubernetes workflows portable and scalable. Authors Josh Patterson, Michael Katzenellenbogen, and Austin Harris demonstrate how this open source platform orchestrates workflows by managing machine learning pipelines. You'll learn how to plan and execute a Kubeflow platform that can support workflows from on-premises to cloud providers including Google, Amazon, and Microsoft. * Dive into Kubeflow architecture and learn best practices for using the platform * Understand the process of planning your Kubeflow deployment * Install Kubeflow on an existing on-premise Kubernetes cluster * Deploy Kubeflow on Google Cloud Platform, AWS, and Azure * Use KFServing to develop and deploy machine learning models « Building models is a small part of the story when it comes to deploying machine learning applications. The entire process involves developing, orchestrating, deploying, and running scalable and portable machine learning workloads -- a process Kubeflow makes much easier. This practical book shows data scientists, data engineers, and platform architects how to plan and execute a Kubeflow project to make their Kubernetes workflows portable and scalable. Authors Josh Patterson, Michael Katzenellenbogen, and Austin Harris demonstrate how this open source platform archestrates workflows by managing machine learning pipelines. You'll learn how to plan and execute a Kubeflow platform that can support workflows from on-premise to cloud providers including Google, Amazon, and Microsoft. »--Page 4 de la couverture