Deep Learning in Production

Build, train, deploy, scale and maintain deep learning models. Understand ML infrastructure and MLOps using hands-on examples. Deep Learning research is advancing rapidly over the past years. Frameworks and libraries are constantly been developed and updated. However, we still lack standardized solutions on how to serve, deploy and scale Deep Learning models. Deep Learning infrastructure is not very mature yet. This book accumulates a set of best practices and approaches on how to build robust and scalable machine learning applications. It covers the entire lifecycle from data processing and training to deployment and maintenance. It will help you understand how to transfer methodologies that are generally accepted and applied in the software community, into Deep Learning projects. It's an excellent choice for researchers with a minimal software background, software engineers with little experience in machine learning, or aspiring machine learning engineers. What you will learn? - Best practices to write Deep Learning code - How to unit test and debug Machine Learning code - How to build and deploy efficient data pipelines - How to serve Deep Learning models - How to deploy and scale your application - What is MLOps and how to build end-to-end pipelines Who is this book for? - Software engineers who are starting out with deep learning - Machine learning researchers with limited software engineering background - Machine learning engineers who seek to strengthen their knowledge - Data scientists who want to productionize their models and build customer-facing applications What tools you will use? Tensorflow, Flask, uWSGI, Nginx, Docker, Kubernetes, Tensorflow Extended, Google Cloud, Vertex AI Preface Acknowledgements About this Book Welcome to Deep Learning in Production Is this book for me? What is the book’s goal? Will this be difficult to learn? Why should you read this book? How to use this book? How is the book structured? Do I need to know anything else before I get started? Designing a Machine Learning System Machine learning: phase zero Data engineering Model engineering DevOps engineering Putting it all together Tackling a real-life problem Setting up a Deep Learning Workstation Laptop setup Laptop requirements Operating system Frameworks and libraries Development tools Terminal Version control Python package and environment management IDE / code editor Other tools Writing and Structuring Deep Learning Code Best practices Project structure Object-oriented programming Configuration Type checking Documentation Unit testing Basics of unit testing Unit tests in Python Tests in Tensorflow Mocking Test coverage Test example cases Integration / acceptance tests Debugging How to a debug deep learning project? Python’s debugger Debugging data with schema validation Logging Python’s Logging module Useful Tensorflow debugging and logging functions Data Processing ETL: Extract, Transform, Load Data reading Loading from multiple sources Parallel data extraction Processing Loading Iterators Optimizing a data pipeline Batching Prefetching Caching Streaming Training Building a trainer Creating a custom training loop Training checkpoints Saving the trained model Visualizing the training with Tensorboard Model validation Training in the cloud Getting started with cloud computing Creating a VM instance Connecting to the VM instance Transferring files to the VM instance Running the training remotely Accessing training data from a remote environment Distributed training Data vs model parallelism Training in a single machine Synchronous training Asynchronous training Model parallelism Serving Preparing the model Building the model’s inference function Creating a web application using Flask Basics of modern web applications Exposing the deep learning model using Flask Creating a client Serving with uWSGI and Nginx Basic Terminology Designing a serving system Setting up a uWSGI server with Flask Setting up Nginx as a reverse proxy Serving with model servers Tensorflow Serving vs Flask Export a Tensorflow model Install Tensorflow Serving Load a model Multiple versions support Multiple models support Batching inferences Deploying Containerizing using Docker and Docker Compose What is a container? What is Docker Setting up Docker Building a deep learning Docker image Running a deep learning Docker container Creating an Nginx container Defining multi-container Docker apps using Docker Compose Deploying in a production environment Using containers in Google Cloud Allowing network traffic to the instance Deploying in Google Cloud Continuous Integration and Delivery (CI / CD) Scaling A journey from 1 to millions of users First iterations of the machine learning app Vertical vs horizontal scaling Autoscaling Cache mechanisms Monitoring alerts Retraining machine learning models Model A/B testing Offline inference Growing with Kubernetes What is Kubernetes? Getting started with Kubernetes Deploying with Google Kubernetes Engine Scaling with Kubernetes Updating the application Monitoring the application Running a (re)training job Using Kubernetes with GPUs Model A/B testing Building an End-to-End Pipeline MLOps Basic principles MLOps levels Building a pipeline using TFX TFX glossary Data ingestion Data validation Feature engineering Train the model Validate model Push model Build a TFX pipeline Run a TFX pipeline MLOps with Vertex AI and Google Cloud Hands on Vertex AI Experimenting with notebooks Loading data Training the model Deploying to Vertex AI Creating a pipeline More end-to-end solutions Where to Go from Here Appendix List of Figures Index About the Author