Modern DevOps Practices - Second Edition: Implement, secure, and manage applications on the public cloud by leveraging cutting-edge tools

Use GitLab CI/CD pipelines for automating and deploying different steps of your software development lifecycle using best practices and troubleshooting methods. Key Features Reap the power of GitLab CI/CD pipelines at every stage of your software development lifecycle Learn how GitLab makes Git easier to use and more powerful when committing and reviewing code Cement your understanding using hands-on tutorials and extensive self-assessment exercises Purchase of the print or Kindle book includes a free eBook in the PDF format Book Description Developers and release engineers understand the high stakes involved in building, packaging, and deploying code correctly. Ensuring that your code is functionally correct, fast, and secure is a time-consuming and complex task. Code implementation, development, and deployment can be conducted efficiently using GitLab CI/CD pipelines. Automating DevOps with GitLab CI/CD Pipelines begins with the basics of Git and GitLab, showing how to commit and review code. You'll learn to set up GitLab Runners for executing and autoscaling CI/CD pipelines and creating and configuring pipelines for many software development lifecycle steps. You'll also discover where to find pipeline results in GitLab, and how to interpret those results. Through the course of the book, you'll become well-equipped with deploying code to different environments, advancing CI/CD pipeline features such as connecting GitLab to a Kubernetes cluster and using GitLab with Terraform, triggering pipelines and improving pipeline performance and using best practices and troubleshooting tips for uncooperative pipelines. In-text examples, use cases, and self-assessments will reinforce the important CI/CD, GitLab, and Git concepts, and help you prepare for interviews and certification exams related to GitLab. By the end of this book, you'll be able to use GitLab to build CI/CD pipelines that automate all the DevOps steps needed to build and deploy high-quality, secure code. What you will learn Gain insights into the essentials of Git, GitLab, and DevOps Understand how to create, view, and run GitLab CI/CD pipelines Explore how to verify, secure, and deploy code with GitLab CI/CD pipelines Configure and use GitLab Runners to execute CI/CD pipelines Explore advanced GitLab CI/CD pipeline features like DAGs and conditional logic Follow best practices and troubleshooting methods of GitLab CI/CD pipelines Implement end-to-end software development lifecycle workflows using examples Who this book is for This book is for DevOps/DevSecOps engineers, application developers, release engineers, quality assurance engineers, security engineers, SREs, and sysadmins looking to implement fast, secure and automated software development lifecycle tasks using continuous integration and continuous delivery (CI/CD) pipelines in GitLab. Basic knowledge of major stages of the software development life cycle and DevOps processes will be helpful Modern DevOps Practices Contributors About the author About the reviewer Preface Who this book is for What this book covers To get the most out of this book Download the example code files Conventions used Get in touch Share your thoughts Download a free PDF copy of this book Part 1:Modern DevOps Fundamentals 1 The Modern Way of DevOps What is DevOps? Introduction to cloud computing Understanding modern cloud-native applications Modern DevOps versus traditional DevOps The need for containers The matrix of hell Virtual machines Containers It works on my machine Container architecture Container networking Containers and modern DevOps practices Migrating from virtual machines to containers Discovery Application requirement assessment Container infrastructure design Containerizing the application Testing Deployment and rollout What applications should go in containers? Breaking the applications into smaller pieces Are we there yet? Summary Questions Answers 2 Source Code Management with Git and GitOps Technical requirements What is source code management? A crash course on Git Installing Git Initializing your first Git repository Staging code changes Displaying commit history Amending the last commit Understanding remote repositories Creating a remote Git repository Setting up authentication with the remote Git repository Connecting the local repository to the remote repository Pushing changes from the local repository to the remote repository Pulling and rebasing your code Git branches Creating and managing Git branches Working with pull requests What is GitOps? Why GitOps? The principles of GitOps Branching strategies and the GitOps workflow The push model The pull model Structuring the Git repository Git versus GitOps Summary Questions Answers 3 Containerization with Docker Technical requirements Installing Docker Introducing Docker storage drivers and volumes Docker data storage options Mounting volumes Docker storage drivers Configuring a storage driver Running your first container Running containers from versioned images Running Docker containers in the background Troubleshooting containers Putting it all together Restarting and removing containers Docker logging and logging drivers Container log management Logging drivers Configuring logging drivers Typical challenges and best practices to address these challenges with Docker logging Docker monitoring with Prometheus Challenges with container monitoring Installing Prometheus Configuring cAdvisor and the node exporter to expose metrics Configuring Prometheus to scrape metrics Launching a sample container application Metrics to monitor Declarative container management with Docker Compose Deploying a sample application with Docker Compose Creating the docker-compose file Docker Compose best practices Summary Questions Answers 4 Creating and Managing Container Images Technical requirements Docker architecture Understanding Docker images The layered filesystem Image history Understanding Dockerfiles, components, and directives Can we use ENTRYPOINT instead of CMD? Are RUN and CMD the same? Building our first container Building and managing Docker images Single-stage builds Multi-stage builds Managing Docker images Flattening Docker images Optimizing containers with distroless images Performance Security Cost Understanding Docker registries Hosting your private Docker registry Other public registries Summary Questions Answers Part 2:Container Orchestration and Serverless 5 Container Orchestration with Kubernetes Technical requirements What is Kubernetes, and why do I need it? Kubernetes architecture Installing Kubernetes (Minikube and KinD) Installing Minikube Installing KinD Understanding Kubernetes pods Running a pod Using port forwarding Troubleshooting pods Ensuring pod reliability Pod multi-container design patterns Summary Questions Answers 6 Managing Advanced Kubernetes Resources Technical requirements Spinning up GKE The need for advanced Kubernetes resources Kubernetes Deployments ReplicaSet resources Deployment resources Kubernetes Deployment strategies Kubernetes Services and Ingresses ClusterIP Service resources NodePort Service resources LoadBalancer Service resources Ingress resources Horizontal Pod autoscaling Managing stateful applications StatefulSet resources Managing Persistent Volumes Kubernetes command-line best practices, tips, and tricks Using aliases Using kubectl bash autocompletion Summary Questions Answers 7 Containers as a Service (CaaS) and Serverless Computing for Containers Technical requirements The need for serverless offerings Amazon ECS with EC2 and Fargate ECS architecture Installing the AWS and ECS CLIs Spinning up an ECS cluster Creating task definitions Scheduling EC2 tasks on ECS Scaling tasks Querying container logs from CloudWatch Stopping tasks Scheduling Fargate tasks on ECS Scheduling services on ECS Browsing container logs using the ECS CLI Deleting an ECS service Load balancing containers running on ECS Other CaaS services Open source CaaS with Knative Knative architecture Spinning up GKE Installing Knative Deploying a Python Flask application on Knative Load testing your app on Knative Summary Questions Answers Part 3:Managing Config and Infrastructure 8 Infrastructure as Code (IaC) with Terraform Technical requirements Introduction to IaC Installing Terraform Terraform providers Authentication and authorization with Azure Using the Azure Terraform provider Terraform variables Providing variable values Terraform workflow terraform init Creating the first resource – Azure resource group terraform fmt terraform validate terraform plan terraform apply terraform destroy Terraform modules Managing Terraform state Using the Azure Storage backend Terraform workspaces Inspecting resources Inspecting state files Cleaning up Terraform output, state, console, and graphs terraform output Managing Terraform state terraform console Terraform dependencies and graphs Cleaning up resources Summary Questions Answers 9 Configuration Management with Ansible Technical requirements Introduction to configuration management Setting up Ansible Setting up inventory Connecting the Ansible control node with inventory servers Installing Ansible in the control node Setting up an inventory file Setting up the Ansible configuration file Ansible tasks and modules Introduction to Ansible playbooks Checking playbook syntax Applying the first playbook Ansible playbooks in action Updating packages and repositories Installing application packages and services Configuring applications Combining playbooks Executing playbooks Designing for reusability Ansible variables Sourcing variable values Jinja2 templates Ansible roles Summary Questions Answers 10 Immutable Infrastructure with Packer Technical requirements Immutable infrastructure with HashiCorp’s Packer When to use immutable infrastructure Installing Packer Creating the Apache and MySQL playbooks Building the Apache and MySQL images using Packer and Ansible provisioners Prerequisites Defining the Packer configuration The Packer workflow for building images Creating the required infrastructure with Terraform Summary Questions Answers Part 4:Delivering Applications with GitOps 11 Continuous Integration with GitHub Actions and Jenkins Technical requirements The importance of automation Introduction to the sample microservices-based blogging application – Blog App Building a CI pipeline with GitHub Actions Creating a GitHub repository Creating a GitHub Actions workflow Scalable Jenkins on Kubernetes with Kaniko Spinning up Google Kubernetes Engine Creating the Jenkins CaC (JCasC) file Installing Jenkins Running our first Jenkins job Automating a build with triggers Building performance best practices Aim for faster builds Always use post-commit triggers Configure build reporting Customize the build server size Ensure that your builds only contain what you need Parallelize your builds Make use of caching Use incremental building Optimize testing Use artifact management Manage application dependencies Utilize Infrastructure as Code Use containerization to manage build and test environments Utilize cloud-based CI/CD Monitor and profile your CI/CD pipelines Pipeline optimization Implement automated cleanup Documentation and training Summary Questions Answers 12 Continuous Deployment/Delivery with Argo CD Technical requirements The importance of CD and automation CD models and tools Simple deployment model Complex deployment models The Blog App and its deployment configuration Continuous declarative IaC using an Environment repository Creating and setting up our Environment repository Introduction to Argo CD Installing and setting up Argo CD Terraform changes The Kubernetes manifests Argo CD Application and ApplicationSet Accessing the Argo CD Web UI Managing sensitive configurations and Secrets Installing the Sealed Secrets operator Installing kubeseal Creating Sealed Secrets Deploying the sample Blog App Summary Questions Answers 13 Securing and Testing Your CI/CD Pipeline Technical requirements Securing and testing CI/CD pipelines Revisiting the Blog Application Container vulnerability scanning Installing Anchore Grype Scanning images Managing secrets Creating a Secret in Google Cloud Secret Manager Accessing external secrets using External Secrets Operator Setting up the baseline Installing external secrets with Terraform Testing your application within the CD pipeline CD workflow changes Binary authorization Setting up binary authorization Release gating with pull requests and deployment to production Merging code and deploying to prod Security and testing best practices for modern DevOps pipelines Adopt a DevSecOps culture Establish access control Implement shift left Manage security risks consistently Implement vulnerability scanning Automate security Test automation within your CI/CD pipelines Manage your test data effectively Test all aspects of your application Implement chaos engineering Monitor and observe your application when it is being tested Effective testing in production Documentation and knowledge sharing Summary Questions Answers Part 5:Operating Applications in Production 14 Understanding Key Performance Indicators (KPIs) for Your Production Service Understanding the importance of reliability Understanding SLIs, SLOs, and SLAs SLIs SLOs SLAs Error budgets Disaster recovery, RTO, and RPO Running distributed applications in production Summary Questions Answers 15 Implementing Traffic Management, Security, and Observability with Istio Technical requirements Setting up the baseline Revisiting the Blog App Introduction to service mesh Introduction to Istio Traffic management Security Observability Developer-friendly Understanding the Istio architecture The control plane architecture The data plane architecture Installing Istio Enabling automatic sidecar injection Using Istio ingress to allow traffic Securing your microservices using Istio Creating secure ingress gateways Enforcing TLS within your service mesh Managing traffic with Istio Traffic shifting and canary rollouts Traffic mirroring Observing traffic and alerting with Istio Accessing the Kiali dashboard Monitoring and alerting with Grafana Summary Questions Answers Appendix: The Role of AI in DevOps What is AI? The role of AI in the DevOps infinity loop Code development Software testing and quality assurance Continuous integration and delivery Software operations Summary Index Why subscribe? Other Books You May Enjoy Packt is searching for authors like you Share your thoughts Download a free PDF copy of this book Enhance DevOps workflows by integrating the functionalities of Git, Docker, Kubernetes, Argo CD, Ansible, Terraform, Istio, and more with the help of practical examples and expert tipsKey FeaturesExplore containers as a service (CaaS) and infrastructure automation in the public cloudSecure and ship software continuously to production with DevOps, GitOps, SecOps, and automationOperate distributed and scalable microservices apps in the cloud with a modern service meshPurchase of the print or Kindle book includes a free PDF eBookBook DescriptionDevOps and the cloud have changed how we look at software development and operations like never before, leading to the rapid growth of various DevOps tools, techniques, and practices. This updated edition helps you pick up the right tools by providing you with everything you need to get started with your DevOps journey. The book begins by introducing you to modern cloud-native architecture, and then teaches you about the architectural concepts needed to implement the modern way of application development. The next set of chapters helps you get familiarized with Git, Docker, Kubernetes, Ansible, Terraform, Packer, and other similar tools to enable you to build a base. As you advance, you'll explore the core elements of cloud integration—AWS ECS, GKE, and other CaaS services. The chapters also discuss GitOps, continuous integration, and continuous delivery—GitHub actions, Jenkins, and Argo CD—to help you understand the essence of modern app delivery. Later, you'll operate your container app in production using a service mesh and apply AI in DevOps. Throughout the book, you'll discover best practices for automating and managing your development lifecycle, infrastructure, containers, and more. By the end of this DevOps book, you'll be well-equipped to develop and operate applications using modern tools and techniques.What you will learnExplore modern DevOps practices with Git and GitOpsMaster container fundamentals with Docker and KubernetesBecome well versed in AWS ECS, Google Cloud Run, and KnativeDiscover how to efficiently build and manage secure Docker imagesUnderstand continuous integration with Jenkins on Kubernetes and GitHub ActionsGet to grips with using Argo CD for continuous deployment and deliveryManage immutable infrastructure on the cloud with Packer, Terraform, and AnsibleOperate container applications in production using Istio and learn about AI in DevOpsWho this book is forIf you are a software engineer, system administrator, or operations engineer looking to step into the world of DevOps within public cloud platforms, this book is for you. Existing DevOps engineers will also find this book helpful as it covers best practices, tips, and tricks for implementing DevOps with a cloud-native mindset. Although no containerization experience is necessary, a basic understanding of the software development life cycle and delivery will help you get the most out of this book. Enhance DevOps workflows by integrating the functionalities of Git, Docker, Kubernetes, Argo CD, Ansible, Terraform, Istio, and more with the help of practical examples and expert tips Key Features Explore containers as a service (CaaS) and infrastructure automation in the public cloud Secure and ship software continuously to production with DevOps, GitOps, SecOps, and automation Operate distributed and scalable microservices apps in the cloud with a modern service mesh Book Description DevOps and the cloud have changed how we look at software development and operations like never before, leading to the rapid growth of various DevOps tools, techniques, and practices. This updated edition helps you pick up the right tools by providing you with everything you need to get started with your DevOps journey. The book begins by introducing you to modern cloud-native architecture, and then teaches you about the architectural concepts needed to implement the modern way of application development. The next set of chapters helps you get familiarized with Git, Docker, Kubernetes, Ansible, Terraform, Packer, and other similar tools to enable you to build a base. As you advance, you’ll explore the core elements of cloud integration—AWS ECS, GKE, and other CaaS services. The chapters also discuss GitOps, continuous integration, and continuous delivery—GitHub actions, Jenkins, and Argo CD—to help you understand the essence of modern app delivery. Later, you’ll operate your container app in production using a service mesh and apply AI in DevOps. Throughout the book, you’ll discover best practices for automating and managing your development lifecycle, infrastructure, containers, and more. By the end of this DevOps book, you'll be well-equipped to develop and operate applications using modern tools and techniques. What you will learn Explore modern DevOps practices with Git and GitOps Master container fundamentals with Docker and Kubernetes Become well versed in AWS ECS, Google Cloud Run, and Knative Discover how to efficiently build and manage secure Docker images Understand continuous integration with Jenkins on Kubernetes and GitHub Actions Get to grips with using Argo CD for continuous deployment and delivery Manage immutable infrastructure on the cloud with Packer, Terraform, and Ansible Operate container applications in production using Istio and learn about AI in DevOps Who this book is for If you are a software engineer, system administrator, or operations engineer looking to step into the world of DevOps within public cloud platforms, this book is for you. Existing DevOps engineers will also find this book helpful as it covers best practices, tips, and tricks for implementing DevOps with a cloud-native mindset. Although no containerization experience is necessary, a basic understanding of the software development life cycle and delivery will help you get the most out of this book. Discover how the DevOps approach with Scaled Agile Framework helps you develop and deliver high-quality, secured solutions with a reduced risk of production failures with this step-by-step guide Key Features Explore the five elements of the CALMR approach to avoid product development challenges Use value stream management to introduce systems thinking and flow for product development Demonstrate how the CD pipeline combines practices and technology to optimize your value stream Purchase of the print or Kindle book includes a free eBook in the PDF format Book Description Product development and release faces overlapping challenges due to the combined pressure of delivering high-quality products in shorter time-to-market cycles, along with maintaining proper operation and ensuring security in a complex high-tech environment. This calls for new ways of overcoming these challenges from design to development, to release, and beyond. SAFe® for DevOps Practitioners helps you use a DevOps approach with the Scaled Agile Framework and details how value streams help you resolve these challenges using examples and use cases. The book begins by explaining how the CALMR approach makes DevOps effective in resolving product development roadblocks. Next, you'll learn to apply value stream management to establish a value stream that enables product development flow, measure its effectiveness through appropriate feedback loops, and find ways of improving it. Finally, you'll get to grips with implementing a continuous delivery pipeline that optimizes the value stream through four phases during release on demand. This book complements the latest SAFe DevOps courses, and you'll find it useful while studying for the SAFe DevOps Practitioner (SDP) certification. By the end of this DevOps book, you'll have gained a clear understanding of how to achieve continuous execution and release on demand using DevOps and SAFe. What you will learn Understand the important elements of the CALMR approach Discover how to organize around value using value stream mapping Measure your value stream using value stream metrics Improve your value stream with continuous learning Use continuous exploration to design high-quality and secure features Prevent rework and build in quality using continuous integration Automate delivery with continuous deployment Measure successful outcomes with Release on Demand Who this book is for This book is for IT professionals such as DevOps and DevSecOps practitioners, SREs, and managers who are interested in implementing DevOps practices using the Scaled Agile Framework (SAFe) approach. Basic knowledge of DevOps and agile software development lifecycle and methodology will be helpful