PRO SERVERLESS DATA HANDLING WITH MICROSOFT AZURE : architecting etl and data driven applications... in the cloud

Design and build architectures on the Microsoft Azure platform specifically for data-driven and ETL applications. Modern cloud architectures rely on serverless components more than ever, and this book helps you identify those components of data-driven or ETL applications that can be tackled using the technologies available on the Azure platform. The book shows you which Azure components are best suited to form a strong foundation for data-driven applications in the Microsoft Azure Cloud. If you are a solution architect or a decision maker, the conceptual aspects of this book will help you gain a deeper understanding of the underlying technology and its capabilities. You will understand how to develop using Azure Functions, Azure Data Factory, Logic Apps, and to employ serverless databases in your application to achieve the best scalability and design. If you are a developer, you will benefit from the hands-on approach used throughout this book. Many practical examples and architectures applied in real-world projects will be valuable to you on your path to serverless success. What You Will Learn• Know what services are available in Microsoft Azure that can deal with large amounts of data• Design modern data applications based on serverless technology in the cloud• Transform and present data without the use of infrastructure• Employ proven design patterns for rapid implementation of serverless data applications• Choose the correct set of development tools for the services you are using• Understand the term "serverless" and how it can be a benefit• Identify scenarios in which serverless is not the best option available Table of Contents About the Authors About the Technical Reviewer Acknowledgments Introduction Part I: The Basics Chapter 1: Azure Basics The Different Cloud Service Models Infrastructure as a Service (IaaS) Platform as a Service (PaaS) Software as a Service (SaaS) Cloud Model Responsibilities The Structure of Microsoft Azure Azure Geographies Azure Regions Azure Availability Zones Azure Account Azure Subscription Azure Resource Groups Azure Resource Manager Creating and Naming the Resources Creating Resources Naming Resources Overview of Data Services Data Categories Azure Data Services Summary Chapter 2: Serverless Computing Cloud Software Delivery Serverless Delivery The Cost of Perfection Handling Data Chapter 3: Data-Driven Applications ETL the Classic Way Transformation: What Does That Mean? Different Data Models for Different Applications OLTP: The Relational Model Table Key Relationship OLAP: Star and Snowflake Schemas Modern Data Warehouses and Data Applications Part II: Hands-On Chapter 4: Azure Functions The Flavors of Azure Functions Triggers and Bindings Creating Your First Azure Function Creating the Azure Resources Creating the Function A Look at the Code Testing the Function Deploying Your Function Handling State The Basics The Code Running It in the Cloud Chapter 5: Logic Apps Principles of Code-Free Implementation Creating a Logic App The Logic Apps UI Chapter 6: Azure Data Factory The Building Blocks of ADF Working with Azure Data Factory Creating an ADF Using Azure CLI Preparing Resources Creating a Pipeline Parametrizing Your Pipeline Creating a Data Flow Best Practices Using Git Using Azure Key Vault Chapter 7: Database and Storage Options Relational and Non-Relational Data Explained Storage Accounts Storage Account Basics Creating a Storage Account Using Azure Table Storage Azure Queue Storage Cosmos DB Use Cases for Cosmos DB Accounts Azure SQL DB Serverless Creating a Serverless SQL Database When to Choose What? Chapter 8: IoT Hub, Event Hub, and Streaming Data IoT Hub Event Hub Service Bus Stream Analytics Chapter 9: Power BI Power BI Service and Power BI Desktop Building Data Visualizations with Power BI Reports Visualizing Data Streams Sharing Content Licensing of Power BI Part III: Design Practices Chapter 10: Achieving Resiliency What Is Resiliency? How Is Resiliency Ensured? Different Areas to Be Resilient Patterns That Support Resiliency Choosing the Right Services for Resiliency Achieving Resiliency Chapter 11: Queues, Messages, and Commands Messages Events Commands Scenarios for Events and Commands Implementing the Scenario Chapter 12: Processing Streams of Data Streaming Data—What Is It About? Stream Processing: Lambda Architecture Implementing a Lambda Architecture in Azure There’s More... Chapter 13: Monitoring Serverless Applications Monitoring and Alerting Serverless and Monitoring Implementing Monitoring Implementing Alerting Part IV: Putting It All Together Chapter 14: Tools and Helpers Visual Studio Code Azure Data Studio Docker / Docker Desktop Azure CLI PowerShell Bicep / ARM Templates Azure Storage Explorer Azure DevOps dbatools Azure Quickstart Templates Git Git Kraken Chocolatey Azure Data Community Useful Visual Studio Code Plugins Chapter 15: Data-Loading Patterns Data-Loading Patterns for Flat Files Data-Loading Patterns for REST APIs Data-Loading Patterns for Databases Data-Loading Patterns for Data Streams Chapter 16: Data Storage Patterns Relational Databases Storage Accounts Non-Relational Databases Chapter 17: Architecture for a Modern Data-Driven Application REST API, Tracking & Transaction Data Communicating with the Shops Data Warehousing and Analytics Index Design and build architectures on the Microsoft Azure platform specifically for data-driven and ETL applications. Modern cloud architectures rely on serverless components more than ever, and this book helps you identify those components of data-driven or ETL applications that can be tackled using the technologies available on the Azure platform. The book shows you which Azure components are best suited to form a strong foundation for data-driven applications in the Microsoft Azure Cloud.If you are a solution architect or a decision maker, the conceptual aspects of this book will help you gain a deeper understanding of the underlying technology and its capabilities. You will understand how to develop using Azure Functions, Azure Data Factory, Logic Apps, and to employ serverless databases in your application to achieve the best scalability and design. If you are a developer, you will benefit from the hands-on approach used throughout this book. Many practical examples andarchitectures applied in real-world projects will be valuable to you on your path to serverless success. What You Will LearnKnow what services are available in Microsoft Azure that can deal with large amounts of dataDesign modern data applications based on serverless technology in the cloudTransform and present data without the use of infrastructureEmploy proven design patterns for rapid implementation of serverless data applicationsChoose the correct set of development tools for the services you are usingUnderstand the term'serverless'and how it can be a benefitIdentify scenarios in which serverless is not the best option availableWho This Book Is ForArchitects and decision makers who want to understand how modern architectures are designed and how to modernize their applications. The book is aimed at the developer who needs a steppingstone to quickly implement a serverless data application. And the book is for any IT professional who seeks a head start to serverless computing for data-heavy applications on the Azure platform.