Big Data : A Tutorial-Based Approach

Big Data: A Tutorial-Based Approach explores the tools and techniques used to bring about the marriage of structured and unstructured data. It focuses on Hadoop Distributed Storage and MapReduce Processing by implementing (i) Tools and Techniques of Hadoop Eco System, (ii) Hadoop Distributed File System Infrastructure, and (iii) efficient MapReduce processing. The book includes Use Cases and Tutorials to provide an integrated approach that answers the ‘What’, ‘How’, and ‘Why’ of Big Data. Features Identifies the primary drivers of Big Data Walks readers through the theory, methods and technology of Big Data Explains how to handle the 4 V’s of Big Data in order to extract value for better business decision making Shows how and why data connectors are critical and necessary for Agile text analytics Includes in-depth tutorials to perform necessary set-ups, installation, configuration and execution of important tasks Explains the command line as well as GUI interface to a powerful data exchange tool between Hadoop and legacy r-dbms databases Cover 1 Half Title 2 Title Page 4 Copyright Page 5 Dedication 6 Contents 8 List of Tutorials 14 List of Figures/Illustrations 16 Foreword 18 Preface 20 Acknowledgements 24 Author 26 Chapter 1: Introduction to Big Data 28 OVERVIEW 28 RAPID GROWTH OF BIG DATA 28 BIG DATA DEFINITION 30 BIG DATA PROJECTS 31 BUSINESS VALUE OF BIG DATA 32 Chapter 2: Big Data Implementation 36 OVERVIEW 36 HIGH-LEVEL TASKS TO IMPLEMENT INFORMATICA BDM, CLOUDERA HIVE, AND TABLEAU 37 BIG DATA TRIGGERS DIGITAL TRANSFORMATION OF THE PRODUCTION MODEL 38 BIG DATA CHALLENGES AND ASSOCIATED USE CASES 40 HADOOP INFRASTRUCTURE: OVERVIEW 41 HADOOP INFRASTRUCTURE: DEFINED 42 Hyperconverged Hadoop Infrastructure 42 Compute Hardware Components 43 Network Hardware Components 44 Storage Hardware Architecture and Components 46 HADOOP ECO SYSTEM 47 HADOOP: JVM FRAMEWORK 49 HADOOP DISTRIBUTED FILE PROCESSING 49 MAPREDUCE SOFTWARE 53 MAPREDUCE SOFTWARE INSTALLATION 54 MAPREDUCE PROCESSING 55 Chapter 3: Big Data Use Cases 60 OVERVIEW 60 BIG DATA USE CASE: HEALTH 60 BIG DATA USE CASE: MANUFACTURING 62 BIG DATA USE CASE: INSURANCE 63 Chapter 4: Big Data Migration 66 OVERVIEW 66 CHALLENGES IN MIGRATING ORACLE DATA USING SQOOP 68 WHERE IS SQOOP USED? 68 SQOOP COMMANDS 69 HIVE ARGUMENTS USED BY SQOOP 70 APACHE SQOOP ARCHITECTURE 71 APACHE SQOOP COMMAND LINE INTERFACE 72 Chapter 5: Big Data Ingestion, Integration, and Management 76 OVERVIEW 76 INFORMATICA: MATURE AND COMPREHENSIVE BIG DATA SOLUTION 77 INFORMATICA DATA INTEGRATION 79 Chapter 6: Big Data Repository 86 OVERVIEW 86 DATA REPOSITORY LAYER 88 HIVE BIG DATA WAREHOUSE 89 SLOWLY CHANGING DIMENSION IN HIVE 90 HIVE METADATA: DEFINITIONS 92 INTEGRATED USE OF DATA INTEGRATION, DATA MANAGEMENT, AND DATA VISUALIZATION TOOLS 99 Chapter 7: Big Data Visualization 102 OVERVIEW 102 VARIABLE TYPES 110 Numbers 110 Strings 112 Factors 113 SUCCESS FACTORS FOR TABLEAU 114 TABLEAU: STEP FORWARD IN DATA ANALYTICS 115 TABLEAU CONNECTORS FOR DATA SOURCES 120 TABLEAU DATA ENGINE TUNING 120 TABLEAU TUNING FEATURES 127 Fast Interactive Query Engine 127 Strategically Utilize Live Connections versus Extracts 127 Curate Data from the Data Lake 127 Optimize Data Extracts 128 Customize Tableau Connection Performance 129 Chapter 8: Structured and Un-Structured Data Analytics 130 OVERVIEW 130 TEXT ANALYTICS AS MEANS TO EXTRACT VALUE FROM UN-STRUCTURED DATA 131 MAJOR PLAYERS IN TEXT ANALYTICS 132 Decision Maker 132 Domain Expert 133 Linguist 133 Data Scientists 133 Conclusion 134 FROM DATA TO ACTION 134 CONCLUSION 141 Chapter 9: Data Virtualization 142 OVERVIEW 142 Conclusion: Flexibility and Agility 150 Pre-Installation Steps to Set Up Denodo Development Environment 151 CONCLUSION 163 Chapter 10: Cloud Computing 164 OVERVIEW 164 A QUICK GLANCE AT CLOUD COMPUTING 166 Software as a Service (SaaS) 166 Platform as a Service (PaaS) 166 Infrastructure as a Service (IaaS) 167 CLOUD COMPUTING VERSUS HADOOP PROCESSING 168 CLOUD SERVICE MOST SUITED FOR BIG DATA 169 Infrastructure as a Service (IaaS) 169 Advantages of IaaS 170 CONCLUSION 170 SELF-ASSESSMENT QUIZ 172 ANSWERS TO THE SELF-ASSESSMENT QUIZ 180 REFERENCES 190 INDEX 194 Cover......Page 1 Half Title......Page 2 Title Page......Page 4 Copyright Page......Page 5 Dedication......Page 6 Contents......Page 8 List of Tutorials......Page 14 List of Figures/Illustrations......Page 16 Foreword......Page 18 Preface......Page 20 Acknowledgements......Page 24 Author......Page 26 RAPID GROWTH OF BIG DATA......Page 28 BIG DATA DEFINITION......Page 30 BIG DATA PROJECTS......Page 31 BUSINESS VALUE OF BIG DATA......Page 32 OVERVIEW......Page 36 HIGH-LEVEL TASKS TO IMPLEMENT INFORMATICA BDM, CLOUDERA HIVE, AND TABLEAU......Page 37 BIG DATA TRIGGERS DIGITAL TRANSFORMATION OF THE PRODUCTION MODEL......Page 38 BIG DATA CHALLENGES AND ASSOCIATED USE CASES......Page 40 HADOOP INFRASTRUCTURE: OVERVIEW......Page 41 Hyperconverged Hadoop Infrastructure......Page 42 Compute Hardware Components......Page 43 Network Hardware Components......Page 44 Storage Hardware Architecture and Components......Page 46 HADOOP ECO SYSTEM......Page 47 HADOOP DISTRIBUTED FILE PROCESSING......Page 49 MAPREDUCE SOFTWARE......Page 53 MAPREDUCE SOFTWARE INSTALLATION......Page 54 MAPREDUCE PROCESSING......Page 55 BIG DATA USE CASE: HEALTH......Page 60 BIG DATA USE CASE: MANUFACTURING......Page 62 BIG DATA USE CASE: INSURANCE......Page 63 OVERVIEW......Page 66 WHERE IS SQOOP USED?......Page 68 SQOOP COMMANDS......Page 69 HIVE ARGUMENTS USED BY SQOOP......Page 70 APACHE SQOOP ARCHITECTURE......Page 71 APACHE SQOOP COMMAND LINE INTERFACE......Page 72 OVERVIEW......Page 76 INFORMATICA: MATURE AND COMPREHENSIVE BIG DATA SOLUTION......Page 77 INFORMATICA DATA INTEGRATION......Page 79 OVERVIEW......Page 86 DATA REPOSITORY LAYER......Page 88 HIVE BIG DATA WAREHOUSE......Page 89 SLOWLY CHANGING DIMENSION IN HIVE......Page 90 HIVE METADATA: DEFINITIONS......Page 92 INTEGRATED USE OF DATA INTEGRATION, DATA MANAGEMENT, AND DATA VISUALIZATION TOOLS......Page 99 OVERVIEW......Page 102 Numbers......Page 110 Strings......Page 112 Factors......Page 113 SUCCESS FACTORS FOR TABLEAU......Page 114 TABLEAU: STEP FORWARD IN DATA ANALYTICS......Page 115 TABLEAU DATA ENGINE TUNING......Page 120 Curate Data from the Data Lake......Page 127 Optimize Data Extracts......Page 128 Customize Tableau Connection Performance......Page 129 OVERVIEW......Page 130 TEXT ANALYTICS AS MEANS TO EXTRACT VALUE FROM UN-STRUCTURED DATA......Page 131 Decision Maker......Page 132 Data Scientists......Page 133 FROM DATA TO ACTION......Page 134 CONCLUSION......Page 141 OVERVIEW......Page 142 Conclusion: Flexibility and Agility......Page 150 Pre-Installation Steps to Set Up Denodo Development Environment......Page 151 CONCLUSION......Page 163 OVERVIEW......Page 164 Platform as a Service (PaaS)......Page 166 Infrastructure as a Service (IaaS)......Page 167 CLOUD COMPUTING VERSUS HADOOP PROCESSING......Page 168 Infrastructure as a Service (IaaS)......Page 169 CONCLUSION......Page 170 SELF-ASSESSMENT QUIZ......Page 172 ANSWERS TO THE SELF-ASSESSMENT QUIZ......Page 180 REFERENCES......Page 190 INDEX......Page 194 "This book explores the tools and techniques to bring about the marriage of structured and unstructured data. It focuses on Hadoop Distributed Storage and MapReduce Processing by implementing (i) Tools and Techniques of Hadoop Eco System, (ii) Hadoop Distributed File System Infrastructure, and (iii) efficient MapReduce processing. The book includes Use Cases and Tutorials to provide an integrated approach that answers the 'What', 'How', and 'Why' of Big Data"-- Provided by publisher