Practical concurrent Haskell : with big data applications

Learn to use the APIs and frameworks for parallel and concurrent applications in Haskell. This book will show you how to exploit multicore processors with the help of parallelism in order to increase the performance of your applications. __Practical Concurrent Haskell__ teaches you how concurrency enables you to write programs using threads for multiple interactions. After accomplishing this, you will be ready to make your move into application development and portability with applications in cloud computing and big data. You'll use MapReduce and other, similar big data tools as part of your Haskell big data applications development. **What You'll Learn** * Program with Haskell * Harness concurrency to Haskell * Apply Haskell to big data and cloud computing applications * Use Haskell concurrency design patterns in big data * Accomplish iterative data processing on big data using Haskell * Use MapReduce and work with Haskell on large clusters **Who This Book Is For**Those with at least some prior experience with Haskell and some prior experience with big data in another programming language such as Java, C#, Python, or C++. Contents at a Glance 4 Contents 6 About the Authors 14 About the Technical Reviewer 15 Part I: Haskell Foundations. General Introductory Notions 16 Chapter 1: Introduction 17 What Is Haskell? 17 A Little Bit of Haskell History 19 The Cloud and Haskell 20 Book Structure 23 Summary 25 Chapter 2: Programming with Haskell 26 Functional vs. Object-Oriented Programming 26 Language Basics 27 Arithmetic 28 Pairs, Triples, and Much More 30 Lists 31 Source Code Files 34 Functions 34 if-else 35 Types 36 Simple vs. Polymorphic Types 37 Parametric Polymorphism 37 Ad hoc Polymorphism 37 Type Classes 37 Function Types 38 Data Types 38 Input/Output (IO) Mechanisms 39 Modules 43 :load/:reload 44 :module 44 :import 44 Operators Used as Sections and Infix 45 Local Declarations 46 Partial Application 46 Pattern Matching 47 Guards 48 Instance Declarations 49 Rules for the Head 49 Rules for the Context 50 Rules for Instance Termination 50 Other Lists 50 Arrays 51 Immutable Arrays 51 Mutable Arrays 52 Finite Maps 52 Layout Principles and Rules 53 xmonad 53 wxHaskell 54 The Final Word on Lists 54 Advanced Types 55 Monads 57 Other Advanced Techniques 57 map, filter, takeWhile 59 Lambdas 59 Summary 59 Chapter 3: Parallelism and Concurrency with Haskell 60 Annotating the Code for Parallelism 61 Parallelism for Dataflow 62 Concurrent Servers for a Network 64 Threads for Parallel Programming 66 Threads and MVars 68 Distributed Programming 70 Socket Server 70 System.IO for Sockets 71 Concurrency 71 Communication Between Threads 72 The Final Code 73 Running the Server 75 Eval Monad for Parallelism 75 Summary 78 Chapter 4: Strategies Used in the Evaluation Process 79 Redexes and Lazy Evaluation 79 Parallel Strategies in Haskell 84 Scan Family 85 Skeletons 87 Summary 88 Chapter 5: Exceptions 89 Errors 89 Using the error Function 90 Maybe 90 Either 93 Exceptions 94 Lazy Evaluation and Exceptions 94 The handle Function 95 Input/Output Exceptions 96 The throw Function 96 Dynamic Exceptions 96 Summary 98 Chapter 6: Cancellation 99 Asynchronous Exceptions 100 Using Asynchronous Exceptions with mask 102 Extending the bracket Function 105 Writing Safe Channels Using Asynchronous Exceptions 105 timeout Variants 108 Catching Asynchronous Exceptions 109 mask and forkIO Operations 111 Summary 112 Chapter 7: Transactional Memory Case Studies 113 Transactions 113 Introducing Transactional Memory 113 Software Transactional Memory 114 Software Transactional Memory in Haskell 114 A Bank Account Example 117 Transactional Memory Version 120 Blocking and Choice 121 Summary 124 Chapter 8: Debugging Techniques Used in Big Data 125 Data Science 125 Big Data 126 Characteristics 126 Tools 127 Storage and Management 128 Cleaning 128 Data Mining 128 Languages 129 Haskell for Big Data 129 hspark 129 Hadron 130 Cloud Haskell 130 ZeroMQ 131 The Krapsh Library 131 The Developer’s Perspective 132 The User’s Perspective 132 Haskell vs. Data Science 132 Debugging Tehniques 134 Stack Trace 138 Printf and Friends 139 The Safe Library 140 Offline Analysis of Traces 140 Haskell Tracer HAT 140 Hoed: The Lightweight Haskell Tracer and Debugger 140 Dynamic Breakpoints in GHCi 140 Source-Located Errors 141 Other Tricks 142 Locating a Failure in a Library Function 142 Mysterious Parse Errors 142 Infinite Loops 142 Summary 143 Part II: Haskell for Big Data and Cloud Computing 144 Chapter 9: Haskell in the Cloud 145 Processes and Messages 145 Processes 146 Create the First Node 146 Topologies 146 The simplelocalnet Topology 147 Master-Slave Configuration 147 Obtaining Information About Processes 148 Messages to Processes 148 Serialization 149 Starting and Locating Processes 150 Fault Tolerance 151 Process Lifetime 152 Receiving and Matching 153 Monad Transformers Stack 156 Generic Processes 158 Client-Server Example 161 Matching Without Blocking 166 Unexpected Messages 166 Hiding Implementation Details 167 Messages Within Channels 168 Reply Channels 169 Input (Control) Channels 170 Summary 174 Chapter 10: Haskell in Big Data 175 More About Big Data 175 Data Generation 175 Data from Companies 175 IoT Data 176 Biomedical Data 176 Data Generation from Other Fields 176 Data Collection 177 Data Storage 177 Database Technology 178 Models and Tools 178 MapReduce in Haskell 179 Polymorphic Implementation 182 Distributed k-means 183 Summary 185 Chapter 11: Concurrency Design Patterns 186 Active Object 187 Balking Pattern 189 Barrier 190 Disruptor 192 Double-Checked Locking 196 Guarded Suspension 197 Monitor Object 198 Reactor Pattern 199 Scheduler Pattern 199 Thread Pool Pattern 200 Summary 203 Chapter 12: Large-Scale Design in Haskell 204 The Type System 204 Purity 204 Monads for Structuring 204 Type Classes and Existential Types 204 Concurrency and Parallelism 205 Use of FFI 205 The Profiler 205 Time Profiling 205 Space Profiling 205 QuickCheck 206 Refactor 210 Summary 212 Chapter 13: Designing a Shared Memory Approach for Hadoop Streaming Performance 213 Hadoop 213 More About MapReduce 214 Hadoop Distributed File System 214 How Hadoop Works 215 Stage 1 215 Stage 2 215 Stage 3 215 Hadoop Streaming 216 An Improved Streaming Model 216 Hadoop Streaming in Haskell 219 Haskell-Hadoop Library 219 Hadron 220 Summary 228 Chapter 14: Interactive Debugger for Development and Portability Applications Based on Big Data 229 Approaches to Run-Time Type Reconstruction 230 Run-Time Type Inference 230 RTTI and New Types 232 Termination and Efficiency 232 Practical Concerns 233 Implementation in Haskell 233 Summary 237 Chapter 15: Iterative Data Processing on Big Data 239 Programming Model 239 Loop-Aware Task Scheduling 242 Inter-Iteration Locality 242 Experimental Tests and Implementation 243 Summary 243 Chapter 16: MapReduce 244 Incremental and Iterative Techniques 244 Iterative Computation in MapReduce 248 Incremental Iterative Processing on MRBGraph 252 Summary 252 Chapter 17: Big Data and Large Clusters 253 Programming Model 253 Master Data Structures 253 Fault Tolerance 254 Worker Failures 254 Master Failures 254 Locality 254 Task Granularity 254 Backup Tasks 255 Partitioning Function 255 Implementation of Data Processing Techniques 255 Summary 258 Bibliography 259 Index 267 Front Matter ....Pages i-xv Front Matter ....Pages 1-1 Introduction (Stefania Loredana Nita, Marius Mihailescu)....Pages 3-11 Programming with Haskell (Stefania Loredana Nita, Marius Mihailescu)....Pages 13-46 Parallelism and Concurrency with Haskell (Stefania Loredana Nita, Marius Mihailescu)....Pages 47-65 Strategies Used in the Evaluation Process (Stefania Loredana Nita, Marius Mihailescu)....Pages 67-76 Exceptions (Stefania Loredana Nita, Marius Mihailescu)....Pages 77-86 Cancellation (Stefania Loredana Nita, Marius Mihailescu)....Pages 87-100 Transactional Memory Case Studies (Stefania Loredana Nita, Marius Mihailescu)....Pages 101-112 Debugging Techniques Used in Big Data (Stefania Loredana Nita, Marius Mihailescu)....Pages 113-131 Front Matter ....Pages 133-133 Haskell in the Cloud (Stefania Loredana Nita, Marius Mihailescu)....Pages 135-164 Haskell in Big Data (Stefania Loredana Nita, Marius Mihailescu)....Pages 165-175 Concurrency Design Patterns (Stefania Loredana Nita, Marius Mihailescu)....Pages 177-194 Large-Scale Design in Haskell (Stefania Loredana Nita, Marius Mihailescu)....Pages 195-203 Designing a Shared Memory Approach for Hadoop Streaming Performance (Stefania Loredana Nita, Marius Mihailescu)....Pages 205-220 Interactive Debugger for Development and Portability Applications Based on Big Data (Stefania Loredana Nita, Marius Mihailescu)....Pages 221-230 Iterative Data Processing on Big Data (Stefania Loredana Nita, Marius Mihailescu)....Pages 231-235 MapReduce (Stefania Loredana Nita, Marius Mihailescu)....Pages 237-245 Big Data and Large Clusters (Stefania Loredana Nita, Marius Mihailescu)....Pages 247-252 Back Matter ....Pages 253-266 Learn to use the APIs and frameworks for parallel and concurrent applications in Haskell. This book will show you how to exploit multicore processors with the help of parallelism in order to increase the performance of your applications. Practical Concurrent Haskell teaches you how concurrency enables you to write programs using threads for multiple interactions. After accomplishing this, you will be ready to make your move into application development and portability with applications in cloud computing and big data. You'll use MapReduce and other, similar big data tools as part of your Haskell big data applications development. What You'll Learn Program with Haskell Harness concurrency to Haskell Apply Haskell to big data and cloud computing applications Use Haskell concurrency design patterns in big data Accomplish iterative data processing on big data using Haskell Use MapReduce and work with Haskell on large clusters Who This Book Is For Those with at least some prior experience with Haskell and some prior experience with big data in another programming language such as Java, C#, Python, or C .