Mastering Regular Expressions, Third Edition

Regular expressions are an extremely powerful tool for manipulating text and data. They are now standard features in a wide range of languages and popular tools, including Perl, Python, Ruby, Java, VB.NET and C# (and any language using the .NET Framework), PHP, and MySQL. If you don't use regular expressions yet, you will discover in this book a whole new world of mastery over your data. If you already use them, you'll appreciate this book's unprecedented detail and breadth of coverage. If you think you know all you need to know about regular expressions, this book is a stunning eye-opener. As this book shows, a command of regular expressions is an invaluable skill. Regular expressions allow you to code complex and subtle text processing that you never imagined could be automated. Regular expressions can save you time and aggravation. They can be used to craft elegant solutions to a wide range of problems. Once you've mastered regular expressions, they'll become an invaluable part of your toolkit. You will wonder how you ever got by without them. Yet despite their wide availability, flexibility, and unparalleled power, regular expressions are frequently underutilized. Yet what is power in the hands of an expert can be fraught with peril for the unwary. Mastering Regular Expressions will help you navigate the minefield to becoming an expert and help you optimize your use of regular expressions. Mastering Regular Expressions, Third Edition, now includes a full chapter devoted to PHP and its powerful and expressive suite of regular expression functions, in addition to enhanced PHP coverage in the central "core" chapters. Furthermore, this edition has been updated throughout to reflect advances in other languages, including expanded in-depth coverage of Sun's java.util.regex package, which has emerged as the standard Java regex implementation.Topics include: A comparison of features among different versions of many languages and tools How the regular expression engine works Optimization (major savings available here!) Matching just what you want, but not what you don't want Sections and chapters on individual languages Written in the lucid, entertaining tone that makes a complex, dry topic become crystal-clear to programmers, and sprinkled with solutions to complex real-world problems, Mastering Regular Expressions, Third Edition offers a wealth information that you can put to immediate use. Reviews of this new edition and the second edition: "There isn't a better (or more useful) book available on regular expressions." --Zak Greant, Managing Director, eZ Systems "A real tour-de-force of a book which not only covers the mechanics of regexes in extraordinary detail but also talks about efficiency and the use of regexes in Perl, Java, and .NET...If you use regular expressions as part of your professional work (even if you already have a good book on whatever language you're programming in) I would strongly recommend this book to you." --Dr. Chris Brown, Linux Format "The author does an outstanding job leading the reader from regex novice to master. The book is extremely easy to read and chock full of useful and relevant examples...Regular expressions are valuable tools that every developer should have in their toolbox. Mastering Regular Expressions is the definitive guide to the subject, and an outstanding resource that belongs on every programmer's bookshelf. Ten out of Ten Horseshoes." --Jason Menard, Java Ranch Table of Contents 3 1 Introduction to Regular Expressions 12 Solving Real Problems 13 Regular Expressions as a Language 15 The Filename Analogy 15 The Language Analogy 16 The Regular-Expression Frame of Mind 17 If You Have Some Regular-Expression Experience 17 Searching Text Files: Egrep 17 Egrep Metacharacters 19 Start and End of the Line 19 Character Classes 20 Matching any one of several characters 20 Reading *cat$, ^$, and ^ 21 Negated character classes 21 Matching Any Character with Dot 22 Alternation 24 Matching any one of several subexpressions 24 Ignoring Differences in Capitalization 25 Word Boundaries 26 In a Nutshell 27 Optional Items 28 Other Quantifiers: Repetition 29 Making a Subexpression Optional 31 Defined range of matches: intervals 31 Parentheses and Backreferences 31 The Great Escape 33 Expanding the Foundation 34 Linguistic Diversification 34 The Goal of a Regular Expression 34 A Few More Examples 34 Variable names 35 A string within double quotes 35 Dollar amount (with optional cents) 35 An HTTP/HTML URL 36 An HTML tag 37 Time of day, such as "9:17 am" or "12:30 pm" 37 Regular Expression Nomenclature 38 Regex 38 Matching 38 Metacharacter 38 Flavor 38 Extending the Time Regex to Handle a 24–Hour Clock 39 Subexpression 40 Character 40 Improving on the Status Quo 41 Summary 43 Personal Glimpses 44 2 Extended Introductory Examples 46 About the Examples 47 A Short Introduction to Perl 48 Matching Text with Regular Expressions 49 Toward a More Real-World Example 51 Side Effects of a Successful Match 51 Intertwined Regular Expressions 54 A short aside—metacharacters galore 55 Non-Capturing Parentheses: (?:•••) 56 Generic "whitespace" with \s 58 Intermission 60 Modifying Text with Regular Expressions 61 Example: Form Letter 61 Example: Prettifying a Stock Price 62 Automated Editing 64 A Small Mail Utility 64 A Sample Email Message 65 A Warning About .* 67 Real-world problems, real-world solutions 69 The "real" real world 70 Adding Commas to a Number with Lookaround 70 Lookaround doesn't "consume" text 71 A few more lookahead examples 72 Back to the comma example... 75 Word boundaries and negative lookaround 76 Commafication without lookbehind 78 Text-to-HTML Conversion 78 Cooking special characters 79 Separating paragraphs 80 "Linkizing" an email address 81 Matching the username and hostname 82 Putting it together 84 "Linkizing" an HTTP URL 85 Building a regex library 87 Why '$' and '@' sometimes need to be escaped 88 That Doubled-Word Thing 88 Moving bits around: operators, functions, and objects 91 3 Overview of Regular Expression Features and Flavors 93 Regular Expressions and Cars 93 In This Chapter 94 A Casual Stroll Across the Regex Landscape 95 The Origins of Regular Expressions 95 Grep's metacharacters 96 Grep evolves 96 Egrep evolves 96 Other species evolve 97 POSIX—An attempt at standardization 97 Henry Spencer's regex package 98 Perl evolves 98 A partial consolidation of flavors 100 Versions as of this book 101 At a Glance 101 Care and Handling of Regular Expressions 103 Integrated Handling 104 Procedural and Object-Oriented Handling 105 Regex handling in Java 105 A procedural example 105 Regex handling in VB and other .NET languages 106 Regex handling in PHP 107 Regex handling in Python 107 Why do approaches differ? 107 A Search-and-Replace Example 108 Search and replace in Java 108 Search and replace in VB.NET 109 Search and replace in PHP 109 Search and Replace in Other Languages 110 Awk 110 Tcl 110 GNU Emacs 110 Care and Handling: Summary 111 Strings, Character Encodings, and Modes 111 Strings as Regular Expressions 111 Strings in Java 112 Strings in VB.NET 113 Strings in C# 113 Strings in PHP 113 Strings in Python 114 Strings in Tcl 114 Regex literals in Perl 115 Character-Encoding Issues 115 Richness of encoding-related support 116 Unicode 116 Characters versus combining-character sequences 117 Multiple code points for the same character 118 Unicode 3.1+ and code points beyond U+FFFF 119 Unicode line terminator 119 Regex Modes and Match Modes 120 Case-insensitive match mode 120 Free-spacing and comments regex mode 121 Dot-matches-all match mode (a.k.a., "single-line mode") 121 Enhanced line-anchor match mode (a.k.a., "multiline mode") 122 Literal-text regex mode 123 Common Metacharacters and Features 123 Character Representations 125 Character shorthands 125 These are machine dependent? 125 Octal escape—\num 126 Hex and Unicode escapes: \xnum, \x{num}, \unum, \Unum, ... 127 Control characters: \ cchar 127 Character Classes and Class-Like Constructs 128 Normal classes: [a-z] and [^a-z] 128 Almost any character: dot 129 Dot versus a negated character class 129 Exactly one byte 130 Unicode combining character sequence: \X 130 Class shorthands: \w, \d, \s, \W, \D, \S 130 Unicode properties, scripts, and blocks: \p{Prop}, \P{Prop} 131 Scripts 132 Blocks 134 Other properties/qualities 134 Simple class subtraction: [[a-z] – [aeiou]] 135 Full class set operations: [[a-z] && [^aeiou]] 135 Class subtraction with set operators 136 Mimicking class set operations with lookaround 136 POSIX bracket-expression "character class": [[:alpha:]] 137 POSIX bracket-expression "collating sequences": [[. span-ll.]] 138 POSIX bracket-expression "character equivalents": [[=n=]] 138 Emacs syntax classes 138 Anchors and Other "Zero-Width Assertions" 139 Start of line/string: ^, \A 139 End of line/string: $, \Z, \z 139 Start of match (or end of previous match): \G 140 End of previous match, or start of the current match? 141 Advanced Use of \G with Perl 142 Word boundaries: \b, \B, \ , ... 143 Lookahead (?=•••), (?!•••); Lookbehind, (?