System Programming Essentials with Go : System Calls, Networking, Efficiency, and Security Practices with Practical Projects in Golang

Go beyond web development to learn system programming, building secure, concurrent, and efficient applications with Go's unique system programming capabilities You’ll learn how Go, unlike traditional system programming languages (C/C++), lets you focus on the problem by prioritizing readability and elevating developer experience with features like automatic garbage collection and built-in concurrency primitives, which remove the burden of low-level memory management and intricate synchronization.Through hands-on projects, you'll master core concepts like file I/O, process management, and inter-process communication to automate tasks and interact with your system efficiently. You'll delve into network programming in Go, equipping yourself with the skills to build robust, distributed applications. This book goes beyond the basics by exploring modern practices like logging and tracing for comprehensive application monitoring, and advance to distributed system design using Go to prepare you to tackle complex architectures.- Understand the fundamentals of system programming using Go.- Grasp the concepts of goroutines, channels, data races, and managing concurrency in Go.- Manage file operations and inter-process communication (IPC).- Handle USB drives and Bluetooth devices and monitor peripheral events for hardware automation.- Familiarize yourself with the basics of network programming and its application in Go.- Implement logging, tracing, and other telemetry practices.- Construct distributed cache and approach distributed systems using Go. Who this book is forThis book is for software engineers looking to expand their understanding of system programming concepts. Professionals with a coding foundation seeking profound knowledge of system-level operations will also greatly benefit. Additionally, individuals interested in advancing their system programming skills, whether experienced developers or those transitioning to the field, will find this book indispensable. Cover Title Page Copyright and Credits Contributors Table of Contents Preface Part 1: Introduction Chapter 1: Why Go? Choosing Go Concurrency and goroutines Concurrency Goroutines CSP-inspired model Share by communication Interacting with the OS Tooling go build go test go run go vet go fmt Cross-platform development with Go Summary Chapter 2: Refreshing Concurrency and Parallelism Technical requirements Understanding goroutines WaitGroup Changing shared state Managing data races Atomic operations Mutexes Making sense of channels How to use channels An unbuffered channel Buffered channels The guarantee of delivery Latency State and signaling State Signaling Choosing your synchronization mechanism Summary Part 2: Interation with the OS Chapter 3: Understanding System Calls Technical requirements Introduction to system calls The catalog of services and identification Information exchange The syscall package A closer look at the os and x/sys packages x/sys package – low-level system calls Operating system functionality Portability Everyday system calls Tracing system calls Tracing specific system calls Developing and testing a CLI program Standard streams File descriptors Creating a CLI application Redirections and standard streams Making it testable Summary Chapter 4: File and Directory Operations Technical requirements Identifying unsafe file and directory permissions Files and permissions Scanning directories in Go Understanding file paths Using the path/filepath package Traversing directories Symbolic links and unlinking files Symbolic links – the shortcut of the file world Unlinking files – the great escape act Calculating directory size Finding duplicate files Optimizing filesystem operations Summary Chapter 5: Working with System Events Managing system events What are signals? The os/signal package Task scheduling in Go Why schedule? Basic scheduling Handling timer signals File monitoring Inotify fsnotify File rotation Process management Execution and timeouts Execute and control process execution time Building a distributed lock manager in Go Summary Chapter 6: Understanding Pipes in Inter-Process Communication Technical requirements What are pipes in IPC? Why are pipes important? Pipes in Golang The mechanics of anonymous pipes Navigating named pipes (Mkfifo()) Best practices – guidelines for using pipes Efficient data handling Error handling and resource management Security considerations Performance optimization Developing a log processing tool Summary Chapter 7: Unix Sockets Introduction to Unix sockets Creating a Unix socket Going a little deeper into socket creation Creating the client Inspecting the socket with lsof Building a chat server The complete chat client Serving HTTP under UNIX domain sockets Client HTTP request line HTTP request header Empty line signifying end of headers The textproto package Performance Other common use cases Summary Part 3: Performance Chapter 8: Memory Management Technical requirements Garbage collection Stack and heap allocation The GC algorithm GOGC GC pacer GODEBUG Memory ballast GOMEMLIMIT Memory arenas Using memory arenas Summary Chapter 9: Analyzing Performance Escape analysis Stack and pointers Pointers Stack Heap How can we analyze? Benchmarking your code Writing your first benchmark Memory allocations Common pitfalls CPU profiling Memory profiling Profiling memory over time Preparing to explore the trade-offs Summary Part 4: Connected Apps Chapter 10: Networking The net package TCP sockets HTTP servers and clients HTTP verbs HTTP status codes Putting it all together Securing the connection Certificates Advanced networking UDP versus TCP Summary Chapter 11: Telemetry Technical requirements Logs Zap versus slog Logging for debugging or monitoring? What to log? What not to log? Traces Effective tracing Distributed tracing Metrics What metric should we use? The OTel project OTel Summary Chapter 12: Distributing Your Apps Technical requirements Go Modules The routine using modules CI Caching Static analysis Releasing your application Summary Part 5: Going Beyond Chapter 13: Capstone Project – Distributed Cache Technical requirements Understanding distributed caching System requirements Requirements Design and trade-offs Creating the project Thread safety Choosing the right approach Adding thread safety The interface TCP HTTP Others Eviction policies Sharding Summary Chapter 14: Effective Coding Practices Technical requirements Reusing resources Using sync.Pool in a network server Using sync.Pool for JSON marshaling Executing tasks once singleflight Effective memory mapping API usage Advanced usage with protection and mapping flags Avoiding common performance pitfalls Leaking with time.After Defer in for loops Maps management Resource management Handling HTTP bodies Channel mismanagement Summary Chapter 15: Stay Sharp with System Programming Real-world applications Dropbox’s leap of faith HashiCorp – Go from day one Grafana Labs – visualizing success with Go Docker – building a container revolution with Go SoundCloud – from Ruby to Go Navigating the system programming landscape Go release notes and blog Community Contribution Experimentation Resources for continued learning Advanced Programming in the UNIX Environment by W. Richard Stevens Learn C Programming - Second Edition: A beginner’s guide to learning the most powerful and general-purpose programming language with ease Linux Kernel Programming - Second Edition: A comprehensive and practical guide to kernel internals, writing modules, and kernel synchronization Linux System Programming Techniques: Become a proficient Linux system programmer using expert recipes and techniques Operating Systems: Design and Implementation by Andrew S. Tanenbaum Unix Network Programming by W. Richard Stevens Linux System Programming Techniques: Become a proficient Linux system programmer using expert recipes and techniques Mastering Embedded Linux Programming - Third Edition: Create fast and reliable embedded solutions with Linux 5.4 and the Yocto Project 3.1 (Dunfell) Modern Operating Systems by Andrew S. Tanenbaum The Art of UNIX Programming by Eric S. Raymond Your system programming journey Appendix Hardware Automation Automation in system programming USB Application The goal The /proc/mounts file Reading the files on the flash drive Partitions versus blocks versus devices versus disks Open source to the rescue! Interacting with USB events Bluetooth Detecting the smartwatch Locking the screen XDG dilemma The Wayland conundrum Summary Index Go beyond the web, learn system programming with Go, and build efficient, secure applicationsKey FeaturesGet to grips with system programming concepts in Go with application examplesGain expert guidance on essential topics like file operations, process management, and network programmingLearn how to develop modern, functional applications from scratchPurchase of the print or Kindle book includes a free PDF eBookBook DescriptionAlex Rios, a seasoned Go developer and active community builder, shares his 15 years of expertise in designing large-scale systems through this book. It masterfully cuts through complexity, enabling you to build efficient and secure applications with Go's streamlined syntax and powerful concurrency features. In this book, you'll learn how Go, unlike traditional system programming languages (C/C++), lets you focus on the problem by prioritizing readability and elevating developer experience with features like automatic garbage collection and built-in concurrency primitives, which remove the burden of low-level memory management and intricate synchronization. Through hands-on projects, you'll master core concepts like file I/O, process management, and inter-process communication to automate tasks and interact with your system efficiently. You'll delve into network programming in Go, equipping yourself with the skills to build robust, distributed applications. This book goes beyond the basics by exploring modern practices like logging and tracing for comprehensive application monitoring, and advance to distributed system design using Go to prepare you to tackle complex architectures. By the end of this book, you'll emerge as a confident Go system programmer, ready to craft high-performance, secure applications for the modern world.What you will learnUnderstand the fundamentals of system programming using GoGrasp the concepts of goroutines, channels, data races, and managing concurrency in GoManage file operations and inter-process communication (IPC)Handle USB drives and Bluetooth devices and monitor peripheral events for hardware automationFamiliarize yourself with the basics of network programming and its application in GoImplement logging, tracing, and other telemetry practicesConstruct distributed cache and approach distributed systems using GoWho this book is forThis book is for software engineers looking to expand their understanding of system programming concepts. Professionals with a coding foundation seeking profound knowledge of system-level operations will also greatly benefit. Additionally, individuals interested in advancing their system programming skills, whether experienced developers or those transitioning to the field, will find this book indispensable.