Processing for Android : Create Mobile, Sensor-aware, and XR Applications Using Processing

Learn how to use the Processing programming language and environment to create Android applications with ease. This book covers the basics of the Processing language, allowing users to effectively program interactive graphics in 2D, 3D, and Extended Reality (XR). It also details the application of these techniques to different types of Android devices (smartphones, tablets, wearables, and smartwatches). This updated edition walks you through the entire process of creating an app, from the initial idea to release of the final app via the Google Play App Store. Over the course of the book, you’ll learn to write engaging apps driven by user interaction and sensor data. A comprehensive series of hands-on projects, ranging from simple sketches to more complex projects involving shaders, VR, and AR will give you the firsthand experience you need to begin developing your own projects. And once you have your Processing projects completed, you’ll be able to upload them to the Google Play store to be shared with the world! What You Will Learn Write apps and live wallpapers for smartphones and tablets Design and implement interactive watch faces Create Extended Reality (VR/AR) experiences Incorporate GLSL shaders into your Processing apps Integrate Processing sketches into larger apps and Android Studio Learn how to write your own libraries to share with the community Export projects as completed apps ready to distribute through Google Play Store Who This Book Is For Artists, designers, students, researchers, and hobbyists who are not necessarily Android experts, but are looking to write mobile apps that make creative use of interactive graphics, sensor data, and extended reality. Table of Contents About the Author About the Technical Reviewer Acknowledgments Introduction Part I: First Steps with Processing for Android Chapter 1: Getting Started with the Android Mode What Is the Processing Project? The Processing Language The Processing Development Environment Extending Processing The Contribution Manager Processing for Android Installing the Android Mode Interface of the Android Mode Running a Sketch on a Device Running a Sketch in the Emulator Summary Chapter 2: The Processing Language A Programming Sketchbook for Artists and Designers The Structure of a Processing Sketch Drawing with Code Coordinates Form Color Applying Geometric Transformations Responding to User Input Coding a “Vine Drawing” App Summary Chapter 3: From Sketch to Play Store Sketching and Debugging Getting Information from the Console Getting more information with logcat Using the Integrated Debugger Reporting Processing Bugs Preparing a Sketch for Release Adjusting for Device’s DPI Using the Emulator Setting Icons and Bundle Name Setting Package Name and Version Exporting As Signed Bundle Summary Part II: Drawing and Interaction Chapter 4: Drawing Graphics and Text Drawing Shapes More Types of Shapes Curve Shapes Using Bézier Curves Shape Attributes Shape Styles Drawing Images Drawing Text Loading and Creating Fonts Text Attributes Scaling Text Renderers in Processing The PShape Class Creating PShapes Loading Shapes from SVG Texturing Shapes Summary Chapter 5: Touchscreen Interaction Touch Events in Android Basic Touch Events Multi-touch Events Touch-Based Interaction Item Selection Scrolling Swipe and Pinch Using the Keyboard Summary Chapter 6: Live Wallpapers Live Wallpapers Writing and Installing Live Wallpapers Using Multiple Home Screens Handling Permissions Drawing with Particle Systems Simulating Natural Motion An Image-Flow Wallpaper Loading, Resizing, and Cropping Images Putting Everything Together Wrapping the Project Up Summary Part III: Sensors Chapter 7: Reading Sensor Data Sensors in Android Devices Accelerometer Gyroscope Magnetometer Location Accessing Sensors from Processing Creating a Sensor Manager Adding a Sensor Listener Reading Data from the Sensor Reading Data from Other Sensors The Ketai Library Installing Ketai Using Ketai Event Handlers in Ketai Summary Chapter 8: Driving Graphics and Sound with Sensor Data Using Ketai to Read Sensor Data Measuring Acceleration Shake Detection Step Counter Audio-visual Mapping of Step Data Playing Audio Using the Magnetic Sensor Creating a Compass App The Gyroscope Controlling Navigation with the Gyroscope Summary Chapter 9: Geolocation Location Data in Android Using Location API in Processing Location Permissions Accessing Data from Multiple Threads Location with Ketai Using Additional Location Data A Street View Collage Using Google Street View Image API Making a Simple Image Collage Putting Everything Together Summary Part IV: Wearables and Watch Faces Chapter 10: Wearable Devices From Activity Trackers to Smartwatches Smartwatches Running Watch Face Sketches Using a Watch Using the Emulator Displaying Time Counting Steps Designing for Smartwatches Adjusting for Screen Shape Watch Face Preview Icons Summary Chapter 11: Visualizing Time From Sundials to Smartwatches Using Time to Control Motion Square vs. Round Watch Faces Working with a Watch Face Concept Elapsed/Remaining Time Adding Interaction Loading and Displaying Images Summary Chapter 12: Visualizing Physical Activity Body Sensors Step Counter Heart Rate Visualizing Physical Activity in Real Time Simple Step Counter Accessing the Heart Rate Sensor Visualizing Step Count Data A Beating Heart Sensor Debugging Grow a Tree As You Exercise! Generating a Tree with a Particle System Incorporating Step Count Data Tweaking the Watch Face Blooming the Tree Summary Part V: 3D and Shaders Chapter 13: 3D in Processing The P3D Renderer Hello World in 3D 3D Transformations Combining Transformations 3D Shapes Custom Shapes The Camera Immediate vs. Retained Rendering Custom PShape Objects Loading OBJ Shapes Summary Chapter 14: Lighting and Texturing Making 3D Scenes more Realistic with Lights and Textures Light Sources and Material Properties Texture Mapping Putting Everything Together Summary Chapter 15: GLSL Shaders What Is a Shader? The Graphics Pipeline: Vertex and Pixel Shaders The PShader Class Anatomy of a Simple Shader Defining Custom Uniforms Types of Shaders in Processing Color Shaders Texture Shaders Light Shaders Texlight Shaders Image Postprocessing Filters Day-to-Night Earth Shader and Live Wallpaper Summary Part VI: Extended Reality: VR and AR Chapter 16: Basics of VR Brief History of VR Google Cardboard Hardware Requirements VR in Processing Stereo Rendering Monoscopic Rendering VR Interaction Eye and World Coordinates The Line of Sight Selecting a Shape with Screen Coordinates Bounding Box Selection Movement in VR Automatic Movement Unconstrained Movement Summary Chapter 17: Basics of AR A Brief History of AR AR on Android Requirements Getting Started Drawing Trackables Using Anchors Hitting Trackables Adding a User Interface Using Custom Shaders in AR Drawing in AR Summary Chapter 18: Creating XR Experiences Challenges with XR Apps Drawing in VR Initial Sketches A Simple VR UI Drawing in 3D Flying Around Final Tweaks and Packaging Intro Text Icons and Package Export Summary Part VII: Appendixes Chapter 19: Appendix A: Gradle and Android Studio Integration Google Tools for Android Development Exporting a Sketch As a Gradle Project Importing into Android Studio Adding a Processing Sketch to a Layout Chapter 20: Appendix B: Processing Libraries for Android Extending Processing with Libraries Installing Libraries with the Contribution Manager Installing Libraries Manually Writing New Libraries Index