Resumen: This book has two main goals: to provide a unifed and structured overview of this growing field, as well as to propose a corresponding software framework, the OpenTL library, developed by the author and his working group at TUM-Informatik. The main objective of this work is to show, how most real-world application scenarios can be naturally cast into a common description vocabulary, and therefore implemented and tested in a fully modular and scalable way, through the defnition of a layered, object-oriented software architecture. The resulting architecture covers in a seamless way all processing levels, from raw data acquisition up to model-based object detection and sequential localization, and defines, at the application level, what we call the tracking pipeline. Within this framework, extensive use of graphics hardware (GPU computing) as well as distributed processing, allows real-time performances for complex models and sensory systems MODEL-BASED VISUALTRACKING: The OpenTL Framework......Page 5 CONTENTS......Page 7 PREFACE......Page 13 CHAPTER 1: INTRODUCTION......Page 17 1.1 OVERVIEW OF THE PROBLEM......Page 18 1.1.1 Models......Page 19 1.1.2 Visual Processing......Page 21 1.2 GENERAL TRACKING SYSTEM PROTOTYPE......Page 22 1.3 THE TRACKING PIPELINE......Page 24 CHAPTER 2: MODEL REPRESENTATION......Page 28 2.1.1 Internal Camera Model......Page 29 2.1.2 Nonlinear Distortion......Page 32 2.1.3 External Camera Parameters......Page 33 2.1.4 Uncalibrated Models......Page 34 2.1.5 Camera Calibration......Page 36 2.2.1 Shape Model and Pose Parameters......Page 42 2.2.2 Appearance Model......Page 50 2.2.3 Learning an Active Shape or Appearance Model......Page 53 2.3 MAPPING BETWEEN OBJECT AND SENSOR SPACES......Page 55 2.3.1 Forward Projection......Page 56 2.3.2 Back-Projection......Page 57 2.4 OBJECT DYNAMICS......Page 59 2.4.1 Brownian Motion......Page 63 2.4.3 Oscillatory Model......Page 65 2.4.4 State Updating Rules......Page 66 2.4.5 Learning AR Models......Page 68 3.1 PREPROCESSING......Page 71 3.2 SAMPLING AND UPDATING REFERENCE FEATURES......Page 73 3.3 MODEL MATCHING WITH THE IMAGE DATA......Page 75 3.3.1 Pixel-Level Measurements......Page 78 3.3.2 Feature-Level Measurements......Page 80 3.3.3 Object-Level Measurements......Page 83 3.3.4 Handling Mutual Occlusions......Page 84 3.4 DATA FUSION ACROSS MULTIPLE MODALITIES AND CAMERAS......Page 86 3.4.2 Multicamera Fusion......Page 87 3.4.3 Static and Dynamic Measurement Fusion......Page 88 3.4.4 Building a Visual Processing Tree......Page 93 CHAPTER 4: EXAMPLES OF VISUAL MODALITIES......Page 94 4.1 COLOR STATISTICS......Page 95 4.1.1 Color Spaces......Page 96 4.1.2 Representing Color Distributions......Page 101 4.1.3 Model-Based Color Matching......Page 105 4.1.4 Kernel-Based Segmentation and Tracking......Page 106 4.2 BACKGROUND SUBTRACTION......Page 109 4.3 BLOBS......Page 112 4.3.1 Shape Descriptors......Page 113 4.3.2 Blob Matching Using Variational Approaches......Page 120 4.4 MODEL CONTOURS......Page 128 4.4.1 Intensity Edges......Page 130 4.4.2 Contour Lines......Page 135 4.4.3 Local Color Statistics......Page 138 4.5 KEYPOINTS......Page 142 4.5.1 Wide-Baseline Matching......Page 144 4.5.2 Harris Corners......Page 145 4.5.3 Scale-Invariant Keypoints: Detection, Description, and Matching......Page 149 4.5.4 Matching Strategies for Invariant Keypoints......Page 154 4.6.1 Motion History Images......Page 156 4.6.2 Optical Flow......Page 158 4.7 TEMPLATES......Page 163 4.7.1 Pose Estimation with AAM......Page 167 4.7.2 Pose Estimation with Mutual Information......Page 174 CHAPTER 5: RECURSIVE STATE-SPACE ESTIMATION......Page 178 5.1 TARGET-STATE DISTRIBUTION......Page 179 5.2 MLE AND MAP ESTIMATION......Page 182 5.2.1 Least-Squares Estimation......Page 183 5.2.2 Robust Least-Squares Estimation......Page 184 5.3.1 Kalman and Information Filters......Page 188 5.3.2 Extended Kalman and Information Filters......Page 189 5.3.3 Unscented Kalman and Information Filters......Page 192 5.4 MONTE CARLO FILTERS......Page 196 5.4.1 SIR Particle Filter......Page 197 5.4.2 Partitioned Sampling......Page 201 5.4.3 Annealed Particle Filter......Page 203 5.4.4 MCMC Particle Filter......Page 205 5.5 GRID FILTERS......Page 208 CHAPTER 6: EXAMPLES OF TARGET DETECTORS......Page 213 6.1 BLOB CLUSTERING......Page 214 6.1.1 Localization with Three-Dimensional Triangulation......Page 215 6.2.1 AdaBoost Algorithm for Object Detection......Page 218 6.2.2 Example: Face Detection......Page 219 6.3 GEOMETRIC HASHING......Page 220 6.4 MONTE CARLO SAMPLING......Page 224 6.5 INVARIANT KEYPOINTS......Page 227 7.1 FUNCTIONAL ARCHITECTURE OF OpenTL......Page 230 7.2 BUILDING A TUTORIAL APPLICATION WITH OpenTL......Page 232 7.2.1 Setting the Camera Input and Video Output......Page 233 7.2.2 Pose Representation and Model Projection......Page 236 7.2.3 Shape and Appearance Model......Page 240 7.2.4 Setting the Color-Based Likelihood......Page 243 7.2.5 Setting the Particle Filter and Tracking the Object......Page 248 7.2.6 Tracking Multiple Targets......Page 251 7.2.7 Multimodal Measurement Fusion......Page 253 7.3 OTHER APPLICATION EXAMPLES......Page 256 A.1 POINT CORRESPONDENCES......Page 267 A.1.2 Algebraic Error......Page 269 A.1.3 2D-2D and 3D-3D Transforms......Page 270 A.1.4 DLT Approach to 3D-2D Projections......Page 272 A.2 LINE CORRESPONDENCES......Page 275 A.2.1 2D-2D Line Correspondences......Page 276 A.3 POINT AND LINE CORRESPONDENCES......Page 277 A.4 COMPUTATION OF THE PROJECTIVE DLT MATRICES......Page 278 B.1 POSES WITHOUT ROTATION......Page 281 B.1.1 Pure Translation......Page 282 B.1.3 Translation and Nonuniform Scale......Page 283 B.2 PARAMETERIZING ROTATIONS......Page 284 B.3.1 Similarity (Roto-translation and Uniform Scale)......Page 288 B.3.2 Rotation and Uniform Scale......Page 289 B.3.4 Pure Rotation......Page 290 B.4 AFFINITY......Page 291 B.5 POSES WITH ROTATION AND NONUNIFORM SCALE......Page 293 B.6 GENERAL HOMOGRAPHY: THE DLT ALGORITHM......Page 294 NOMENCLATURE......Page 297 BIBLIOGRAPHY......Page 301 INDEX......Page 311 Color plates......Page 319
This book has two main goals: to provide a unifed and structured overview of this growing field, as well as to propose a corresponding software framework, the OpenTL library, developed by the author and his working group at TUM-Informatik.
The main objective of this work is to show, how most real-world application scenarios can be naturally cast into a common description vocabulary, and therefore implemented and tested in a fully modular and scalable way, through the defnition of a layered, object-oriented software architecture.The resulting architecture covers in a seamless way all processing levels, from raw data acquisition up to model-based object detection and sequential localization, and defines, at the application level, what we call the tracking pipeline. Within this framework, extensive use of graphics hardware (GPU computing) as well as distributed processing, allows real-time performances for complex models and sensory systems.
Since the early days of computer vision, the field of visual object tracking has greatly evolved, along with the available imaging devices and computing hardware. This book provides a unified overview of the growing subject. Additionally, it proposes the algorithmic foundation of a general-purpose software architecture, the OpenTL library.
