Motion deblurring : algorithms and systems

A comprehensive guide to restoring images degraded by motion blur, bridging the traditional approaches and emerging computational photography-based techniques, and bringing together a wide range of methods emerging from basic theory as well as cutting-edge research. It encompasses both algorithms and architectures, providing detailed coverage of practical techniques by leading researchers. From an algorithms perspective, blind and non-blind approaches are discussed, including the use of single or multiple images; projective motion blur model; image priors and parametric models; high dynamic range imaging in the irradiance domain; and image recognition in blur. Performance limits for motion deblurring cameras are also presented. From a systems perspective, hybrid frameworks combining low-resolution-high-speed and high-resolution-low-speed cameras are described, along with the use of inertial sensors and coded exposure cameras. Also covered is an architecture exploiting compressive sensing for video recovery. A valuable resource for researchers and practitioners in computer vision, image processing, and related fields. Half Title Title Page Imprints Contents List of contributors Preface 1 Mathematical models and practical solvers for uniform motion deblurring 1.1 Non-blind deconvolution 1.1.1 Regularized approaches 1.1.2 Iterative approaches 1.1.3 Recent advancements 1.1.4 Variable splitting solver 1.1.5 A few results 1.2 Blind deconvolution 1.2.1 Maximum marginal probability estimation 1.2.2 Alternating energy minimization 1.2.3 Implicit edge recovery 1.2.4 Explicit edge prediction for very large PSF estimation 1.2.5 Results and running time 2 Spatially varying image deblurring 2.1 Review of image deblurring methods 2.2 A unified camera shake blur model 2.2.1 Blur matrices 2.2.2 Spatially-varying deconvolution 2.3 Single image deblurring using motion density functions 2.3.1 Optimization formulation 2.3.2 Our system for MDF-based deblurring 2.3.3 Experiments and results 2.4 Image deblurring using inertial measurement sensors 2.4.1 Deblurring using inertial sensors 2.4.2 Deblurring system 2.4.3 Results 2.5 Generating sharp panoramas from motion-blurred videos 2.5.1 Motion and duty cycle estimation 2.5.2 Experiments 2.5.3 Real videos 2.6 Discussion 3 Hybrid-imaging for motion deblurring 3.1 Introduction 3.2 Fundamental resolution tradeoff 3.3 Hybrid-imaging systems 3.4 Shift invariant PSF image deblurring 3.4.1 Parametric motion computation 3.4.2 Shift invariant PSF estimation 3.4.3 Image deconvolution 3.4.4 Examples – shift invariant PSF 3.4.5 Shift-invariant PSF optimization 3.4.6 Examples – optimized shift invariant PSF 3.5 Spatially-varying PSF image deblurring 3.5.1 Examples – spatially varying PSF 3.6 Moving object deblurring 3.7 Discussion and summary 4 Efficient, blind, spatially-variant deblurring for shaken images 4.1 Introduction 4.2 Modelling spatially-variant camera shake blur 4.2.1 Components of camera motion 4.2.2 Motion blur and homographies 4.2.3 Camera calibration 4.2.4 Uniform blur as a special case 4.3 The computational model 4.4 Blind estimation of blur from a single image 4.4.1 Updating the blur kernel 4.4.2 Updating the latent image 4.5 Efficient computation of the spatially-variant model 4.5.1 A locally-uniform approximation for camera shake 4.5.2 Updating the blur kernel 4.5.3 Updating the latent image fast, non-iterative non-blind deconvolution 4.6 Single-image deblurring results 4.6.1 Limitations and failures 4.7 Implementation 4.8 Conclusion 5 Removing camera shake in smartphones without hardware stabilization 5.1 Introduction 5.2 Image acquisition model 5.2.1 Space-invariant model 5.2.2 Space-variant model 5.3 Inverse problem 5.3.1 MAP and beyond 5.3.2 Getting more prior information 5.3.3 Patch based 5.4 Pinhole camera model 5.5 Smartphone application 5.5.1 Space-invariant implementation 5.5.2 Space-variant implementation 5.6 Evaluation 5.7 Conclusions 6 Multi-sensor fusion for motion deblurring 6.1 Introduction 6.2 Hybrid-speed sensor 6.3 Motion deblurring 6.3.1 Motion flow estimation 6.3.2 Motion warping 6.3.3 PSF estimation and motion deblurring 6.4 Depth map super-resolution 6.4.1 Initial depth estimation 6.4.2 Joint bilateral upsampling 6.4.3 Results and discussion 6.5 Extensions to low-light imaging 6.5.1 Sensor construction 6.5.2 Processing pipeline 6.5.3 Preliminary results 6.6 Discussion and summary 7 Motion deblurring using fluttered shutter 7.1 Related work 7.2 Coded exposure photography 7.3 Image deconvolution 7.3.1 Motion model 7.4 Code selection 7.5 Linear solution for deblurring 7.5.1 Background estimation 7.5.2 Motion generalization 7.6 Resolution enhancement 7.7 Optimized codes for PSF estimation 7.7.1 Blur estimation using alpha matting 7.7.2 Motion from blur 7.7.3 Code selection 7.7.4 Results 7.8 Implementation 7.9 Analysis 7.9.1 Noise analysis 7.9.2 Resolution analysis 7.10 Summary 8 Richardson–Lucy deblurring for scenes under a projective motion path 8.1 Introduction 8.2 Related work 8.3 The projective motion blur model 8.4 Projective motion Richardson–Lucy 8.4.1 Richardson–Lucy deconvolution algorithm 8.4.2 Projective motion Richardson–Lucy algorithm 8.4.3 Gaussian noise 8.5 Motion estimation 8.6 Experiment results 8.6.1 Convergence analysis 8.6.2 Noise analysis 8.6.3 Qualitative and quantitative analysis 8.6.4 Comparisons with spatially invariant method 8.6.5 Real examples 8.7 Discussion and conclusion 8.7.1 Conventional PSF representation versus projective motion blur model 8.7.2 Limitations 8.7.3 Running time analysis 9 HDR imaging in the presence of motion blur 9.1 Introduction 9.2 Existing approaches to HDRI 9.2.1 Spatially-varying pixel exposures 9.2.2 Multiple exposures (irradiance) 9.2.3 Multiple exposures (direct) 9.3 CRF, irradiance estimation and tone-mapping 9.3.1 Estimation of inverse CRF and irradiance 9.3.2 Tone-mapping 9.4 HDR imaging under uniform blurring 9.5 HDRI for non-uniform blurring 9.5.1 Image accumulation 9.5.2 TSF and its estimation 9.5.3 PSF estimation 9.5.4 Irradiance image recovery 9.6 Experimental results 9.7 Conclusions and discussions 10 Compressive video sensing to tackle motion blur 10.1 Introduction 10.1.1 Video compressive sensing to handle complex motion 10.2 Related work 10.3 Imaging architecture 10.3.1 Programmable pixel compressive camera 10.3.2 Prototype P2C2 10.3.3 P2C2 as a underdetermined linear system 10.4 High-speed video recovery 10.4.1 Transform domain sparsity 10.4.2 Brightness constancy as temporal redundancy 10.5 Experimental results 10.5.1 Simulation on high-speed videos 10.5.2 Results on P2C2 prototype datasets 10.6 Conclusions 11 Coded exposure motion deblurring for recognition 11.1 Motion sensitivity of iris recognition 11.2 Coded exposure 11.2.1 Sequence selection for image capture 11.2.2 Blur estimation 11.2.3 Deblurring 11.3 Coded exposure performance on iris recognition 11.3.1 Synthetic experiments 11.3.2 Real image experiments 11.4 Barcodes 11.5 More general subject motion 11.6 Implications of computational imaging for recognition 11.7 Conclusion 12 Direct recognition of motion-blurred faces 12.1 Introduction 12.1.1 Related work 12.2 The set of all motion-blurred images 12.2.1 Convolution model for blur 12.2.2 The set of all blurred images versus the set of motion-blurred images 12.3 Bank of classifiers approach for recognizing motion-blurred faces 12.4 Experimental evaluation 12.4.1 Sensitivity analysis of the BoC approach 12.4.2 Performance evaluation on synthetically generated motion-blurred images 12.4.3 Performance evaluation on the real dataset REMOTE 12.5 Discussion 13 Performance limits for motion deblurring cameras 13.1 Introduction 13.2 Performance bounds for flutter shutter cameras 13.2.1 Optimal flutter shutter performance 13.3 Performance bound for motion-invariant cameras 13.3.1 Space–time analysis 13.3.2 Optimal motion-invariant performance 13.4 Simulations to verify performance bounds 13.5 Role of image priors 13.6 When to use computational imaging 13.6.1 Rule of thumb 13.7 Relationship to other computational imaging systems 13.7.1 Example computational cameras 13.8 Summary and discussion Index This comprehensive guide to the restoration of images degraded by motion blur brings together a wide range of approaches to the problem, blending basic theory with cutting-edge research. It encompasses both algorithms and architectures, providing detailed coverage of practical techniques by leading researchers.