Unrolling the Shutter: CNN to Correct Motion Distortions

Vijay Rengarajan, Yogesh Balaji, A.N. Rajagopalan
Computer Vision and Pattern Recognition (CVPR) 2017

Abstract


Row-wise exposure delay present in CMOS cameras is responsible for skew and curvature distortions known as the rolling shutter (RS) effect while imaging under camera motion. Existing RS correction methods resort to using multiple images or tailor scene-specific correction schemes. We propose a convolutional neural network (CNN) architecture that automatically learns essential scene features from a single RS image to estimate the row-wise camera motion and undo RS distortions back to the time of first-row exposure. We employ long rectangular kernels to specifically learn the effects produced by the row-wise exposure. Experiments reveal that our proposed architecture performs better than the conventional CNN employing square kernels. Our single-image correction method fares well even operating in a frame-by-frame manner against video-based methods and performs better than scene-specific correction schemes even under challenging situations.


Publication


Unrolling the Shutter: CNN to Correct Motion Distortions
Vijay Rengarajan, Yogesh Balaji, A.N. Rajagopalan
Computer Vision and Pattern Recognition (CVPR)
Honolulu, July 2017
Paper
Supplementary
Poster

Code


GitHub

Data


Building Training Dataset
Face Training Dataset
Test Dataset (Building, Face, Synthetic Motion)

Results


Given a single rolling shutter distorted image, our RowColCNN maps it to camera motion which is used to rectify the distortions. ![alt text](../img/rs_rect_cnn_eg.png "Examples")