Method and apparatus for correcting distortion in an image due to rotational motion of an image capture device occurring while the image is being captured

• US 9,082,187 B1
• Filed: 06/02/2014
• Issued: 07/14/2015
• Est. Priority Date: 07/13/2010
• Status: Expired due to Fees

• Current Assignee: Marvell Asia Pte Limited (Marvell Technology Group Limited)
• Original Assignee: Marvell International Limited (Marvell Technology Group Limited)
• Inventors: Bledsoe, James Daren
• Primary Examiner(s): Haskins, Twyler
• Assistant Examiner(s): TEJANO, DWIGHT ALEX C
• Application Number:
  US14/293,242
• Time in Patent Office: 407 Days
• Field of Search: 3482081-20811
• US Class Current: 1/1
• CPC Class Codes: G06T 5/80 Geometric correction
• CPC Class Codes: H04N 23/6812 based on additional sensors, e.g. acceleration sensors
• CPC Class Codes: H04N 5/2628 Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation

Abstract

A system is operable to correct for distortion in an image. The distortion in the image was caused by rotation of an image capture device while the image was being captured by the image capture device. The system includes an orientation sensor configured to perform a first measurement concurrently with the image capture device capturing the image. The system further includes a rotation module configured to generate, based on the first measurement, a rotation matrix. The system further includes a correction module configured to, based on the rotation matrix, correct the distortion in the image caused by the rotation of the image capture device. The system further includes a restoration module configured to selectively reverse the correction of the distortion in the image based on (i) the image as corrected by the correction module and (ii) the rotation matrix.

Claims

1. A system operable to correct for distortion in a captured image, the distortion in the captured image having been caused by a rotation of an image capture device while the captured image was being captured by the image capture device, the system comprising:

an orientation sensor configured to perform a first measurement concurrently with the image capture device capturing the captured image;

a rotation module configured to generate, based on the first measurement, a rotation matrix;

a correction module configured to, based on the rotation matrix, produce a corrected image by correcting the distortion in the captured image caused by the rotation of the image capture device; and

a restoration module configured to selectively revert the corrected image back to the captured image based on (i) the corrected image and (ii) the rotation matrix.

2. The system of claim 1, further comprising a storage module configured to store an image file, wherein the image file includes the corrected image and at least one of (i) the rotation matrix and (ii) correction coefficients derived from the rotation matrix.

3. The system of claim 1, further comprising an image capture sensor, wherein the captured image is captured by the image capture sensor.

4. The system of claim 3, wherein the rotation matrix represents a rotational difference between (i) an orientation of the image capture sensor during capture of the captured image and (ii) a reference orientation.

5. The system of claim 4, wherein the reference orientation corresponds to a gravity vector.

6. The system of claim 4, wherein the first measurement corresponds to the orientation of the image capture sensor during capture of the captured image.

7. The system of claim 4, wherein the reference orientation represents an orientation of the image capture sensor during capture of a reference image.

8. The system of claim 7, wherein the captured image and the reference image are part of a panorama.

9. The system of claim 7, wherein the captured image and the reference image are frames in a video.

10. The system of claim 1, further comprising:

an accelerometer configured to perform a second measurement concurrently with the image capture device capturing the captured image; and

a feedback module configured to, based on the second measurement, transform the first measurement into a corrected first measurement to correct for drift of the orientation sensor,

wherein the rotation module is configured to generate the rotation matrix based on the corrected first measurement.

11. The system of claim 10, wherein the orientation sensor comprises a gyroscope.

12. The system of claim 1, wherein the rotation matrix includes a direction cosine matrix.

13. The system of claim 1, further comprising a display module, wherein the display module is configured to display

(i) the corrected image, and

(ii) a control allowing a user to direct the restoration module to revert the corrected image to the captured image.

14. A method for correcting distortion in a captured image, the distortion in the captured image having been caused by a rotation of an image capture device while the captured image was being captured by the image capture device, the method comprising:

performing a first measurement using an orientation sensor concurrently with the captured image being captured;

generating, based on the first measurement, a rotation matrix;

producing a corrected image by correcting, based on the rotation matrix, the distortion in the captured image caused by the rotation of the image capture device; and

selectively reverting the corrected image back to the captured image based on (i) the corrected image and (ii) the rotation matrix.

15. The method of claim 14, further comprising storing an image file, wherein the image file includes the corrected image and at least one of (i) the rotation matrix and (ii) correction coefficients derived from the rotation matrix.

16. The method of claim 14, wherein the rotation matrix represents a rotational difference between (i) an orientation of the image capture device during capture of the captured image and (ii) a reference orientation.

17. The method of claim 16, wherein the reference orientation corresponds to a gravity vector.

18. The method of claim 16, wherein the first measurement corresponds to the orientation of the image capture device during capture of the captured image.

19. The method of claim 16, wherein the reference orientation represents an orientation of the image capture device during capture of a reference image.

20. The method of claim 19, wherein the captured image and the reference image are part of a panorama.

21. The method of claim 19, wherein the captured image and the reference image are frames in a video.

22. The method of claim 14, further comprising:

performing a second measurement using an accelerometer concurrently with the image capture device capturing the captured image; and

based on the second measurement, transforming the first measurement into a corrected first measurement to correct for drift of the orientation sensor,

wherein the orientation sensor comprises a gyroscope, and

wherein the rotation matrix is generated based on the corrected first measurement.

23. The method of claim 14, wherein the rotation matrix includes a direction cosine matrix.