Image rotation correction and restoration using gyroscope and accelerometer

• US 8,743,219 B1
• Filed: 07/12/2011
• Issued: 06/03/2014
• Est. Priority Date: 07/13/2010
• Status: Expired due to Fees

• Current Assignee: Marvell International Limited (Marvell Technology Group Limited)
• Original Assignee: Marvell International Limited (Marvell Technology Group Limited)
• Inventors: Bledsoe, James Daren
• Primary Examiner(s): Haskins, Twyler
• Assistant Examiner(s): TEJANO, DWIGHT ALEX C
• Application Number:
  US13/180,813
• Time in Patent Office: 1,057 Days
• Field of Search: None
• US Class Current: 348/208.4
• CPC Class Codes: G06T 5/80 Geometric correction
• CPC Class Codes: H04N 23/6812 based on additional sensors, e.g. acceleration sensors
• CPC Class Codes: H04N 5/2628 Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation

Abstract

An apparatus includes an image capture sensor, a gyroscope, an accelerometer, a feedback module, a rotation module, a correction module, and a restoration module. The image capture sensor captures an image. The gyroscope makes a first measurement that indicates an orientation of the image capture sensor relative to a reference orientation. The accelerometer makes a second measurement. The feedback module generates a corrected first measurement to correct for drift of the gyroscope in response to the second measurement. The rotation module generates a rotation matrix based on the corrected first measurement. The correction module corrects the image captured by the image capture sensor based on the rotation matrix. The restoration module selectively restores the image captured by the image capture sensor based on (i) the image as corrected by the correction module and (ii) the rotation matrix.

Claims

1. An apparatus comprising:

an image capture sensor configured to capture an image;

a gyroscope configured to make a first measurement, wherein the first measurement indicates an orientation of the image capture sensor relative to a reference orientation;

an accelerometer configured to make a second measurement;

a feedback module configured to, in response to the second measurement, generate a corrected first measurement to correct for drift of the gyroscope;

a rotation module configured to generate, based on the corrected first measurement, a rotation matrix;

a correction module configured to, based on the rotation matrix, correct the image captured by the image capture sensor; and

a restoration module configured to selectively restore the image captured by the image capture sensor based on (i) the image as corrected by the correction module and (ii) the rotation matrix.

2. The apparatus of claim 1, wherein the rotation matrix represents a rotational difference between (i) the orientation of the image capture sensor during capture of the image and (ii) the reference orientation.

3. The apparatus of claim 1, wherein the rotation matrix includes a direction cosine matrix.

4. The apparatus of claim 1, further comprising:

a display module, wherein the display module is configured to display

(i) the image as corrected by the correction module, and

(ii) an option for causing the restoration module to restore the image captured by the image capture sensor.

5. A hand-held device comprising the apparatus of claim 1.

6. The hand-held device of claim 5, comprising at least one of:

a digital camera;

a video camera;

a tablet; and

a smartphone.

7. The apparatus of claim 1, further comprising:

an integrated circuit, wherein the integrated circuit includes

the rotation module, and

the correction module.

8. The apparatus of claim 1, wherein the reference orientation represents the orientation of the image capture sensor during capture of a reference image.

9. The apparatus of claim 8, wherein the image captured by the image capture sensor and the reference image are part of a panorama.

10. The apparatus of claim 8, wherein the image captured by the image capture sensor and the reference image are frames in a video.

11. The apparatus of claim 1, further comprising:

a storage module configured to store an image file, wherein the image file includes the image as corrected by the correction module and at least one of

the rotation matrix, and

correction coefficients derived from the rotation matrix.

19. The apparatus of claim 1, wherein the reference orientation corresponds to a gravity vector.

12. A method comprising:

capturing an image with an image capture sensor;

making a first measurement using a gyroscope, wherein the first measurement indicates an orientation of the image capture sensor relative to a reference orientation;

making a second measurement using an accelerometer;

in response to the second measurement, generating a corrected first measurement to correct for drift of the gyroscope;

generating, based on the corrected first measurement, a rotation matrix;

based on the rotation matrix, correcting the image captured by the image capture sensor; and

selectively restoring the image captured by the image capture sensor based on (i) the image as corrected and (ii) the rotation matrix.

13. The method of claim 12, wherein the rotation matrix represents a rotational difference between (i) the orientation of the image capture sensor during capture of the image and (ii) the reference orientation.

14. The method of claim 13, wherein the reference orientation represents an orientation of the image capture sensor during capture of a prior image.

15. The method of claim 12, wherein the rotation matrix includes a direction cosine matrix.

16. The method of claim 12, wherein the reference orientation represents the orientation of the image capture sensor during capture of a reference image.

17. The method of claim 16, wherein the image captured by the image capture sensor and the reference image are part of a panorama.

18. The method of claim 12, wherein the reference orientation corresponds to a gravity vector.