System for camera motion compensation

• US 8,493,454 B1
• Filed: 02/17/2010
• Issued: 07/23/2013
• Est. Priority Date: 02/17/2010
• Status: Active Grant

• Current Assignee: Ambarella International LP (Ambarella Incorporated)
• Original Assignee: Ambarella Incorporated
• Inventors: Kohn, Leslie D., Uz, Kamil Metin
• Primary Examiner(s): NGUYEN, LUONG TRUNG
• Application Number:
  US12/707,174
• Time in Patent Office: 1,252 Days
• Field of Search: None
• US Class Current: 348/208.6
• CPC Class Codes: H04N 23/6812 based on additional sensors, e.g. acceleration sensors
• CPC Class Codes: H04N 23/6815 by distinguishing pan or tilt from motion
• CPC Class Codes: H04N 23/6842 by controlling the scanning position, e.g. windowing
• CPC Class Codes: H04N 23/689 Motion occurring during a rolling shutter mode

Abstract

An apparatus including one or more motion sensors, a first circuit, and a second circuit. The one or more motion sensors may be configured to generate motion samples representing motion of a camera. The first circuit may be configured to record a plurality of the motion samples from the one or more motion sensors while capturing image data with the camera. The second circuit may be configured to transfer information about the motion of the camera based on the plurality of motion samples. The information about the motion of the camera may be transferred within a bitstream carrying the image data captured by the camera.

Claims

1. An apparatus comprising:

an interface circuit configured to retrieve (i) image data and (ii) information about camera motion, wherein said information about said camera motion is either encapsulated within a bitstream carrying said image data or stored in a first file associated with a second file containing said image data; and

a motion compensation circuit configured to perform camera motion compensation on said image data using said information about said camera motion, wherein said motion compensation circuit is enabled to alter motion compensation performed on a current frame based on said information about said camera motion from a future frame.

2. The apparatus according to claim 1, wherein said motion compensation circuit uses said information about said camera motion to improve playback quality.

3. The apparatus according to claim 1, wherein said motion compensation circuit uses said information about said camera motion to improve crop window placement.

4. The apparatus according to claim 1, wherein said motion compensation circuit uses said information about said camera motion to improve rolling shutter related distortion.

5. The apparatus according to claim 1, wherein said information about said camera motion comprises a plurality of motion samples from one or more motion sensors of a camera.

6. The apparatus according to claim 5, wherein said plurality of motion samples is recorded while capturing said image data with said camera.

7. The apparatus according to claim 5, wherein said information about said camera motion comprises original motion samples recorded from said one or more motion sensors.

8. The apparatus according to claim 5, wherein said information about said camera motion comprises corrections made by said camera during acquisition of said image data.

9. The apparatus according to claim 5, wherein said information about said camera motion comprises residual motion information recorded from said one or more motion sensors and corrections made by said camera prior to transferring or storing said image data.

10. The apparatus according to claim 1, wherein said information about said camera motion comprises one or more samples from gyroscopic sensors, inertial sensors, or both gyroscopic and inertial sensors.

11. The apparatus according to claim 1, wherein said information about said camera motion for a respective frame is encapsulated with the respective frame.

12. The apparatus according to claim 1, wherein said information about said camera motion for a respective frame is encapsulated with a frame acquired one or more frame times later than the respective frame.

13. The apparatus according to claim 1, wherein said motion compensation circuit is separate from a camera that captured said image data and said information about said camera motion.

14. The apparatus according to claim 13, wherein said apparatus comprises a playback device, a video editing system, or a transcoder.

15. The apparatus according to claim 1, wherein said motion compensation circuit is integrated with a camera that captured said image data and said information about said camera motion.

16. The apparatus according to claim 1, wherein said information about said camera motion comprises horizontal, vertical, or horizontal and vertical motion samples.

17. The apparatus according to claim 1, wherein said information about said camera motion comprises at least one of angular motion information and linear motion information.

18. The apparatus according to claim 1, wherein said information about said camera motion encapsulated within said bitstream carrying said image data or stored in said first file comprises:

a first field comprising all or a subset of horizontal and vertical motion samples acquired during a frame time;

a second field comprising crop window parameters when applicable;

a third field comprising residual motion information;

a fourth field comprising exposure time; and

a fifth field comprising a focal distance of a lens of the camera whose motion the information is about.

19. The apparatus according to claim 1, wherein said camera motion compensation further comprises correction for a slow drift in a zero point of a gyroscopic sensor based upon detection of gyroscopic sensor parameter drift.