Video stabilization system and method

• US 5,973,733 A
• Filed: 08/30/1996
• Issued: 10/26/1999
• Est. Priority Date: 05/31/1995
• Status: Expired due to Term

• Current Assignee: Texas Instruments, Inc.
• Original Assignee: Texas Instruments, Inc.
• Inventors: Gove, Robert J.
• Primary Examiner(s): Faile, Andrew
• Assistant Examiner(s): Vu, Ngoc-Yen, VU, NGOC YEN T
• Application Number:
  US08/707,045
• Time in Patent Office: 1,152 Days
• Field of Search: 348/135, 348/142, 348/143, 348/154, 348/155, 348/169, 348/207, 348/208, 348/699, 348/700
• US Class Current: 348/208.13
• CPC Class Codes: H04N 23/68 for stable pick-up of the scene, e.g. compensating for camera body vibrations
• CPC Class Codes: H04N 23/6811 based on the image signal
• CPC Class Codes: H04N 23/682 Vibration or motion blur correction
• CPC Class Codes: H04N 5/145 Movement estimation for video coding H04N19/51
• CPC Class Codes: H04N 5/77 between a recording apparatus and a television camera

Abstract

A system (26) for stabilizing a video recording of a scene (20, 22, & 24) made with a video camera (34) is provided. The video recording may include video data (36) and audio (38) data. The system (26) may include source frame storage (64) for storing source video data (36) as a plurality of sequential frames. The system (26) may also include a processor (50) for detecting camera movement occurring during recording and for modifying the video data (36) to compensate for the camera movement. Additionally the system (26) may include destination frame storage (70) for storing the modified video data as plurality of sequential frames.

Claims

1. A method for stabilizing a video recording of a scene made with a video camera, comprising the steps of:

separating video data of the video recording into a plurality of frames;

dividing each frame into a plurality of blocks;

determining for each frame, a motion vector for each block representing direction and magnitude of motion in the block, said motion vectors being determined from a comparison of each block in a first one of the frames and a second one of the frames;

comparing the motion vectors for each block in one of the plurality of frames with the motion vectors for each block in another of the plurality of frames adjacent to the one frame; and

detecting camera movement when the motion vectors for the one frame are different from motion vectors for an adjacent frame; and

modifying the video data to compensate for the camera movement.

2. The method of claim 1 wherein the modifying step further comprises warping the video data to compensate for camera movement.

3. The method of claim 1 wherein the modifying step further comprises interpolating the video data to compensate for camera movement.

4. The method of claim 1 wherein the modifying step further comprises warping and interpolating the video data to compensate for camera movement.

5. The method of claim 1 wherein the modifying step further comprises warping the video data to compensate for camera movement, the warping step further comprising:

determining a source address for the video data;

determining a destination address for the video data; and

translating the video data from the source address to the destination address.

6. The method of claim 1 wherein the modifying step further comprises interpolating the video data to compensate for camera movement, the interpolating step further comprising stretching the video data for the scene and filling in missing portions of the scene with one of prior and subsequent video data.

7. The method of claim 1 wherein the modifying step further comprises interpolating the video data to compensate for camera movement, the interpolating step further comprising:

stretching the video data for the scene; and

filling in missing portions of the scene with one of prior and subsequent video data.

8. The method of claim 1 further comprising the steps of:

separating the video data from audio of the video recording data prior to the detecting step; and

recombining the video data with the audio data after the modifying step.

9. The method of claim 1, further comprising the step of analyzing the motion vectors to detect rotation indicating camera movement prior to said modifying step.

10. The method of claim 1 further comprising the step of analyzing the motion vectors to detect excessive zoom, wherein said modifying step also compensates for excessive zoom.

11. A method for stabilizing a video recording of a scene made with a video camera, the video recording including video data and audio data, the method comprising the steps of:

separating the video data from the audio data;

detecting camera movement occurring during recording by,

separating the video data into a plurality of frames,

dividing each frame into a plurality of blocks,

determining a motion vector for each block of each frame, the motion vector representing direction and magnitude in the block, said motion vectors being determined from a comparison of each block in a first one of the frames and a second one of the frames;

analyzing the motion vectors for each block over a plurality of frames; and

determining camera movement when motion vectors for one frame in the plurality of frames are different from motion vectors for adjacent frames in the plurality of frames;

modifying the video data to compensate for the camera movement by warping the video data, the warping step further comprising the steps of,

determining a source address for the video data,

determining a destination address for the video data, and

translating the video data from the source address to the destination address; and

recombining the video data with the audio data after the modifying step.

12. The method of claim 11 wherein the modifying step further comprises interpolating the video data to compensate for camera movement, the interpolating step further comprising filling in missing portions of the scene with one of prior and subsequent video data.

13. The method of claim 12 wherein the modifying step further comprises interpolating the video data to compensate for camera movement, the interpolating step further comprising:

stretching the video data for the scene; and

filling in missing portions of the scene with one of prior and subsequent video data.

14. A system for stabilizing a video recording of a scene made by a video camera comprising:

a source frame storage for storing a plurality of frames of video data of the video recording;

a processor coupled to said source frame storage for dividing each frame into a plurality of blocks and determining a motion vector for each block in said plurality of frames, said motion vectors being determined from a comparison of each block in a first one of said plurality of frames and a second one of said plurality of frames, said processor comparing motion vectors for each block in one of the plurality of frames with the motion vectors for each block in an adjacent frame, detecting camera movement when the motion vectors for the one frame are different from the motion vectors in the adjacent frame and modifying said video data to compensate for said camera movement.

15. The system of claim 14 further comprising a destination memory storage for storing the video data processed by said processor, said destination memory being distinct from said source frame storage.

16. The system of claim 14 wherein said video recording includes an audio signal and further comprising means for separating said audio signal from said video data prior to said video stabilization system, delaying said audio signal and synchronizing said delayed audio signal with said processed video data.

17. The system of claim 14 further comprising interpolating means for interpolating said video data to compensate for camera movement, said interpolating means filing in portions of the scene with portions of one of prior and subsequent video data.