Image processing apparatus and a method to compensate for shaking during image capture

• US 7,015,952 B2
• Filed: 09/10/2001
• Issued: 03/21/2006
• Est. Priority Date: 09/28/2000
• Status: Expired due to Term

• Current Assignee: Toshiba Electronic Devices & Storage Corporation (Toshiba Corporation), Toshiba Infrastructure Systems & Solutions Corporation (Toshiba Corporation)
• Original Assignee: Kabushiki Kaisha Toshiba (Toshiba Corporation)
• Inventors: Okada, Ryuzo
• Primary Examiner(s): Vu, Ngoc-Yen
• Assistant Examiner(s): Vieaux, Gary C.
• Application Number:
  US09/948,635
• Publication Number:
  US 20020036692A1
• Time in Patent Office: 1,653 Days
• Field of Search: 3482083-2086, 348/208.14, 348/208.15, 348/208, 348/208.99, 348/208.1, 348/155, 348/699, 348/113, 348/118, 348/154, 340/933, 340/937, 396/52, 396/421, 396/55
• US Class Current: 348/208.1
• CPC Class Codes: H04N 23/68 for stable pick-up of the scene, e.g. compensating for camera body vibrations
• CPC Class Codes: H04N 23/6811 based on the image signal
• CPC Class Codes: H04N 23/682 Vibration or motion blur correction

Abstract

An image processing apparatus comprises an image capture device which generates a video image containing an image shake caused by a shake of the image capture device, a region selector which selects a given image region used for detecting the image shake, a shake detector which detects an amount of the image shake within the given image region, and an image compensator which compensates the video image based on the amount of the image shake to cancel the image shake.

Claims

1. An image processing apparatus, comprising:

an image capture device configured to generate a video image containing an image shake caused by a shake of the image capture device;

a region selector configured to select, from the video image, a laterally elongated region for detecting a vertical shake but not for detecting a lateral shake or a vertically elongated region for detecting a lateral shake but not for detecting a vertical shake;

a shake detector configured to detect an amount of the image shake within the laterally elongated region or the vertically elongated region; and

an image compensator configured to compensate the video image based on the amount of the image shake to cancel the image shake.

2. An apparatus according to claim 1, wherein the region selector selects an image region including a vanishing point corresponding to an infinite distance in a direction in which the image capture device is moved as the given image region.

3. An image processing apparatus provided for a mobile object movable in real space, comprising:

an image capture device mounted on the mobile object and configured to generate a video image containing an image shake caused by a shake of the image capture device;

a region selector configured to select, from the video image, a laterally elongated region for detecting a vertical shake but not for detecting a lateral shake or a vertically elongated region for detecting a lateral shake but not for detecting a vertical shake;

a shake detector configured to detect an amount of the image shake within the laterally elongated region or the vertically elongated region and

an image compensator configured to compensate the video image based on the amount of the image shake to cancel the image shake.

4. An apparatus according to claim 3, wherein the region selector selects an image region including a vanishing point corresponding to an infinite distance of a direction in which the mobile object is moving as the given image region.

5. An apparatus according to claim 3, wherein the region selector selects a region such that a variation of the image caused by translation of the mobile object other than the shake of the image capture device caused by the unevenness of a road surface or steering of the mobile object is minute.

6. An apparatus according to claim 3, wherein the shake detector detects a motion vector of each pixel in the laterally elongated region or vertically elongated region, and computes the amount of shake of the video image by analyzing the motion vector.

7. An apparatus according to claim 3, wherein the shake detector comprises a motion vector detector and a shake-amount computation device, the motion vector detector detecting a motion vector of each pixel in the laterally elongated region or vertically elongated region, using at least two frames of the video image, and the shake-amount computation device computing the amount of image shake from the motion vector detected by the motion vector detector.

8. An apparatus according to claim 7, wherein the shake detector sets a reference frame of the video image, and the motion vector detector computes the motion vector of each pixel using the reference frame and a current frame of the video image.

9. An apparatus according to claim 8, wherein the reference frame is updated at one or both of a time when a magnitude of the motion vector exceeds a predetermined value and a time when a difference between a current time and a time when the reference frame is captured becomes larger than a predetermined value.

10. An apparatus according to claim 3, wherein the image capture device comprises a television camera mounted on a vehicle as the mobile object.

11. An image processing method, comprising:

generating a video image containing an image shake;

selecting, from the video image, a laterally elongated region for detecting a vertical shake but not for detecting a lateral shake or a vertically elongated region for detecting a lateral shake but not for detecting a vertical shake;

detecting an amount of the image shake within the laterally elongated region or the vertically elongated region; and

compensating the video image based on the amount of the image shake to cancel the image shake.

12. An image processing method, comprising:

generating a video image containing an image shake caused by a shake of the image capture device mounted on a mobile object moving on a road;

selecting, from the video image, a laterally elongated region for detecting a vertical shake but not for detecting a lateral shake or a vertically elongated region for detecting a lateral shake but not for detecting a vertical shake;

detecting an amount of the image shake within the laterally elongated region or the vertically elongated region; and

compensating the video image based on the amount of the image shake to cancel the image shake.

13. A method according to claim 12, wherein the step of selecting the laterally elongated region or vertically elongated region selects an image region including a vanishing point corresponding to an infinite distance in a direction in which the mobile object is moving.

14. A method according to claim 12, wherein the step of selecting the laterally elongated region or vertically elongated region selects a region wherein the image shake accompanying parallel displacement of the mobile object other than the shake of the image capture device caused by the unevenness of a road surface or steering of the mobile object is minute.

15. A method according to claim 12, wherein the step of detecting an amount of the image shake includes detecting a motion vector of each pixel in the the laterally elongated region or the vertically elongated region, and computing the amount of shake of the video image by analyzing the motion vector.

16. A method according to claim 12, wherein the step of detecting an amount of the image shake includes detecting a motion vector of each pixel in the the laterally elongated region or the vertically elongated region, using at least two frames of the video image, and computing the amount of image shake from the motion vector detected by the motion vector detector.

17. A computer readable program stored in a storing medium, the program comprising:

means for instructing a computer to input a video image containing an image shake caused by a shake of an image capture device mounted on a mobile object moving on a road;

means for instructing the computer to select, from the video image, a laterally elongated region for detecting a vertical shake but not for detecting a lateral shake or a vertically elongated region for detecting a lateral shake but not for detecting a vertical shake;

means for instructing the computer to detect an amount of the image shake within the laterally elongated region or the vertically elongated region; and

means for instructing to compensate the video image based on the amount of the image shake to cancel the image shake.