CAMERA SHAKE CORRECTION DEVICE, CAMERA SHAKE CORRECTION METHOD AND IMAGING DEVICE

• US 20080111889A1
• Filed: 11/12/2007
• Published: 05/15/2008
• Est. Priority Date: 11/13/2006
• Status: Active Grant

• Current Assignee: Xacti Corporation (Office Support KK)
• Original Assignee: Sanyo Electric Company Limited (Panasonic Holdings Corporation)
• Inventors: Yoshida, Hiroaki, Yamada, Akihiko, FUJITA, Hideto, Hatanaka, Haruo
• Granted Patent: US 8,134,604 B2
• Time in Patent Office: Days
• US Class Current: 348/208.5
• CPC Class Codes: H04N 23/611 where the recognised objects include parts of the human body
• CPC Class Codes: H04N 23/68 for stable pick-up of the scene, e.g. compensating for camera body vibrations
• CPC Class Codes: H04N 23/6811 based on the image signal
• CPC Class Codes: H04N 23/6815 by distinguishing pan or tilt from motion
• CPC Class Codes: H04N 23/682 Vibration or motion blur correction
• CPC Class Codes: H04N 23/683 performed by a processor, e.g. controlling the readout of an image memory
• CPC Class Codes: H04N 23/687 by shifting the lens or sensor position
• CPC Class Codes: H04N 5/145 Movement estimation for video coding H04N19/51

Abstract

Movement of an imaging device is detected using a camera shake detection sensor or a motion vector obtained from an image signal, and camera shake correction is performed by shifting an extraction frame (valid photographing region) within a photographable maximum region based on the motion detection result. When movement of the imaging device is causing a face region located at the edge of the extraction frame to move towards out of the frame, it is determined that the movement is due to camera shake, and the extraction frame is moved so that the face region can stay within the frame. Meanwhile, when movement of the imaging device is causing the face region to move inward within the frame, it is determined that the movement is due to a pan or tilt operation, and position control is performed so that the deliberate movement is not inhibited.

Claims

1. A camera shake correction device comprising:

a detection unit configured to detect movement of an imaging device, and to output motion data representing the detected movement;

an object-region extraction unit configured to extract a target object region from a photographed image obtained by an imaging unit in the imaging device, and to output position data of the extracted object region for each frame of the photographed image; and

a correction unit configured to perform camera shake correction on the photographed image based on the position data of the object region and the motion data.

2. The camera shake correction device as claimed in claim 1, further comprising:

a discrimination unit configured to receive the position data of the extracted object region for each frame and the motion data, and, to detect movement of the imaging device between adjacent frames based on the position data of the object region for each frame and on the motion data, and to determine whether the movement of the imaging device is camera shake,

wherein said correction unit performs the camera shake correction processing on the camera shake based on the determination of the discrimination unit.

3. The camera shake correction device as claimed in claim 2, wherein the correction unit aborts execution of the camera shake correction processing when the discrimination unit determines that the movement of the imaging device is not camera shake.

4. The camera shake correction device as claimed in claim 2, wherein the discrimination unit determines that movement of the imaging device is camera shake upon detecting, based on the position data of the object region for each frame and the motion data, movement of the imaging device that causes the object region to move outward from an imaging region of the imaging unit, whereas the discrimination unit determines that the movement of the imaging device is not camera shake upon detecting movement of the imaging device causing the object region to move inward within the imaging region of the imaging unit.

5. The camera shake correction device as claimed in claim 1, wherein the object-region extraction unit detects an object of a predetermined type or an object having a predetermined image characteristic from each photographed image, and extracts a region including the entire object or a part of the object as the object region.

6. The camera shake correction device as claimed in claim 1, wherein the object-region extraction unit extracts a face region corresponding to a human face from the photographed image obtained by the imaging unit in the imaging device, and outputs position data of the extracted face region for each frame of the captured image.

7. The camera shake correction device as claimed in claim 1, wherein the object-region extraction unit classifies regions of the photographed image into those with relatively high color saturation and those with relatively low color saturation, and extracts the regions with relatively high color saturation as the object region.

8. The camera shake correction device as claimed in claim 1, wherein the object-region extraction unit classifies regions of the photographed image into those with a relatively large number of edge components and those with a relatively small number of edge components, and extracts the regions with a relatively large number of edge components as the object region.

9. The camera shake correction device as claimed in claim 1, wherein the object-region extraction unit classifies regions of the photographed image into regions with a relatively large amount of contrast and those with a relatively small amount of contrast, and extracts the regions with a relatively large amount of contrast as the object region.

10. The camera shake correction device as claimed in claim 1, wherein the object-region extraction unit compares between a registered pattern and a photographed image, and extracts the object region on the basis of the comparison result.

11. The camera shake correction device as claimed in claim 1, wherein, based on an image signal in a motion detection region provided in each photographed image, the detection unit detects movement between the photographed images as movement of the imaging device, and outputs the detected movement of the imaging device as the motion data, and, based on position data of the object region extracted by the object-region extraction unit, the detection unit excludes the object region from the motion detection region, and outputs motion data representing movement of the imaging device.

12. The camera shake correction device as claimed in claim 11, wherein the object-region extraction unit extracts a face region corresponding to a human face from the photographed image obtained by the imaging unit in the imaging device, and outputs position data of the extracted face region for each frame of the photographed image.

13. The camera shake correction device as claimed in claim 12, wherein the detection unit excludes the face region from the motion detection region only when transition takes place from a frame in which the face region is not included in the captured image to a frame in which the face region is included in the photographed image.

14. The camera shake correction device as claimed in claim 8, wherein the detection unit excludes the face region from the motion detection region upon detecting a change in the direction of a face corresponding to the extracted face region.

15. An imaging device comprising:

an imaging unit configured to photograph a subject, and output a photographed image; and

a correction unit configured to detect camera shake of the imaging device based on the photographed image, to perform camera shake correction processing, and to output a valid photographed image which is a partial image of the photographed image, said correction unit comprising;

a detection unit configured to detect movement of the imaging device, and to output motion data representing the movement;

an object-region extraction unit configured to extract a target object region from a photographed image obtained by the imaging unit in the imaging device, and to output position data of the extracted object region for each frame of the photographed image;

a discrimination unit configured to receive the position data of the extracted object region for each frame and the motion data, and, based on the position data of the object region for each frame and the motion data, to detect movement of the imaging device between adjacent frames, and to determine whether the movement of the imaging device is camera shake, and

a correction unit configured to perform camera shake correction for the photographed image on the basis of the discrimination result of the discrimination unit, to extract a valid photographed image from the photographed image, and to output the valid photographed image.

16. The imaging device as claimed in claim 15, wherein the discrimination unit determines that movement of the imaging device is camera shake upon detecting, based on the position data of the object region for each frame and the motion data, the movement of the imaging device that causes the object region to move outward from an imaging region of the imaging unit, whereas the discrimination unit determines that movement of the imaging device is not camera shake upon detecting the movement of the imaging device causing the object region to move inward within the imaging region of the imaging unit.

17. The imaging device as claimed in claim 16, wherein

the correction unit performs camera shake correction of the photographed image based on the discrimination result of the discrimination unit, and,

when the object region is located at the edge of the valid photographed region, upon detecting the object region moving outward from the partial image of the imaging unit, the discrimination unit extracts the partial image so that the object region is included in the partial image, and outputs the partial image, whereas the discrimination unit aborts execution of the camera shake correction upon detecting the object region moving inward within the valid photographed image of the imaging unit.

18. A camera shake correction method comprising the steps of:

detecting movement of an imaging device, and outputting motion data representing the movement;

extracting a target object region from a photographed image obtained by an imaging unit in the imaging device;

outputting position data of the extracted object region for each frame of the photographed image; and

performing camera shake correction of the photographed image based on the position data of the object region and the motion data.

19. The method as claimed in claim 18, further comprising:

receiving the position data of the extracted object region for each frame and the motion data;

detecting movement of the imaging device between adjacent frames based on the position data of the object region for each frame and on the motion data; and

determining whether the movement of the imaging device is camera shake.

wherein said performing camera shake correction performs the camera shake correction processing on the camera shake based on the determination of the discrimination unit.

20. The method as claimed in claim 19, wherein the performing camera shake correction includes aborting execution of the camera shake correction processing when the movement of the imaging device is not camera shake.