Camera shake correction device, camera shake correction method and imaging device

• US 8,134,604 B2
• Filed: 11/12/2007
• Issued: 03/13/2012
• Est. Priority Date: 11/13/2006
• Status: Expired due to Fees

• Current Assignee: Xacti Corporation (Office Support KK)
• Original Assignee: Sanyo Electric Company Limited (Panasonic Holdings Corporation)
• Inventors: Fujita, Hideto, Hatanaka, Haruo, Yoshida, Hiroaki, Yamada, Akihiko
• Primary Examiner(s): Chan, Jason
• Assistant Examiner(s): Vieaux, Gary C
• Application Number:
  US11/938,401
• Publication Number:
  US 20080111889A1
• Time in Patent Office: 1,583 Days
• Field of Search: 348/208.3, 348/208.14, 348/204.6, 348/208.4, 348/208.6, 382/118
• US Class Current: 348/208.6
• CPC Class Codes: H04N 23/611 where the recognised objects include parts of the human body
• CPC Class Codes: H04N 23/68 for stable pick-up of the scene, e.g. compensating for camera body vibrations
• CPC Class Codes: H04N 23/6811 based on the image signal
• CPC Class Codes: H04N 23/6815 by distinguishing pan or tilt from motion
• CPC Class Codes: H04N 23/682 Vibration or motion blur correction
• CPC Class Codes: H04N 23/683 performed by a processor, e.g. controlling the readout of an image memory
• CPC Class Codes: H04N 23/687 by shifting the lens or sensor position
• CPC Class Codes: H04N 5/145 Movement estimation for video coding H04N19/51

Abstract

Movement of an imaging device is detected using a camera shake detection sensor or a motion vector obtained from an image signal, and camera shake correction is performed by shifting an extraction frame (valid photographing region) within a photographable maximum region based on the motion detection result. When movement of the imaging device is causing a face region located at the edge of the extraction frame to move towards out of the frame, it is determined that the movement is due to camera shake, and the extraction frame is moved so that the face region can stay within the frame. Meanwhile, when movement of the imaging device is causing the face region to move inward within the frame, it is determined that the movement is due to a pan or tilt operation, and position control is performed so that the deliberate movement is not inhibited.

Claims

1. A camera shake correction device comprising:

a detection unit configured to detect movement of an imaging device, and to output motion data representing the detected movement;

an object-region extraction unit configured to extract a target object region from a photographed image obtained by an imaging unit in the imaging device; and

a correction unit configured to perform camera shake correction on the photographed image, wherein the correction unit performs the camera shake correction on the photographed image based on a moving direction of the target object region in the photographed image wherein when the moving direction of the target object image is outward of the photographed image, the correction unit performs the camera shake correction but when the moving direction of the target object image is inward of the photographed image, the correction unit does not perform the camera shake correction.

2. The camera shake correction device as claimed in claim 1, further comprising:

a discrimination unit configured to receive the motion data of the extracted object region and to determine whether the movement of the imaging device is camera shake or intentional moving of the camera.

3. The camera shake correction device as claimed in claim 2, wherein the correction unit aborts execution of the camera shake correction processing when the discrimination unit determines that the movement of the imaging device is not camera shake.

4. The camera shake correction device as claimed in claim 1, wherein the object-region extraction unit detects an object of a predetermined type or an object having a predetermined image characteristic from each photographed image, and extracts a region including the entire object or a part of the object as the object region.

5. The camera shake correction device as claimed in claim 1, wherein the object-region extraction unit extracts a face region corresponding to a human face from the photographed image obtained by the imaging unit in the imaging device, and outputs position data of the extracted face region.

6. A camera shake correction device comprising:

a detection unit configured to detect movement of an imaging device, and to output motion data representing the detected movement;

an object-region extraction unit configured to extract a target object region from a photographed image obtained by an imaging unit in the imaging device, and to output position data of the extracted object region for each frame of the photographed image;

a correction unit configured to perform camera shake correction on the photographed image based on the position data of the object region and the motion data; and

a discrimination unit configured to receive the position data of the extracted object region for each frame and the motion data, and, to detect movement of the imaging device between adjacent frames based on the position data of the object region for each frame and on the motion data, and to determine whether the movement of the imaging device is camera shake,

wherein said correction unit performs the camera shake correction processing on the camera shake based on the determination of the discrimination unit, wherein the discrimination unit determines that movement of the imaging device is camera shake upon detecting, based on the position data of the object region for each frame and the motion data, movement of the imaging device that causes the object region to move outward from an imaging region of the imaging unit, whereas the discrimination unit determines that the movement of the imaging device is not camera shake upon detecting movement of the imaging device causing the object region to move inward within the imaging region of the imaging unit.

7. An imaging device comprising:

an imaging unit configured to photograph a subject, and output a photographed image; and

a correction unit configured to detect camera shake of the imaging device based on the photographed image, to perform camera shake correction processing, and to output a valid photographed image which is a partial image of the photographed image, said correction unit comprising;

a detection unit configured to detect movement of the imaging device, and to output motion data representing the movement;

an object-region extraction unit configured to extract a target object region from a photographed image obtained by the imaging unit in the imaging device, and to output position data of the extracted object region for each frame of the photographed image;

a discrimination unit configured to receive the position data of the extracted object region for each frame and the motion data, and, based on the position data of the object region for each frame and the motion data, to detect movement of the imaging device between adjacent frames, and to determine whether the movement of the imaging device is camera shake, and

an extraction unit configured to perform camera shake correction for the photographed image on the basis of the discrimination result of the discrimination unit, to extract a valid photographed image from the photographed image, and to output the valid photographed image,

wherein the discrimination unit determines that movement of the imaging device is camera shake upon detecting, based on the position data of the object region for each frame and the motion data, the movement of the imaging device that causes the object region to move outward from an imaging region of the imaging unit, whereas the discrimination unit determines that movement of the imaging device is not camera shake upon detecting the movement of the imaging device causing the object region to move inward within the imaging region of the imaging unit.

8. The imaging device as claimed in claim 7, wherein

the extraction unit performs camera shake correction of the photographed image based on the discrimination result of the discrimination unit, and,

when the object region is located at the edge of the valid photographed region, upon detecting the object region moving outward from the partial image of the imaging unit, the discrimination unit extracts the partial image so that the object region is included in the partial image, and outputs the partial image, whereas the discrimination unit aborts execution of the camera shake correction upon detecting the object region moving inward within the valid photographed image of the imaging unit.

9. A camera shake correction method comprising the steps of:

detecting movement of an imaging device, and outputting motion data representing the movement;

extracting a target object region from a photographed image obtained by an imaging unit in the imaging device; and

performing camera shake correction of the photographed image, the camera shake correction on the photographed image is performed based on a moving direction of the target object region in the photographed image wherein when the moving direction of the target object image is outward of the photographed image, the correction unit performs the camera shake correction but when the moving direction of the target object image is inward of the photographed image, the correction unit does not perform the camera shake correction.

10. The method as claimed in claim 9, wherein the performing camera shake correction includes aborting execution of the camera shake correction processing when the movement of the imaging device is not camera shake.