IMAGE TAKING APPARATUS, IMAGE PROCESSING APPARATUS, IMAGE PROCESSING METHOD, AND IMAGE PROCESSING PROGRAM

• US 20080186386A1
• Filed: 10/25/2007
• Published: 08/07/2008
• Est. Priority Date: 11/30/2006
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: OKADA, Miyuki, Nomura, Hirofumi, Tsujimura, Takashi
• Granted Patent: US 8,228,390 B2
• Time in Patent Office: Days
• US Class Current: 348/208.4
• CPC Class Codes: H04N 21/42204 User interfaces specially adapted for controlling a client device through a remote control device; Remote control devices therefor constructive details of casings for the remote control device H01H9/0235; user interfaces for controlling a tuning device of a television receiver through a remote control H03J9/00; remote control of peripheral devices connected to a television receiver through the remote control device of the television receiver H04B1/205
• CPC Class Codes: H04N 21/4316 for displaying supplemental content in a region of the screen, e.g. an advertisement in a separate window
• CPC Class Codes: H04N 21/47 End-user applications
• CPC Class Codes: H04N 23/68 for stable pick-up of the scene, e.g. compensating for camera body vibrations
• CPC Class Codes: H04N 5/77 between a recording apparatus and a television camera
• CPC Class Codes: H04N 5/772 the recording apparatus and the television camera being placed in the same enclosure
• CPC Class Codes: H04N 9/8047 using transform coding
• CPC Class Codes: H04N 9/8227 the additional signal being at least another television signal

Abstract

In an image taking apparatus, an imaging unit senses an image of a subject, and a face detection unit detects a face in the image sensed by the imaging unit. A motion detection unit detects an amount of face motion indicating an amount of shake of an area of the face detected by the face detection unit in the image sensed by the imaging unit and also detects an amount of background motion indicating an amount of shake of an area other than the face area. A shake correction unit corrects the shake of the image sensed by the imaging unit according to the amount of face motion and the amount of background motion detected by the motion detection unit.

Claims

1. An image taking apparatus comprising:

imaging means for sensing an image of a subject;

face detection means for detecting a face in the image sensed by the imaging means;

motion detection means for detecting an amount of face motion indicating an amount of shake of an area of the face detected, by the face detection means, in the image sensed by the imaging means and also detecting an amount of background motion indicating an amount of shake of an area other than the face area; and

shake correction means for correcting the shake of the image sensed by the imaging means according to the amount of face motion and the amount of background motion detected by the motion detection means.

2. The image taking apparatus according to claim 1, further comprising correction value calculation means for calculating the correction value of the shake of the image to be corrected by the shake correction means, in accordance with the amount of face motion and the amount of background motion detected by the motion detection means.

3. The image taking apparatus according to claim 2, wherein the correction value calculation means calculates the correction value in accordance with the amount of face motion and the amount of background motion detected by the motion detection means and in accordance with predetermined weighting factors.

4. The image taking apparatus according to claim 3, wherein the correction value calculation means calculates the correction value by applying a greater weighting factor to the amount of face motion detected by the motion detection means than to the amount of background motion detected by the motion detection means.

5. The image taking apparatus according to claim 3, wherein:

the face detection means further detects feature information associated with the face detected in the image sensed by the imaging means; and

the correction value calculation means calculates the weighting factors in accordance with the feature information associated with the face detected by the face detection means.

6. The image taking apparatus according to claim 5, wherein the feature information associated with the face includes one or more pieces of information selected from the group consisting of information indicating a face area size, information indicating coordinates of the face area, information indicating the likelihood of being a face, information indicating the degree of facing forward, information indicating the degree of tilt of the face, information indicating the degree of smiling, information indicating the degree of being serious, and information indicating the degree of closing of eyes.

7. The image taking apparatus according to claim 5, wherein in the calculation of the correction value, the correction value calculation means increases the weighting factor for blocks located close to the face detected by face detection means.

8. The image taking apparatus according to claim 1, wherein the motion detection means divides the image sensed by the imaging means into a plurality of areas such that the face detected by the face detection means is included in one of the areas, detects the amount of face motion from the area including the face, and detects the amount of background motion from areas other than the area including the face.

9. The image taking apparatus according to claim 1, further comprising vibration detection means for detecting an amount of shake due to a vibration of the image taking apparatus, wherein

the shake correction means corrects the shake of the image sensed by the imaging means in accordance with the amount of face motion and the amount of background motion detected by the motion detection means and the amount of shake detected by the vibration detection means.

10. The image taking apparatus according to claim 5, further comprising camera control value detection means for detecting the degree of whether focus and/or exposure are set optimally for the face detected by the face detection means, based on the feature information associated with the face, wherein

the correction value calculation means calculates the correction value for the face in accordance with the degree of whether focus and/or exposure are set optimally for the face, detected by the camera control value detection means.

11. The image taking apparatus according to claim 1, further comprising setting means for setting whether detection of a face in the image sensed by the imaging means should be performed.

12. The image taking apparatus according to claim 3, further comprising

specific person storage means for storing specific person information associated with a face of a specific person;

and

specific person identification means for determining whether the face detected by the face detection means is identical to the face of the specific person stored in the specific person storage means, wherein

the correction value calculation means calculates the correction value such that if the specific person identification means determines that the face detected by the face detection means is identical to the face of the specific person stored in the specific person storage means, the correction value calculation means modifies the weighting factor for each area depending on whether the area includes the face of the specific person.

13. The image taking apparatus according to claim 2, wherein the correction value calculation means calculates a first average value indicating the average of the amounts of face motion detected by the motion detection means, calculates the correction value for the area of the face detected by the face detection means in accordance with the first average value, calculates a second average value indicating the average of the amounts of background motion detected by the motion detection means, and calculates the correction value for areas other than the area of the face detected by the face detection means in accordance with the second average value.

14. The image taking apparatus according to claim 13, wherein if a plurality of faces are detected by the face detection means, the correction value calculation means calculates the average value of the amounts of face motion for each of the faces detected by the face detection means and calculates the correction value in accordance with the calculated average value.

15. The image taking apparatus according to claim 1, wherein the shake correction means corrects the shake of the image sensed by the imaging means such that the area of the face detected by the face detection means is moved toward the center of the image.

16. The image taking apparatus according to claim 1, wherein the motion detection means detects the amount of face motion of the area of the face, detected by the face detection means, in the image sensed by the imaging means on the basis of the correlation between the face detected by the face detection means from the current image detected by the imaging means and a face detected by the face detection means from a previous image detected by the imaging means a predetermined time before the current image.

17. An image processing apparatus comprising:

image input means for inputting an image;

face detection means for detecting a face in the image input via the image input means;

motion detection means for detecting an amount of face motion indicating an amount of shake of an area of the face detected, by the face detection means, in the image input via the image input means and also detecting an amount of background motion indicating an amount of shake of an area other than the face area; and

shake correction means for correcting the shake of the image input via the image input means according to the amount of face motion and the amount of background motion detected by the motion detection means.

18. A method of processing an image, comprising the steps of:

inputting an image;

detecting a face in the image input in the image inputting step;

detecting an amount of face motion indicating an amount of shake of an area of the face detected, in the face detection step, in the image input in the image inputting step, and detecting an amount of background motion indicating an amount of shake of an area other than the face area; and

correcting the shake of the image input in the image inputting step according to the amount of face motion and the amount of background motion detected in the motion detection step.

19. A program executable by a computer to perform a process comprising the steps of:

inputting an image;

detecting a face in the image input in the image inputting step;

detecting an amount of face motion indicating an amount of shake of an area of the face detected, in the face detection step, in the image input in the image inputting step, and detecting an amount of background motion indicating an amount of shake of an area other than the face area; and

correcting the shake of the image input in the image inputting step according to the amount of face motion and the amount of background motion detected in the motion detection step.

20. An image taking apparatus comprising:

an imaging unit adapted to sense an image of a subject;

a face detection unit adapted to detect a face in the image sensed by the imaging unit;

a motion detection unit adapted to detect an amount of face motion indicating an amount of shake of an area of the face detected, by the face detection unit, in the image sensed by the imaging unit and also detect an amount of background motion indicating an amount of shake of an area other than the face area; and

a shake correction unit adapted to correct the shake of the image sensed by the imaging unit according to the amount of face motion and the amount of background motion detected by the motion detection unit.

21. An image processing apparatus comprising:

an image input unit adapted to input an image;

a face detection unit adapted to detect a face in the image input via the image input unit;

a motion detection unit adapted to detect an amount of face motion indicating an amount of shake of an area of the face detected, by the face detection unit, in the image input via the image input unit and also detect an amount of background motion indicating an amount of shake of an area other than the face area; and

a shake correction unit adapted to correct the shake of the image input via the image input unit according to the amount of face motion and the amount of background motion detected by the motion detection unit.