IMAGE PROCESSING APPARATUS AND IMAGE PICKUP APPARATUS

• US 20140211045A1
• Filed: 01/27/2014
• Published: 07/31/2014
• Est. Priority Date: 01/31/2013
• Status: Active Grant

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Tanaka, Shin
• Granted Patent: US 9,313,419 B2
• Time in Patent Office: Days
• US Class Current: 348/239
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 7/50 Depth or shape recovery
• CPC Class Codes: G06T 7/60 Analysis of geometric attributes
• CPC Class Codes: G06V 40/168 Feature extraction; Face representation
• CPC Class Codes: H04N 23/959 by adjusting depth of field during image capture, e.g. maximising or setting range based on scene characteristics
• CPC Class Codes: H04N 5/2621 Cameras specially adapted for the electronic generation of special effects during image pickup, e.g. digital cameras, camcorders, video cameras having integrated special effects capability

Abstract

An image processing apparatus has: a depth map acquiring unit configured to acquire a depth map that records information indicating a depth distance corresponding to each point on a photographed image; an in-focus position acquiring unit configured to acquire an in-focus position on the photographed image; a reference distance determining unit configured to acquire a depth distance corresponding to the in-focus position from the depth map and sets the acquired depth distance as a reference distance; and an image processing unit configured to perform image processing on the photographed image by using the depth map and the reference distance.

Claims

1. An image processing apparatus comprising:

a depth map acquiring unit configured to acquire a depth map that records information indicating a depth distance corresponding to each point on a photographed image;

an in-focus position acquiring unit configured to acquire an in-focus position on the photographed image;

a reference distance determining unit configured to acquire a depth distance corresponding to the in-focus position from the depth map and sets the acquired depth distance as a reference distance; and

an image processing unit configured to perform image processing on the photographed image by using the depth map and the reference distance.

2. The image processing apparatus according to claim 1, wherein the image processing unit comprises a blurred image generating unit configured to generate an image, in which an object other than an object existing at the reference distance is blurred, by adding a blur to the photographed image in accordance with a deviation of a depth distance with respect to the reference distance.

3. The image processing apparatus according to claim 1, wherein the image processing unit comprises a distinguished image generating unit configured to generate a distinguished image by distinguishing a subject included in a specific distance range from another subject, which is included in the photographed image and is not included in the specific distance range, by using the depth map and the reference distance.

4. The image processing apparatus according to claim 1, wherein the reference distance determining unit acquires depth distances of a plurality of points corresponding to the in-focus position and determines the reference distance based on the depth distances of the plurality of points.

5. The image processing apparatus according to claim 1, wherein the in-focus position acquiring unit acquires the in-focus position from autofocus information at the time of photography of the photographed image.

6. The image processing apparatus according to claim 5, wherein the autofocus information is information on a range-finding point that was used during automatic focusing.

7. The image processing apparatus according to claim 1, wherein the information indicating a depth distance is image-plane-side information.

8. The image processing apparatus according to claim 1, further comprising

a tracking unit configured to track an object in focus, when the focus has been fixed during automatic focusing, to update a position of the object on an image, wherein

the in-focus position acquiring unit sets the position of the object on the image, which is obtained by the tracking unit at the time of photography of the photographed image, as the in-focus position.

9. The image processing apparatus according to claim 1, wherein the in-focus position acquiring unit detects a prescribed object from the photographed image by analyzing the photographed image and sets a position on the photographed image at which the prescribed object had been detected as the in-focus position.

10. The image processing apparatus according to claim 9, wherein the prescribed object is a face of a person.

11. The image processing apparatus according to claim 2, wherein when a plurality of in-focus positions are acquired by the in-focus position acquiring unit, the blurred image generating unit generates an image in which all objects existing at depth distances corresponding to the plurality of in-focus positions are not blurred.

12. The image processing apparatus according to claim 1, wherein the depth map contains depth distances in the form of multivalued information.

13. The image processing apparatus according to claim 1, wherein the depth map is generated by a depth from defocus (DFD) method or a stereo method.

14. An image pickup apparatus comprising an imaging unit and the image processing apparatus according to claim 1, wherein

the image processing apparatus performs image processing on a photographed image that is photographed by the imaging unit.

15. An image processing method in which an image processing apparatus executes the steps of:

acquiring a depth map that records information indicating a depth distance corresponding to each point on a photographed image;

acquiring an in-focus position on the photographed image;

acquiring a depth distance corresponding to the in-focus position from the depth map and setting the acquired depth distance as a reference distance; and

performing image processing on the photographed image by using the depth map and the reference distance.

16. The image processing method according to claim 15, wherein in the step of performing image processing, an image, in which an object other than an object existing at the reference distance is blurred, is generated by adding a blur in accordance with a deviation of a depth distance with respect to the reference distance to the photographed image.

17. The image processing method according to claim 15, wherein in the step of performing image processing, a distinguished image is generated by distinguishing a subject included in a specific distance range from background, which is included in the photographed image and is not included in the specific distance range, by using the depth map and the reference distance.

18. A non-transitory computer readable storage medium storing a program that causes an image processing apparatus to execute the respective steps of the image processing method according to claim 15.

19. An image processing apparatus comprising:

a depth map acquiring unit configured to acquire a depth map that records information indicating a depth distance corresponding to each point on a photographed image;

an in-focus position acquiring unit configured to acquire an in-focus position on the photographed image; and

a reference distance determining unit configured to acquire a depth distance corresponding to the in-focus position from the depth map and sets the acquired depth distance as a reference distance.