Method for creating a depth map for auto focus using an all-in-focus picture and two-dimensional scale space matching

• US 20070019883A1
• Filed: 07/19/2005
• Published: 01/25/2007
• Est. Priority Date: 07/19/2005
• Status: Abandoned Application

• Current Assignee: Sony Corporation (Sony Group Corp.), Sony Electronics Inc. (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.), Sony Electronics Inc. (Sony Group Corp.)
• Inventors: Nakamura, Makibi, Wong, Earl
• Application Number:
  US11/185,611
• Publication Number:
  US 20070019883A1
• Time in Patent Office: Days
• US Class Current: 382/276
• CPC Class Codes: G06T 7/571 from focus

Abstract

An imaging acquisition system that generates a depth map for a picture of a three dimension spatial scene from the estimated blur radius of the picture is described. The system generates an all-in-focus reference picture of the three dimension spatial scene. The system uses the all-in-focus reference picture to generate a two-dimensional scale space representation. The system computes the picture depth map for a finite depth of field using the two-dimensional scale space representation.

Claims

1. A computerized method comprising:

generating a two-dimensional scale space representation from an all-in-focus reference picture of a three dimensional spatial scene; and

computing a picture depth map based on the two-dimensional scale space representation and a finite depth of field picture of the three dimensional spatial scene, wherein an entry in the picture depth map has a corresponding entry in a picture scale map.

2. The computerized method of claim 1, further comprising generating the all-in-focus reference picture, wherein generating the all-in-focus reference picture comprises:

capturing a plurality of pictures of the three dimensional spatial scene, wherein a plurality of objects of the three dimensional spatial scene are in focus in at least one picture from the plurality of pictures;

determining a sharpest block from each block group in the plurality of pictures; and

copying the sharpest block from each block group into the all-in-focus reference pictures.

3. The computerized method of claim 1, wherein the generating the picture scale map comprises:

matching each block in the finite depth of field picture to a closest corresponding block in the two-dimensional scale space representation; and

copying the blur value associated with the closest corresponding block into the corresponding entry of the picture scale map.

4. The computerized method of claim 1, wherein the generating the two-dimensional scale space representation comprises applying a family of parametric convolving kernels to the all-in-focus reference picture.

5. The computerized method of claim 4, wherein the family of parametric convolving kernels is selected from the group consisting of a gaussian and a pillbox.

6. The computerized method of claim 1, wherein the two-dimensional scale space representation is a sequence of progressively blurred pictures of the all-in-focus reference picture.

7. The computerized method of claim 6, wherein each picture in the sequence of progressively blurred pictures has a known blur value.

8. The method of claim 1, further comprising:

applying a clustering algorithm to the depth map.

9. The computerized method of claim 1, wherein the computing the picture depth map comprises:

generating the picture scale map entry from the finite depth of field picture and the two-dimensional scale space representation; and

calculating, from the picture scale map entry, the picture depth map entry using the equation

$d_o=\frac{\mathrm{fD}}{D-f-2<br><br>{\mathrm{rf}}_{\mathrm{number}}},$

where f is the camera lens focal length, D the distance between the image plane inside the camera and the lens, r is the blur radius of the image on the image plane and f number is the f_number of the camera lens.

10. A machine readable medium having executable instructions to cause a processor to perform a method comprising:

generating a two-dimensional scale space representation from an all-in-focus reference picture of a three dimensional spatial scene; and

computing a picture depth map based on the two-dimensional scale space representation and a finite depth of field picture of the three dimensional spatial scene, wherein an entry in the picture depth map has a corresponding entry in a picture scale map.

11. The machine readable medium of claim 10, further comprising generating the all-in-focus reference picture, wherein generating the all-in-focus reference picture comprises:

capturing a plurality of pictures of the three dimensional spatial scene, wherein a plurality of objects of the three dimensional spatial scene are in focus in at least one picture from the plurality of pictures;

determining a sharpest block from each block group in the plurality of pictures; and

copying the sharpest block from each block group into the all-in-focus reference pictures.

12. The machine readable medium of claim 10, wherein the generating the picture scale map comprises:

matching each block in the finite depth of field picture to a closest corresponding block in the two-dimensional scale space representation; and

copying the blur value associated with the closest corresponding block into the corresponding entry of the picture scale map.

13. The machine readable medium of claim 10, wherein the generating the two-dimensional scale space representation comprises applying a family of parametric convolving kernels to the all-in-focus reference picture.

14. The machine readable medium of claim 10 wherein the computing the picture depth map comprises:

generating a picture scale map from the finite depth of field picture and the two-dimensional scale space representation; and

calculating, from a picture scale map entry, the picture depth map entry using the equation

$d_o=\frac{\mathrm{fD}}{D-f-2<br><br>{\mathrm{rf}}_{\mathrm{number}}},$

where f is the camera lens focal length, D the distance between the image plane inside the camera and the lens, r is the blur radius of the image on the image plane and f number is the f_number of the camera lens.

15. An apparatus comprising:

means for generating a two-dimensional scale space representation from an all-in-focus reference picture of a three dimensional spatial scene; and

means for computing a picture depth map based on the two-dimensional scale space representation and a finite depth of field picture of the three dimensional spatial scene, wherein an entry in the picture depth map has a corresponding entry in a picture scale map.

16. The apparatus of claim 15, further comprising means for generating the all-in-focus reference picture, wherein the means for generating the all-in-focus reference picture comprises:

means for capturing a plurality of pictures of the three dimensional spatial scene, wherein a plurality of objects of the three dimensional spatial scene are in focus in at least one picture from the plurality of pictures;

means for determining a sharpest block from each block group in the plurality of pictures; and

means for copying the sharpest block from each block group into the all-in-focus reference pictures.

17. A system comprising:

a processor;

a memory coupled to the processor though a bus; and

a process executed from the memory by the processor to cause the processor to generate a two-dimensional scale space representation from an all-in-focus reference picture of a three dimensional spatial scene and to compute a picture depth map based on the two-dimensional scale space representation and a finite depth of field picture of the three dimensional spatial scene, wherein an entry in the picture depth map has a corresponding entry in a picture scale map.

18. The system of claim 17, wherein the process further causes the processor to generate the all-in-focus reference picture, the all-in-focus reference picture generation comprises:

capturing a plurality of pictures of the three dimensional spatial scene, wherein a plurality of objects of the three dimensional spatial scene are in focus in at least one picture from the plurality of pictures;

determining a sharpest block from each block group in the plurality of pictures; and

copying the sharpest block from each block group into the all-in-focus reference pictures.

19. The system of claim 17, wherein the generating the picture scale map comprises:

matching each block in the finite depth of field picture to a closest corresponding block in the two-dimensional scale space representation; and

copying the blur value associated with the closest corresponding block into the corresponding entry of the picture scale map.

20. The system of claim 17, wherein the generating the two-dimensional scale space representation comprises applying a family of parametric convolving kernels to the all-in-focus reference picture.