Method of and apparatus for generating a depth map utilized in autofocusing

• US 7,711,201 B2
• Filed: 06/22/2006
• Issued: 05/04/2010
• Est. Priority Date: 06/22/2006
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.), Sony Electronics Inc. (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.), Sony Electronics Inc. (Sony Group Corp.)
• Inventors: Wong, Earl Q., Nakamura, Makibi, Kushida, Hidenori, Triteyaprasert, Soroj
• Primary Examiner(s): Lu; Tom Y
• Application Number:
  US11/473,694
• Publication Number:
  US 20070297784A1
• Time in Patent Office: 1,412 Days
• Field of Search: 382/254, 382/255
• US Class Current: 382/254
• CPC Class Codes: G02B 27/0075 with means for altering, e.g. increasing, the depth of field or depth of focus
• CPC Class Codes: G03B 13/30 indicating depth of field
• CPC Class Codes: G03B 3/02 moving lens along baseboard

Abstract

A method of and an apparatus for determining a depth map utilizing a movable lens and an image sensor are described herein. The depth information is acquired by moving the lens a short distance and acquiring multiple images with different blur quantities. An improved method of simulating gaussian blur and an approximation equation that relates a known gaussian blur quantity to a known pillbox quantity are used in conjunction with a non-linear blur difference equation. The blur quantity difference is able to be calculated and then used to determine the depth map. Many applications are possible using the method and system described herein, such as autofocusing, surveillance, robot/computer vision, autonomous vehicle navigation, multi-dimensional imaging and data compression.

Claims

1. An apparatus for determining depth information comprising:

a. a lens for acquiring an image signal;

b. an image sensor for receiving the image signal, wherein one or more of the lens and the image sensor are movable in relation to each other; and

c. a processing module coupled to the image sensor for determining depth information by using a first blur quantity and a second blur quantity to determine a non-linear blur difference.

2. The apparatus as claimed in claim 1 wherein the lens, the image sensor and the processing module are contained within an imaging device.

3. The apparatus as claimed in claim 2 wherein the imaging device is selected from the group consisting of a camera, a video camera, a camcorder, a digital camera, a cell phone and a PDA.

4. The apparatus as claimed in claim 1 wherein the lens is movable and acquires the image signal at a first position and then at a second position.

5. The apparatus as claimed in claim 1 wherein the processing module simulates gaussian blur.

6. The apparatus as claimed in claim 5 wherein the processing module simulates gaussian blur using iterations of a convolution kernel.

7. The apparatus as claimed in claim 1 wherein the processing module relates a gaussian blur quantity to a pillbox blur quantity.

8. A method of determining depth information comprising:

a. receiving a first image signal with a first blur quantity at an image sensor after the first image signal passes through a lens at a first position;

b. receiving a second image signal with a second blur quantity at the image sensor after the second image signal passes through the lens at a second position;

c. computing a non-linear blur difference using the first blur quantity and the second blur quantity; and

d. generating a depth map from the non-linear blur difference.

9. The method as claimed in claim 8 wherein the method is performed within an imaging device.

10. The method as claimed in claim 9 wherein the imaging device is selected from the group consisting of a camera, a video camera, a camcorder, a digital camera, a cell phone and a PDA.

11. The method as claimed in claim 8 further comprising simulating gaussian blur.

12. The method as claimed in claim 11 wherein simulating gaussian blur uses iterations of a convolution kernel.

13. The method as claimed in claim 8 further comprising relating a gaussian blur quantity to a pillbox blur quantity.

14. A method calculating depth information comprising:

a. simulating a gaussian blur quantity using a convolution kernel;

b. relating the gaussian blur quantity to a pillbox blur quantity;

c. computing a non-linear blur difference; and

d. generating a depth map from the non-linear blur difference.

15. The method as claimed in claim 14 wherein the method is performed within an imaging device.

16. The method as claimed in claim 15 wherein the imaging device is selected from the group consisting of a camera, a video camera, a camcorder, a digital camera, a cell phone and a PDA.

17. The method as claimed in claim 14 wherein simulating gaussian blur uses iterations of a convolution kernel.

18. A capture and display device comprising:

a. a lens for receiving an image signal;

b. an image sensor for receiving the received image signal, wherein one or more of the lens and the image sensor are movable in relation to each other; and

c. a program coupled to the image sensor for determining depth information by using a first blur quantity and a second blur quantity to determine a non-linear blur difference.

19. The capture and display device as claimed in claim 18 wherein the lens, the image sensor and the program are contained within an imaging device.

20. The capture and display device as claimed in claim 19 wherein the imaging device is selected from the group consisting of a camera, a video camera, a camcorder, a digital camera, a cell phone and a PDA.

21. The capture and display device as claimed in claim 18 wherein the lens is movable and acquires the image signal at a first position and then at a second position.

22. The capture and display device as claimed in claim 18 wherein the program simulates gaussian blur.

23. The capture and display device as claimed in claim 22 wherein the program simulates gaussian blur using iterations of a convolution kernel.

24. The capture and display device as claimed in claim 18 wherein the program relates a gaussian blur quantity to a pillbox blur quantity.

25. An apparatus for determining depth information comprising:

a. a lens for receiving an image signal;

b. an image sensor for receiving the received image signal, wherein one or more of the lens and the image sensor are movable in relation to each other; and

c. a processing module coupled to the image sensor for determining depth information by using a first blur quantity and a second blur quantity to determine a non-linear blur difference, wherein a gaussian blur is simulated.

26. The apparatus as claimed in claim 25 wherein the lens, the image sensor and the processing module are contained within an imaging device.

27. The apparatus as claimed in claim 26 wherein the imaging device is selected from the group consisting of a camera, a video camera, a camcorder, a digital camera, a cell phone and a PDA.

28. The apparatus as claimed in claim 25 wherein the image sensor is movable and acquires the image signal at a first position and then at a second position.

29. The apparatus as claimed in claim 25 wherein the processing module simulates the gaussian blur using iterations of a convolution kernel.

30. The apparatus as claimed in claim 25 wherein the processing module relates the gaussian blur quantity to a pillbox blur quantity.