ALGORITHMS FOR ESTIMATING PRECISE AND RELATIVE OBJECT DISTANCES IN A SCENE

• US 20110142287A1
• Filed: 12/16/2009
• Published: 06/16/2011
• Est. Priority Date: 12/16/2009
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.), Sony Electronics Inc. (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.), Sony Electronics Inc. (Sony Group Corp.)
• Inventors: Murakami, Yoshihiro, Kushida, Hidenori, Wong, Earl, Sugiura, Mamoru, Nakamura, Maikibi, Triteyaprasert, Soroj, Li, Pingshan
• Granted Patent: US 8,229,172 B2
• Time in Patent Office: Days
• US Class Current: 382/106
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10148 Varying focus
• CPC Class Codes: G06T 7/571 from focus

Abstract

A two picture matching curve information is able to be used to determine precise object distance or relative object distance in a scene. Acquiring two images with different blur information in addition to the curve information enables a device to determine distance information of objects in a scene. The distance information is able to be used in image processing including generating a depth map which is then able to be used in many imaging applications.

Claims

1. A method implemented on a device comprising:

a. acquiring a first image of a scene;

b. acquiring a second image of the scene; and

c. utilizing curve information to determine a device-to-object distance of an object in the scene.

2. The method of claim 1 wherein the curve information is precalculated.

3. The method of claim 1 wherein utilizing the curve information includes:

a. determining a number of convolutions used to blur one of the first image and the second image to a blurriness of the other of the first image and the second image;

b. using the number of convolutions to determine an object-to-focus position distance based on the curve information;

c. computing a device-to-focus position distance; and

d. adding the object-to-focus position distance and the device-to-focus position distance to determine the device-to-object distance.

4. The method of claim 1 wherein the curve information includes multiple curves.

5. The method of claim 1 further comprising generating the curve information.

6. The method of claim 5 wherein generating the curve information includes acquiring multiple images at different blur quantities.

7. The method of claim 1 wherein the first image and the second image have a different blur amount.

8. The method of claim 7 wherein the different blur amount is achieved by changing the focus position between acquiring the first image and the second image.

9. The method of claim 1 further comprising generating a depth map.

10. The method of claim 9 further comprising storing the depth map.

11. The method of claim 10 further comprising utilizing the depth map to perform an application.

12. The method of claim 11 wherein the application is selected from the group consisting of auto focus, auto exposure, zoom setting, aperture setting, flash setting, shutter speed, white balance, noise reduction, gamma correction, motion estimation, image/video compression, generating blur, quality improvement, generating a 3-D image, shadow removal and object segmentation.

13. The method of claim 1 wherein the device is selected from the group consisting of a personal computer, a laptop computer, a computer workstation, a server, a mainframe computer, a handheld computer, a personal digital assistant, a cellular/mobile telephone, a smart appliance, a gaming console, a digital camera, a digital camcorder, a camera phone, an iPod®

, a video player, a DVD writer/player, a television and a home entertainment system.

14. A system implemented on a device configured for determining distances of objects in an image, the system comprising:

a. an image acquisition module configured for acquiring a first image and a second image;

b. a curve generation module operatively coupled to the image acquisition module, the curve generation module configured for generating curve information; and

c. a distance calculation module operatively coupled to the curve generation module, the distance calculation module configured for calculating a device-to-object distance of an object within the image.

15. The system of claim 14 wherein the distance calculation module utilizes the curve information by:

a. determining a number of convolutions used to blur one of the first image and the second image to a blurriness of the other of the first image and the second image;

b. using the number of convolutions to determine an object-to-focus position distance based on the curve information;

c. computing a device-to-focus position distance; and

d. adding the object-to-focus position distance and the device-to-focus position distance to determine the device-to-object distance.

16. The system of claim 14 wherein the curve information is determined by acquiring target data of multiple images at different blur quantities.

17. The system of claim 14 wherein the curve information includes multiple curves.

18. The system of claim 14 wherein the first image and the second image have a different blur amount.

19. The system of claim 18 wherein the different blur amount is achieved by changing the focus position between acquiring the first image and the second image.

20. The system of claim 14 further comprising a depth map generation module operatively coupled to the distance calculation module, the depth map generation module configured for generating a depth map.

24. The system of claim 14 wherein the device is selected from the group consisting of a personal computer, a laptop computer, a computer workstation, a server, a mainframe computer, a handheld computer, a personal digital assistant, a cellular/mobile telephone, a smart appliance, a gaming console, a digital camera, a digital camcorder, a camera phone, an iPod®

, a video player, a DVD writer/player, a television and a home entertainment system.

21. The system of claim 21 wherein the depth map is stored.

22. The system of claim 22 wherein the depth map is utilized to perform an application.

23. The system of claim 23 wherein the application is selected from the group consisting of auto focus, auto exposure, zoom setting, aperture setting, flash setting, shutter speed, white balance, noise reduction, gamma correction, motion estimation, image/video compression, generating blur, quality improvement, generating a 3-D image, shadow removal and object segmentation.

25. A device comprising:

a. a memory for storing an application, the application configured for;

i. acquiring a first image of a scene;

ii. acquiring a second image of the scene; and

iii. utilizing curve information to determine a distance of one or more objects in the scene; and

b. a processing component coupled to the memory, the processing component configured for processing the application.

26. The device of claim 25 wherein utilizing the curve information includes:

a. determining a number of convolutions used to blur one of the first image and the second image to a blurriness of the other of the first image and the second image;

b. using the number of convolutions to determine an object-to-focus position distance based on the curve information;

c. computing a device-to-focus position distance; and

d. adding the object-to-focus position distance and the device-to-focus position distance to determine the device-to-object distance.

27. The device of claim 25 wherein the curve information is predetermined.

28. The device of claim 25 wherein the curve information includes multiple curves.

29. The device of claim 25 wherein the application is further configured for generating the curve information.

30. The device of claim 29 wherein generating the curve information includes acquiring multiple images at different blur quantities.

31. The device of claim 25 wherein the first image and the second image have a different blur amount.

32. The device of claim 31 wherein the different blur amount is achieved by changing the focus position between acquiring the first image and the second image.

33. The device of claim 25 wherein the application is further configured for generating a depth map.

37. The device of claim 25 wherein the device is selected from the group consisting of a personal computer, a laptop computer, a computer workstation, a server, a mainframe computer, a handheld computer, a personal digital assistant, a cellular/mobile telephone, a smart appliance, a gaming console, a digital camera, a digital camcorder, a camera phone, an iPod®

, a video player, a DVD writer/player, a television and a home entertainment system.

34. The device of claim 34 wherein the depth map is stored.

35. The device of claim 35 wherein the application is further configured for utilizing the depth map to perform an imaging application.

36. The device of claim 36 wherein the imaging application is selected from the group consisting of auto focus, auto exposure, zoom setting, aperture setting, flash setting, shutter speed, white balance, noise reduction, gamma correction, motion estimation, image/video compression, generating blur, quality improvement, generating a 3-D image, shadow removal and object segmentation.

38. A method of generating curve information on a device comprising:

a. acquiring a first image and a second image of a test object in a scene with a changed focus position for the second image;

b. computing a change in blur between the first image and the second image; and

c. repeating a and b for a plurality of different focus positions to generate the curve information.

39. The method of claim 38 further comprising identifying the test object in the scene.

40. The method of claim 38 wherein the image of the test object is acquired for a fixed zoom and aperture.

41. The method of claim 38 wherein generating the curve information occurs while calibrating the device.

42. The method of claim 38 wherein the curve information is stored on the device.

43. The method of claim 38 wherein the device is selected from the group consisting of a personal computer, a laptop computer, a computer workstation, a server, a mainframe computer, a handheld computer, a personal digital assistant, a cellular/mobile telephone, a smart appliance, a gaming console, a digital camera, a digital camcorder, a camera phone, an iPod®

, a video player, a DVD writer/player, a television and a home entertainment system.