System and method for efficiently performing a depth map recovery procedure

• US 20060029270A1
• Filed: 08/03/2004
• Published: 02/09/2006
• Est. Priority Date: 08/03/2004
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.), Sony Electronics Inc. (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.), Sony Electronics Inc. (Sony Group Corp.)
• Inventors: Wong, Earl, Berestov, Alexander
• Granted Patent: US 7,333,652 B2
• Time in Patent Office: Days
• US Class Current: 382/154
• CPC Class Codes: G06T 7/593 from stereo images

Abstract

A system and method for efficiently performing a depth map recovery procedure includes an imaging device that is implemented in a single-lens stereo-shutter configuration for simultaneously capturing overlaid images corresponding to a photographic target. A depth map generator is configured to analyze the overlaid images to determine disparity values corresponding to separation distances between matching points in the overlaid images. The depth map generator then utilizes the disparity values to calculate depth values that correspond to locations in the photographic target. The depth map generator may then utilize the foregoing depth values for creating a depth map corresponding to the photographic target.

Claims

1. A system for performing a depth map recovery procedure, comprising:

an imaging device implemented with a single-lens stereo-shutter configuration for simultaneously capturing overlaid images corresponding to a photographic target; and

a depth map generator configured to analyze said overlaid images to determine disparity values corresponding to separation distances between matching points in said overlaid images.

2. The system of claim 1 wherein said depth map generator utilizes said disparity values to calculate depth values that correspond to said matching points, said depth map generator utilizing said depth values for creating a depth map of said photographic target.

3. The system of claim 1 wherein said depth values are functions of said disparity values, said depth values representing physical distances from said single-lens stereo-shutter configuration to said physical locations in said photographic target.

4. The system of claim 1 wherein said single-lens stereo-shutter configuration and said depth map generator are implemented in an electronic camera device with an auto-focus module that utilizes said depth values from said depth map for performing an auto-focus procedure.

5. The system of claim 4 wherein said electronic camera device captures one or more final images after said auto-focus procedure has been performed by said auto-focus module.

6. The system of claim 1 wherein an electronic device utilizes said depth map for performing at least one of a machine vision procedure or an object tracking procedure.

7. The system of claim 1 wherein said single-lens stereo-shutter configuration includes a stereo shutter that is positionable adjacent to a single lens for passing reflected light from said photographic target only through a first opening and a second opening to an image sensor to thereby create a first overlaid image and a second overlaid image of said overlaid images.

8. The system of claim 7 wherein said stereo shutter is implemented as a non-translucent rectangular object that covers all of said single lens except for an extreme left portion corresponding to said first opening and an extreme right portion corresponding to said second opening.

9. The system of claim 7 wherein said stereo shutter is implemented as a non-translucent object that covers all of said single lens except for a first circular portion corresponding to said first opening and a second circular portion corresponding to said second opening.

10. The system of claim 1 wherein said depth map generator performs a feature matching procedure to identify said matching points in said overlaid images, said feature matching procedure identifying said matching points by locating corresponding features in said overlaid images.

11. The system of claim 1 wherein said depth map generator performs a correlation matching procedure to identify said matching points in said overlaid images, said correlation matching procedure identifying said matching points by calculating correlation relationships in said overlaid images.

12. The system of claim 1 wherein said depth map generator performs an epipolar matching procedure to identify said matching points in said overlaid images, said epipolar matching procedure identifying said matching points by analyzing pixel rows in said overlaid images.

13. The system of claim 1 wherein said depth map generator enters a depth map mode to thereby activate said single-lens stereo-shutter configuration.

14. The system of claim 13 wherein an image sensor device simultaneously captures said overlaid images through said single-lens stereo-shutter configuration.

15. The system of claim 14 wherein each set of said overlaid images includes a first overlaid image of said photographic target and a second overlaid image of said photographic target, said first overlaid image being offset from said second overlaid image distances represented by said disparity values.

16. The system of claim 14 wherein said depth map generator performs a matching procedure to identify multiple pairs of said matching points in said overlaid images, each of said multiple pairs of said matching points corresponding to a different specific location in said photographic target.

17. The system of claim 16 wherein said depth map generator determines said disparity values by calculating said separation distances between said multiple pairs of said matching points from said overlaid images.

18. The system of claim 17 wherein said depth map generator utilizes said disparity values to calculate corresponding ones of said depth values that each represents a physical distance from said single-lens stereo-shutter configuration to said different specific location in said photographic target.

19. The system of claim 18 wherein said depth map generator calculates each of said depth values according to a formula:

Z=(f*b)/(d−

b)

where said Z is said depth value, said d is said disparity value, said b is a baseline value representing a baseline distance between a first opening and a second opening in said single-lens stereo-shutter configuration, and said f is a focal distance from said single-lens stereo-shutter configuration to said image sensor device.

20. The system of claim 18 wherein said depth map generator creates said depth map by populating an image of said photographic target with said depth values corresponding to different physical locations in said photographic target.

21. A method for performing a depth map recovery procedure, comprising:

implementing an imaging device with a single-lens stereo-shutter configuration for simultaneously capturing overlaid images corresponding to a photographic target; and

analyzing said overlaid images with a depth map generator to determine disparity values corresponding to separation distances between matching points in said overlaid images.

22. The method of claim 21 wherein said depth map generator utilizes said disparity values to calculate depth values that correspond to said matching points, said depth map generator utilizing said depth values for creating a depth map of said photographic target.

23. The method of claim 21 wherein said depth values are functions of said disparity values, said depth values representing physical distances from said single-lens stereo-shutter configuration to said physical locations in said photographic target.

24. The method of claim 21 wherein said single-lens stereo-shutter configuration and said depth map generator are implemented in an electronic camera device with an auto-focus module that utilizes said depth values from said depth map for performing an auto-focus procedure.

25. The method of claim 24 wherein said electronic camera device captures one or more final images after said auto-focus procedure has been performed by said auto-focus module.

26. The method of claim 21 wherein an electronic device utilizes said depth map for performing at least one of a machine vision procedure or an object tracking procedure.

27. The method of claim 21 wherein said single-lens stereo-shutter configuration includes a stereo shutter that is positionable adjacent to a single lens for passing reflected light from said photographic target only through a first opening and a second opening to an image sensor to thereby create a first overlaid image and a second overlaid image of said overlaid images.

28. The method of claim 27 wherein said stereo shutter is implemented as a non-translucent rectangular object that covers all of said single lens except for an extreme left portion corresponding to said first opening and an extreme right portion corresponding to said second opening.

29. The method of claim 27 wherein said stereo shutter is implemented as a non-translucent object that covers all of said single lens except for a first circular portion corresponding to said first opening and a second circular portion corresponding to said second opening.

30. The method of claim 21 wherein said depth map generator performs a feature matching procedure to identify said matching points in said overlaid images, said feature matching procedure identifying said matching points by locating corresponding features in said overlaid images.

31. The method of claim 21 wherein said depth map generator performs a correlation matching procedure to identify said matching points in said overlaid images, said correlation matching procedure identifying said matching points by calculating correlation relationships in said overlaid images.

32. The method of claim 21 wherein said depth map generator performs an epipolar matching procedure to identify said matching points in said overlaid images, said epipolar matching procedure identifying said matching points by analyzing pixel rows in said overlaid images.

33. The method of claim 21 wherein said depth map generator enters a depth map mode to thereby activate said single-lens stereo-shutter configuration.

34. The method of claim 33 wherein an image sensor device simultaneously captures said overlaid images through said single-lens stereo-shutter configuration.

35. The method of claim 34 wherein each set of said overlaid images includes a first overlaid image of said photographic target and a second overlaid image of said photographic target, said first overlaid image being offset from said second overlaid image distances represented by said disparity values.

36. The method of claim 34 wherein said depth map generator performs a matching procedure to identify multiple pairs of said matching points in said overlaid images, each of said multiple pairs of said matching points corresponding to a different specific location in said photographic target.

37. The method of claim 36 wherein said depth map generator determines said disparity values by calculating said separation distances between said multiple pairs of said matching points from said overlaid images.

38. The method of claim 37 wherein said depth map generator utilizes said disparity values to calculate corresponding ones of said depth values that each represents a physical distance from said single-lens stereo-shutter configuration to said different specific location in said photographic target.

39. The method of claim 38 wherein said depth map generator calculates each of said depth values according to a formula:

Z=(f*b)/(d−

b)

where said Z is said depth value, said d is said disparity value, said b is a baseline value representing a baseline distance between a first opening and a second opening in said single-lens stereo-shutter configuration, and said f is a focal distance from said single-lens stereo-shutter configuration to said image sensor device.

40. The method of claim 38 wherein said depth map generator creates said depth map by populating an image of said photographic target with said depth values corresponding to different physical locations in said photographic target.

41. A system for performing a depth map recovery procedure, comprising:

means for simultaneously capturing overlaid images corresponding to a photographic target; and

means for analyzing said overlaid images to determine disparity values corresponding to separation distances between matching points in said overlaid images, said means for analyzing then utilizing said disparity values to calculate depth values that correspond to said matching points, said means for analyzing finally creating a depth map of said photographic target by utilizing said depth values.

42. A system for performing a depth map recovery procedure, comprising:

an imaging device implemented to simultaneously capture overlaid images corresponding to a photographic target; and

a depth map generator configured to analyze said overlaid images to determine disparity values corresponding to separation distances between matching points from said overlaid images, said depth map generator then utilizing said disparity values to create a depth map of said photographic target.