Method and apparatus for enhanced stereo vision

• US 8,718,387 B1
• Filed: 12/12/2011
• Issued: 05/06/2014
• Est. Priority Date: 11/11/2011
• Status: Active Grant

• Current Assignee: Golden Edge Holding Corporation
• Original Assignee: Edge Technologies, Inc
• Inventors: El Dokor, Tarek, Cluster, Jordan
• Primary Examiner(s): Patel, Jayesh A
• Assistant Examiner(s): Kholdebarin, Iman K
• Application Number:
  US13/316,606
• Time in Patent Office: 876 Days
• Field of Search: 382/154, 382/128, 382/224, 382/225, 382/173, 348/234, 348/169, 348/222.1, 345/419
• US Class Current: 382/225
• CPC Class Codes: G06F 18/23 Clustering techniques
• CPC Class Codes: G06T 2200/04 involving 3D image data
• CPC Class Codes: G06T 2207/10021 Stereoscopic video; Stereoscopic image sequence
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/20112 Image segmentation details
• CPC Class Codes: G06T 7/00 Image analysis
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/174 involving the use of two or more images
• CPC Class Codes: G06T 7/187 involving region growing; involving region merging; involving connected component labelling
• CPC Class Codes: G06T 7/593 from stereo images
• CPC Class Codes: G06V 20/64 Three-dimensional objects
• CPC Class Codes: H05K 999/99 dummy group

Abstract

A method and apparatus for segmenting an image are provided. The method may include the steps of clustering pixels from one of a plurality of images into one or more segments, determining one or more unstable segments changing by more than a predetermined threshold from a prior of the plurality of images, determining one or more segments transitioning from an unstable to a stable segment, determining depth for one or more of the one or more segments that have changed by more than the predetermined threshold, determining depth for one or more of the one or more transitioning segments, and combining the determined depth for the one or more unstable segments and the one or more transitioning segments with a predetermined depth of all segments changing less than the predetermined threshold from the prior of the plurality of images.

Claims

1. A method for segmenting an image, comprising the steps of:

clustering pixels from one of a plurality of images into one or more segments;

determining one or more unstable segments changing by more than a predetermined threshold from a prior of the plurality of images;

determining one or more transitioning segments transitioning from an unstable to a stable segment;

determining depth only for one or more of the one or more unstable segments that have changed by more than the predetermined threshold, and

for one or more of the one or more transitioning segments; and

combining the determined depth for the one or more unstable segments and the one or more transitioning segments with a predetermined depth of all segments changing less than the predetermined threshold, and thus comprising other than the one or more unstable segments and one or more transitioning segments, from the prior of the plurality of images.

2. The method of claim 1, wherein the segments changing less than the predetermined threshold comprise one or more residual segments.

3. The method of claim 1 wherein the transitioning segments comprise one or more mesostable segments.

4. The method of claim 1, wherein the pixels are clustered from a stereo pair of images.

5. The methods of claim 1, wherein the predetermined depth comprises a depth map.

6. The method of claim 5, wherein the determined depth for the one or more unstable and transitioning segments replaces a prior corresponding depth value in the depth map.

7. The method of claim 1, further comprising the step of generating a depth map associated with the one of the plurality of images in accordance with the determined depth for the one or more unstable segments, the one or more transitioning segments, and the predetermined depth of all segments changing less than the predetermined threshold.

8. The method of claim 7, further comprising the step of repeating the process for generating the depth map of a next sequential frame from the plurality of acquired images, such that only the changes in depth are computed.

9. The method of claim 8, wherein a depth map associated with a next frame is generated by starting with the depth map associated with the one of the plurality of images, and recalculating depth associated with one or more unstable segments or portions of such segments that have changed by more than the predetermined threshold and one or more transitioning segments.

10. The method of claim 1, wherein the predetermined threshold may be learned.

11. The method of claim 1, further comprising the step of combining one or more transitioning segments with a stable segment to produce a different stable segment.

12. The method of claim 11, further comprising the step of computing the depth of the different stable segment if larger than the stable segment by more than a predetermined amount.

13. A non-transitory computer program stored to a non-volatile computer medium, the computer program causing a multi-purpose computer to segment an image by performing the steps of:

clustering pixels from one of a plurality of images into one or more segments;

determining one or more unstable segments changing by more than a predetermined threshold from a prior of the plurality of images;

determining one or more transitioning segments transitioning from an unstable to a stable segment;

determining depth only for one or more of the one or more unstable segments that have changed by more than the predetermined threshold, and for one or more of the one or more transitioning segments; and

combining the determined depth for the one or more unstable segments and the one or more transitioning segments with a predetermined depth of all segments changing less than the predetermined threshold, and thus comprising other than the one or more unstable segments and one or more transitioning segments, from the prior of the plurality of images.

14. The computer program of claim 13, wherein the segments changing less than the predetermined threshold comprise one or more residual segments.

15. The computer program of claim 13, wherein the transitioning segments comprise one or more mesostable segments.

16. The computer program of claim 13, wherein the segments are clustered from a stereo pair of images.

17. The computer program of claim 13, wherein the predetermined depth comprises a depth map.

18. The computer program of claim 17, wherein the determined depth for the one or more unstable and transitioning segments replaces a prior corresponding depth value in the depth map.

19. The computer program of claim 13, further comprising the step of generating a depth map associated with the one of the plurality of images in accordance with the determined depth for the one or more unstable segments, the one or more transitioning segments, and the predetermined depth of all segments changing less than the predetermined threshold.

20. The computer program of claim 19, further comprising the step of repeating the process for generating the depth map of a next sequential frame from the plurality of images.

21. The computer program of claim 20, wherein a depth map associated with a next frame is generated by starting with the depth map associated with the one of the plurality of images, and recalculating depth associated with one or more segments that have changed more than the predetermined threshold and one or more transitioning segments.

22. The computer program of claim 21, wherein the predetermined threshold is learned.

23. The computer program of claim 13, wherein one or more of the steps performed by the multi-purpose computer are performed by a graphical processing unit.

24. The computer program of claim 13, further comprising the step of assigning cluster numbers to the one or more segments according to their classification as unstable, transitioning or stable.

25. The computer program of claim 24, further comprising the step of changing the cluster number of a segment when the segment transitions from one classification to another.

26. A method for computing a depth map of an image, comprising the steps of:

determining one or more unstable pixels changing by more than a predetermined threshold from one of a plurality of images;

determining one or more transitioning pixels transitioning from an unstable to a stable state from the one of the plurality of images;

determining depth only for one or more of the one or more unstable pixels;

and for one or more of the one or more transitioning pixels; and

combining the determined depth for the one or more unstable pixels and the one or more transitioning pixels with a predetermined depth of all pixels changing less than the predetermined threshold, and thus comprising other than the one or more unstable pixels and one or more transitioning pixels to form the depth map.

27. The method of claim 26, wherein the determined depth for the one or more unstable pixels and the one or more transitioning pixels replaces a prior corresponding depth value in the depth map.

28. The method of claim 26, further comprising the step of repeating the process for generating the depth map of a next frame from the plurality of images.

29. The method of claim 26, wherein a depth map associated with the next frame is generated by starting with the depth map associated with the one of the plurality of images, and recalculating depth associated with one or more of the unstable and transitioning pixels.

30. The method of claim 26, wherein the predetermined threshold is learned.