Apparatus and Method for Segmenting an Image

• US 20140219559A1
• Filed: 01/07/2014
• Published: 08/07/2014
• Est. Priority Date: 09/02/2010
• Status: Active Grant

• Current Assignee: Golden Edge Holding Corporation
• Original Assignee: Edge Technologies, Inc
• Inventors: El Dokor, Tarek, King, Joshua, Cluster, Jordan, Holmes, James Edward
• Granted Patent: US 10,586,334 B2
• Time in Patent Office: Days
• US Class Current: 382/173
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 2207/10021 Stereoscopic video; Stereoscopic image sequence
• CPC Class Codes: G06T 2207/20112 Image segmentation details
• CPC Class Codes: G06T 2207/20228 Disparity calculation for image-based rendering
• CPC Class Codes: G06T 7/10 Segmentation; Edge detection motion-based segmentation G06T7/215
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/174 involving the use of two or more images
• CPC Class Codes: G06T 7/187 involving region growing; involving region merging; involving connected component labelling
• CPC Class Codes: G06T 7/194 involving foreground-background segmentation
• CPC Class Codes: G06T 7/50 Depth or shape recovery
• CPC Class Codes: G06T 7/55 from multiple images
• CPC Class Codes: G06T 7/593 from stereo images
• CPC Class Codes: H04N 13/128 Adjusting depth or disparity
• CPC Class Codes: H04N 2013/0081 Depth or disparity estimation from stereoscopic image signals
• CPC Class Codes: H04N 2013/0092 Image segmentation from stereoscopic image signals

Abstract

A method and system for segmenting a plurality of images. The method comprises the steps of segmenting the image through a novel clustering technique that is, generating a composite depth map including temporally stable segments of the image as well as segments in subsequent images that have changed. These changes may be determined by determining one or more differences between the temporally stable depth map and segments included in one or more subsequent frames. Thereafter, the portions of the one or more subsequent frames that include segments including changes from their corresponding segments in the temporally stable depth map are processed and are combined with the segments from the temporally stable depth map to compute their associated disparities in one or more subsequent frames. The images may include a pair of stereo images acquired through a stereo camera system at a substantially similar time.

Claims

1. (canceled)

2. A method for performing segmentation of an image, comprising the steps of:

assigning a unique cluster number to each pixel in an image frame, the cluster numbers being assigned in accordance with a predetermined sequence;

connecting two pixels to form a larger cluster, and assigning a cluster number from one of the two unique cluster numbers to the other pixel, if it is determined that the pixels do not differ by more than a predetermined threshold amount in one or more of one or more characteristics.

3. The method of claim 2, wherein the predetermined sequence comprises an ascending number sequence.

4. The method of claim 3, wherein the assigned cluster number is the lower of the two unique cluster numbers.

5. The method of claim 2, wherein the predetermined sequence comprises a descending number sequence.

6. The method of claim 5, wherein the assigned cluster number is the higher of the two unique cluster numbers.

7. The method of claim 2, wherein the step of connecting two pixels is performed for each pair of pixels in the image located spatially adjacent.

8. The method of claim 2, wherein the step of connecting two pixels is performed for each pair of pixels in the image located within a predetermined distance from each other.

9. The method of claim 2, further comprising the steps of:

connecting a third pixel to the formed larger cluster; and

assigning a cluster number from one of the cluster number of the third pixel and the cluster number of all of the pixels in the larger cluster to each of the third pixel and all of the pixels in the larger cluster, if it is determined that the third pixel and the pixels of the larger cluster do not differ by more than a predetermined threshold amount in one or more of one or more characteristics.

10. The method of claim 2, wherein one or more large clusters comprising at least two pixels similar with respect to the one or more characteristics are formed.

11. The method of claim 10, further comprising the step of defining a key pixel for each large cluster from among the at least two pixels therein, and utilizing that key pixel as the unique cluster number.

12. The method of claim 2, further comprising the step of connecting a plurality of clusters via their cluster numbers.

13. The method of claim 12, wherein the plurality of clusters are further connected in accordance with one or more predetermined thresholds.

14. The method of claim 12, further comprising the step of defining a stopping criterion beyond which the cluster does not extend.

15. The method of claim 2, wherein the one or more characteristics comprise values that may be selected from among the group of a red, green and blue color space;

Hue;

Saturation Value;

a single color channel; and

a predetermined colorspace, or combination thereof.

16. The method of claim 2, wherein the one or more characteristics comprise a depth value calculated from a disparity value.

17. The method of claim 2, further comprising the steps of:

segmenting each of the one or more clusters into sub-regions in accordance with one or more characteristics of the segments and pixels thereof; and

calculating the disparity of each of the sub-regions.

18. The method of claim 17, wherein each of the one or more clusters are segmented into the sub regions by breaking up the clusters across rows and columns of pixels in an image.

19. (canceled)

20. (canceled)

21. (canceled)

22. (canceled)

23. (canceled)

24. (canceled)

25. (canceled)

26. (canceled)

27. (canceled)

28. A computer program stored in a non-transitory storage medium for performing segmentation of an image, the computer program, when implemented on a processor, causing the processor to perform the steps of:

assigning a unique cluster number to each pixel in an image frame, the cluster numbers being assigned in accordance with a predetermined sequence;

connecting two pixels to form a larger cluster, and assigning a cluster number from one of the two unique cluster numbers to the other pixel, if it is determined that the pixels do not differ by more than a predetermined threshold amount in one or more of one or more characteristics.

29. The computer program of claim 28, wherein the predetermined sequence comprises an ascending number sequence.

30. The computer program of claim 29, wherein the assigned cluster number is the lower of the two unique cluster numbers.

31. The computer program of claim 28, wherein the predetermined sequence comprises a descending number sequence.

32. The computer program of claim 31, wherein the assigned cluster number is the higher of the two unique cluster numbers.

33. The computer program of claim 28, wherein the step of connecting two pixels is performed for each pair of pixels in the image located spatially adjacent.

34. The computer program of claim 28, wherein the step of connecting two pixels is performed for each pair of pixels in the image located within a predetermined distance from each other.

35. The computer program of claim 28, further comprising the steps of:

connecting a third pixel to the formed larger cluster; and

assigning a cluster number from one of the cluster number of the third pixel and the cluster number of all of the pixels in the larger cluster to each of the third pixel and all of the pixels in the larger cluster, if it is determined that the third pixel and the pixels of the larger cluster do not differ by more than a predetermined threshold amount in one or more of one or more characteristics.

36. The computer program of claim 28, wherein one or more large clusters comprising at least two pixels similar with respect to the one or more characteristics are formed.

37. The computer program of claim 36, further comprising the step of defining a key pixel for each large cluster from among the at least two pixels therein, and utilizing that key pixel as the unique cluster number.

38. The computer program of claim 28, further comprising the step of connecting a plurality of clusters via their cluster numbers.

39. The computer program of claim 38, wherein the plurality of clusters are further connected in accordance with one or more predetermined thresholds.

40. The computer program of claim 39, further comprising the step of defining a stopping criterion beyond which the cluster does not extend.

41. The computer program of claim 28, wherein the one or more characteristics comprise values that may be selected from among the group of a red, green and blue color space;

Hue;

Saturation Value;

a single color channel; and

a predetermined colorspace, or combination thereof.

42. The computer program of claim 28, wherein the one or more characteristics comprise a depth value calculated from a disparity value.

43. The computer program of claim 28, further comprising the steps of:

segmenting each of the one or more clusters into sub-regions in accordance with one or more characteristics of the segments and pixels thereof; and

calculating the disparity of each of the sub-regions.

44. The computer program of claim 43, wherein each of the one or more clusters is segmented into the sub regions by breaking up the clusters across rows and columns of pixels in an image.

45. (canceled)

46. (canceled)

47. (canceled)

48. (canceled)

49. (canceled)

50. (canceled)

51. (canceled)

52. (canceled)