Method and apparatus for improved depth-map estimation

• US 9,672,609 B1
• Filed: 11/11/2011
• Issued: 06/06/2017
• Est. Priority Date: 11/11/2011
• Status: Active Grant

• Current Assignee: Golden Edge Holding Corporation
• Original Assignee: Edge Technologies, Inc
• Inventors: Cluster, Jordan, El Dokor, Tarek
• Primary Examiner(s): Vaughn, Jr., William C
• Assistant Examiner(s): BELAI, NAOD W
• Application Number:
  US13/294,481
• Time in Patent Office: 2,034 Days
• Field of Search: 324754, 348 46, 382173, 382154, 382128, 382224, 382225, 382134
• CPC Class Codes: G06F 18/23 Clustering techniques
• CPC Class Codes: G06T 2200/04 involving 3D image data
• CPC Class Codes: G06T 2207/10021 Stereoscopic video; Stereoscopic image sequence
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/20112 Image segmentation details
• CPC Class Codes: G06T 7/00 Image analysis
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/174 involving the use of two or more images
• CPC Class Codes: G06T 7/187 involving region growing; involving region merging; involving connected component labelling
• CPC Class Codes: G06T 7/593 from stereo images
• CPC Class Codes: G06V 20/64 Three-dimensional objects
• CPC Class Codes: H05K 999/99 dummy group

Abstract

A method and apparatus for segmenting an image are provided. An image acquisition and segmentation apparatus may include a sensor, a lens having a variable distance to the sensor, and a processor for acquiring an image and segmenting one or more segments from the acquired image. The lens may be moved relative to the sensor in accordance with a microelectronic machine in order to provide a shallow depth of field and a resulting blurred background, therefore easing a segmentation of one or more foreground objects in an acquired image.

Claims

1. A method for segmentation of an image, comprising the steps of:

acquiring the image employing an image acquisition apparatus with a shallow depth of field, the focus of the image apparatus being on one or more foreground objects in the image;

clustering pixels from the one or more foreground objects in the acquired image into one or more segments;

determining one or more unstable segments changing by more than a predetermined threshold from one or more prior images;

determining depth only for one or more of the one or more unstable segments that have changed by more than the predetermined threshold; and

updating a depth map by updating only the segments having their depth determined.

2. The method of claim 1, wherein one or more background segments are determined in accordance with one or more from the group of a large-scale low pass filtering kernels, and an appropriate scale-space and orientation decomposition.

3. The method of claim 2, wherein the step of determining the one or more background segments further comprises the steps of:

defining a candidate segment to have its disparity evaluated; and

defining the candidate segment in accordance with a space frequency decomposition employing a basis function operator.

4. The method of claim 1, further comprising a step of moving a lens of the image acquisition apparatus relative to a sensor of the image acquisition apparatus in order to adjust a depth of field thereof.

5. The method of claim 4, wherein the lens of the image acquisition is moved by one or more microelectronic machines (MEMs).

6. The method of claim 1, further comprising a step of shallowing the depth of field of the image acquisition apparatus by adjusting an aperture thereof.

7. The method of claim 1, further comprising the step of shallowing the depth of field of the image acquisition apparatus by adjusting a focal length thereof.

8. The method of claim 1, further comprising the steps of:

determining a quality of segmentation of the acquired image; and

shallowing a depth of field of the image acquisition apparatus when it is determined that the quality of segmentation of the acquired image is insufficient.

9. The method of claim 8, wherein the shallowing of the depth of field is achieved by movement of a lens of the image acquisition apparatus relative to a sensor of the image acquisition apparatus.

10. The method of claim 1, wherein the step of updating the depth map is performed in accordance with an embedded GPU.

11. The method of claim 10, wherein the design of the CPU s selected from the group of an ARM-MALI design and an Imagination Technologies design.

12. The method of claim 10, further comprising the step of moving a lens of the image acquisition apparatus by one or more microelectronic machines (MEMs) relative to a sensor of the image acquisition apparatus in order to provide a shallow depth of field.

13. An image acquisition and segmentation apparatus, comprising:

a sensor;

a lens having a variable distance to the sensor; and

a processor for acquiring an image by the sensor and lens employing a shallow depth of field, the focus of the image apparatus being on one or more foreground objects in the image, clustering pixels from the one or more foreground objects in the acquired image into one or more segments, determining one or more unstable segments changing by more than a predetermined threshold from one or more prior ages, determining one or more transitioning segments transitioning from an unstable to a stable segment from the one or more prior images, determining depth only for one or more of the one or more unstable segments that have changed by more than the predetermined threshold, and updating a depth map by updating only the segments having their depth determined.

14. The apparatus of claim 13, further comprising a microelectronic machine for moving the lens relative to the sensor.

15. The apparatus of claim 13, wherein the processor determines a quality of segmentation of the acquired image and adjusts a distance between the lens and the sensor in order to change a depth of field of the image acquisition and segmentation apparatus.

16. The apparatus of claim 15, wherein the distance is adjusted to provide a blurred background.

17. The apparatus of claim 15, wherein the distance is adjusted to place a first segment in the acquired image in an in-focus portion of the acquired image.

18. The apparatus of claim 17, wherein the distance is adjusted to place a second segment in a next acquired image in an in-focus portion of the next acquired image.

19. The apparatus of claim 15, wherein the distance is adjusted to provide a blurred portion of the acquired image.