DEPTH IMAGE GENERATION UTILIZING DEPTH INFORMATION RECONSTRUCTED FROM AN AMPLITUDE IMAGE

• US 20160247286A1
• Filed: 08/11/2014
• Published: 08/25/2016
• Est. Priority Date: 02/07/2014
• Status: Abandoned Application

• Current Assignee: Avago Technologies General IP PTE Limited (Broadcom, Inc.)
• Original Assignee: LSI Corporation (Broadcom, Inc.)
• Inventors: Mazurenko, Ivan L., Radovanovic, Nikola, Parkhomenko, Denis V., Kholodenko, Alexander B., Parfenov, Denis V.
• Application Number:
  US14/378,119
• Publication Number:
  US 20160247286A1
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10048 Infrared image
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/521 from laser ranging, e.g. using interferometry; from the projection of structured light
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16

Abstract

An image processing system comprises an image processor having image processing circuitry and an associated memory. The image processor is configured to identify a region of interest in an amplitude image, to detect one or more relatively low gradient regions in the region of interest, to reconstruct depth information for said one or more relatively low gradient regions, to extend the reconstructed depth information beyond said one or more relatively low gradient regions to additional pixels of the region of interest, and to generate a depth image utilizing at least portions of the reconstructed depth information and the extended reconstructed depth information. The image processor in some embodiments is adapted for coupling to an active lighting image sensor, such as an infrared sensor that does not provide depth information corresponding to the amplitude image, or an SL or ToF sensor that provides depth information corresponding to the amplitude image.

Claims

1. A method comprising steps of:

identifying a region of interest in an amplitude image;

detecting one or more relatively low gradient regions in the region of interest;

reconstructing depth information for said one or more relatively low gradient regions in the region of interest;

extending the reconstructed depth information beyond said one or more relatively low gradient regions to additional pixels of the region of interest; and

generating a depth image utilizing at least portions of the reconstructed depth information and the extended reconstructed depth information;

wherein the steps are implemented in an image processor comprising a processor coupled to a memory.

2. The method of claim 1 wherein said one or more relatively low gradient regions comprise regions having respective gradients at or below a specified gradient threshold.

3. The method of claim 1 wherein said one or more relatively low gradient regions comprise regions having respective substantially zero gradients.

4. The method of claim 1 wherein identifying a region of interest comprises generating a binary region of interest mask in which pixels within the region of interest all have a first binary value and pixels outside the region of interest all have a second binary value complementary to the first binary value.

5. The method of claim 1 wherein reconstructing depth information for said one or more relatively low gradient regions comprises reconstructing depth information for said one or more relatively low gradient regions but not for other portions of the region of interest.

6. The method of claim 1 wherein the amplitude image comprises a luminance image generated by an active lighting imager.

7. The method of claim 6 wherein the active lighting imager comprises one of an active lighting infrared image sensor, an SL image sensor and a ToF image sensor.

8. The method of claim 1 wherein reconstructing depth information for said one or more relatively low gradient regions in the region of interest comprises:

determining at least one coefficient that relates amplitude to depth for a particular image sensor configuration; and

utilizing the coefficient to estimate one or more depth values from one or more corresponding amplitude values for respective pixels of said one or more relatively low gradient regions.

9. The method of claim 8 wherein the determined coefficient comprises an element of a coefficient function that relates amplitude to depth for a given pixel as a function of a distance between that pixel and an approximate center of an image frame in the particular image sensor configuration.

10. The method of claim 1 wherein extending the reconstructed depth information beyond said one or more relatively low gradient regions to additional pixels of the region of interest comprises computing one or more depth values for pixels outside of said one or more relatively low gradient regions as a function of depth values determined for pixels within said one or more relatively low gradient regions.

11. The method of claim 10 wherein a depth value for a given pixel outside of said one or more relatively low gradient regions is computed utilizing a mean of a plurality of depth values for respective pixels within said one or more relatively low gradient regions.

12. The method of claim 1 wherein identifying a region of interest comprises identifying the region of interest utilizing the amplitude image and a corresponding coarse depth image.

13. The method of claim 1 wherein generating a depth image utilizing at least portions of the reconstructed depth information and the extended reconstructed depth information further comprises:

generating a reconstructed depth image comprising at least portions of the reconstructed depth information and the extended reconstructed depth information; and

combining a coarse depth image with the reconstructed depth image.

14. An article of manufacture comprising a computer-readable storage medium having computer program code embodied therein, wherein the computer program code when executed in the image processor causes the image processor to perform the method of claim 1.

15. An apparatus comprising:

an image processor adapted for coupling to an image sensor;

the image processor comprising image processing circuitry and an associated memory;

wherein the image processor is configured;

to identify a region of interest in an amplitude image;

to detect one or more relatively low gradient regions in the region of interest;

to reconstruct depth information for said one or more relatively low gradient regions in the region of interest;

to extend the reconstructed depth information beyond said one or more relatively low gradient regions to additional pixels of the region of interest; and

to generate a depth image utilizing at least portions of the reconstructed depth information and the extended reconstructed depth information.

16. An integrated circuit comprising the apparatus of claim 15.

17. An imager comprising:

an image sensor; and

an image processor coupled to the image sensor;

wherein the image processor is configured;

to identify a region of interest in an amplitude image;

to detect one or more relatively low gradient regions in the region of interest;

to reconstruct depth information for said one or more relatively low gradient regions in the region of interest;

to extend the reconstructed depth information beyond said one or more relatively low gradient regions to additional pixels of the region of interest; and

to generate a depth image utilizing at least portions of the reconstructed depth information and the extended reconstructed depth information.

18. The imager of claim 17 wherein the image sensor comprises an active lighting infrared image sensor that does not provide depth information corresponding to the amplitude image.

19. The imager of claim 17 wherein the image sensor comprises an SL or ToF image sensor that provides depth information corresponding to the amplitude image.

20. An image processing system comprising the imager of claim 17.