Hybrid imaging sensor for structured light object capture

• US 10,574,909 B2
• Filed: 08/08/2016
• Issued: 02/25/2020
• Est. Priority Date: 08/08/2016
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Price, Raymond Kirk, Bleyer, Michael, Zhao, Jian, Demandolx, Denis
• Primary Examiner(s): Becker, Joseph W
• Application Number:
  US15/231,179
• Publication Number:
  US 20180041718A1
• Time in Patent Office: 1,296 Days
• CPC Class Codes: G01S 17/46 Indirect determination of position data
• CPC Class Codes: G01S 17/89 for mapping or imaging
• CPC Class Codes: G01S 7/4816 of receivers alone
• CPC Class Codes: G01S 7/4863 Detector arrays, e.g. charge-transfer gates
• CPC Class Codes: G01S 7/4868 Controlling received signal intensity or exposure of sensor
• CPC Class Codes: G01S 7/4914 of detector arrays, e.g. charge-transfer gates
• CPC Class Codes: G01S 7/4918 Controlling received signal intensity, gain or exposure of sensor
• CPC Class Codes: H04N 13/254 in combination with electromagnetic radiation sources for illuminating objects
• CPC Class Codes: H04N 13/257 Colour aspects
• CPC Class Codes: H04N 23/11 for generating image signals from visible and infrared light wavelengths

Abstract

A three dimensional imaging system includes a hybrid imaging sensor with infrared wavelength photoreceptors and visible wavelength photoreceptors integrated in one photoreceptor array. The three dimensional imaging system includes a bandpass filter to filter incoming light before the light is received at the hybrid imaging sensor to reduce crosstalk between the infrared wavelength photoreceptors and visible wavelength photoreceptors. The infrared photoreceptors receive infrared light provided by a structured light source and the visible wavelength photoreceptors receive ambient visible light. The hybrid imaging sensor collects infrared image data and visible light image data concurrently and/or simultaneously.

Claims

1. A system for three-dimensional imaging, the system comprising:

a structured light source, the structured light source configured to emit an emitted light in a structured light pattern having a plurality of infrared dots, the emitted light being in an infrared wavelength range centered around a peak wavelength between about 750 nm and about 1000 nm;

a hybrid imaging sensor having a plurality of domains, at least one domain of the plurality of domains including four photoreceptors,

at least a first visible wavelength photoreceptor configured to detect ambient light in a visible wavelength range,

at least a second visible wavelength photoreceptor configured to detect ambient light in the visible wavelength range, and

at least one infrared wavelength photoreceptor configured to detect infrared light in the infrared wavelength range, the at least one infrared wavelength photoreceptor having an infrared wavelength photoreceptor width and the structured light pattern, a dot diameter of the imaged infrared dots at the hybrid imaging sensor is at least 3 times an infrared wavelength photoreceptor width, and wherein the hybrid imaging sensor is configured to detect a portion of the emitted light as a reflected light; and

a bandpass filter configured to pass light in the visible wavelength range and in the infrared wavelength range and attenuate at least a portion of the light outside of the visible wavelength range and the infrared wavelength range.

2. The system of claim 1, the bandpass filter attenuating light both greater than and less than the infrared wavelength range.

3. The system of claim 1, the at least one visible wavelength photoreceptor comprising a plurality of visible wavelength photoreceptors, a first visible wavelength photoreceptor configured to detect red light, a second visible wavelength photoreceptor configured to detect blue light, and a third visible wavelength photoreceptor configured to detect green light.

4. The system of claim 1, the bandpass filter passing infrared light in a range no greater than 75 nm wide and centered on the peak wavelength of the structured light source.

5. The system of claim 1, the hybrid imaging sensor configured to detect infrared light data and visible light data concurrently.

6. The system of claim 1, the hybrid imaging sensor having a resolution of at least 640×

480.

7. A system for three-dimensional imaging, the system comprising:

a structured light source, the structured light source configured to emit a structured light in a structured light pattern having a plurality of infrared dots, the structured light being in an infrared wavelength range centered around a peak wavelength between about 750 nm and about 1000 nm;

a hybrid imaging sensor having a plurality of domains, at least one domain of the plurality of domains including,

at least one first visible wavelength photoreceptor configured to detect ambient light in a visible wavelength range,

at least a second visible wavelength photoreceptor configured to detect ambient light in the visible wavelength range, and

at least one infrared wavelength photoreceptor configured to detect infrared light in the infrared wavelength range, the at least one infrared wavelength photoreceptor having an infrared wavelength photoreceptor width and the structured light pattern, a dot diameter of the imaged infrared dots at the hybrid imaging sensor is at least 3 times an infrared wavelength photoreceptor width, and wherein the hybrid imaging sensor is configured to detect a portion of the emitted light as a reflected light; and

a bandpass filter configured to pass light in the visible wavelength range to the at least one visible wavelength photoreceptor and in the infrared wavelength range to the at least one infrared wavelength photoreceptor; and

one or more processors in data communication with the hybrid imaging sensor.

8. The system of claim 7, the bandpass filter having a notch filter configured to attenuate light in the infrared wavelength range to the at least one visible wavelength photoreceptor.

9. The system of claim 8, the notch filter attenuates at least 50% of the light in the infrared wavelength range.

10. The system of claim 7, the hybrid imaging sensor configured to capture a frame including image data from the at least one visible wavelength photoreceptor and at least one infrared wavelength photoreceptor simultaneously.

11. The system of claim 7, the infrared wavelength photoreceptor being 25% or less of an active area of the domain.

12. The system of claim 7, the at least one visible wavelength photoreceptor configured to provide visible image data and the at least one infrared wavelength photoreceptor configured to provide infrared image data, and the one or more processors configured to subtract infrared image data from the visible image data.

13. The system of claim 7, the structured light source having a beam divergence that is at least three times an imaging system angular resolution.

14. A method of three dimensional imaging, the method comprising:

providing a structured light in a structured light pattern with a structured light source, the structured light pattern having a plurality of infrared dots

receiving incoming light with an imaging system, the incoming light including visible light and infrared light, at least a portion of the infrared light including reflected structured light, where the imaging system includes at least one infrared wavelength photoreceptor having an infrared wavelength photoreceptor width and the structured light pattern, a dot diameter of the imaged infrared dots at the hybrid imaging sensor is at least 3 times an infrared wavelength photoreceptor width;

filtering the incoming light to attenuate light at least between 650 nm and 750 nm; and

collecting visible image data regarding the visible light with at least one visible wavelength photoreceptor and infrared image data regarding the infrared light with at least one infrared wavelength photoreceptor concurrently.

15. The method of claim 14, further comprising emitting a structured light in an infrared wavelength range with a structured light source.

16. The method of claim 14, further comprising attenuating the infrared light with a notch filter applied to the at least one visible wavelength photoreceptor.

17. The method of claim 14, further comprising subtracting at least a portion of the infrared image data from the visible image data.

18. The method of claim 14, wherein collecting the visible image data and the infrared image data occurs simultaneously.

19. The method of claim 14, further comprising upsampling the infrared image data to a resolution of the visible image data.