RGBZ (red, green, blue, z-depth) filter system usable with sensor systems, including sensor systems with synthetic mirror enhanced three-dimensional imaging

• US 7,375,803 B1
• Filed: 05/18/2007
• Issued: 05/20/2008
• Est. Priority Date: 05/18/2006
• Status: Expired due to Fees

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Canesta Incorporated (Microsoft Corporation)
• Inventors: Bamji, Cyrus
• Primary Examiner(s): Tarcza; Thomas H.
• Assistant Examiner(s): Ratcliffe; Luke D
• Application Number:
  US11/804,653
• Time in Patent Office: 368 Days
• Field of Search: 356/4.01
• US Class Current: 356/4.01
• CPC Class Codes: G01S 17/89 for mapping or imaging
• CPC Class Codes: G01S 7/4816 of receivers alone
• CPC Class Codes: G02B 27/1013 for colour or multispectral image sensors, e.g. splitting an image into monochromatic image components on respective sensors spectral imaging systems G01J
• CPC Class Codes: G02B 27/141 using dichroic mirrors
• CPC Class Codes: G02B 27/145 having sequential partially reflecting surfaces
• CPC Class Codes: G02B 5/201 in the form of arrays
• CPC Class Codes: H04N 2209/047 using multispectral pick-up elements
• CPC Class Codes: H04N 23/11 for generating image signals from visible and infrared light wavelengths
• CPC Class Codes: H04N 25/131 including elements passing infrared wavelengths
• CPC Class Codes: H04N 25/135 based on four or more different wavelength filter elements
• CPC Class Codes: H04N 25/705 Pixels for depth measurement, e.g. RGBZ
• CPC Class Codes: H04N 5/2226 Determination of depth image, e.g. for foreground/background separation determining depth by image analysis in general G06T7/50; segmentation by image analysis in general G06T7/10

Abstract

For use with an imaging system, in one aspect, the present invention implements a filter system that provides a precise narrow band pass for IR-NIR wavelengths, while also providing individual dye-type filters that can be disposed above individual RGB pixels to create dedicated red, green, and blue pixel responses. An interference dichotic filter provides a non-critical first pass band that adequately encompassed RGB wavelengths, and provides a more critical IR-NIR pass band for Z-pixels. The individual dye-type filters have individual R, G, B pass band shapes that fall within the interference filter. Preferably a dye-type NIR-IR filter is also in a common optical path with the other filters to contribute to the overall filtering. In a second aspect, the invention provides an RGB-Z system with dynamically updated image renderings to better enable a viewer of the image to acquire a sense of depth.

Claims

1. For use in a sensor system having a first plurality of detectors responsive to a broad first range of wavelengths dividable into at least a first sub-range and a second sub-range, and having a second plurality of detectors responsive to a narrower second range of longer wavelengths, said first plurality of detectors and said second plurality of detectors sharing a common optical path, a method of filtering incoming optical energy falling upon said sensor system comprising:

(a) inputting said incoming optical energy to an interference filter disposed in said common optical path, said interference filter having a broad first pass band passing at least said first range of wavelengths, and having a more sharply defined second pass band that defines said second narrower range of longer wavelengths;

(b) subjecting optical energy exiting said interference filter as input to at least a first dye filter having a band pass defining said first sub-range, and as input to a second dye filter having a band pass defining said second sub-range, and as input to a third dye filter having a band pass encompassing said sharply defined second pass band;

said first dye filter disposed above at least one detector in said first plurality of detectors responsive to said first sub-range, said second dye filter disposed above at least one detector in said first plurality of detectors responsive to said second sub-range, and said third dye filter disposed above at least one detector in said second plurality of detectors;

wherein detectors in said first sub-range, detectors in said second sub-range, and detectors in said second range of longer wavelengths each receive optical energy having wavelengths defined by an intersection of a band pass of a respective dye filter and a band pass of said interference filter.

2. The method of claim 1, wherein said sensor system is an RGB-Z system, and:

step (b) further includes subjecting optical energy exiting said interference filter as input to a fourth dye filter having a band pass defining a third sub-range, said third sub-range being within said broad first pass band of said interference filter;

said fourth dye filter is disposed above at least one detector in said first plurality of detectors responsive to said fourth sub-range;

wherein detectors in said third sub-range receive optical energy having wavelengths defined by an intersection of a band pass of said fourth dye filter and a band pass of said interference filter.

3. The method of claim 1, wherein said first sub-range and said second sub-range are identical such that said sensor system responds solely to gray scale.

4. The method of claim 1, wherein said broad first pass band of said interference filter passes wavelengths from about 400 nm to about 700 nm.

5. The method of claim 1, wherein said more sharply defined second pass band of said interference filter passes wavelengths from about 800 nm to about 830 nm.

6. The method of claim 1, wherein said first sub-range passes wavelength selected from a range consisting of (i) blue spectra, (ii) green spectra, (iii) red spectra, (iv) 400 nm to about 500 nm, (v) about 475 nm to about 575 nm, and (vi) about 550 nm to about 650 nm.

7. The method of claim 1, wherein said third dye filter passes wavelengths selected from a group consisting of (i) IR wavelengths, (ii) NIR wavelengths, and (iii) wavelengths in a range of about 700 nm to about 900 nm.

8. The method of claim 1, wherein at least one of said first dye filter, said second dye filter, and said third dye filter are fabricated directly upon an integrated circuit array containing at least one of said first plurality of detectors and said second plurality of detectors.

9. For use in a sensor system having a first plurality of detectors responsive to a broad first range of wavelengths dividable into at least a first sub-range and a second sub-range, and having a second plurality of detectors responsive to a narrower second range of longer wavelengths, said first plurality of detectors and said second plurality of detectors sharing a common optical path, a filter system to filter incoming optical energy falling upon said sensor system, said filter system comprising:

an interference filter disposed in said common optical path so as to receive as input incoming optical energy falling upon said filter system, said interference filter having a broad first pass band passing at least said first range of wavelengths, and having a more sharply defined second pass band that defines said second narrower range of longer wavelengths;

at least a first dye filter having a band pass defining said first sub-range;

at least a second dye filter having a band pass defining said second sub-range; and

a third dye filter having a band pass encompassing said sharply defined second pass band;

said first, said second, and said third dye filter disposed so as to receive as input optical energy exiting said interference filter such that said first dye filter is disposed above at least one detector in said first plurality of detectors responsive to said first sub-range, said second dye filter is disposed above at least one detector in said first plurality of detectors responsive to said second sub-range, and said third dye filter disposed above at least one detector in said second plurality of detectors;

wherein detectors in said first sub-range, detectors in said second sub-range, and detectors in said second range of longer wavelengths each receive optical energy having wavelengths defined by an intersection of a band pass of a respective dye filter and a band pass of said interference filter.

10. The filter system of claim 9, further including:

a fourth dye filter, having a band pass defining a third sub-range falling within said broad first pass band of said interference filter, disposed so as to receive as input optical energy exiting said interference filter and disposed above at least one detector in said first plurality of detectors responsive to said third sub-range;

wherein detectors in said third sub-range receive optical energy having wavelengths defined by an intersection of a band pass of said fourth dye filter and a band pass of said interference filter.

11. The filter system of claim 9, wherein said first sub-range and said second sub-range are equal such that said sensor system responds to gray scale.

12. The filter system of claim 9, wherein said broad first pass band of said interference filter passes wavelengths from about 400 nm to about 700 nm.

13. The filter system of claim 9, wherein said more sharply defined second pass band of said interference filter passes wavelengths from about 800 nm to about 830 nm.

14. The filter system of claim 9, wherein said first sub-range passes wavelength selected from a range consisting of (i) blue spectra, (ii) green spectra, (iii) red spectra, (iv) 400 nm to about 500 nm, (v) about 475 nm to about 575 nm, and (vi) about 550 nm to about 650 nm.

15. The filter system of claim 9, wherein said third dye filter passes wavelengths selected from a group consisting of (i) IR wavelengths, (ii) NIR wavelengths, and (iii) wavelengths in a range of about 700 nm to about 900 nm.

16. The filter system of claim 9, wherein at least one of said first dye filter, said second dye filter, and said third dye filter are fabricated directly upon an integrated circuit array containing at least one of said first plurality of detectors and said second plurality of detectors.

17. The filter system of claim 10, wherein said sensor system is an RGB-system.

18. For use with an RGB-Z system that renders an image of a scene for viewing by a user, a method of dynamically displaying a rendering of said screen responsive to movement of the head of a user viewing said image, the method including the following steps:

(a) acquiring RGB-Z of target objects in said scene

(b) acquiring positional information as to location of a head of said user, where said location defines a user vantage point;

(c) dynamically updating renderings of said image using data acquired at step (a) and step (b) such that positional change of said target objects in said display are such that target objects farther from said user vantage point move more slowly than target objects closer to said user vantage point;

wherein said user viewing a display of said scene acquires a measure of depth associated with said target objects;

wherein an updated said display reflects said scene from a new vantage point of said user.

19. The method of claim 18, wherein said RGB-Z system is employed in a motor vehicle, and further including (d) displaying a rendering of a reference image.

20. The method of claim 18, wherein at least two of steps (a), (b) and (c) are carried out under command of a computer.