RANGE CAMERA

• US 20160178512A1
• Filed: 12/18/2014
• Published: 06/23/2016
• Est. Priority Date: 12/18/2014
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Rausch, Kameron, Walker, Ben, Hall, Michael Anthony, Ahmed, Abdelrehim, Cheng, Beibei, Hoof, Jonathan, Kennett, Daniel
• Granted Patent: US 9,958,383 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: A61B 5/0077 Devices for viewing the surface of the body, e.g. camera, magnifying lens
• CPC Class Codes: A61B 5/44 Detecting, measuring or recording for evaluating the integumentary system, e.g. skin, hair or nails A61B5/117 takes precedence
• CPC Class Codes: A61B 5/441 Skin evaluation, e.g. for skin disorder diagnosis
• CPC Class Codes: G01N 21/4738 Diffuse reflection precedence is given to G01N21/55 - G01N21/57 if specular component is taken into consideration, e.g. also for testing fluids, fibrous materials
• CPC Class Codes: G01N 21/4785 Standardising light scatter apparatus; Standards therefor
• CPC Class Codes: G01S 17/46 Indirect determination of position data
• CPC Class Codes: G01S 17/89 for mapping or imaging
• CPC Class Codes: G01S 7/497 Means for monitoring or calibrating
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds

Abstract

An active illumination range camera that acquires a range and a picture image of a scene and provides a reflectance for a feature in the scene responsive to a distance for the feature provided by the range image, a registered irradiance for the feature provided by the picture image and registered irradiance for a region of a calibration surface corresponding to the feature provided by an image of the calibration surface acquired by the range camera.

Claims

1. A range camera operable to determine distances to and reflectance for features of a scene that the range camera images, the range camera comprising:

a light source operable to transmit light to illuminate a scene that the range camera images;

a photosensor having light sensitive pixels;

optics having an optical axis and optical center configured to receive light reflected from the transmitted light by features in the scene and image the reflected light on a plurality of the pixels, each of which plurality of pixels registers reflected light from a feature of the scene that the optics images on the pixel to provide a registered scene irradiance (RSI) for the feature;

a calibration image comprising a registered calibration irradiance (RCI) for each of the light sensitive pixels that provides a measure of irradiance that the pixel registers responsive to light reflected from light transmitted by the light source by a region of a calibration surface having known shape, reflectance, and location that the range camera images on the pixel; and

a controller operable to determine a reflectance for a feature of the scene imaged on a first pixel of the photosensor comprised in the plurality of pixels responsive to the RSI for the feature and an RCI for a region of the calibration surface imaged on a second pixel of the photosensor corresponding to the feature imaged on the first pixel.

2. The range camera according to claim 1 wherein the feature imaged on the first pixel and the corresponding region of the calibration surface imaged on the second pixel are imaged on their respective pixels with light they reflect from light transmitted along a same direction by the light source.

3. The range camera according to claim 1 wherein the determined reflectance for the feature imaged on the first pixel is a function of a ratio RSI/RCI times reflectance ρ

_O of the corresponding region of the calibration surface.

4. The range camera according to claim 1 wherein the range camera determines a distance vector R for the feature that defines a location of the feature in a field of view (FOV) of the camera relative to the optics and a distance vector R′

that defines a location for the feature imaged on the first pixel in the FOV relative to the light source and uses R and R′

to determine a location on the calibration surface of the corresponding region of the calibration surface.

5. The range camera according to claim 4 wherein the range camera determines a distance vector R_O for the corresponding region of the calibration surface in the FOV of the range camera relative to the optics, and a distance vector R_O′

that defines a location of the corresponding region in the FOV relative to a location of the light source responsive to the determined location of the corresponding region of the calibration surface.

6. The range camera according to claim 5 wherein the reflectance for the feature imaged on the first pixel is proportional to ρ

_O(RSI/RCI)[(|R|/|R_O|)(|R|/|R′

_O|)]³ where ρ

_O is reflectance of the corresponding region of the calibration surface.

7. The range camera according to claim 6 wherein the calibration surface is oriented perpendicular to an optical axis of the camera at a fixed distance Z_O from the optical center along the optical axis and the feature imaged on the first pixel is located at a distance Z from the optical center along the optical axis.

8. The range camera according to claim 7 wherein the reflectance for the feature imaged on the first pixel is proportional to ρ

_O(Z_O/Z)⁴(RSI/RCI)[(|R|/|R_O|)(|R′

|/|R′

_O|)]³ where ρ

_O is reflectance of the corresponding region of the calibration surface.

9. The range camera according to claim 1 wherein the controller is operable to determine a tilt factor TF for the feature imaged on the first pixel and use the tilt factor to determine the reflectance for the feature, wherein TF=[Z/(|R′

|({circumflex over (R)}·

{circumflex over (Z)}))][({circumflex over (R)}·

{circumflex over (η

)})/({circumflex over (R)}′

·

{circumflex over (η

)})] and {circumflex over (η

)} is a unit normal to a surface of the feature imaged on the first pixel, “

Z”

is distance of the feature from the optical center along the optical axis, and {circumflex over (R)} and {circumflex over (R)}′

are unit vectors pointing from the feature to the optical center and the light source respectively.

10. The range camera according to claim 9 wherein the reflectance for the feature imaged on the first pixel is proportional to TFρ

_O(Z_O/Z)⁴(RSI/RCI)[(|R|/|R_O|)(|R′

|/|R′

_O|)]³ where ρ

_O is a known reflectance of the region imaged on the second pixel.

11. The range camera according to claim 9 wherein the controller controls the range camera to acquire a range image of the scene and determines {circumflex over (η

)} responsive to a gradient of a surface of the feature defined by the range image.

12. The range camera according to claim 1 wherein the reflectance of the calibration surface is substantially the same for all regions of the calibration surface having an RCI comprised in the calibration image.

13. The range camera according to claim 1 wherein the calibration surface is planar.

14. A method of determining reflectance for a feature of a scene, the method comprising:

using an active lighting range camera to acquire a range image and a picture image of the scene responsive to light transmitted by the range camera to illuminate the scene that is reflected by features in the scene back to the range camera;

determining a location of the feature in the FOV responsive to the range image;

determining registered irradiance of the range camera by light reflected from the transmitted light by the feature responsive to the picture image;

determining a corresponding region of a calibration surface imaged in a calibration image acquired by the range camera;

determining registered irradiance of the range camera by light reflected from the transmitted light by the corresponding region responsive to the calibration image; and

using the determined registered irradiances from the feature and the corresponding calibration region and known reflectance of the calibration surface to determine the reflectance of the feature.

15. The method according to claim 14 wherein determining the corresponding region of the calibration surface comprises determining a region of the calibration surface for which light from the range camera that illuminates the region to acquire the calibration image is transmitted along a same direction that the range camera illuminates the feature to acquire the picture image.

16. The method according to claim 14 wherein determining the reflectance of the feature comprises multiplying the known reflectance by a ratio equal to the registered irradiance from the feature divided by the registered irradiance from the corresponding region.

17. The method according to claim 16 and comprising determining a unit normal {circumflex over (η

)} to a surface of the feature responsive to the range image and using the unit normal to determine the reflectance.

18. The method according to claim 17 wherein using the normal comprises determining a first distance vector from an optical center of the range camera to the feature responsive to the range image and determining a first scalar product of the normal with the first distance vector.

19. The method according to claim 18 wherein using the normal comprises determining a second distance vector from a location from which the range camera transmits light to illuminate the scene to the feature responsive to the range image and determining a second scalar product of the normal with the second distance vector.

20. The method according to claim 19 wherein determining the reflectance comprises determining a ratio between the first scalar product and the second scalar product.

21. A range camera operable to determine distances to and reflectance for features of a scene that the range camera images, the range camera comprising:

a light source operable to transmit light to illuminate a scene that the range camera images;

a photosensor having a plurality of light sensitive pixels;

a look up table (LUT) having for each pixel of the plurality of pixels a set of values for a first feature having known reflectance that is imaged on the pixel from a known first distance from the camera and illuminated by light at known intensity from the light source, the set of values comprising a registered calibration irradiance (RCI) for light reflected by the first feature from the illuminating light; and

a controller that determines a reflectance of a second feature imaged on the pixel from a second distance responsive to square of the ratio of the first distance divided by the second distance, the RCI, and a registered scene irradiance (RSI) that the pixel provides for light from the light source that is reflected by the second feature and imaged on the pixel.

22. The range camera according to claim 21 wherein the LUT comprises at least one coefficient of a power series that provides an expected registered irradiance that the pixel generates for light from the light source reflected by the first feature for a displacement of the first feature from the first distance.

23. The range camera according to claim 22 wherein the controller determines the reflectance responsive to a coefficient of the at least one coefficient.