INTENSITY-MODULATED LIGHT PATTERN FOR ACTIVE STEREO

• US 20140307055A1
• Filed: 06/11/2013
• Published: 10/16/2014
• Est. Priority Date: 04/15/2013
• Status: Abandoned Application

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Kang, Sing Bing, Szeliski, Richard S., Georgiou, Andreas
• Application Number:
  US13/915,626
• Publication Number:
  US 20140307055A1
• Time in Patent Office: Days
• US Class Current: 348/47
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: B29C 64/00 Additive manufacturing, i.e. manufacturing of three-dimensional [3D] objects by additive deposition, additive agglomeration or additive layering, e.g. by 3D printing, stereolithography or selective laser sintering
• CPC Class Codes: B29C 64/386 Data acquisition or data processing for additive manufacturing
• CPC Class Codes: G01B 11/22 for measuring depth
• CPC Class Codes: G01B 11/25 by projecting a pattern, e.g. one or more lines, moiré fringes on the object G01B11/255 takes precedence ; image analysis for depth or shape recovery G06T7/50
• CPC Class Codes: G01B 11/2513 with several lines being projected in more than one direction, e.g. grids, patterns
• CPC Class Codes: G01B 11/2527 with phase change by in-plane movement of the patern
• CPC Class Codes: G01B 11/2545 with one projection direction and several detection directions, e.g. stereo
• CPC Class Codes: G02B 27/4205 having a diffractive optical element [DOE] contributing to image formation, e.g. whereby modulation transfer function MTF or optical aberrations are relevant
• CPC Class Codes: G02B 27/4233 having a diffractive element [DOE] contributing to a non-imaging application diffusers having a diffractive element G02B5/0252; filters having a diffractive element G02B5/203; systems for controlling the direction of light having diffractive elements G02B26/0808; scanning systems having diffractive elements G02B26/106; beam shaping systems using diffractive optical elements G02B27/0944; beam splitting or combining systems operating by diffraction G02B27/1086
• CPC Class Codes: G02B 27/44 Grating systems; Zone plate systems G02B27/46 takes precedence
• CPC Class Codes: G02B 5/1895 such optical elements having dioptric power
• CPC Class Codes: G06F 11/3024 where the computing system component is a central processing unit [CPU]
• CPC Class Codes: G06F 12/00 Accessing, addressing or allocating within memory systems or architectures digital input from, or digital output to record carriers, e.g. to disk storage units, G06F3/06
• CPC Class Codes: G06F 12/02 Addressing or allocation; Relocation program address sequencing G06F9/00; arrangements for selecting an address in a digital store G11C8/00
• CPC Class Codes: G06F 12/0207 with multidimensional access, e.g. row/column, matrix
• CPC Class Codes: G06F 12/0292 using tables or multilevel address translation means G06F12/023 takes precedence; address translation in virtual memory systems G06F12/10
• CPC Class Codes: G06F 2218/12 Classification; Matching
• CPC Class Codes: G06F 3/0653 Monitoring storage devices or systems
• CPC Class Codes: G06F 3/0659 Command handling arrangements, e.g. command buffers, queues, command scheduling
• CPC Class Codes: G06F 3/0683 : Plurality of storage devices
• CPC Class Codes: G06F 9/3004 : to perform operations on memory
• CPC Class Codes: G06F 9/30043 : LOAD or STORE instructions; Clear instruction
• CPC Class Codes: G06F 9/30127 : Register windows
• CPC Class Codes: G06T 1/60 : Memory management
• CPC Class Codes: G06T 2207/30244 : Camera pose
• CPC Class Codes: G06T 7/00 : Image analysis
• CPC Class Codes: G06T 7/586 : from multiple light sources, e.g. photometric stereo
• CPC Class Codes: G06V 20/64 : Three-dimensional objects
• CPC Class Codes: H04N 13/128 : Adjusting depth or disparity
• CPC Class Codes: H04N 13/239 : using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/25 : using two or more image sensors with different characteristics other than in their location or field of view, e.g. having different resolutions or colour pickup characteristics; using image signals from one sensor to control the characteristics of another sensor
• CPC Class Codes: H04N 13/254 : in combination with electromagnetic radiation sources for illuminating objects
• CPC Class Codes: H04N 13/271 : wherein the generated image signals comprise depth maps or disparity maps
• CPC Class Codes: H04N 17/002 : for television cameras
• CPC Class Codes: H04N 2013/0081 : Depth or disparity estimation from stereoscopic image signals
• CPC Class Codes: H04N 23/11 : for generating image signals from visible and infrared light wavelengths
• CPC Class Codes: H04N 23/56 : provided with illuminating means
• CPC Class Codes: H04N 25/131 : including elements passing infrared wavelengths
• CPC Class Codes: H04N 25/611 : Correction of chromatic aberration
• CPC Class Codes: H04N 5/33 : Transforming infrared radiation cameras or camera modules for generating image signals from infrared radiation H04N23/20; circuitry of SSIS for transforming infrared radiation into image signals H04N25/20

Abstract

The subject disclosure is directed towards projecting light in a pattern in which the pattern contains components (e.g., spots) having different intensities. The pattern may be based upon a grid of initial points associated with first intensities and points between the initial points with second intensities, and so on. The pattern may be rotated relative to cameras that capture the pattern, with captured images used active depth sensing based upon stereo matching of dots in stereo images.

Claims

1. A system comprising, a projector that projects a light pattern towards a scene, the projector including a laser and a diffractive optical component, the diffractive optical component configured to output the light pattern as a plurality of sets of sub-patterns, each set corresponding to a different range of intensities.

2. The system of claim 1 wherein the sets comprise differently-sized sub-patterns, including large-sized sub-patterns and at least one set comprising smaller-sized sub-patterns.

3. The system of claim 2 wherein the sets of differently-sized sub-patterns are interleaved via subdivision.

4. The system of claim 1 wherein at least some of the sub-patterns comprise triangles corresponding to points.

5. The system of claim 1 wherein one or more of the sub-patterns comprise points corresponding to positions that form an approximate regular polygon, based upon repositioning points of a substantially regular polygon with additional random repositioning of at least some points thereof.

6. The system of claim 1 wherein at least one of the points that form a sub-pattern is randomly or pseudo-randomly assigned in intensity value within a range of intensities associated with that sub-pattern.

7. The system of claim 1 further comprising a pair of stereo cameras that capture the projected pattern in stereo images, and further comprising an image processing component configured to process stereo images into depth maps.

8. The system of claim 7 wherein the light pattern comprises infrared light for sensing by infrared stereo cameras, or visible light for sensing by color stereo cameras.

9. The system of claim 7 wherein the stereo cameras and the projector are incorporated into a single device.

10. The system of claim 7 wherein the diffractive optical component is configured to project a rotated pattern relative to an orientation of the stereo cameras.

11. A method comprising, generating a grid comprising a first set of points, associating each point in the first set with an intensity value that is within a first intensity range, adding a second set of points between subsets of points of the first set of points, associating each point in the second set with an intensity value that is within a second intensity range, and encoding a diffractive optical component based upon the first set of points and the second set of points.

12. The method of claim 11 further comprising, projecting light through the diffractive optical component to project a pattern comprising a first set of spots each within the first intensity range, and the second set of spots each within the second intensity range.

13. The method of claim 11 wherein projecting the light comprises projecting the pattern rotated relative to orientation of a camera.

14. The method of claim 11 further comprising, adding a third set of points between subsets of points of the second set of points, associating each point in the third set with an intensity value that is within a third intensity range, and wherein encoding the diffractive optical component further comprises encoding the diffractive optical based upon the third set of points.

15. The method of claim 11 wherein the grid is triangular, and wherein generating the gird comprises arranging the first set of points into triangles based upon the grid, and wherein adding the second set of points comprises subdividing at least some of the triangles.

16. The method of claim 11 wherein associating each point in the first set with an intensity value that is within a first intensity range comprises randomly or pseudo-randomly selecting an intensity value within the first range for at least some of the points in the first set.

17. The method of claim 11 wherein associating each point in the second set with an intensity value that is within a second intensity range comprises randomly or pseudo-randomly selecting an intensity value within the second range for at least some of the points in the second set.

18. The method of claim 11 further comprising, randomly or pseudo-randomly positioning at least some of the points in the first set relative to the grid.

19. The method of claim 11 further comprising, randomly or pseudo-randomly randomly positioning at least some of the points in the second set relative to their positions between subsets of points of the first set of points.

20. A method comprising, projecting light through a diffractive optical component to project a pattern comprising a first set of spots corresponding to a the first intensity range, and a second set of spots corresponding to a second intensity range, wherein the positions of the spots of the first set are based upon an initial grid layout and the positions of the spots of the second set of spots are based upon the positions of the spots of the first set, sensing the first set of spots and the second set of spots as left and right stereo camera images, and processing the images to correlate spots in the left image with spots in the right image, in which scanlines of the images are not aligned with the initial grid layout.