Method and system for object reconstruction

• US 8,400,494 B2
• Filed: 03/14/2006
• Issued: 03/19/2013
• Est. Priority Date: 10/11/2005
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: PrimeSense Ltd (Apple Inc.)
• Inventors: Shpunt, Alexander, Garcia, Javier, Zalevsky, Zeev, Malzels, Aviad
• Primary Examiner(s): DONAGHUE, LARRY D
• Application Number:
  US11/991,994
• Publication Number:
  US 20100177164A1
• Time in Patent Office: 2,562 Days
• Field of Search: 348/46
• US Class Current: 348/46
• CPC Class Codes: G01B 11/162 by speckle- or shearing interferometry
• CPC Class Codes: G01B 11/2441 using interferometry
• CPC Class Codes: G01B 11/2513 with several lines being projected in more than one direction, e.g. grids, patterns
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 2207/20056 Discrete and fast Fourier transform, [DFT, FFT]
• CPC Class Codes: G06T 7/521 from laser ranging, e.g. using interferometry; from the projection of structured light
• CPC Class Codes: H01L 23/528 Geometry or layout of the interconnection structure H01L27/0207 takes precedence; algorithms G06F30/00
• CPC Class Codes: H01L 23/535 including internal interconnections, e.g. cross-under constructions internal lead connections H01L23/481
• CPC Class Codes: H01L 29/0688 characterised by the particular shape of a junction between semiconductor regions
• CPC Class Codes: H01L 29/42328 with at least one additional gate other than the floating gate and the control gate, e.g. program gate, erase gate or select gate
• CPC Class Codes: H01L 29/788 with floating gate H01L29/78391 takes precedence
• CPC Class Codes: H01L 2924/0002 Not covered by any one of groups H01L24/00, H01L24/00 and H01L2224/00
• CPC Class Codes: H04N 13/20 Image signal generators
• CPC Class Codes: H04N 13/254 in combination with electromagnetic radiation sources for illuminating objects
• CPC Class Codes: H10B 41/10 characterised by the top-view layout
• CPC Class Codes: H10B 41/35 with a cell select transistor, e.g. NAND

Abstract

A system and method are presented for use in the object reconstruction. The system comprises an illuminating unit, and an imaging unit (see FIG. 1). The illuminating unit comprises a coherent light source and a generator of a random speckle pattern accommodated in the optical path of illuminating light propagating from the light source towards an object, thereby projecting onto the object a coherent random speckle pattern. The imaging unit is configured for detecting a light response of an illuminated region and generating image data. The image data is indicative of the object with the projected speckles pattern and thus indicative of a shift of the pattern in the image of the object relative to a reference image of said pattern. This enables real-time reconstruction of a three-dimensional map of the object.

Claims

1. A system for use in the object reconstruction, the system comprising:

(a) an illuminating unit, comprising a coherent light source and a generator of a random speckle pattern accommodated in the optical path of illuminating light propagating from the light source towards an object, thereby projecting onto the object a coherent random speckle pattern; and

(b) an imaging unit for detecting a light response of an illuminated region and generating image data,

the image data being indicative of the object with the projected speckles pattern and thus indicative of a shift of the pattern in the image of the object relative to a reference image of said pattern, the system thereby enabling real-time reconstruction of a three-dimensional map of the object.

2. The system of claim 1, wherein the pattern generator comprises a light diffuser that diffuses light impinging thereon by addition of random phase.

3. The system of claim 2, wherein said light diffuser comprises a piece of ground glass.

4. The system of claim 1, wherein the image data is indicative of the image of object with the speckle pattern projected thereon, acquired with the optical path of illuminating light being substantially normal to the main object plane and at a substantially the same distance from the pattern generator and the imaging unit to said main object plane.

5. The system of claim 1 comprising a control unit configured for storing reference data indicative of a reference image of said coherent random speckle pattern.

6. The system of claim 5, wherein said reference data is indicative of the image of the speckle pattern acquired at a reference plane oriented substantially normally to said optical path of illuminating light propagation and at a substantially the same distance from the pattern generator and the imaging unit.

7. The system of claim 5, wherein the control unit is preprogrammed for processing and analyzing the image data utilizing said data indicative of the reference image for determining a correlation between the images.

8. The system of claim 6, wherein the control unit is preprogrammed for processing and analyzing the image data utilizing said data indicative of the reference image for determining a correlation between the images.

9. The system of claim 1, configured to define a measured volume formed by intersection between the illuminating light propagation and the light collection of the imaging unit, to enable imaging of the object distanced from the pattern generator a distance larger than a Rayleigh distance, each point in said measured volume defining an angle of intersection between the illuminating and collected light propagation paths.

10. The system of claim 1, wherein the pattern generator and the imaging unit are oriented so as to provide substantially equal distances from the pattern generator and the imaging unit to the object.

11. The system of claim 9, wherein the pattern generator and the imaging unit are oriented so as to provide substantially equal distances from the pattern generator and the imaging unit to the object.

12. The system of claim 10, wherein the pattern generator and the imaging unit are located within a fraction of the Rayleigh distance of the illuminating light from each other.

13. The system of claim 8, wherein the control unit is preprogrammed for decomposing the data indicative of the image into Mellin and logarithmic decompositions, thereby obtaining the correlation which is invariant to scaling and projection distortion of the speckle patterns.

14. The system of claim 10, configured such that a minimal value of an average speckle feature size, Δ

x_cam, on a pixel plane of the imaging unit is determined as;

15. The system of claim 1, comprising an adjustor configured for reducing the illumination brightness variation between different regions of the field of view of the imaging unit.

16. The system of claim 15, wherein said adjustor comprises a diffractive optical element.

17. The system of claim 16, wherein said diffractive optical element is configured so as to operate together with the pattern generator to adjust the brightness within a plane transversal to the optical path of illuminating light propagation.

18. The system of claim 16, wherein said diffractive optical element is configured to enable reduction of the brightness variability along an axis substantially parallel to the illuminating light propagation.

19. The system of claim 16, wherein said diffractive optical element is a phase only element accommodated downstream of the pattern generator with respect to the illuminating light propagation, and is configured to adjust the brightness within a plane transversal to the optical path of illuminating light propagation.

20. The system of claim 19, wherein the illuminating unit is configured as a 4-F Fourier Transformer.

21. The system of claim 1, wherein the illuminating unit is configured as a 4-F Fourier Transformer, comprising a diffractive optical element which is configured as a phase only element, a Fourier transform of which produces an elliptic cross section of intensity, and is accommodated between two converging lenses.

22. The system of claim 1, wherein the imaging unit comprises a single light detector defining a stationary field of view.

23. A system for use in the object reconstruction, the system comprising:

(a) an illuminating unit comprising a coherent light source and a light diffuser unit accommodated in the optical path of illuminating light propagating from the light source towards an object thus projecting onto the object a coherent random speckle pattern; and

(b) an imaging unit comprising a single light detector defining a stationary field of view, for detecting a light response of an illuminated region and generating image data,

the image data being indicative of the object with the projected speckles pattern and thus indicative of a shift of the pattern in the image of the object relative to a to reference image of said pattern, the system thereby enabling real-time reconstruction of a three-dimensional map of the object.

24. A system for use in the object reconstruction, the system comprising:

(a) an illuminating unit comprising a coherent light source and a light diffuser is unit accommodated in the optical path of illuminating light propagating from the light source towards an object thus projecting onto the object a coherent random speckle pattern; and

an imaging unit for detecting a light response of an illuminated region and generating image data, the imaging unit and the diffuser being oriented so as to provide substantially equal distances from the diffuser and the imaging unit to the object,

the image data being indicative of the object with the projected species pattern and thus indicative of a shift of the pattern in the image of the object relative to a reference image of said pattern, the system thereby enabling real-time reconstruction of a three-dimensional map of the object.

25. A system for use in the object reconstruction, the system comprising:

(a) an illuminating unit comprising a coherent light source and a light diffuser unit accommodated in the optical path of illuminating light propagating from the light source towards an object thus projecting onto the object a coherent random speckle pattern; and

(b) an imaging unit for detecting a light response of an illuminated region and generating image data, the image data being indicative of the object with the projected speckles pattern and thus indicative of a shift of the pattern in the image of the object relative to a reference image of said pattern,

(c) a control unit configured for storing reference data indicative of a reference image of said coherent random speckle pattern.

26. A projection module for use in a system for object reconstruction, the projection module comprising a coherent light source and a light diffuser configured for that diffusing light impinging thereon by addition of random phase, thereby producing a random speckle pattern for projecting it onto an object thereby enabling imaging of the object with the projected pattern and real time processing image data for reconstructing a three-dimensional map of the object.

27. A method for use in real-time reconstruction of an object, the method comprising:

(i) projecting onto the object a random speckle pattern formed by illuminating coherent light;

(ii) detecting a light response from the object and generating image data of the object with the projected speckles pattern;

(iii) processing the image data to determine a shift of the pattern in the image of the object relative to a reference image of said pattern, thereby determining a three-dimensional map of the object.

28. The method of claim 27, wherein the object is located at a distance from a pattern generator substantially equal to a distance between the object and a light detector.

29. The method of claim 28, wherein the main object plane is oriented substantially normally to an axis of illuminating light propagation.

30. The method of claim 27, wherein said reference data is indicative of the image of the speckle pattern acquired at a reference plane oriented substantially normally to said optical path of illuminating light propagation and at a substantially the same distance from the diffuser and the imaging unit.

31. The method of claim 27, wherein said processing comprises determination of correlation between said image and the reference image.

32. The method of claim 27, wherein intersection between the illuminating light propagation and the light collection defines a certain measured volume enabling imaging of the object within the measured volume distanced from a speckle pattern generator a distance larger than a Rayleigh distance, each point in said measured volume defining an angle of intersection between the illuminating and collected light propagation paths.

33. The method of claim 31, wherein said processing comprises decomposing the data indicative of the image into Mellin and/or logarithmic decompositions, thereby obtaining the correlation which is invariant to scaling and projection distortion of the speckle patterns.

34. The method of claim 27, wherein a minimal value of an average speckle feature size, Δ

x_cam, on a pixel plane of a light detector is determined as;

35. The method of claim 27, comprising reducing the illumination brightness variation between different regions of the field of view of a light detector.

36. The method of claim 35, comprising projecting said pattern through a diffractive optical element.

37. The method of claim 27, wherein said detection of the light response is carried out by a single light detector defining a stationary field of view.

38. The method of claim 27, comprising inputting the data indicative of the three-dimensional map of the object to a Man Machine Interface.

39. The method of claim 38, comprising using the Man Machine Interface for Gaming.

40. The method of claim 38, comprising using the Man Machine Interface for Fitness.

41. The method of claim 27, comprising using the data indicative of the 3D map of the object for bio-medical analysis.

42. The method of claim 41, wherein said bio-medical analysis comprises endoscopy.

43. The method of claim 27, carried out by a mobile phone device.

44. The method of claim 27, comprising utilizing said 3D map of the object for range estimation between vehicles and/or proximity alarm.

45. The method of claim 27, comprising utilizing said 3D map of the object for intrusion alarm.