Error compensation in three-dimensional mapping

• US 9,330,324 B2
• Filed: 07/05/2012
• Issued: 05/03/2016
• Est. Priority Date: 10/11/2005
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Cohen, Daniel, Rais, Dmitri, Sali, Erez, Galezer, Niv, Shpunt, Alexander
• Primary Examiner(s): Spinks, Antoinette
• Assistant Examiner(s): Quiett, Carramah J
• Application Number:
  US13/541,775
• Publication Number:
  US 20120281240A1
• Time in Patent Office: 1,398 Days
• Field of Search: 348/222.1, 348/335, 382/275, 382/285
• US Class Current: 1/1
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 7/521 from laser ranging, e.g. using interferometry; from the projection of structured light
• CPC Class Codes: G06V 10/145 Illumination specially adapted for pattern recognition, e.g. using gratings
• CPC Class Codes: G06V 2201/12 Acquisition of 3D measurements of objects
• CPC Class Codes: H04N 13/254 in combination with electromagnetic radiation sources for illuminating objects
• CPC Class Codes: H04N 13/271 wherein the generated image signals comprise depth maps or disparity maps

Abstract

A method for forming a three-dimensional (3D) map of an object, including illuminating the object from a light source so as to project a pattern onto the object, capturing an image of the pattern using an array of detector elements, and processing the captured image so as to measure respective offsets of elements of the pattern in the captured image relative to a reference pattern, the offsets including at least a first offset of a first element of the pattern and a second offset of a second element of the pattern, measured respectively in first and second, mutually-perpendicular directions in a plane of the array. The method further includes computing a correction factor in response to the first offset, applying the correction factor to the second offset so as to find a corrected offset, and computing depth coordinates of the object in response to the corrected offset.

Claims

1. A method for forming a three-dimensional (3D) map of an object, comprising:

illuminating the object from a light source so as to project a pattern onto the object;

capturing an image of the pattern on the object using an array of detector elements;

processing the captured image so as to measure respective offsets of elements of the pattern in the captured image relative to a reference pattern, the offsets comprising at least a first offset of a first element of the pattern and a second offset of a second element of the pattern, measured respectively in first and second, mutually-perpendicular directions in a plane of the array;

computing a geometrical correction factor in response to the first offset;

applying the geometrical correction factor to the second offset so as to find a corrected offset; and

computing depth coordinates of the object in response to the corrected offset.

2. The method according to claim 1, wherein the at least first offset of the first element comprises a plurality of first offsets of respective first elements.

3. The method according to claim 2, and wherein computing the geometrical correction factor comprises fitting the plurality of first offsets to an equation so as to determine constants of the equation.

4. The method according to claim 3, wherein the corrected offset comprises a corrected second offset, and wherein applying the geometrical correction factor comprises using at least one of the constants to find the corrected second offset.

5. The method according to claim 3, wherein the equation comprises a two-dimensional linear equation.

6. The method according to claim 3, wherein the equation comprises a two-dimensional quadratic equation.

7. The method according to claim 3, and comprising using the equation to predict a subsequent value of the first offset.

8. The method according to claim 1, and comprising correcting at least one of the offsets for distortion in the image.

9. The method according to claim 8, wherein correcting for the distortion comprises capturing a reference image of the pattern, storing parameters of the reference image in a look-up table, and using the look-up table to correct the at least one of the offsets.

10. The method according to claim 1, and comprising correcting the at least first offset for a geometrical misalignment between the light source and the array of detectors.

11. The method according to claim 10, wherein correcting for the geometrical misalignment comprises using a captured reference image to determine the misalignment.

12. The method according to claim 10, wherein correcting the at least first offset for the geometrical misalignment comprises correcting for a change in the geometrical misalignment.

13. The method according to claim 1, wherein the first offsets comprise a deviation in response to a change of wavelength of the light source, and wherein the geometrical correction factor is configured to correct for the change of wavelength.

14. The method according to claim 1, wherein the first offsets comprise a deviation in response to a change of focal length in optics forming the image, and wherein the geometrical correction factor is configured to correct for the change of focal length.

15. The method according to claim 14, and comprising measuring a temperature of the optics, and determining the change in focal length in response to the temperature.

16. The method according to claim 1, wherein a line defined by the light source and the array is parallel to one of the mutually-perpendicular directions.

17. The method according to claim 1, wherein the pattern comprises primary speckles.

18. The method according to claim 1, wherein the pattern comprises a regular geometric pattern.

19. An apparatus for forming a three-dimensional (3D) map of an object, comprising:

a light source configured to illuminate the object so as to project a pattern onto the object;

an array of detector elements configured to capture an image of the pattern on the object; and

a processor which is configured to;

process the captured image so as to measure respective offsets of elements of the pattern in the captured image relative to a reference pattern, the offsets comprising at least a first offset of a first element of the pattern and a second offset of a second element of the pattern, measured respectively in first and second, mutually-perpendicular directions in a plane of the array,

compute a geometrical correction factor in response to the first offset;

apply the geometrical correction factor to the second offset so as to find a corrected offset; and

compute depth coordinates of the object in response to the corrected offset.

20. The apparatus according to claim 19, wherein the at least first offset of the first element comprises a plurality of first offsets of respective first elements.

21. The apparatus according to claim 20, wherein computing the geometrical correction factor comprises fitting the plurality of first offsets to an equation so as to determine constants of the equation.

22. The apparatus according to claim 21, wherein the corrected offset comprises a corrected second offset, and wherein applying the geometrical correction factor comprises using at least one of the constants to find the corrected second offset.

23. The apparatus according to claim 21, wherein the equation comprises a two-dimensional linear equation.

24. The apparatus according to claim 21, wherein the equation comprises a two-dimensional quadratic equation.

25. The apparatus according to claim 21, and comprising using the equation to predict a subsequent value of the first offset.

26. The apparatus according to claim 19, wherein the processor is configured to correct at least one of the offsets for distortion in the image.

27. The apparatus according to claim 26, wherein correcting for the distortion comprises capturing a reference image of the pattern, storing parameters of the reference image in a look-up table, and using the look-up table to correct the at least one of the offsets.

28. The apparatus according to claim 19, and comprising correcting the at least first offset for a geometrical misalignment between the light source and the array of detectors.

29. The apparatus according to claim 28, wherein correcting for the geometrical misalignment comprises using a captured reference image to determine the misalignment.

30. The apparatus according to claim 28, wherein correcting the at least first offset for the geometrical misalignment comprises correcting for a change in the geometrical misalignment.

31. The apparatus according to claim 19, wherein the first offsets comprise a deviation in response to a change of wavelength of the light source, and wherein the geometrical correction factor is configured to correct for the change of wavelength.

32. The apparatus according to claim 19, and comprising optics forming the image, and wherein the first offsets comprise a deviation in response to a change of focal length in the optics, and wherein the geometrical correction factor is configured to correct for the change of focal length.

33. The apparatus according to claim 32, and comprising a temperature sensor configured to measure a temperature of the optics, and wherein the processor is configured to determine the change in focal length in response to the temperature.

34. The apparatus according to claim 19, wherein a line defined by the light source and the array is parallel to one of the mutually-perpendicular directions.

35. The apparatus according to claim 19, wherein the pattern comprises primary speckles.

36. The apparatus according to claim 19, wherein the pattern comprises a regular geometric pattern.