3D scanning using shadows

• US 7,106,898 B2
• Filed: 12/06/2000
• Issued: 09/12/2006
• Est. Priority Date: 12/06/1999
• Status: Expired due to Fees

• Current Assignee: California Institute of Technology
• Original Assignee: California Institute of Technology
• Inventors: Perona, Pietro, Bouguet, Jean-Yves
• Primary Examiner(s): CHAWAN, SHEELA C
• Application Number:
  US09/732,506
• Publication Number:
  US 20020024593A1
• Time in Patent Office: 2,106 Days
• Field of Search: 382/1, 382/154, 382/174, 345/419, 345/423, 345/427, 345/581, 345/426, 345/425, 356/3.1
• US Class Current: 382/154
• CPC Class Codes: G01B 11/2504 Calibration devices
• CPC Class Codes: G01B 11/2518 Projection by scanning of the object

Abstract

A system and method of determining 3D information about a 3D scene using shadows that are cast on the 3D object.

Claims

1. A method, comprising:

moving a shadow across a three-dimensional scene;

imaging said moving shadow by separately temporally processing the moving shadow and spatially processing the moving shadow, the temporal processing determining temporal information about the moving shadow and the spatial processing determining shadow information associated with information within said temporal information;

determining three dimensional information about the scene from both the shadow information and from the temporal information; and

wherein said determining comprises triangulating to form information indicative of points on the three-dimensional scene.

2. A method as in claim 1, wherein said imaging comprises using a camera to obtain an image of the moving shadow.

3. A method as in claim 2, further comprising determining a transformation between an image plane of the camera and actual plane comprising the three-dimensional scene.

4. A method as in claim 1, further comprising an initial operation of calibrating a position of a light source.

5. A method as in claim 4, wherein said calibrating a position of the light source comprises imaging an item of known height by defining a position of its shadow, and triangulating a position of the light source.

6. A method as in claim 1, further comprising an initial operation of calibrated a position of a plane on which the three-dimensional scene is located.

7. A method as in claim 1, further comprising converting said projection into actual shadow information.

8. A method as in claim 1, wherein said determining comprises obtaining images of different edges at different locations, and using information about the intersection to form three-dimensional information.

9. A method as in claim 1, further comprising thresholding the temporal processing and the spatial processing to make both refer to the same points.

10. A method, comprising:

moving a shadow across a three-dimensional scene;

imaging said moving shadow by separately temporally processing the moving shadow and spatially processing the moving shadow, the temporal processing determining temporal information about the moving shadow and the spatial processing determining shadow information associated with information within said temporal information;

determining three dimensional information about the scene from both the shadow information and from the temporal information; and

wherein said determining comprises converting information into a dual-space representation, and calculating said information in said dual space representation.

11. A method comprising:

obtaining an image of a moving shadow on a three-dimensional scene using an image acquisition element;

determining a profile of different intensity portions of said moving shadow and using said profile to define an edge of said moving shadow; and

converting said image using additional information, to determine actual three dimensional information about the three-dimensional scene.

12. A method as in claim 11, wherein said additional information is a position of a light source.

13. A method as in claim 12, further comprising obtaining an object of known height, obtaining as shadow of said object, and using said shadow to determine the position of the light source.

14. A method as in claim 11, wherein said additional information is a position of a reference plane.

15. A method as in claim 14, wherein said reference plane is a single reference plane.

16. A method as in claim 14, wherein said additional information about said reference plane includes a position of two different reference planes.

17. A method as in claim 14, further comprising a calibration operation that determines a position of the reference plane.

18. A method as in claim 11, wherein said additional information is information about a shadow of unknown object of known height.

19. A method as in claim 11, wherein said additional information is information from a second light source.

20. A method as in claim 11, wherein said additional information is information from a second shadow.

21. A method as in claim 11, wherein said converting comprises converting a projection of the shadow into actual shadow information.

22. A method as in claim 11, wherein said additional information is information which propagates between edges of the image.

23. A method as in claim 11, wherein said shadow is formed by two separate light sources.

24. A method as in claim 11, wherein said converting comprises defining said shadow as a set of edges ε

, and a set of intersection points pk.

25. A method as in claim 11, wherein said determining the profile comprises determining both spatial information and temporal information of the profile, and said determining an edge of the shadow uses both said spatial and temporal information.

26. A method as in claim 11, wherein said converting comprises calculating values in dual space.

27. A method as in claim 11, wherein said converting comprises determining both temporal information about the moving shadow anti shadow information about the moving shadow at times based on information within said temporal information, and determining said three-dimensional information based on both the shadow information and the temporal information by determining shadow information at each of a plurality of times.

28. A method of imaging a three-dimensional surface, comprising:

projecting a moving shadow across the three-dimensional surface to be imaged;

extracting temporal information from said moving shadow and using said temporal information in a temporal coordinate system to determine a plurality of times;

obtaining an image of the moving shadow in a spatial coordinate system at each of the plurality of times;

wherein each image includes a line of the shadow, including a plurality of points p, which represent points P on the three-dimensional surface;

determining a relationship between the image and the three-dimensional surface at each of the plurality of times; and

converting said image into information indicative of the three-dimensional surface.

29. A method as in claim 28, wherein said converting comprises triangulating between a reference plane of an imaging object and a reference plane of the three-dimensional surface.

30. A method as in claim 29, wherein said triangulating includes determining a position of a light source, and determining a reference plane between said light source and a line of the moving shadow.

31. A method as in claim 30, wherein said determining a position of a light source comprises using an object of known height to triangulate a position of a light source, by obtaining a shadow of the object of known height.

32. A method as in claim 29, wherein said converting comprises determining positions of horizontal and vertical reference planes, and triangulating using said positions.

33. A method as in claim 32, wherein said determining positions comprises determining positions of at least one plane by a calibration operation.

34. A method as in claim 28, wherein said obtaining comprises using a camera to obtain said image, and wherein said converting comprises determining information about the camera reference plane, and converting said image using said information about the camera reference plane.

35. A method as in claim 28, further comprising equalizing information from the temporal coordinate system and the spatial coordinates system using thresholds.

36. A method of imaging a three-dimensional surface, comprising:

projecting a moving shadow across the three-dimensional surface to be imaged;

extracting temporal information from said moving shadow and using said temporal information in a temporal coordinate system to determine a plurality of times;

obtaining an image of the moving shadow in a spatial coordinate system at each of the plurality of times;

determining a relationship between the image and the three-dimensional surface at each of the plurality of times;

converting said image into information indicative of the three-dimensional surface; and

wherein said converting comprises converting the information obtained into dual space, and calculating the values obtained in the dual space representation.

37. A method of imaging a three-dimensional surface, comprising:

projecting a moving shadow across the three-dimensional surface to be imaged;

extracting temporal information from said moving shadow and using said temporal information in a temporal coordinate system to determine a plurality of times;

obtaining an image of the moving shadow in a spatial coordinate system at each of the plurality of times;

determining a relationship between the image and the three-dimensional surface at each of the plurality of times;

converting said image into information indicative of the three-dimensional surface; and

wherein said converting comprises determining three-dimensional information about three points in the image, and determining all other points from said determining three points.

38. An apparatus comprising:

a camera, obtaining an image of a scene, and producing a signal indicative thereof; and

a processor, processing said image to determine a moving shadow in the image, and to determine three-dimensional information about the scene represented by the image, by determining temporal information about the moving shadow in a temporal coordinate system and determining shadow information in a spatial coordinate system and equalizing the temporal information and the shadow information to refer to the same points.

39. An apparatus as in claim 38, further comprising a memory, associated with said processor, storing information obtained from camera calibration.

40. An apparatus as in claim 39, wherein said information stored in said memory comprises ground plane information.

41. An apparatus as in claim 39, wherein said memory also stores information indicative of a length of a device used to produce said moving shadow.

42. An apparatus as in claim 39, wherein said memory also stores information about a profile of brightness intensity.

43. An apparatus as in claim 39, wherein said memory also stores information about a threshold of brightness intensity.

44. An apparatus as in claim 43, wherein said processor processes only pixels of the image which have intensity values greater than said specified threshold.

45. An apparatus as in claim 39, wherein said memory stores information about a location of a light source.

46. An apparatus as in claim 39, wherein said processor uses said information in the memory to transform between an image plane of said camera and an actual plane comprising the three-dimensional scene.

47. An apparatus as in claim 38, wherein said processor also operates to determine a profile of the shadow as it moves, and to determine an edge of the shadow by using information from said profile.

48. An apparatus as in claim 47, wherein said processor determines both spatial and temporal information about the profile and determines the edge of the shadow using both said spatial and temporal information.

49. An apparatus comprising:

a camera, obtaining an image of a scene, and producing a signal indicative thereof; and

a processor, processing said image to determine a moving shadow in the image, and to determine three-dimensional information about the scene represented by the image, by determining temporal information about the moving shadow in a temporal coordinate system and determining shadow information in a spatial coordinate system and equalizing the temporal information and the shadow information to refer to the same points; and

wherein said processor carries out an operation to determine information in two orthogonal shadow planes, and determining a position of a light source automatically from said information in said two orthogonal shadow planes.

50. An apparatus comprising:

a camera, obtaining an image of a scene, and producing a signal indicative thereof; and

a processor, processing said image to determine a moving shadow in the image, and to determine three-dimensional information about the scene represented by the image, by determining temporal information about the moving shadow in a temporal coordinate system and determining shadow information in a spatial coordinate system and equalizing the temporal information and the shadow information to refer to the same points;

further comprising a memory, associated with said processor, storing information obtained from camera calibration; and

wherein said memory does not store information about a location of the light source, and wherein said processor carries out an operation to determine information about shadows in two orthogonal shadow planes.

51. A computer-readable storage medium, including instructions in machine readable form, which are executed by a general purpose machine, the set of instructions comprising instructions to:

detect a movement of the shadow in a sequence of two-dimensional images, across the three-dimensional scene; and

use calibration information to determine information about the actual plane of the three-dimensional scene based on the transformation between the image plane of the device acquiring the two-dimensional image, and the three-dimensional scene, wherein said instructions include instructions to determine information in two orthogonal shadow planes, and to determine a position of a light source automatically from said information in said two orthogonal shadow planes.

52. A medium as in claim 51, wherein said instructions include instructions to automatically determine a position of the light source from the information in said image.

53. A medium as in claim 51, wherein said calibration information includes information indicative of a position of a ground plane.

54. A medium as in claim 51, wherein said calibration information includes information indicative of a length of a device being used to produce said moving shadow.

55. A medium as in claim 51, further comprising instructions to determine a threshold of intensity values and accept parts of said image when they are greater than said threshold.

56. A medium as in claim 51, wherein said instructions further comprising instructions to determine a profile of the shadow image as it moves, including at least intensity information about the moving shadow, and using information in the profile to determine an edge of said shadow.

57. A medium as in claim 56, wherein said profile also includes temporal information, and both said intensity information and said temporal information are used to determine said edge of said shadow.

58. A method, comprising:

moving a shadow across a three-dimensional scene;

imaging said moving shadow by determining temporal information about the moving shadow arid determining shadow information associated with times within said temporal information; and

determining three dimensional information about the scene from the shadow information and from the temporal information, wherein said imaging further comprises determining a profile of the shadow image as it moves, that includes at least intensity information about different parts of the moving shadow image, and determining an edge of the shadow image by determining a profile of said shadow and using said profile to determine an edge of said shadow.

59. A method as in claim 58, wherein said determining the profile comprises determining both spatial information and time information of the profile, and said determining an edge of the shadow uses both said spatial and temporal information.

60. A method, comprising:

moving a shadow across a three-dimensional scene;

imaging said moving shadow by determining temporal information about the moving shadow and determining shadow information associated with times within said temporal information; and

determining three dimensional information about the scene from the shadow information and from the temporal information, wherein said determining the profile comprises determining mean values between shadow parts of the image and non-shadow parts of the image, and using said mean values to determine zero crossing points.

61. A method, comprising:

moving a shadow across a three-dimensional scene;

imaging said moving shadow by separately temporally processing the moving shadow and spatially processing the moving shadow, the temporal processing determining temporal information about the moving shadow and the spatial processing determining shadow information associated with information within said temporal information;

determining three dimensional information about the scene from both the shadow information and from the temporal information; and

wherein said determining comprises calculating values in dual space.

62. A method comprising:

obtaining an image of a moving shadow on a three-dimensional scene using an image acquisition element;

determining a profile of different intensity portions of said moving shadow and using said profile to define an edge of said moving shadow; and

converting said image using additional information, to determine actual three dimensional information about the three-dimensional scene, wherein said determining the profile comprises determining both spatial information and temporal information of the profile, and said determining an edge of the shadow uses both said spatial and temporal information,wherein said determining the profile comprises determining mean values between shadow parts of the image and non-shadow parts of the image, and using said mean values to determine zero crossing points.