Method and apparatus for accurate alignment of images in digital imaging systems by matching points in the images corresponding to scene elements

• US 7,006,086 B2
• Filed: 12/19/2001
• Issued: 02/28/2006
• Est. Priority Date: 08/28/2000
• Status: Expired due to Term

• Current Assignee: CogniTens Ltd. (Hexagon AB)
• Original Assignee: CogniTens Ltd. (Hexagon AB)
• Inventors: Albeck, Dan, Segev, Avner, Gershon, Ron
• Primary Examiner(s): Shalwala, Bipin
• Assistant Examiner(s): LEWIS, DAVID LEE
• Application Number:
  US10/025,175
• Publication Number:
  US 20020140670A1
• Time in Patent Office: 1,532 Days
• Field of Search: 345/419, 345/619, 345/420, 345/156, 382/154, 382/276, 382/287, 382/291, 356601-603, 348/370, 358/474, 358/488
• US Class Current: 345/419
• CPC Class Codes: G01C 11/00 Photogrammetry or videogrammetry, e.g. stereogrammetry; Photographic surveying
• CPC Class Codes: G06T 7/521 from laser ranging, e.g. using interferometry; from the projection of structured light
• CPC Class Codes: G06T 7/596 from three or more stereo images

Abstract

A digital imaging system is described that facilitates the location of anchors or targets in images of a scene. In one aspect, the digital imaging system makes use of differences as between the properties of the surfaces of the targets and the properties of the surfaces of the objects that are to be mensurated, reconstructed, etc., to facilitate providing uniform illumination of the targets when recording a set of images of the scene, thereby reducing noise that may arise in connection with determining the locations of the targets if they were illuminated by structured illumination, while contemporaneously providing that the objects can be illuminated by structured illumination when the images are recorded. In this aspect, the digital imaging system can use the positions of the targets in the images to relate a local coordinate system associated with the image set to a global coordinate system. In a second aspect, the digital imaging system makes use of one or more of a plurality of algorithms to determine the locations of targets in the images of the scene in the respective objects. In this aspect, the digital imaging system records two sets of images, including a baseline set and a working set. The baseline set is recorded using uniform illumination, with the baseline set comprising only images of the targets. The working set is recorded using structured illumination, with the working set comprising image of both the targets and the objects. The working set is used in connection with mensuration, virtual reconstruction, etc., and one or more of the algorithms are used to determine the likely positions of the targets in the images in the working image set, and to determine transformations between the baseline and working image set such that the local coordinate system associated with the working image set can be related to the global coordinate system.

Claims

1. A digital imaging system comprising:

A. a scene illumination arrangement configured to illuminate said scene comprising at least one object and at least one target independent of said at least one object, said at least one object and at least one target capable of being illuminated, the scene illumination arrangement being configured to illuminate at least the at least one object and at least one target to facilitate disambiguation between the at least one object and the at least one target;

B. an image recording device configured to record at least one image of a scene when said scene is illuminated by said illumination arrangement,

C. an image processing subsystem configured to process the at least one image to differentiate a part of said image corresponding to said at least one target from another part of said image corresponding to said at least one object in said at least one image and to identify a location of the at least one target appearing in the at least one image with respect to a local coordinate system associated with the location from which the image recording device recorded the at least one image of the scene, and for relating said local coordinate system to a global coordinate system, wherein the location of the said at least one target with respect to said global coordinate system is known independently of said at least one image.

2. A digital imaging system as defined in claim 1 in which

A. the at least one target has a surface configured to reflect light of a selected wavelength;

B. the scene illumination arrangement is configured to illuminate the scene with illumination of the selected wavelength; and

C. the image processing subsystem is configured to identify the location of the image of the at least one target as the location in the at least one image of a region representing the image of light of the wavelength reflected by the surface of the at least one target.

3. A digital imaging system as defined in claim 2 in which the scene illumination arrangement is configured to illuminate the scene with uniform light of the selected wavelength.

4. A digital imaging system as defined in claim 3 in which the selected wavelength comprises a band of wavelengths.

5. A digital imaging system as defined in claim 2 in which the scene illumination arrangement is further configured to illuminate the scene with structured illumination at a further wavelength that is not reflected by the at least one target.

6. A digital imaging system as defined in claim 5 in which the further wavelength comprises a band of wavelengths, the band of wavelengths and the selected wavelength being disjoint.

7. A digital imaging system as defined in claim 1 in which

A. the at least one target has a surface configured to reflect light of a selected wavelength;

B. the scene illumination arrangement is configured to illuminate the scene with uniform illumination of the selected wavelength, and structured illumination of at least one other wavelength; and

C. the image recording device is configured to record two images,

i. one of said images comprising an image recorded while the scene illumination arrangement illuminates the scene with light of the selected wavelength; and

ii. the other of said images comprising an image recorded while the scene illumination arrangement illuminates the scene with light of the at least one other wavelength.

8. A digital imaging system as defined in claim 7 in which the image recording device includes a beam splitter arrangement configured to divide light reflected thereto from the scene into two portions, and two filters, each to allow light of one of the respective wavelengths to pass to an image recording medium.

9. A digital imaging system as defined in claim 1 in which

A. the scene illumination arrangement is configured to illuminate the scene with illumination of having respective selected polarization directions, illumination of one polarization direction being uniform illumination and illumination of the other polarization direction being structured illumination;

B. the at least one target having a surface configured to reflect illumination incident thereon in a manner to preserve the polarization direction and the at least one object having a surface configured to reflect illumination incident thereon in a manner to not preserve the polarization direction;

C. the image recording device is configured to record an image of the scene such that the image of the at least one target reflects uniform illumination incident on the surface thereof; and

D. the image processing subsystem is configured to identify the location of the image of the at least one target in relation to the uniformity of the image thereof as recorded by the image recording device.

10. A digital imaging system as defined in claim 9 in which

A. the scene illumination arrangement is configured to illuminate the scene such that the uniform illumination is in a first selected direction and the structured illumination is in a second selected direction perpendicular to the first selected direction;

and

B. the image recording device is provided with a polarization arrangement configured to pass illumination in the first selected direction to an image recording medium, thereby ensuring that structured illumination reflected from the surface of the at least one target is not depicted in the image recorded by the image recording medium.

11. A digital imaging system as defined in claim 1 in which

A. the at least one target is located in a position in the scene displaced from the position of the at least one object;

B. the scene illumination arrangement is configured to provide uniform illumination in at least one region of the scene in which the at least one target is located, and structured illumination in at least one portion of the scene in which the at least one object is located; and

C. the image processing subsystem is configured to identify the location of the image of the at least one target in relation to the uniformity of the image thereof as recorded by the image recording device.

12. A digital imaging system as defined in claim 11 in which

A. the scene illumination arrangement is configured to provide uniform illumination and structured illumination;

and

B. the image processing subsystem is configured to

(i) enable the scene illumination arrangement to illuminate the scene by uniform illumination, and enable the image recording device to record a first image of the scene, and use the first image to determine the location of the at least one target in the scene; and

(ii) thereafter enable the scene illumination arrangement to illuminate the portion of the scene at which the at least one target is located by uniform illumination and at least one other portion of the scene with structured illumination, and enable the image recording device to record a second image of the scene, and use the second image to determine the location of the at least one target in the scene, thereby to facilitate relating the local coordinate system associated with the location from which the image recording device recorded the second image of the scene to the global coordinate system.

13. A digital imaging system as defined in claim 12 in which the scene illumination arrangement includes a pattern generator configured to selectively generate uniform illumination for at least one portion of the scene and structured illumination for at least a second portion of the scene.

14. A digital imaging system as defined in claim 1 in which

A. the image recording device is configured to record successive images of the scene at points in time separated by a relatively short time interval;

and

B. the scene illumination arrangement is configured to illuminate at least the at least one target with uniform illumination when the image recording device records one of said successive images, and the at least one object with structured illumination when the image recording device records the other of said successive images.

15. A digital imaging system as defined in claim 1 in which

A. the scene illumination arrangement is configured to provide uniform illumination and structured illumination;

and

B. the image processing subsystem is configured to

(i) enable the scene illumination arrangement to illuminate the scene by uniform illumination, and enable the image recording device to record a baseline image of the scene, and determine the location of the at least one target in the baseline image;

(ii) enable the scene illumination arrangement to illuminate the scene by structured illumination, and enable the image recording device to record a working image of the scene, and

(iii) using the location of the at least one target in the baseline image, determine the location of the at least one target in the working image.

16. A digital imaging system as defined in claim 15 in which the image processing subsystem makes use of a selected search methodology, in which the image processing subsystem searches at least a region of the working image proximate the location of the at least one target in the baseline image, in determining the location of the at least one target in the working image.

17. A digital imaging system as defined in claim 16 in which, in the selected search methodology, the image processing subsystem searches at least one region of the working image proximate the location of an edge of the at least one target in the baseline image, in determining the location of the at least one target in the working image.

18. A digital imaging system as defined in claim 16 in which the at least one target has a selected shape, the search methodology reflecting the selected shape.

19. A digital imaging system as defined in claim 16 in which, in the selected search methodology, the image processing subsystem detects at least one edge of the at least one target in the baseline image and in the working image and performs a distance transform operation between them in order to determine the location of at least one target in the working image.

20. A digital imaging system as defined in claim 16 in which, in the selected search methodology, the image processing subsystem detects edges of the at least one target in the baseline and in the working image sets, determines a shape of a contour of the target image in the baseline image set and best fits the shape of the contour with the edge of the target image in the working image set in order to determine the location of at least one target in the working image.

21. A digital imaging system as defined in claim 16 in which the selected search methodology is a least squares fit methodology.

22. A digital imaging system as defined in claim 16 in which the image processing subsystem is further configured to generate a mask representing the image of the target in the baseline image, the image processing subsystem being further configured to use the mask to define the region in the working image in which it performs the selected search methodology.

23. A digital imaging system as defined in claim 22 in which the m ask includes a mask element that is a selected percentage of the size of the target in the baseline image, the mask element defining the region in the working image in which the image processing subsystem performs the selected search methodology.

24. A digital imaging system as defined in claim 15 in which the image processing subsystem is further configured to determine a transformation between the baseline image and the working image, thereby to facilitate relating the local coordinate system associated with the location from which the image recording device recorded the working image of the scene to the global coordinate system.

25. A digital imaging method comprising the steps of:

A. recording at least one image of a scene, said scene comprising at least one object and at least one target independent of said at least one object, said at least one object and at least one target capable of being illuminated, at least said at least one object and at least one target being illuminated in a manner to facilitate disambiguation between the at least one object and the at least one target;

B. processing the at least one image to differentiate a part of the image corresponding to said at least one target from another part of the image corresponding to said at least one object in said at least one image and to identify a location of the at least one target appearing in the at least one image with respect to a local coordinate system associated with a location from which the at least one image of the scene is recorded in step (A); and

C. relating said local coordinate system to a global coordinate system, wherein the location of the said at least one target with respect to said global coordinate system is known independently of said at least one image.

26. A digital imaging method as defined in claim 25 in which

A. the at least one target has a surface configured to reflect light of a selected wavelength;

B. the scene illumination step includes the step of illuminating the scene with illumination of the selected wavelength; and

C. the image processing step includes the step of identifying the location of the image of the at least one target as the location in the at least one image of a region representing the image of light of the wavelength reflected by the surface of the at least one target.

27. A digital imaging method as defined in claim 26 in which the scene illumination step includes the step of illuminating the scene with uniform light of the selected wavelength.

28. A digital imaging method as defined in claim 27 in which the selected wavelength comprises a band of wavelengths.

29. A digital imaging method as defined in claim 26 in which the scene illumination step includes the step of illuminating the scene with structured illumination at a further wavelength that is not reflected by the at least one target.

30. A digital imaging method as defined in claim 29 in which the further wavelength comprises a band of wavelengths, the band of wavelengths and the selected wavelength being disjoint.

31. A digital imaging method as defined in claim 25 in which each of the at least one object and at least one target has a surface configured to reflect light of respective selected wavelengths,

A. the scene illumination step including the step of illuminating the scene with illumination of the respective selected wavelengths;

and

B. the image processing step includes the step of identifying the location of the image of the at least one target as the location in the at least one image of a region representing the image of light of the wavelength reflected by the surface of the at least one target.

32. A digital imaging method as defined in claim 25 in which each of the at least one object and at least one target has a surface configured to reflect light of respective selected wavelengths;

A. the scene illumination step including the step of illuminating the scene with illumination of the respective selected wavelengths; and

C. the image recording step includes the step of is configured to recording two images,

i. one of said images comprising an image of light at the wavelength for which the surface of the target is configured to reflect and

ii. the other of said images comprising an image at the wavelength for which the surface of the object is configured to reflect.

33. A digital imaging method as defined in claim 32 in which the image recording step includes the steps of

A. dividing light reflected from the scene into two portions, and

B. filtering each portion in such a manner as to allow light of one of the respective wavelengths to pass to an image recording medium.

34. A digital imaging method as defined in claim 25 in which the at least one target has a surface configured to reflect illumination incident thereon in a manner to preserve the polarization direction and the at least one object having a surface configured to reflect illumination incident thereon in a manner to not preserve the polarization direction,

A. the scene illumination step including the step of illuminating the scene with illumination of having respective selected polarization directions, illumination of one polarization direction being uniform illumination and illumination of the other polarization direction being structured illumination;

B. the image recording step includes the step of recording an image of the scene such that the image of the at least one target reflects uniform illumination incident on the surface thereof; and

C. the image processing step includes the step of identifying the location of the image of the at least one target in relation to the uniformity of the image thereof as recorded by the image recording device.

35. A digital imaging method as defined in claim 34 in which

A. the scene illumination step includes the step of illuminating the scene such that the uniform illumination is in a first selected direction and the structured illumination is in a second selected direction perpendicular to the first selected direction;

and

B. the image recording step includes a polarization step in which illumination is passed having a first selected direction to an image recording medium, thereby ensuring that structured illumination reflected from the surface of the at least one target is not depicted in the image recorded by the image recording medium.

36. A digital imaging method as defined in claim 34 in which the at least one target is located in a position in the scene displaced from the position of the at least one object;

A. the scene illumination step includes the step of providing uniform illumination in at least one region of the scene in which the at least one target is located, and structured illumination in at least one portion of the scene in which the at least one object is located; and

B. the image processing step includes the step of identifying the location of the image of the at least one target in relation to the uniformity of the image thereof as recorded by the image recording device.

37. A digital imaging method as defined in claim 36 in which

A. initially

(i) in connection with the scene illumination step, illuminating the scene by uniform illumination, and

(ii) in connection with the image recording step, recording a first image of the scene, and

(iii) in connection with the image processing step, using the first image to determine the location of the at least one target in the scene;

and

B. thereafter

(i) in connection with the scene illumination step, illuminating the portion of the scene at which the at least one target is located by uniform illumination and at least one other portion of the scene with structured illumination, and

(ii) in connection with the image recording step, recording a second image of the scene,

(iii) in connection with the image processing step, using the second image to determine the location of the at least one target in the scene, thereby to facilitate relating the local coordinate system associated with the location from which the image recording device recorded the second image of the scene to the global coordinate system.

38. A digital imaging method as defined in claim 37 in which the scene illumination step includes the step of using a pattern generator configured to selectively generate uniform illumination for at least one portion of the scene and structured illumination for at least a second portion of the scene.

39. A digital imaging method as defined in claim 25 in which

A. the image recording step includes the step of recording successive images of the scene at points in time separated by a relatively short time interval;

and

B. the scene illumination step includes the step of illuminating at least the at least one target with uniform illumination when one of said successive images is recorded, and the at least one object with structured illumination when the other of said successive images is recorded.

40. A digital imaging method as defined in claim 25 in which

A. the scene illumination step includes the steps of providing uniform illumination and structured illumination;

and

B. in connection with the scene illumination step, illuminating the scene by uniform illumination, and

(i) in connection with the image recording step, recording a baseline image of the scene,

(ii) in connection with the image processing step, using the baseline image to determine the location of the at least one target in the scene; and

C. in connection with the scene illumination step, illuminating the scene with structured illumination;

(i) in connection with the image recording step, recording a working image of the scene, and

(ii) in connection with the image processing step, using the location of the at least one target in the baseline image in connection with determining the location of the at least one target in the working image.

41. A digital imaging method as defined in claim 40 in which the image processing step includes the step of using a selected search methodology, in which the at least a region of the working image is searched proximate the location of the at least one target in the baseline image, in determining the location of the at least one target in the working image.

42. A digital imaging method as defined in claim 41 in which, in the selected search methodology, a search is performed in at least one region of the working image proximate the location of an edge of the at least one target in the baseline image, in determining the location of the at least one target in the working image.

43. A digital imaging method as defined in claim 41 in which the at least one target has a selected shape, the search methodology reflecting the selected shape.

44. A digital imaging method as defined in claim 41 in which, in the selected search methodology, the image processing step includes the step of detecting at least one edge of the at least one target in the baseline image and in the working image and performing a distance transform operation between them in order to determine the location of at least one target in the working image.

45. A digital imaging method as defined in claim 41 in which, in the selected search methodology, the image processing method includes the step of detecting at least one edge of the at least one target in the baseline image and in the working image, determining a shape of a contour of the target image in the baseline image set and best fits the shape of the contour with the edge of the target image in the working image set in order to determine the location of at least one target in the working image.

46. A digital imaging method as defined in claim 41 in which the selected search methodology is a least squares fit methodology.

47. A digital imaging method as defined in claim 40 in which the image processing step includes the steps of generating a mask representing the image of the target in the baseline image, and using the mask to define the region in the working image in which it performs the selected search methodology.

48. A digital imaging method as defined in claim 47 in which the mask includes a mask element that is a selected percentage of the size of the target in the baseline image, the mask element defining the region in the working image in which the selected search methodology is performed.

49. A digital imaging method as defined in claim 40 further comprising the step of determining a transformation between the baseline image and the working image, thereby to facilitate relating the local coordinate system associated with the location from which the image recording device recorded the working image of the scene to the global coordinate system.