Apparatus and method for obtaining three-dimensional positional data from a two-dimensional captured image

• US 7,075,661 B2
• Filed: 02/22/2002
• Issued: 07/11/2006
• Est. Priority Date: 02/23/2001
• Status: Expired due to Fees

• Current Assignee: Industrials Control Systems Limited
• Original Assignee: Industrial Control Systems Limited
• Inventors: Petty, Phillip Stephen, Petty, John Michael
• Primary Examiner(s): Toatley, Jr., Gregory J.
• Assistant Examiner(s): Nguyen, Sang H.
• Application Number:
  US10/468,953
• Publication Number:
  US 20040128102A1
• Time in Patent Office: 1,600 Days
• Field of Search: 356600-625, 250/559.22, 250/559.05, 382/111, 382/321, 382/779, 396/106, 396/431, 396/50, 438/774, 438/761, 438/766, 438/239, 438/207.09
• US Class Current: 356/603
• CPC Class Codes: G01S 17/46 Indirect determination of position data
• CPC Class Codes: G01S 17/875 for determining attitude
• CPC Class Codes: G01S 3/784 using a mosaic of detectors
• CPC Class Codes: G01S 5/163 Determination of attitude using inertial means G01C9/00; control of attitude G05D1/49

Abstract

Methods and apparatus for introducing a plurality of optical markers to a field of view, capturing a two-dimensional image of said field of view on an image plate comprising a pixel array, determining a set of marker origin offset values and using said offset values to establish a set of orientation values describing the relationship between the field of view and the image plate are disclosed. These orientation values are used to relate the area of image captured by each pixel in the image plate to the real world. By applying projective geometry, vector analysis and trigonometrical surveys an image analysis is conducted to establish a three-dimensional positional data set describing the three-dimensional position of regions of reflectivity in the captured two-dimensional image. From this data set a three-dimensional model of the field of view can be reconstructed.

Claims

1. A method of acquiring an image of a field of view whereby positional data are obtained for at least one region of reflectivity contained in said field of view, said field of view containing a plurality of said regions of reflectivity, said method comprising the steps of:

projecting into said field of view at least two detectable markers, each marker produced by emitting an electromagnetic beam incident on at least one reference plane comprised by said field of view, said reference plane comprising said at least one region of reflectivity, said beams each having an origin;

capturing an image of said field of view on an image plate, said image comprising said markers; and

selecting a set of beam configuration characteristics,

wherein at least one spatial relationship between at least one of the origins of said beams and said image plate at image capture, is determined,

identifying said at least two markers in the captured image; and

using said at least one spatial relationship to determine a set of offset values relating said image plate to said origin; and

using the identification of said markers, said offset values, said selected beam configuration characteristics and trigonometric calculations to determine a set of orientation values describing the orientation of said image plate to each of selected regions of reflectivity in the field of view, said selected regions of reflectivity not being comprised by said reference plane.

2. A method as claimed in claim 1, wherein said beam configuration characteristics comprise at least one spatial separation between said origins.

3. A method as claimed in claim 1, wherein said beam configuration characteristics comprise at least one angular separation between beams projected from common or distinct origins.

4. A method as claimed in claim 1, wherein said beam configuration characteristics comprise at least one spatial separation between origins and at least one angular separation of beams projected from a common or distinct origin.

5. A method as claimed in claim 1, said method configured for the obtaining of said positional data in one, two or three-dimensions.

6. A method as claimed in claim 1, said method further comprising the step of:

using said orientation values to reconstruct a positional data set describing the position of at least one said selected region of reflectivity.

7. A method as claimed in claim 6, wherein said step of using said orientation values to reconstruct a positional data set describing the position of at least one said selected region of reflectivity comprises the steps of:

using said orientation values to evaluate each pixel of data in the real world by determining the degree of pan and/or twist and/or tilt for each pixel of data; and

associating each said pixel of data with a said selected region of reflectivity and determining the real world position of said selected region of reflectivity.

8. A method as claimed in claim 6, wherein said step of using said orientation values to reconstruct a positional data set describing the position of at least one said selected region of reflectivity comprises the steps of:

using said orientation values to evaluate each pixel of data in the real world by determining the degree of pan and/or twist and/or tilt for each pixel of data; and

associating each said pixel of data with a said selected region of reflectivity and determining the real world position of said selected region of reflectivity; and

repeating said evaluation of each pixel of data for a plurality of heights above a striking plane.

9. A method as claimed in claim 6 wherein said step of using said orientation values to reconstruct a positional data set describing the position of at least one said selected region of reflectivity comprises the steps of:

using said orientation values to evaluate each pixel of data in the real world by determining the degree of pan and/or twist and/or tilt for each pixel of data; and

associating each said pixel of data with a said selected region of reflectivity and determining the real world position of said selected region of reflectivity; and

selecting a pixel of data representing a said selected region of reflectivity having known three dimensional positional data and scanning to all three-dimensional planes to test tht each pixel of data matches a set of three dimensional coordinates.

10. A method as claimed in claim 6, wherein said step of using said orientation values to reconstruct a positional data set describing the position of at least one said selected region of reflectivity comprises the steps of:

using said orientation values to evaluate each pixel of data in the real world by determining the degree of pan and/or twist and/or tilt for each pixel of data; and

associating each said pixel of data with a said selected region of reflectivity and determining the real world position of said selected region of reflectivity; and

said step of using said orientation values to reconstruct a positional data set comprises the step of reconstructing a three-dimensional data set describing the three-dimensional position of said selected regions of reflectivity, said step comprising the steps of;

applying an edge detection function to the two-dimensional captured image; and

checking the three-dimensional data describing said selected regions of reflectivity against the edge detection results to establish a finalised set of three-dimensional data describing the three dimensional real world position of each said selected region of reflectivity.

11. A method as claimed in claim 1, wherein said orientation values comprise a pan of said image plate with respect to said selected region of reflectivity, a pan defined as rotation about the z axis.

12. A method as claimed in claim 1, wherein said orientation values comprise a twist of said image plate with respect to said selected region of reflectivity, a twist defined as rotation about the y axis.

13. A method as claimed in claim 1, wherein said orientation values comprise a tilt of said image plate with respect to a said selected region of reflectivity, a tilt defined as rotation about the x axis.

14. A method as claimed in claim 1, wherein said orientation values comprise a height of said image plate with respect to a said selected region of reflectivity.

15. A method as claimed in claim 1, wherein said step of identifying at least two markers in the captured image comprises the steps of:

searching for said markers in the captured image to obtain a first set of identified markers; and

filtering said first set of identified markers to remove false markers.

16. A method as claimed in claim 15, said method comprising the step of:

determining the position of said markers in the image by implementing a best fit analysis.

17. A method as claimed in claim 1, wherein said step of determining a set of offset values comprises the step of:

determining a set of offset coefficients, each offset coefficient describing a function of the image plate—

origin spatial relationships.

18. A method as claimed in claim 17, wherein said step of determining a set of offset values further comprises the step of:

calculating an angular mounting error of the image plate in relation to the origin of each beam.

19. The method as claimed in claim 1, said method further comprising the steps of:

using said orientation values to determine a distance from said image plate to each said selected region of reflectivity.

20. A method as claimed in claim 1, wherein said method further comprises the step of:

using respective said orientation values to determine the perpendicular height difference between a said selected region of reflectivity and a striking plane.

21. A method as claimed in claim 1 wherein said image plate comprises an array of elements each capable of generating a pixel of data.

22. A method as claimed in claim 21, wherein said image plate comprises a CCD or CMOS array.

23. A method as claimed in claim 1, wherein said step of identifying said markers in the captured image comprises the step of:

conducting a search of the captured two-dimensional image to identify pixels receiving reflected radiation of a particular wavelength.

24. A method as claimed in claim 1, said method further comprising the step of:

mapping three-dimensional positional data obtained to reconstruct a virtual three-dimensional representation of the field of view.

25. A method as claimed in claim 1, wherein said at least two detectable markers comprise at least one first marker projected from a first origin and at least one second marker projected from a second origin.

26. A method as claimed in claim 1, wherein two said markers are projected into said field of view, each marker projected from a distinct origin, said markers intersected by the optical axis of said image plate, said beam configuration characteristics comprising the spatial separation of each origin to the image plate centre.

27. A method as claimed in claim 1, wherein at least three said markers are projected into said field of view, each marker projected from a distinct origin, said beam configuration characteristics comprising the spatial separation between each origin.

28. A method as claimed in claim 1, wherein at least three said markers are projected into said field of view, at least one marker projected from a first origin and at least two said markers projected from a second common origin, said beam configuration characteristics comprising at least one spatial separation between said origins and an angular separation of the beams projected from said second common origin.

29. A method of acquiring an image of a field of view, said method comprising the distance between at least one region of reflectivity contained in a field of view and an image plate, said method comprising the steps of:

projecting into said field of view at least one detectable marker said marker produced by emitting at least one electromagnetic beam incident on at least one reference plane comprised by said field of view, said reference plane comprising at least one region of reflectivity, said beam having an origin;

capturing an image of said field of view on an image plate, said image comprising said markers; and

selecting a set of beam configuration characteristics,

wherein at least one spatial relationship between said origins of said beam and said image plate, at image capture, is determined,

identifying said marker in the captured image; and

using said at least one spatial relationship to determine a set of offset values relating said image plate to said origins; and

using the identification of said marker, said offset values, said selected beam configuration characteristics and trigonometric calculations to determine a set of orientation values describing the orientation of said image plate to each of selected regions of reflectivity in the field of view, said selected region of reflectivity not being comprised by said reference plane.

30. A method as claimed in claim 29, further comprising the step of

using said orientation values to determine the distance between said image plate and one or more of said selected regions of reflectivity.

31. A method as claimed in claim 29, wherein said image plate comprises an array of elements each capable of generating a pixel of data.

32. A method of acquiring and processing an image of a field of view to obtain positional data describing the real world position of at least one region of reflectivity contained in the image, said image comprising a representation of said field of view into which at least two detectable markers have been projected, each marker produced by incidence of at least one electromagnetic beam on at least one reference plane in said field of view, said reference plane comprising at least one region of reflectivity in the imaged field of view, said beams each having an origin said method comprising the steps of:

selecting a set of beam configuration characteristics;

determining at least one spatial relationship between the origin of each said beam and the image plate at image capture;

identifying said at least two markers in the captured image; and

using said at least one spatial relationship to determine a set of offset values relating said image plate to each said origin(s); and

using the identification of said markers, said offset values, said selected beam configuration characteristics and trigonometric calculations to determine a set of orientation values describing the orientation of said image plate to each of selected regions of reflectivity in the field of view, said selected regions of reflectivity not being comprised by said reference plane.

33. A method imaging a field of view for use in obtaining positional data describing the real world position of at least one region of reflectivity contained in said field of view, wherein said field of view contains a plurality of regions of reflectivity, said method comprising the steps of:

projecting into said field of view at least two detectable markers, each marker produced by emitting at least one electromagnetic beam incident on at least one reference plane comprised by said field of view, said reference plane comprising at least one region of reflectivity, said beams having at least one origin; and

capturing an image of said field of view on an image plate, said image comprising said markers,

wherein said image plate and said origins are arranged in fixed positions relative to each other at image capture, wherein by selecting a set of beam configuration characteristics an analysis of the captured image can be performed comprising the identification of the markers in the image, the determination of a set of offset values relating the image plate to said at least one origin and using the identified markers, offset values, selected beam configuration characteristics and trigonometric calculations to determine a set of orientation values describing the orientation of the image plate to each of selected regions of reflectivity in the field of view, said selected regions of reflectivity not being comprised by said reference plane.

34. An apparatus for acquiring an image of a field of view. Said apparatus configured to project at least first and second detectable markers onto said field of view, wherein said markers are captured as part of said obtained image of said field of view formed on an image plate, said apparatus comprising:

at least one electromagnetic radiation source configured for emission of electromagnetic radiation;

beam producing means configured to operate on said electromagnetic radiation to produce at least two electromagnetic beams, each beam having an origin, said beams producing said detectable markers on incidence with a reference plane comprising a region of reflectivity in said field of view; and

location means to locate an image plate, said location means located at a pre-determined position,

wherein;

the angular projection of each said beam from each respective origin and the positions of each said origin within said unit are pre-determined, such that the spatial relationship between said origins and/or said origins and said means to locate an image plate are thereby also pre-determined; and

said projected beams are configured to project said markers onto said field of view to provide information for the analysis of an image of said field of view and said markers captured at said image plate in order to establish a set of orientation values describing the orientation of the image plate, at image capture, to selected regions of reflectivity in said field of view, said selected regions of reflectivity not being comprised by said reference plane.

35. An apparatus as claimed in claim 34, wherein said markers form a predefined pattern in said field of view.

36. An apparatus as claimed in claim 34, wherein said electromagnetic radiation comprises radiation of a wavelength in the range 10⁻

15 m to 10<sup>″

6</sup> m.

37. An apparatus as claimed in claim 34, wherein said image plate comprises a CCD or CMOS array.

38. An apparatus as claimed in claim 34, wherein said image plate is comprised by a digital camera, said means to locate said image plate configured to engage said digital camera for the location of said digital camera in fixed spatial and angular orientation on said projection unit.

39. An apparatus as claimed in claim 34, wherein said electromagnetic radiation comprises visible light of a wavelength in the range 400 nm to 700 nm.

40. An apparatus as claimed in claim 34, wherein said projection unit further comprises a body portion, said body portion defining a fixed location of said origins and a fixed location of said means to locate said image plate.

41. An apparatus as claimed in claim 34, wherein said orientation values enable the reconstruction of a positional data set describing the real world position of said selected regions of reflectivity in said field of view in one, two or three-dimensions.

42. An apparatus as claimed in claim 34, wherein said projection unit further comprises means to focus said beams.