Methods and apparatus for using image data to determine camera location and orientation

• US 5,699,444 A
• Filed: 03/31/1995
• Issued: 12/16/1997
• Est. Priority Date: 03/31/1995
• Status: Expired due to Term

• Current Assignee: Jade Pixel LLC (Intellectual Ventures LLC)
• Original Assignee: Synthonics Incorporated (Synthonics Technologies, Inc.)
• Inventors: Palm, Charles S.
• Primary Examiner(s): Boudreau, Leo
• Assistant Examiner(s): KELLEY, CHRISTOPHER S
• Application Number:
  US08/414,651
• Time in Patent Office: 991 Days
• Field of Search: 382/106, 382/154, 382/285, 348/42, 348/47, 348/48, 348/50, 348/139, 356/3
• US Class Current: 382/106
• CPC Class Codes: G01C 11/06 by comparison of two or more pictures of the same area
• CPC Class Codes: G06T 2207/30204 Marker
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 7/55 from multiple images
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06V 10/147 Details of sensors, e.g. sensor lenses fingerprint or palmprint sensors G06V40/13; vascular sensors G06V40/145; eye sensors G06V40/19
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/246 Calibration of cameras
• CPC Class Codes: H04N 2013/0081 Depth or disparity estimation from stereoscopic image signals

Abstract

Methods and apparatus for accurately surveying and determining the physical location of objects in a scene are disclosed which use image data captured by one or more cameras and three points from the scene which may either be measured after the images are captured or may be included in the calibrated target placed in the scene at the time of image capture. Objects are located with respect to a three dimensional coordinate system defined with reference to the three points. The methods and apparatus permit rapid set up and capture of precise location data using simple apparatus and simple image processing. The precise location and orientation of the camera utilized to capture each scene is determined from image data, from the three point locations and from optical parameters of the camera.

Claims

1. A method of measuring the absolute three dimensional location of a point D with respect to a coordinate system defined using three points, A, B and C, separated by known distances using image data comprising:

a. capturing two images of a scene containing the points of B, C and D, using one or more cameras of known principal distance,

b. determining the location and orientation of said one or more cameras at the time each of said images was captured with reference to said coordinate system using 3 to 5 points from said images, principal distance and said known distances,

c. using the locations of the one or more cameras at the time the images were captured to determine the location of said point D from image data.

8. The method of claim 1 used to measure the distance between two points.

9. The method of claim 1 used to measure distances in a vertical direction.

10. The method of claim 1 used to locate physical position accurately of objects visible in said images.

11. The method of claim 1 used to create a three dimensional wireframe representation or a three dimensional surface model comprising 3 or 4 vertices surface element.

12. The method of claim 1 used to document the as built condition of an object.

2. A method of measuring the absolute three dimensional location of a point D with respect to a coordinate system defined using three points, A, B and C, separated by known distances using image data comprising:

a. capturing two images of a scene containing the points A, B, C and D, using one or more cameras of known principal distance,

b. determining the location and orientation of said one or more cameras at the time each of said images was captured with reference to said coordinate system using information derived from said images, principal distance and said known distances, and

c. using the locations of the one or more cameras at the time the images were captured to determine the location of said point D from image data by

c1. defining an auxiliary coordinate system with origin along the line joining the locations of the cameras,

c2. defining the center point of each image as an origin of a set of image reference axes pointing in X'"'"', Y'"'"' and Z'"'"' directions, respectively,

c3. measuring offset in at least one of the X'"'"' and Y'"'"' directions of a point on the first image and of a corresponding point of a second image,

c4. determining the angles formed between a line joining point D, the principal point of the objective and the image of point D on one of the X'"'"' or Y'"'"' planes for each of the images,

c5. determining a distance h representing a distance of point d to a line joining principle points of said one or more cameras used to capture said two images using the measured offsets, the focal length and the angles,

c6. determining the X'"'"' and Y'"'"' coordinates of point D in the auxiliary coordinate system, and

c7. transforming coordinates (X'"'"', Y'"'"', h) of the auxiliary coordinate system to a representation in said coordinate system defined using said three points, A, B and C.

3. A method of measuring the absolute three dimensional location of a point D with respect to a coordinate system defined using three points, A, B and C, separated by known distances using image data comprising:

a. capturing two images of a scene containing the points A, B, C and D, usinq one or more cameras of known principal distance,

b. determining the location and orientation of said one or more cameras at the time each of said images was captured with reference to said coordinate system using information derived from said images, principal distance and said known distances by,

b1. representing the distance between point A, B and C and the principal point of a camera O as a viewing pyramid,

b2. modifying the representation of the pyramid to a three triangle flattened representation,

b3. selecting a low estimate Ob¹ for one interior side of a first triangle of said flattened representation,

b4. solving the first triangle using image data, principal distance and said known distances, yielding, inter alia, a first calculated value for length OA, given estimate Ob¹,

b5. solving the second triangle using results obtained,

b6. solving the third triangle using results obtained, yielding, inter alia, a second calculated value for length OA,

b7. subtracting the second calculated value for length OA from the first calculated value for length OA to produce a difference value,

b8. revising the value of estimate Ob¹ by adding said difference value to achieve a revised estimate,

b9. iterating steps d-h using the revised estimate until said difference value is less than a desired accuracy, and

b10. deriving values for camera location using one or more sets of values for distances OA, OB and OC, and

c. using the locations of the one or more cameras at the time the images were captured to determine the location of said point D from image data.

4. The method of claim 3 in which the step of deriving values for camera location using one or more sets of values for distances OA, OB and OC comprises solving simultaneously equations for spheres centered at points A, B and C with respective radii of OA, OB and OC.

5. The method of claim 3, further comprising:

k. determining the orientation of one of more of said cameras by calculating the azimuthal and elevational adjustment required to direct the camera to the location of point A.

6. The method of claim 5, further comprising:

l. determining the orientation of one of more of said cameras by calculating the amount of rotation about the optical axis required to align point B once the camera points at point A.

7. The method of claim 5 further comprising iterating steps k and l until the degree of alignment is within the desired degree of accuracy.

13. A method of measuring the absolute three dimensional location O of a camera with respect to a coordinate system defined using three points, A, B and C, separated by known distances using image data comprising:

a. capturing an image of a scene containing the points A, B, and C, using a camera,

b. determining the principal distance of said camera,

c. determining the location of said camera at the time said image was captured with reference to said coordinate system using 3 to 5 points from said image, principal distance and said known distances.

14. A method of measuring the absolute three dimensional location O of a camera with respect to a coordinate system defined using three points, A, B and C, separated by known distances using image data comprising:

a. capturing an images of a scene containing the points A, B, and C, using a camera of known principal distance,

b. determining the location of said camera at the time said image was captured with reference to said coordinate system using 3 to 5 points from said image, principal distance and said known distances.

15. A method of measuring distance including vertical height comprising:

a. measuring the absolute three dimensional location of points D, E and F with respect to a coordinate system defined using three points, A, B and C, separated by known distances using image data by;

a1. capturing two images of a scene containing the points A, B, C, D, E and F, using one or more cameras of known principal distance,

a2. determining the location and orientation of said one or more cameras at the time each of said images was captured with reference to said coordinate system using points A, B, and C from said images, principal distance and said known distances,

a3. using the locations of the one or more cameras at the time the images were captured to determine the locations of said points D, E and F from image data,

b. determining distances between points D, E and F, and

c. using the location of said points D, E and F and the location of one or more cameras at the time images were captured to determine the location of other points.

16. The method of claim 15 in which the locations of points D, E and F are used to determine the location of said other points using image data from images different from those used to determine the location of points D, E and F.

17. Apparatus for measuring the absolute three dimensional location of a point D with respect to a coordinate system defined using three points, A, B and C, separated by known distances using image data comprising:

a. one or more cameras for capturing images of a scene containing the points A, B, C and D,

b. means for storing images captured by said one or more cameras,

c. means for processing stored images to determine the location and orientation of said one or more cameras at the time each of said images was captured with reference to said coordinate system, using 3 to 5 points from said images, principal distance and said known distances,

d. means for using the locations of said one or more cameras at the time the images were captured to determine the location of said point D from image data.

18. Apparatus as claimed in claim 17 in which the location of point D is stored in a database utilized to store a three dimensional wireframe representation.

19. Apparatus as claimed in claim 18 in which the location of point D is stored in a database of locations of points surveyed.