Navigation by a video-camera sensed ground array

• US 4,866,626 A
• Filed: 09/18/1987
• Issued: 09/12/1989
• Est. Priority Date: 09/18/1987
• Status: Expired due to Fees

• Current Assignee: Honeywell Incorporated (Honeywell International Inc.)
• Original Assignee: Honeywell Incorporated (Honeywell International Inc.)
• Inventors: Egli, Werner H.
• Primary Examiner(s): Lall, Parshotam S.
• Assistant Examiner(s): Black, Thomas G.
• Application Number:
  US07/098,341
• Time in Patent Office: 725 Days
• Field of Search: 364/449, 364/450, 364/428, 364/439, 364/460, 364/458, 364/456, 342/33-35, 340/94.8, 340/951"954
• US Class Current: 701/207
• CPC Class Codes: G01C 21/005 with correlation of navigation data from several sources, e.g. map or contour matching G01C21/30 takes precedence
• CPC Class Codes: G01S 5/16 using electromagnetic waves other than radio waves

Abstract

Navigation by interpretation of images of parallelogram corner points of a ground beacon array. A video camera senses images of the beacon array and a computer interprets the images to provide attitude and location of the platform, on which the camera is mounted, relative to the array.

Claims

1. A method of navigation by a video-camera sensed array comprising:

sensing beacons of an array with a video camera, wherein the array is located externally to the camera, and the beacons are situated so as to represent corners of a parallelogram;

detecting locations of images of the beacons on an image plane of the camera;

translating the locations of the images into pixel addresses;

processing the pixel addresses in a manner so as to determine a position and an attitude of the camera relative to the beacons of the array, wherein said processing comprises;

defining the images on the image plane of the camera with vectors;

connecting first and second images with a first line;

connecting third and fourth images with a second line;

extending the first and second lines until they meet at a first point;

connecting second and third images with a third line;

connecting first and fourth images with a fourth line;

extending the first and second lines until they meet at a second point;

connecting the first and second points with a line which is an image of a horizon;

connecting the first point and a center point of a focal lens of the camera with a fifth line;

connecting the second point and the center point of the focal lens of the camera with a sixth line;

defining a first plane with the fifth and sixth lines, said plane being parallel with a second plane containing the beacons and defining orientation of the second plane;

determining distance between the first and second planes;

determining orientation of the second plane; and

determining location and orientation of the camera relative to the beacons;

displaying the position and the attitude of the camera relative to the beacons;

storing the position and attitude; and

comparing the position and attitude with formerly stored positions and attitudes.

2. Method of claim 1 further comprising:

designating the beacons of the array forming the corners of a parallelogram as A, B, C and D vectors, respectively, artibrarily for identification;

designating the first, second, third and fourth images of the beacons on the image plane of the camera, as a, b, c and d vectors having one-to-one correspondence to vectors A, B, C and D, respectively, the vectors being defined in terms of x, y and z coordinates having an origin at the center point of the focal lens of the camera, having an i as a unit x-axis vector, perpendicular to the image plane and y and z axes in the image plane;

designating the first point by a vector u; and

designating the second point by a vector v.

3. Method of claim 2 further comprising:

designating

vector quantity <

s>

=(a+b+c+d)/4,

vector quantity d1=(a-b)/2,

vector quantity d2=(b-c)/2,

vector quantity d3=(c-d)/2,

vector quantity d4=(d-a)/2,

vector quantity s1=(a+b)/2, and

vector quantity s3=(c+d)/2;

expressing vector u as s1+G*d1 wherein G is a scalar

expressing vector u as s3+H*d3, wherein H is a scalar;

eliminating vector u from set of equations

u=s1+G*d1 and

u=s3+H*d3;

vector multiplying by d3 and by d1;

scalar multiplying by the unit x-axis vector i;

expressing the results as triple scalar products denoted by Q'"'"'s in view that all d'"'"'s are in a y-z plane, where

D1Y denotes y-component of d1,

D1Z denotes z-component of d1,

D2Y denotes y-component of d2,

D2Z denotes z-component of d2,

D3Y denotes y-component of d3,

D3Z denotes z-component of d3,

D4Y denotes y-component of d4, and

D4Z denotes z-component of d4, resulting in

Q31=D3Y*D1Z-D3Z*D1Y

Q342=D3Y*(D4Z-D2Z)-D3Z*(D4Y-D2Y)

Q142=D1Y*(D4Z-D2Z)-D1Z*(D4Y-D2Y);

expressing u as equal to

<

s>

+(Q342*d1+Q142*d3)/(2*Q31);

deriving from v vector properties

triple scalar products of

Q42=D4Y*D2Z-D4Z*D2Y

Q413=D4Y*(D1Z-D3Z)-D4Z*(D1Y-D3Y)

Q213=D2Y*(D1Z-D3Z)-D2Z*(D1Y-D3Y)

expressing v=<

s>

(Q413*d2+Q213*d4)/(2*Q42);

describing second plane of vectors u and v by defining a direction normal N to the second plane as N=u×

v; and

determining a first perpendicular distance DP1 along direction N from the center point of the focal lens of the camera to an object plane containing the beacons, by DP1=L1/(|N|*|a/(a.N)-b/(b.N)|

where A, B, C and D are scalar multiples of vectors a, b, c and d, respectively,

L1=|A-B|+|C-D|.

4. Method of claim 3 further comprising determining a second perpendicular distance DP2 along the direction N from the center point of the focal lens of the camera to the object plane containing the beacons, by DP2=L2/(|N|*|b/(b.N)-C/(C.N)| where L2=|B-C|=|D-A|.

5. Method of claim 4 further comprising determining a third perpendicular distance DP as a best estimate by root mean squaring DP1 and DP2 as DP=SQRT((DP1**2+DP2**2)/2).

6. Method of claim 3 further comprising:

determining a reference point within the array of beacons;

expressing the reference point in an earth-fixed reference frame; and

defining an attitude of the camera relative to the reference point.

7. Method of claim 6 wherein said determining the reference point, PREFR, comprises:

noting that PREFR=(A+B+C+D)/4 which expresses target location in a camera reference frame;

defining sides of the beacon array by vectors A-B, B-C, C-D and D-A equal to D1, D2, D3 and D4, respectively, expressed in the camera frame;

expressing D1 and D2 in an earth-fixed frame as D1E and D2E, respectively, having the relationships of D1=MTR'"'"'*D1E D2=MTR'"'"'*D2E

where D1=D3, D2=D4, MTR'"'"'=Transpose/Inverse of MTR and MTR is a matrix of the beacon array attitude relative to the camera frame;

defining MD as a 3 by 3 matrix having three columns comprising vectors D1, D2 and D1×

D2, respectively;

defining MDE as a 3 by 3 matrix having three columns comprising vectors D1E, D2E and D1E×

D2E, respectively;

equating MD to MTR'"'"'*MDE; and

solving for MTR with MTR=MDE*(1/MD)

where 1/MD is the inverse of matrix MD.

8. Method of claim 7 comprising determining vector distance of the center point of the focal lens of the camera (CRANGE) from the reference point in earth-fixed coordinates by multiplying the reference point by -1 and then by camera attitude matrix,

CRANGE=MCAM*PREPR.

9. Method of claim 7 comprising:

least-squares best-fitting a rotation matrix to MTR according to

MTRBF=(((1/(MTR*MTR'"'"'))**1/2)*MTR where

MTR'"'"'=transpose of MTR,

1/(MTR*MTR'"'"')=inverse of matrix

MTR*MTR'"'"', and matrix (1/(MTR*MTR'"'"'))**1/2 multiplied by itself yields

1/(MTR*MTR'"'"');

expressing MTR*MTR'"'"' as I+e where I is an identity matrix to which MTR*MTR'"'"' approaches equality;

expanding (1/(MTR'"'"'MTR'"'"'))**1/2 according to a Taylors series;

solving for MTRBF with a first order expression of the Taylors series resulting in MTRBF=(3*MTR-MTR*MTR'"'"'*MTR)/2;

solving for an attitude of the camera, MCAM, relative to an earth-fixed frame, by taking the inverse of MTRBF which is MCAM=(3*MTR'"'"'-MTR'"'"'*MTR*MTR'"'"')/2; and

solving for the vector distance, CRANGE, of the camera in earth-fixed coordinates by multiplying PREFR by -1 and then by the attitude of the camera matrix as

CRANGE=-MCAM*PREFR.

10. A navigation device comprising:

sensing means for detecting objects of a ground array, wherein the ground array is a parallelogram in a first plane, having a first, second, third and fourth objects at the first, second, third and fourth corners, respectively, of the parallelogram, and said sensing means comprises;

a focal lens;

an image plane situated at a focal point of said lens; and

addressing means, connected to said image plane for addressing image plane pixel outputs;

preprocessing means, connected to said sensing means, for preprocessing data from said sensing means;

computing means, connected to said preprocessing means, for processing preprocessed data from said preprocessing means into navigational information, including position and attitude of said sensing means relative to the ground array which lies in an earth-fixed frame of reference, wherein said computing means determines;

first, second, third and fourth object vectors that define the first, second, third and fourth objects, respectively;

first, second, third and fourth image points on said image plane, corresponding to the first, second, third and fourth objects, respectively;

first, second, third and fourth image vectors that define image points on said image plane corresponding to the first, second, third and fourth objects;

a coordinate frame having an origin and a reference relative to said sensing means such that the first, second, third and fourth image vectors are positive scalar multiples of the first, second, third and fourth object vectors, respectively;

a fifth image point by an intersection of a first line through the first and second image points and of a second line through the third and fourth image points;

a sixth image point by an intersection of a third line through the second and third object image points and of a fourth line through the first and fourth image points;

a horizon image by a fifth line through the fifth and sixth image points;

a second plane by a sixth line from the fifth image point through the origin of the coordinate frame and by a seventh line from the sixth image point through the origin of the coordinate frame, wherein the second plane defines an orientation of the first plane containing the ground array;

distance as a range vector between the origin of the coordinate frame and a point in the first plane by exploiting that the first, second, third and fourth objects are at some same perpendicular distance from the origin of the coordinate frame, and that the distances between the objects of the array are known; and

an orientation matrix of the first plane that yields location and orientation of said sensing means relative to the first plane.

11. Apparatus of claim 10 wherein said computing means further determines position and orientation of a platform upon which said sensing means is mounted.

12. Apparatus of claim 11 wherein said sensing means is a video camera.

13. Apparatus of claim 12 wherein said computing means is a microcomputer.

14. Method of navigation by a video-camera sensed array comprising:

sensing, with an image plane in a camera, first, second, third and fourth object image points of first, second, third and fourth objects, respectively, in an array located externally to the camera, wherein the first, second, third and fourth objects are located at first, second, third and fourth corners, respectively, of a parallelogram;

determining first, second, third and fourth image vectors that define the first, second, third and fourth object image points, respectively, according to location on the image plane;

processing information of the first, second, third and fourth image vectors to determine position and attitude of the camera relative to the first, second, third and fourth objects in the array;

determining first, second, third and fourth object vectors that define the first, second third and fourth objects;

determining a coordinate frame having reference to the camera in conventional orientation and having an origin at a center of the camera'"'"'s focal lens and such that the first, second, third and fourth image vectors are positive scalar multiples of the first, second, third and fourth object vectors, respectively;

determining a fifth image point by an intersection of a first line through the first and second object image points and of a second line through the third and fourth object image points;

determining a sixth image point by an intersection of a third line through the second and third object image points and of fourth line through the first and fourth object image points;

determining a horizon image by a fifth line through the fifth and sixth image points;

determining, parallel to a second plane containing the parallelogram, a first plane, by a sixth line from the fifth image point through the origin of the coordinate frame reference and by a seventh line from the sixth image point through the origin of the coordinate frame of reference, wherein said determining the first plane defines the orientation of the second plane containing the parallelogram;

determining distance as a range vector between the origin of the coordinate frame of reference and a point in the second plane by exploiting that the first, second, third and fourth objects are at some same perpendicular distance from the origin of the coordinate frame, that the first, second, third and fourth image vectors are positive scalar multiples of the first, second, third and fourth objects, respectively, and that the magnitudes between the corners of the parallelogram are known; and

determining an orientation matrix of the second plane which yields locations and orientation of the camera relative to the second plane.

15. Method of claim 14 further comprising determining the position and orientation of the platform upon which the camera is mounted.