Landmark navigation for vehicles using blinking optical beacons

• US 7,739,034 B2
• Filed: 04/17/2007
• Issued: 06/15/2010
• Est. Priority Date: 04/17/2007
• Status: Active Grant

• Current Assignee: Harris Corporation (L3Harris Technologies, Inc.)
• Original Assignee: ITT Manufacturing Enterprises, Inc. (ITT, Inc.)
• Inventors: Farwell, Mark Lalon
• Primary Examiner(s): Black, Thomas G
• Assistant Examiner(s): Chen, Shelley
• Application Number:
  US11/736,042
• Publication Number:
  US 20080262718A1
• Time in Patent Office: 1,155 Days
• Field of Search: 701/207
• US Class Current: 701/207
• CPC Class Codes: G05D 1/0234 using optical markers or beacons optical beacons per se G01S1/70
• CPC Class Codes: G05D 1/0244 using reflecting strips

Abstract

A system and method for landmark navigation employing optical beacons deployed at locations throughout a field of operation of a vehicle. The optical beacons emit or reflect an optical signal at a predetermined blink frequency. The locations of the optical beacons may or may not be known to the vehicle. At least one imaging device on the vehicle, such as a digital camera, captures images in the field of operation, and in particular a pair of image frames such that the time interval between the image frames of the pair is equal to one-half of the blink period of the optical signal. Data is generated that represents a difference frame between two image frames captured by the imaging device. Pixel locations of optical beacons in the difference frame are identified. The position and orientation of the vehicle is determined from data representing pixel locations of optical beacons in the difference frame.

Claims

1. A method for landmark navigation of a vehicle, comprising:

a. storing data in the vehicle representing locations of each of a plurality of optical beacons in a field of operation, wherein each optical beacon emits or reflects an optical signal at a predetermined blink frequency and with a 50% duty cycle and such that the plurality of optical beacons are not synchronized;

b. during a position update cycle;

i. capturing a pair of image frames in the field of operation with at least one imaging device on the vehicle, wherein capturing comprises capturing the pair of image frames such that a time interval separating the capture of the image frames in the pair is equal to one-half of a blink period corresponding to the predetermined blink frequency of the optical beacons, and the time of capture for each image in the pair of image frames coincides with emission or reflection by a different set of optical beacons;

ii. generating data representing a difference frame between the two image frames in the pair of image frames;

iii. identifying pixel locations of optical beacons in the difference frame; and

iv. deriving a position and orientation of the vehicle from data representing pixel locations of optical beacons in the difference frame and from the stored data representing the location of each of the optical beacons in the field of operation.

2. The method of claim 1, and further comprising emitting light at said predetermined blink frequency from a light source on the vehicle so as to illuminate an optical beacon, and wherein capturing comprises capturing a pair of image frames with one of the image frames including light reflected by an optical beacon on which light from said light source is incident.

3. The method of claim 1, wherein one or more of the optical beacons comprises a light source that emits an optical signal at said predetermined blink frequency.

4. The method of claim 1, wherein (iv) deriving comprises, for a first difference frame during a first position update cycle, matching pixel locations of optical beacons in the difference frame to actual optical beacons from the stored data representing the locations of each of the optical beacons.

5. The method of claim 4, wherein matching comprises, for difference frames subsequent said first difference frame during position update cycles after the first position update cycle, associating pixel locations of optical beacons in the difference frames to actual optical beacons based on data for closest matched beacon pixel locations determined from difference frames at a previous position update cycle, and generating first beacon match data representative thereof.

6. The method of claim 5, wherein (iv) deriving comprises, during position update cycles after the first position update cycle, for those pixel locations of optical beacons in the difference frames that cannot be matched to actual optical beacons from the difference frame at a previous position update cycle, determining matches to actual optical beacons from the stored data representing the locations of each of the optical beacons, and generating second beacon match data representative thereof.

7. The method of claim 6, wherein generating the second beacon match data comprises computing a three-dimensional pixel to beacon ray intersection based on a position of the vehicle as of a most recent position update cycle, orientation of the imaging device on the vehicle and the stored data representing locations of each of the optical beacons.

8. The method of claim 6, wherein (iv) deriving comprises computing position and orientation of the vehicle for a position update cycle based on the first beacon match data and second beacon match data.

9. The method of claim 8, wherein computing position and orientation of the vehicle comprises minimizing a summation of errors for the three-dimensional pixel to beacon ray intersections represented by the second match data with the position of the optical beacon to which the pixel to beacon ray is matched in the stored data of positions of the optical beacons.

10. The method of claim 6, wherein when a beacon pixel cannot be matched to an actual optical beacon in the stored data, then further comprising storing data for three-dimensional rays for beacon pixels identified over multiple position update cycles at different positions of the vehicle, and analyzing data for three-dimensional rays accumulated over time to determine when several three-dimensional rays point to the same position in space so as to store data for a new optical beacon at that position.

11. A landmark navigation system for a vehicle, comprising:

a. a data storage device that stores data representing a location of each of a plurality of optical beacons in a field of operation, wherein each optical beacon emits or reflects an optical signal at a predetermined blink frequency and with a 50% duty cycle and such that the plurality of optical beacons are not synchronized;

b. at least one imaging device that captures images in a field of view from the vehicle in the field of operation; and

c. a controller connected to the storage device and the at least one imaging device, wherein for each of a plurality of position update cycles, the controller controls the at least one imaging device to capture a pair of image frames such that a time interval separating the capture of the image frames in the pair is equal to one-half of a blink period corresponding to the predetermined blink frequency of the optical beacons and the time of capture for each image in the pair of image frames coincides with emission or reflection by a different set of optical beacons, processes image data for the pair of images frames to generate a difference frame between the two image frames in the pair of image frames, and analyzes the difference frame in order to derive a position and orientation of the vehicle based on the stored data representing the location of each of the optical beacons in the field of operation.

12. The system of claim 11, and further comprising a light source on the vehicle that emits light at said predetermined blink frequency, and wherein the imaging device captures a pair of image frames with one of the image frames including light reflected by an optical beacon on which light from said light source is incident.

13. The system of claim 12, and comprising a plurality of imaging devices, each imaging device being associated with a corresponding one of a plurality of light sources.

14. The system of claim 11, wherein one or more of the optical beacons comprises a light source that emits an optical signal at said predetermined blink frequency.

15. The system of claim 11, wherein the controller analyzes a first difference frame during a first position update cycle to match pixel locations of optical beacons in the difference frame to actual optical beacons from the stored data representing the locations of each of the optical beacons.

16. The system of claim 15, wherein the controller analyzes difference frames subsequent said first difference frame during position update cycles after the first position update cycle, to associate pixel locations of optical beacons in the difference frames to actual optical beacons based on data for closest matched beacon pixel locations determined from difference frames at a previous position update cycle, and generates first beacon match data representative thereof.

17. The system of claim 16, wherein during position update cycles after the first position update cycle, and for those pixel locations of optical beacons in the difference frames that cannot be matched to actual optical beacons from the difference frame at a previous position update cycle, the controller determines matches to actual optical beacons from the stored data representing the locations of each of the optical beacons, and generates second beacon match data representative thereof.

18. The system of claim 17, wherein when a beacon pixel cannot be matched to an actual optical beacon in the stored data, the controller stores data for three-dimensional rays for beacon pixels identified over multiple position update cycles at different positions of the vehicle, and analyzes the data for three-dimensional rays accumulated over time to determine when several three-dimensional rays point to the same position in space so as to store data for a new optical beacon at that position.

19. A method for determining a position of the vehicle based on an optical signal having a predetermined blink frequency and with a 50% duty cycle that is emitted or reflected by each of a plurality of optical beacons positioned in a field of operation of the vehicle and such that the plurality of optical beacons are not synchronized, comprising:

storing data in the vehicle that represents the actual location of one or more of the optical beacons in the field of operation;

generating data representing a difference frame between two image frames captured from the vehicle such that a time interval separating the capture of the two image frames is equal to one-half of a blink period of the optical signal and the time of capture for each image in the pair of image frames coincides with emission or reflection by a different set of optical beacons;

identifying pixel locations of optical beacons in the difference frame; and

computing a position and orientation of the vehicle and from said stored data and from data representing pixel locations of optical beacons in the difference frame.

20. The method of claim 19, and further comprising emitting light at said predetermined blink frequency from a light source on the vehicle so as to illuminate an optical beacon, and capturing the two image frames with one of the image frames including light reflected by an optical beacon on which light from said light source is incident.

21. The method of claim 19, wherein one of more of the optical beacons comprises a light source that emits an optical signal at said predetermined blink frequency, and further comprising capturing the two image frames with an imaging device on the vehicle.

22. A tangible computer readable medium storing instructions that, when executed by a computer, cause the computer to determine a position of the vehicle based on an optical signal having a predetermined blink frequency and with a 50% duty cycle that is emitted or reflected by each of a plurality of optical beacons positioned in a field of operation of the vehicle and such that the plurality of optical beacons are not synchronized, the instructions cause the computer to perform operations comprising:

storing data in the vehicle that represents the actual location of one or more of the optical beacons in the field of operation;

generating data representing a difference frame between two image frames captured from the vehicle such that a time interval separating the capture of the two image frames is equal to one-half of a blink period of the optical signal and the time of capture for each image in the pair of image frames coincides with emission or reflection by a different set of optical beacons;

identifying pixel locations of optical beacons in the difference frame; and

computing a position and orientation of the vehicle and from said stored data and from data representing pixel locations of optical beacons in the difference frame.

23. The tangible computer readable medium of claim 22, wherein the instructions that cause the processor to perform said computing comprise instructions that cause the processor to, for a first difference frame during a first position update cycle, match pixel locations of optical beacons in the difference frame to actual optical beacons from the stored data representing the locations of each of the optical beacons, and for difference frames subsequent said first difference frame during position update cycles after the first position update cycle, associate pixel locations of optical beacons in the difference frames to actual optical beacons based on data for closest matched beacon pixel locations determined from difference frames at a previous position update cycle, and generate first beacon match data representative thereof.

24. The tangible computer readable medium of claim 23, wherein the instructions that cause the processor to compute comprise instructions that cause the processor to, during position update cycles after the first position update cycle, for those pixel locations of optical beacons in the difference frames that cannot be matched to actual optical beacons from the difference frame at a previous position update cycle, determine matches to actual optical beacons from the stored data representing the locations of each of the optical beacons, to generate second beacon match data representative thereof and to compute said position and orientation of the vehicle for a position update cycle based on the first beacon match data and second beacon match data.

25. The tangible computer readable medium of claim 23, wherein the instructions that cause the processor to compute the second beacon match data comprise instructions that cause the processor to compute a three-dimensional pixel to beacon ray intersection based on a position of the vehicle as of a most recent position update cycle, orientation of the imaging device on the vehicle and the stored data representing locations of each of the optical beacons.

26. The tangible computer readable medium of claim 23, and further comprising instructions that cause the processor to, when a beacon pixel cannot be matched to an actual optical beacon in the stored data, store data for three-dimensional rays for beacon pixels identified over multiple position update cycles at different positions of the vehicle, and analyze data for three-dimensional rays accumulated over time to determine when several three-dimensional rays point to the same position in space so as to store data for a new optical beacon at that position.