Camera-based heading-hold navigation

• US 10,724,871 B2
• Filed: 11/03/2017
• Issued: 07/28/2020
• Est. Priority Date: 11/03/2016
• Status: Active Grant

• Current Assignee: Charles Stark Draper Laboratory Incorporated
• Original Assignee: Charles Stark Draper Laboratory Incorporated
• Inventors: Laine, Juha-Pekka J., Bortolami, Simone B., Korn, Jeffrey, Blasche, Gregory P., Jamula, Matthew T., Bohn, Paul A., Dawson, Robin Mark Adrian, Lane, Benjamin F., Hoke, Eric T., Meiser, Daniel M., Kinast, Joseph M., McCarthy, Timothy J., Smith, Stephen P.
• Primary Examiner(s): To, Tuan C
• Application Number:
  US15/803,367
• Publication Number:
  US 20180120122A1
• Time in Patent Office: 998 Days
• Field of Search: 701428
• CPC Class Codes: G01C 17/38 Testing, calibrating, or compensating of compasses
• CPC Class Codes: G01C 21/3602 Input other than that of destination using image analysis, e.g. detection of road signs, lanes, buildings, real preceding vehicles using a camera
• CPC Class Codes: G01C 21/3647 Guidance involving output of stored or live camera images or video streams
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: H04W 4/026 using orientation information, e.g. compass

Abstract

A visual navigation system includes a compass configured to orient a user in a heading direction, an image sensor configured to capture a series of successive navigation images in the heading direction, one or more of the navigation images having at least two reference markers, data storage memory configured to store the series of successive navigation images, a navigation processor configured to identify at least one principal marker and at least one ancillary marker from the at least two reference markers, the principal marker positioned within a principal angle and the ancillary marker positioned within an ancillary angle, which is greater than the principal angle, and to determine heading direction information based on a position of the at least one principal marker and/or the at least one ancillary marker in the successive navigation images, and a user interface configured to provide the heading direction information to the user.

Claims

1. A visual navigation system for navigating toward a heading direction, the system comprising:

a compass providing compass measurements configured to orient a user in the heading direction;

an image sensor, in communication with the compass, the image sensor configured to capture a series of successive navigation images in the heading direction oriented by the compass measurements, one or more of the navigation images having at least two reference markers;

data storage memory configured to store the series of successive navigation images;

a navigation processor, coupled to the data storage memory, configured to;

(a) identify at least one principal marker from the at least two reference markers, the at least one principal marker positioned within a principal angle relative to the heading direction, oriented by the compass measurements, from a position of the image sensor;

(b) identify at least one ancillary marker from the at least two reference markers, the at least one ancillary marker positioned within an ancillary angle relative to the heading direction, oriented by the compass measurements, from the position of the image sensor, the ancillary angle greater than the principal angle; and

(c) determine heading direction information based on a position of the at least one principal marker and/or a position of the at least one ancillary marker in the successive navigation images, such that the position of the at least one principal marker and/or the position of the at least one ancillary marker correct errors in the heading direction based on the compass measurements; and

a user interface, coupled to the navigation processor, configured to provide the heading direction information to the user.

2. The visual navigation system according to claim 1, wherein the data storage memory is further configured to store two or more compass measurements, each compass measurement taken at a different location when traveling in the heading direction, and the navigation processor further includes a navigation filter configured to process the two or more compass measurements, wherein the navigation processor determines the heading direction information further based on the processed two or more compass measurements.

3. The visual navigation system according to claim 2, wherein the navigation filter computes an average of the two or more compass measurements and uses the average along with the position of the at least one principal marker and/or the position of the at least one ancillary marker in the successive navigation images to estimate the heading direction information.

4. The visual navigation system according to claim 1, wherein the navigation processor is further configured to receive landmark information from a landmark map, at least a portion of the landmark information corresponding to one or more of the reference markers, wherein the navigation processor further identifies the at least one principal marker from the landmark information and/or further identifies the at least one ancillary marker from the landmark information, and the navigation processor is configured to determine the heading direction information further based on the landmark information.

5. The visual navigation system according to claim 1, wherein the navigation processor is further configured to determine the heading direction information by comparing (a) the position of the at least one principal marker in the one or more navigation images to a subsequent position of the at least one principal marker in one or more subsequent navigation images taken at a time after the one or more navigation images and/or (b) the position of the at least one ancillary marker in the one or more navigation images to a subsequent position of the at least one ancillary marker in one or more subsequent navigation images taken at a time after the one or more navigation images.

6. The visual navigation system according to claim 1, wherein the navigation processor is further configured to receive inertial navigation information from an inertial measurement unit sensor, wherein the navigation processor is configured to determine the heading direction information further based on the inertial navigation information.

7. The visual navigation system according to claim 6, wherein the inertial navigation information includes a step-count-based estimate for distance traveled by the user or a tread/wheel-turn-based estimate for distance traveled by a vehicle transporting the user.

8. The visual navigation system according to claim 1, wherein the navigation processor is further configured to receive altitude information from a pressure altimeter and configured to receive topographic map information from a topographic map, wherein the navigation processor is further configured to determine navigation information based on the altitude information in conjunction with the topographic map information and configured to determine the heading direction information further based on the navigation information.

9. The visual navigation system according to claim 1, wherein the compass, the image sensor, the data storage memory, the navigation processor, and the user interface are contained within a hand-held device.

10. The visual navigation system according to claim 1, wherein the user interface includes an audio output, visual output, tactile output or any combination thereof.

11. A method for navigating toward a heading direction, the method comprising:

providing a compass having compass measurements configured to orient a user in the heading direction;

capturing a series of successive navigation images in the heading direction oriented by the compass measurements using an image sensor, one or more of the navigation images having at least two reference markers;

storing the series of successive navigation images in a data storage memory;

identifying at least one principal marker from the at least two reference markers, the at least one principal marker positioned within a principal angle relative to the heading direction, oriented by the compass measurements, from a position of the image sensor;

identifying at least one ancillary marker from the at least two reference markers, the at least one ancillary marker positioned within an ancillary angle relative to the heading direction, oriented by the compass measurements, from the position of the image sensor, the ancillary angle greater than the principal angle;

determining heading direction information based on a position of the at least one principal marker and/or a position of the at least one ancillary marker in the successive navigation images, such that the position of the at least one principal marker and/or the position of the at least one ancillary marker correct errors in the heading direction based on the compass measurements; and

providing the heading direction information to a user interface.

12. The method according to claim 11, further comprising:

storing two or more compass measurements in the data storage memory, each compass measurement taken at a different location when traveling in the heading direction; and

processing the two or more compass measurements using a navigation filter, wherein determining the heading direction information is further based on the processed two or more compass measurements.

13. The method according to claim 12, wherein processing the two or more compass measurements includes computing an average of the two or more compass measurements, and determining the heading direction information is further based on the average.

14. The method according to claim 11, further comprising providing landmark information from a landmark map, at least a portion of the landmark information corresponding to one or more of the reference markers, wherein identifying the at least one principal marker further comprises identifying the at least one principal marker from the landmark information and/or identifying the at least one ancillary marker further comprises identifying the at least one ancillary marker from the landmark information, and determining the heading direction information is further based on the landmark information.

15. The method according to claim 11, wherein the determining includes (a) comparing the position of the at least one principal marker in the one or more navigation images to a subsequent position of the at least one principal marker in one or more subsequent navigation images taken at a time after the one or more navigation images and/or (b) comparing the position of the at least one ancillary marker in the one or more navigation images to a subsequent position of the at least one ancillary marker in one or more subsequent navigation images taken at a time after the one or more navigation images.

16. The method according to claim 11, further comprising providing inertial navigation information from an inertial measurement unit, wherein determining the heading direction information is further based on the inertial navigation information.

17. The method according to claim 16, wherein the inertial navigation information includes a step-count-based estimate for distance traveled by the user or a tread/wheel-turn-based estimate for distance traveled by a vehicle transporting the user.

18. The method according to claim 11, further comprising:

providing altitude information from a pressure altimeter;

accessing a topographic map having topographic map information; and

determining navigation information based on the altitude information and the topographic map information, wherein determining the heading direction information is further based on the navigation information.

19. A computer program product including non-transitory computer-readable medium having computer code thereon for navigating toward a heading direction, the computer code comprising:

program code for providing compass measurements from a compass, the compass measurements configured to orient a user in the heading direction;

program code for capturing a series of successive navigation images in the heading direction oriented by the compass measurements using an image sensor, one or more of the navigation images having at least two reference markers;

program code for storing the series of successive navigation images in a data storage memory;

program code for identifying at least one principal marker from the at least two reference markers, the at least one principal marker positioned within a principal angle relative to the heading direction, oriented by the compass measurements, from a position of the image sensor;

program code for identifying at least one ancillary marker from the at least two reference markers, the at least one ancillary marker positioned within an ancillary angle relative to the heading direction, oriented by the compass measurements, from the position of the image sensor, the ancillary angle greater than the principal angle;

program code for determining heading direction information based on a position of the at least one principal marker and/or a position of the at least one ancillary marker in the successive navigation images, such that the position of the at least one principal marker and/or the position of the at least one ancillary marker correct errors in the heading direction based on the compass measurements; and

program code for providing the heading direction information to a user interface.

20. The computer program product according to claim 19, further comprising:

program code for storing two or more compass measurements in the data storage memory, each compass measurement taken at a different location when traveling in the heading direction; and

program code for processing the two or more compass measurements using a navigation filter, wherein program code for determining the heading direction information is further based on the processed two or more compass measurements.

21. The computer program product according to claim 20, wherein the program code for processing the two or more compass measurements includes computing an average of the two or more compass measurements, and the program code for determining the heading direction information is further based on the average.

22. The computer program product according to claim 19, further comprising program code for providing landmark information from a landmark map, at least a portion of the landmark information corresponding to one or more of the reference markers, wherein the program code for identifying the at least one principal marker further comprises program code for identifying the at least one principal marker from the landmark information and/or the program code for identifying the at least one ancillary marker further comprises program code for identifying the at least one ancillary marker from the landmark information, and the program code for determining the heading direction information is further based on the landmark information.

23. The computer program product according to claim 19, wherein the program code for determining includes program code for comparing (a) the position of the at least one principal marker in the one or more navigation images to a subsequent position of the at least one principal marker in one or more subsequent navigation images taken at a time after the one or more navigation images and/or (b) the position of the at least one ancillary marker in the one or more navigation images to a subsequent position of the at least one ancillary marker in one or more subsequent navigation images taken at a time after the one or more navigation images.

24. The computer program product according to claim 19, further comprising program code for providing inertial navigation information from an inertial measurement unit, wherein the program code for determining the heading direction information is further based on the inertial navigation information.

25. The computer program product according to claim 24, wherein the inertial navigation information includes a step-count-based estimate for distance traveled by the user or a tread/wheel-turn-based estimate for distance traveled by a vehicle transporting the user.

26. The computer program product according to claim 19, further comprising:

program code for providing altitude information from a pressure altimeter;

program code for accessing a topographic map having topographic map information; and

program code for determining navigation information based on the altitude information and the topographic map information, wherein the program code for determining the heading direction information is further based on the navigation information.