Egocentric odometry system for maintaining pose alignment between real and virtual worlds

• US 10,325,411 B1
• Filed: 02/01/2018
• Issued: 06/18/2019
• Est. Priority Date: 12/13/2017
• Status: Active Grant

• Current Assignee: Charles Stark Draper Laboratory Incorporated
• Original Assignee: Charles Stark Draper Laboratory Incorporated
• Inventors: Laney, James, Madison, Richard W., Truax, Robert, Steiner, III, Theodore J., Jones, Eric
• Primary Examiner(s): Martello, Edward
• Application Number:
  US15/886,532
• Time in Patent Office: 502 Days
• CPC Class Codes: G01C 21/1652 with ranging devices, e.g. LIDAR or RADAR
• CPC Class Codes: G01C 21/1656 with passive imaging devices, e.g. cameras
• CPC Class Codes: G01C 21/20 Instruments for performing navigational calculations G01C21/24, G01C21/26 take precedence
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/03 Arrangements for converting the position or the displacement of a member into a coded form
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06T 19/003 Navigation within 3D models or images
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00

Abstract

A navigation system provides pose, i.e., location and orientation, solutions using best available location information from two or more location systems. One of the location systems is a fiducial-based location system, which is accurate when a sufficient number of fiducials is recognized. However, when an insufficient number of fiducials is recognized, an odometry-based location system is used. Although the odometry-based location system is subject to drift, when a sufficient number of fiducials is recognized, the fiducial-based location system is used to correct the odometry-based location system. The navigation system provides robust, accurate and timely pose solutions, such as for augmented reality (AR) or virtual reality (VR) systems, without the time-consuming requirement to establish and localize many fiducials or the computational and memory requirements of pure fiducial-based location systems, and without the inherent drift of pure odometry-based location systems.

Claims

1. A navigation system for operation within a reference frame, the navigation system comprising:

a fiducial-based location system configured to use fiducials that are fixed, relative to the reference frame, to provide fiducial-based pose information about the navigation system, relative to the reference frame, wherein the reference frame is moveable within a global reference frame;

an inertial odometry-based location system configured to automatically provide inertial odometry-based pose information about the navigation system;

a third location system, distinct from the inertial odometry-based location system and distinct from the fiducial-based location system, the third location system being configured to automatically provide information about pose of the reference frame, relative to the global reference frame; and

a navigation filter that is;

coupled to the fiducial-based location system to receive the fiducial-based pose information therefrom;

coupled to the inertial odometry-based location system to receive the inertial odometry-based pose information therefrom;

coupled to the third location system to receive therefrom the information about the pose of the reference frame, relative to the global reference frame; and

configured to repeatedly automatically provide pose solutions, wherein;

at least some of the pose solutions are based at least in part on the inertial odometry-based pose information; and

each pose solution that is based at least in part on the inertial odometry-based pose information is compensated for the pose of the reference frame, relative to the global reference frame.

2. A navigation system according to claim 1, wherein the navigation filter is further configured to correct the inertial odometry-based location system based at least in part on the fiducial-based pose information when a sufficient number of the fiducials is recognized by the fiducial-based location system to provide the fiducial-based pose information.

3. A navigation system according to claim 1, wherein the fiducial-based location system comprises an optical fiducial-based location system.

4. A navigation system according to claim 1, wherein the fiducial-based location system comprises a radio frequency fiducial-based location system.

5. A navigation system according to claim 1, wherein the navigation filter is configured to provide the pose solutions to a mixed reality system.

6. A navigation system according to claim 1, wherein the third location system comprises an inertial odometry-based location system.

7. A navigation system according to claim 1, wherein:

the fiducial-based location system comprises a model of an environment;

the model includes identifications of a plurality of physical features within the environment; and

each physical feature of the plurality of physical features represents a respective one of the fiducials.

8. A navigation system according to claim 1, further comprising:

a fourth location system, distinct from the fiducial-based location system, distinct from the inertial odometry-based location system and distinct from the third location system; and

wherein each pose solution is based at least in part on location information from the fourth location system.

9. A navigation system according to claim 8, wherein the fourth location system comprises one or more of:

a global positioning system receiver;

a LiDAR;

an atmospheric pressure-based location system; and

a machine-learning-based localization system.

10. A navigation system according to claim 1,

wherein the navigation filter is configured such that:

at least some of the pose solutions are based at least in part on the fiducial-based pose information; and

each pose solution that is based at least in part on the fiducial-based pose information is based at least in part on the fiducial-based pose information when a sufficient number of the fiducials is recognized by the fiducial-based location system to provide the fiducial-based pose information.

11. A system according to claim 1, wherein the navigation filter is further configured to correct the inertial odometry-based pose information based at least in part on the fiducial-based pose information when a sufficient number of the fiducials is recognized to provide the fiducial-based pose information.

12. A method for operating a navigation system within a reference frame, the method comprising:

automatically providing, by a fiducial-based location system configured to use fiducials that are fixed, relative to the reference frame, fiducial-based pose information about the navigation system, relative to the reference frame, wherein the reference frame is moveable within a global reference frame;

automatically providing, by an inertial odometry-based location system, inertial odometry-based pose information about the navigation system;

automatically providing, by a third location system, distinct from the fiducial-based location system and distinct from the inertial odometry-based location system, information about pose of the reference frame, relative to the global reference frame; and

repeatedly automatically providing, by a navigation filter, pose solutions, wherein at least some of the pose solutions are based at least in part on the inertial odometry-based pose information, including;

automatically compensating each pose solution that is based at least in part on the inertial odometry-based pose information for the pose of the reference frame, relative to the global reference frame.

13. A method according to claim 12, further comprising correcting the inertial odometry-based pose information based at least in part on the fiducial-based pose information when a sufficient number of the fiducials is recognized to provide the fiducial-based pose information.

14. A method according to claim 12, wherein using the fiducials that are fixed, relative to the reference frame, to provide the fiducial-based pose information comprises using an optical fiducial-based location system.

15. A method according to claim 12, wherein using the fiducials that are fixed, relative to the reference frame, to provide the fiducial-based pose information comprises using a radio frequency fiducial-based location system.

16. A method according to claim 12, further comprising providing the pose solutions to a mixed reality system.

17. A method according to claim 12, wherein automatically providing, by the third location system, the information about the pose of the reference frame, relative to the global reference frame, comprises using an inertial odometry-based location system.

18. A method according to claim 12, wherein providing the fiducial-based pose information comprises using a model of an environment, wherein the model includes identifications of a plurality of physical features within the environment, and each physical feature of the plurality of physical features represents a respective one of the fiducials.

19. A method according to claim 12, further comprising:

providing additional location information from a location system, independently of the fiducials, independently of the fiducial-based pose information, independently of the inertial odometry-based pose information and independently of the information about the pose of the reference frame; and

wherein providing the pose solutions comprises providing the pose solutions based at least in part on the additional location information.

20. A method according to claim 19, wherein providing the additional location information from the location system comprises providing the additional location information by one or more of:

a global positioning system receiver;

a LiDAR;

an atmospheric pressure-based location system; and

a machine-learning-based localization system.

22. A method according to claim 12, further comprising:

automatically attempting to recognize, by the fiducial-based location system, a sufficient number of the fiducials to provide the fiducial-based pose information;

when the sufficient number of the fiducials is recognized by the fiducial-based location system, automatically providing, by the fiducial-based location system, the fiducial-based pose information, independently of the inertial odometry-based navigation system;

wherein automatically providing the inertial odometry-based pose information about the navigation system comprises automatically providing, by the inertial odometry-based location system, the inertial odometry-based pose information about the navigation system independently of the fiducials and independently of the fiducial-based pose information; and

wherein repeatedly automatically providing, by the navigation filter, the pose solutions comprises basing a given pose solution at least in part on the fiducial-based pose information when the sufficient number of the fiducials is recognized.

21. A non-transitory computer-readable medium encoded with instructions that, when executed by a processor, establish processes for performing a computer-implemented method for operating a navigation system within a reference frame, the processes comprising:

a process configured to automatically provide, by a fiducial-based location system configured to use fiducials that are fixed, relative to the reference frame, fiducial-based pose information about the navigation system, relative to the reference frame, wherein the reference frame is moveable within a global reference frame;

a process configured to automatically provide, by an inertial odometry-based location system, inertial odometry-based pose information about the navigation system;

a process configured to automatically provide, by a third location system, distinct from the inertial odometry-based location system and distinct from the fiducial-based location system, information about pose of the reference frame, relative to the global reference frame; and

a process configured to repeatedly provide, by a navigation filter, pose solutions, wherein at least some of the pose solutions are based at least in part on the inertial odometry-based pose information, including;

automatically compensating each pose solution that is based at least in part on the inertial odometry-based pose information for the pose of the reference frame.