Ground truth estimation for autonomous navigation

• US 10,612,939 B2
• Filed: 01/02/2014
• Issued: 04/07/2020
• Est. Priority Date: 01/02/2014
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Kikkeri, Harshavardhana, Birchfield, Stanley T., Jalobeanu, Mihai
• Primary Examiner(s): Black, Thomas G
• Assistant Examiner(s): Smith-Stewart, Demetra R
• Application Number:
  US14/146,494
• Publication Number:
  US 20150185027A1
• Time in Patent Office: 2,287 Days
• Field of Search: None
• CPC Class Codes: G01C 25/00 Manufacturing, calibrating, cleaning, or repairing instruments or devices referred to in the other groups of this subclass testing, calibrating or compensating compasses G01C17/38
• CPC Class Codes: G05D 1/0088 characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours using knowledge based models G06N5/00
• CPC Class Codes: G05D 1/027 comprising intertial navigation means, e.g. azimuth detector inertial navigation G01C21/16; inertial navigation combined with non-inertial navigation instruments G01C21/165
• CPC Class Codes: G05D 1/0278 using satellite positioning signals, e.g. GPS

Abstract

A system and method for ground truth estimation of an autonomous navigation system is described. The method includes calibrating a ground truth estimation system for the navigator, by determining a calibration pose of the navigator as disposed in relation to each of a plurality of landmarks during a calibration period. The method also includes directing the navigator to travel to a sequence of waypoints, the waypoints including a selected sequence of the landmarks. The method further includes determining the ground truth estimation based on an accuracy pose of the navigator as disposed in relation to the sequence of landmarks, and the calibration poses for the sequence of landmarks.

Claims

1. A method for ground truth estimation for an autonomous navigation system (navigator), the method comprising:

calibrating a ground truth estimation system for the navigator, by determining a calibration pose of a camera of the navigator as disposed in relation to each of a plurality of landmarks during a calibration period;

directing the navigator to travel to a sequence of waypoints comprising a sequence of the landmarks during an evaluation period; and

determining, using the calibrated ground truth estimation system, a ground truth estimation of the navigator based on an accuracy pose of the camera of the navigator as disposed in relation to the waypoints during the evaluation period, and the calibration poses disposed in relation to the plurality of landmarks.

2. The method recited in claim 1, comprising:

mounting the landmarks in a plurality of spaces above a path of an environment the navigator is navigating; and

mounting an upward-facing camera on the navigator.

3. The method of claim 2, comprising:

mounting the landmarks in a plurality of spaces associated with the path of the environment the navigator is navigating; and

mounting on the navigator, a camera facing in a direction of the spaces as the navigator navigates the path.

4. The method recited in claim 1, comprising generating a map during the calibration period, the map comprising a navigable environment comprising the landmarks, the map comprising an association between the landmarks and locations of the map.

5. The method recited in claim 1, wherein calibrating the ground truth estimation system, and determining the ground truth estimation, comprises:

capturing an image of a landmark from an upward-sensing sensor of the navigator; and

estimating a pose of the upward-sensing sensor in relation to the landmark.

6. The method recited in claim 1, wherein determining the ground truth estimation comprises:

comparing the accuracy poses and the calibration poses; and

determining the ground truth estimation based on a similarity of the accuracy poses to the calibration poses.

7. The method of claim 1, the navigator comprising a robot, the robot traveling during the calibration period and the evaluation period in response to the directing, and in response to a stimulus associated with an environment the navigator is navigating.

8. The method of claim 1, determining a pose comprising estimating a tilt angle and an azimuth angle of an optical axis of the camera mounted on a robot comprising the navigator, the tilt angle and the azimuth angle being with respect to an x,y,z frame centered on wheels of the robot.

9. The method of claim 1, comprising estimating noise in a sensor by running a robot between landmarks multiple times, and calculating a variance of error between different runs for the sensor.

10. The method recited in claim 1, comprising determining a tilt of a floor the navigator is navigating during the calibration period.

11. A system for ground truth estimation, comprising:

a processing unit; and

a system memory, wherein the system memory comprises code configured to direct the processing unit to;

calibrate a ground truth estimation system for an autonomous navigation system (navigator), by determining a calibration pose of a camera of the navigator as disposed in relation to each of a plurality of landmarks during a calibration period, the landmarks mounted on a ceiling above an environment navigated by the navigator;

direct the navigator to travel to a sequence of waypoints comprising a sequence of the landmarks; and

determine, using the calibrated ground truth estimation system, a ground truth estimation of the navigator based on an accuracy pose of the camera of the navigator as disposed in relation to the waypoints during an evaluation period, and the calibration poses disposed in relation to the plurality of landmarks.

12. The system recited in claim 11, the navigator comprising:

an upward-facing sensor;

a navigator processing unit; and

a navigator system memory, comprising code configured to direct the navigator processing unit to;

travel, in response to commands from the ground truth estimation system, to all locations associated with the landmarks during the calibration period;

capture an image of a landmark associated with each of the locations when arriving at each of the locations; and

notify the ground truth estimation system the navigator is stopped and located in proximity with the landmark.

13. The system recited in claim 12, comprising generating a map during the calibration period, the map comprising the environment navigated by the navigator comprising the landmarks, the map comprising an association between the landmarks and the locations.

14. The system recited in claim 12, wherein the ground truth estimation system is calibrated by:

for all the landmarks, capturing an image of each landmark from an upward-sensing sensor of the navigator;

for all the landmarks, estimating a pose of the upward-sensing sensor in relation to each landmark; and

transforming an x-y-z coordinate system of the upward-sensing sensor to an x-y-z coordinate system of the navigator.

15. The system recited in claim 12, the navigator comprising a robot, the robot traveling during the calibration period and the evaluation period in response to the directing, and in response to a stimulus associated with the environment the navigator is navigating.

16. The system recited in claim 15, the environment navigated by the navigator, comprising a floor location directly beneath one landmark, the floor location comprising different surfaces, wherein positioning the navigator directly below the landmark comprises locating a roller of the navigator on all the different surfaces, the ground truth estimation system compensating for the different surfaces in relation to determining a calibration pose associated with the landmark, and in relation to determining an accuracy pose associated with the landmark.

17. The system recited in claim 15, the environment navigated by the navigator, comprising a floor location directly beneath one landmark, the floor location comprising different surfaces, wherein positioning the navigator directly below the landmark comprises locating a roller of the navigator on all the different surfaces, the ground truth estimation system compensating for the different surfaces by directing the robot to exclude the one landmark.

18. The system recited in claim 11, wherein a ground truth is estimated, by:

for a specified set of the landmarks, capturing an image of each landmark from an upward-sensing sensor of the navigator;

for the specified set of the landmarks, estimating a pose of the upward-sensing sensor in relation to each landmark;

comparing the accuracy poses and the calibration poses; and

determining the ground truth estimation based on a similarity of the accuracy poses to the calibration poses.