Localization Within an Environment Using Sensor Fusion

• US 20150283700A1
• Filed: 04/02/2014
• Published: 10/08/2015
• Est. Priority Date: 04/02/2014
• Status: Active Grant

• Current Assignee: The Boeing Co.
• Original Assignee: The Boeing Co.
• Inventors: Trautman, Peter F., Li, Hui, Higgins, Robert P.
• Granted Patent: US 9,427,867 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: B25J 9/16 Programme controls programme controls in general G05B19/00, e.g. numerical pogramme controls G05B19/18; recording or playback systems G05B19/42
• CPC Class Codes: G01C 21/1652 with ranging devices, e.g. LIDAR or RADAR
• CPC Class Codes: G01C 21/1656 with passive imaging devices, e.g. cameras
• CPC Class Codes: G01C 21/28 with correlation of data from several navigational instruments
• CPC Class Codes: G05D 1/0236 in combination with a laser lasers per se H01S
• CPC Class Codes: G05D 1/0248 in combination with a laser lasers per se H01S
• CPC Class Codes: G05D 1/0253 extracting relative motion information from a plurality of images taken successively, e.g. visual odometry, optical flow determining position or orientation from images G06T7/70
• CPC Class Codes: G05D 1/0272 comprising means for registering the travel distance, e.g. revolutions of wheels measuring distance traversed on the ground by vehicles, e.g. using odometers G01C22/00
• CPC Class Codes: G05D 1/0274 using mapping information stored in a memory device navigation using map-matching G01C21/30
• CPC Class Codes: Y10S 901/01 Mobile robot

Abstract

A method and apparatus for guiding a mobile platform within an environment may be provided. A number of first type of data streams and a number of second type of data streams may be generated using a plurality of data systems. A probability distribution may be applied to each of the number of second type of data streams to form a number of modified data streams. The number of first type of data streams and the number of modified data streams may be fused to generate a pose estimate with a desired level of accuracy for the mobile platform with respect to an environment around the mobile platform.

Claims

1. An apparatus comprising:

a plurality of data systems configured to generate a plurality of data streams in which the plurality of data streams includes a number of first type of data streams and a number of second type of data streams;

a modifier configured to apply a probability distribution to each of the number of second type of data streams to form a number of modified data streams; and

a pose estimator located onboard a mobile platform and configured to receive and fuse the number of first type of data streams and the number of modified data streams to generate a pose estimate with a desired level of accuracy for the mobile platform with respect to an environment around the mobile platform;

wherein the number of second type of data streams are generated by a number of off-board data systems of the plurality of data systems.

2. The apparatus of claim 1, wherein at least one of the number of second type of data streams is received from an odometry system in the plurality of data systems.

3. The apparatus of claim 1, wherein an onboard data system in the plurality of data systems is used to observe a landmark in the environment a selected number of times for use in moving the mobile platform to a desired location within the environment.

4. The apparatus of claim 3, wherein increasing the selected number of times that the landmark is observed reduces an overall error in moving the mobile platform to the desired location within the environment.

5. The apparatus of claim 1, wherein the plurality of data systems includes a number of onboard data systems and a number of off-board data systems.

6. The apparatus of claim 5, wherein the number of onboard data systems includes at least one of an inertial measurement unit, a light detection and ranging system, a color and depth odometry system, a wheel odometry system, a visual odometry system, or a localization and mapping system.

7. The apparatus of claim 5, wherein the number of off-board data systems includes at least one of an indoor global positioning system, a motion capture system, or a laser system.

8. The apparatus of claim 1, wherein the pose estimate comprises a position and an orientation of the mobile platform with respect to the environment.

9. The apparatus of claim 1, wherein the mobile platform is a mobile robot and the environment is a manufacturing environment.

10. The apparatus of claim 1, wherein the pose estimator is configured to fuse the number of first type of data streams and the number of modified data streams using a Bayesian estimation algorithm to generate the pose estimate.

11. The apparatus of claim 1 further comprising:

a controller configured to use the pose estimate to guide the mobile platform along a path within the environment.

12. The apparatus of claim 11, wherein the mobile platform comprises:

a movement system configured to be controlled by the controller based on the pose estimate.

25. The apparatus of claim 1, further comprising:

a number of tools to perform a number of operations on an object;

wherein the object is one of a door for an aircraft, a skin panel for the aircraft, a wing for the aircraft, and a fuselage for the aircraft;

wherein each data point included by the first type of data streams includes a measurement of uncertainty based on a probability distribution;

wherein each data point included by the second type of data streams does not include a measurement of uncertainty;

wherein the number of first type of data streams are generated by the number of off-board data systems of the plurality of data systems and by a number of onboard data systems of the plurality of data systems;

wherein the number of second type of data streams are generated by the number of off-board data systems of the plurality of data systems and by the number of onboard data systems of the plurality of data systems;

wherein the number of onboard data systems includes an inertial measurement unit, a light detection and ranging system, a color and depth odometry system, a wheel odometry system, a visual odometry system, and a localization and mapping system; and

wherein the number of off-board data systems includes an indoor global positioning system, a motion capture system, and a laser system.

13. A mobile platform comprising:

a base;

a controller associated with the base and configured to receive data streams from a plurality of data systems in which the data streams include a number of first type of data streams and a number of second type of data streams and in which the controller comprises;

a modifier configured to apply a probability distribution to each of the number of second type of data streams to form a number of modified data streams; and

a pose estimator configured to receive the number of first type of data streams and the number of modified data streams as a plurality of data streams and to fuse the plurality of data streams together to generate a pose estimate with a desired level of accuracy for the mobile platform with respect to an environment around the mobile platform; and

a movement system associated with the base and configured to be controlled by the controller based on the pose estimate to move the mobile platform within the environment;

wherein the number of second type of data streams are generated by a number of off-board data systems of the plurality of data systems.

14. A method for guiding a mobile platform within an environment, the method comprising:

generating a number of first type of data streams and a number of second type of data streams using a plurality of data systems;

applying a probability distribution to each of the number of second type of data streams to form a number of modified data streams; and

fusing the number of first type of data streams and the number of modified data streams to generate a pose estimate with a desired level of accuracy for the mobile platform with respect to the environment around the mobile platform;

wherein the number of second type of data streams are generated by a number of off-board data systems of the plurality of data systems.

15. The method of claim 14 further comprising:

guiding the mobile platform along a path in the environment using the pose estimate generated for the mobile platform.

16. The method of claim 14, wherein applying the probability distribution to the each of the number of second type of data streams to form the number of modified data streams comprises:

applying an empirical covariance to each data point in the each of the number of second type of data streams to form the number of modified data streams.

17. The method of claim 14, wherein fusing the number of first type of data streams and the number of modified data streams comprises:

fusing the number of first type of data streams and the number of modified data streams using a Bayesian estimation algorithm to generate the pose estimate with the desired level of accuracy.

18. The method of claim 14, wherein generating the number of first type of data streams and the number of second type of data streams comprises:

generating the number of second type of data streams using at least one of a color and depth odometry system, a wheel odometry system, a visual odometry system, a light detection and ranging system, or a localizer and mapper.

19. The method of claim 14 further comprising:

observing a current landmark in the environment a selected number of times using an onboard data system, while the mobile platform is at a current location;

identifying an initial relative distance between the current location of the mobile platform and the current landmark; and

moving the mobile platform from the current location to a new location as far as possible towards a desired location without losing the current landmark within a field of view of a sensor system of the onboard data system.

20. The method of claim 19 further comprising:

re-observing the current landmark the selected number of times using the onboard data system;

identifying a new relative distance between the new location of the mobile platform and the current landmark;

computing a difference between the initial relative distance and the new relative distance; and

identifying the new location of the mobile platform in the environment using the difference.

21. The method of claim 20 further comprising:

determining whether the new location is at the desired location within selected tolerances; and

moving the mobile platform closer to the desired location using a new landmark in response to a determination that the new location is not at the desired location within selected tolerances.

22. The method of claim 21, wherein moving the mobile platform closer to the desired location using the new landmark comprises:

searching for the new landmark using the onboard data system;

observing the new landmark the selected number of times using the onboard data system, while the mobile platform is at the new location; and

identifying a relative distance between the new location of the mobile platform and the new landmark.

23. The method of claim 21, wherein moving the mobile platform closer to the desired location using the new landmark further comprises:

re-identifying the new landmark as the current landmark, the new location as the current location, and the relative distance as the initial relative distance; and

repeating the steps of moving the mobile platform from the current location to the new location, identifying the new relative distance between the new location of the mobile platform and the current landmark, computing the difference between the initial relative distance and the new relative distance, identifying the new location of the mobile platform in the environment using the difference, and determining whether the new location is at the desired location within the selected tolerances.

24. The method of claim 23 further comprising:

repeating the step of moving the mobile platform closer to the desired location using the new landmark in response to the determination that the new location is not at the desired location within selected tolerances until the mobile platform reaches the desired location within the selected tolerances.