Sensor fusion

• US 10,279,480 B1
• Filed: 10/21/2016
• Issued: 05/07/2019
• Est. Priority Date: 10/21/2016
• Status: Active Grant

• Current Assignee: X Development LLC (Alphabet Inc.)
• Original Assignee: X Development LLC (Alphabet Inc.)
• Inventors: Holson, Benjamin, Bingham, Jeffrey, Berkowitz, Ben
• Primary Examiner(s): Mancho, Ronnie M
• Application Number:
  US15/331,423
• Time in Patent Office: 928 Days
• Field of Search: 700245, 700258
• CPC Class Codes: B25J 13/089 Determining the position of the robot with reference to its environment
• CPC Class Codes: B25J 19/0008 Balancing devices
• CPC Class Codes: B25J 5/007 mounted on wheels
• CPC Class Codes: B25J 9/162 Mobile manipulator, movable base with manipulator arm mounted on it
• CPC Class Codes: B25J 9/1694 characterised by use of sensors other than normal servo-feedback from position, speed or acceleration sensors, perception control, multi-sensor controlled systems, sensor fusion
• CPC Class Codes: B62D 61/00 Motor vehicles or trailers, characterised by the arrangement or number of wheels, not otherwise provided for, e.g. four wheels in diamond pattern
• CPC Class Codes: G01C 9/00 Measuring inclination, e.g. by clinometers, by levels

Abstract

Methods, apparatus, and computer readable media applicable to robots, such as balancing robots. Some implementations are directed to determining multiple measures of a property of a robot for a given time and determining a final measure of the property of the robot for the given time based on the multiple measures. One or more control commands may be generated based on the final measure of the property and provided to one or more actuators of the robot.

Claims

1. A method performed by one or more processors of a robot, the method comprising:

determining a first measure of a property of the robot for a first time,

wherein determining the first measure is based on first sensor data from a first group of sensors of the robot,

wherein the first group of the sensors includes at least a given sensor of the sensors of the robot, and

wherein the first sensor data for the given sensor directly indicates a known value, for a given variable, that is utilized in determining the first measure;

determining a second measure of the property of the robot for the first time,

wherein determining the second measure is based on second sensor data from a second group of the sensors of the robot,

wherein the second group of the sensors excludes the given sensor and includes at least one additional sensor that is not included in the first group, and

wherein no known value for the given variable is directly indicated by the second sensor data and wherein no known value for the given variable is utilized in determining the second measure;

determining a final measure of the property of the robot for the first time based on the first measure and the second measure;

generating one or more control commands based on the final measure of the property; and

providing the one or more control commands to one or more actuators of the robot to control the one or more actuators based on the one or more control commands.

2. The method of claim 1, wherein the second group of the sensors includes a plurality of the sensors.

3. The method of claim 1, wherein the first group of the sensors consists of the given sensor, and wherein determining the first measure comprises using the known value as the first measure.

4. The method of claim 3, wherein the second group of the sensors includes a plurality of the sensors.

5. The method of claim 4, wherein determining the second measure comprises solving for the given variable based on application of the second sensor data to a model that is applicable to the robot.

6. The method of claim 1, wherein determining the final measure comprises averaging the first measure and the second measure.

7. The method of claim 1, further comprising:

determining a third measure of the property of the robot for the first time,

wherein determining the third measure is based on third sensor data from a third group of the sensors of the robot,

wherein the third group of the sensors excludes the given sensor and excludes the additional sensor, and

wherein no known value for the given variable is directly indicated by the third sensor data and wherein no known value for the given variable is utilized in determining the third measure;

wherein determining the final measure of the property of the robot for the first time is further based on the third measure.

8. The method of claim 1, wherein the robot is a balancing robot and wherein the property of the robot for the first time is a pitch angle of the balancing robot.

9. The method of claim 8, wherein the first group of the sensors consists of the given sensor, wherein the given sensor is an inclinometer, and wherein determining the first measure comprises using the known value as the first measure.

10. The method of claim 8, wherein the second group of the sensors comprise a plurality of sensors that include an inertial measurement unit.

11. The method of claim 8, wherein the second group of the sensors comprise a plurality of sensors that include a first sensor measuring an actuator property for a first actuator and a second sensor measuring the actuator property for a second actuator.

12. A method performed by one or more processors of a balancing robot, the method comprising:

determining a first measure of a property of the balancing robot for a first time,

wherein determining the first measure comprises determining a solution for an unknown variable and using the solution as the first measure,

wherein determining the solution is based on first sensor data from a first group of sensors of the balancing robot,

wherein the first group of the sensors includes at least a given sensor of the sensors of the balancing robot, and

wherein the first sensor data does not directly indicate the solution for the unknown variable;

determining a second measure of the property of the balancing robot for the first time,

wherein determining the second measure comprises determining an additional solution for the unknown variable and using the additional solution as the second measure;

wherein determining the additional solution is based on second sensor data from a second group of the sensors of the balancing robot,

wherein the second group of the sensors excludes the given sensor and includes at least one additional sensor that is not included in the first group, and

wherein the second sensor data does not directly indicate the additional solution for the unknown variable;

determining a final measure of the property of the balancing robot for the first time based on the first measure and the second measure;

generating one or more control commands based on the final measure of the property; and

providing the one or more control commands to one or more actuators of the balancing robot to control the one or more actuators based on the one or more control commands.

13. The method of claim 12, wherein the first group and the second group each include a plurality of the sensors.

14. The method of claim 12, wherein the sensors of the balancing robot do not include any sensors that directly measure the property of the balancing robot.

15. The method of claim 14, wherein the property of the balancing robot is a force applied by an end effector of the balancing robot.

16. The method of claim 12, wherein determining the solution for the unknown variable comprises determining the solution based on application of the first sensor data to a dynamic model of the balancing robot.

17. The method of claim 12, further comprising:

determining a third measure of the property based on a particular sensor of the sensors that directly measure the property of the balancing robot, wherein the particular sensor is not included in the first group and is not included in the second group;

wherein determining the final measure is further based on the third measure.

18. The method of claim 17, wherein determining the final measure comprises:

using the third measure as the final measure when comparison of the third measure to the first measure and the second measure satisfies one or more criteria.

19. The method of claim 17, wherein determining the final measure comprises determining the final measure based on a statistical combination of the first measure, the second measure, and the third measure.

20. A balancing robot, comprising:

a plurality of actuators;

a plurality of sensors, the sensors omitting any sensor that directly measures a particular property of the balancing robot;

one or more processors receiving sensor data from the sensors, wherein the one or more processors are to;

determine a first measure of the particular property of the balancing robot for a first time,

wherein determining the first measure comprises determining a solution for an unknown variable and using the solution as the first measure, and

wherein determining the solution is based on first sensor data from a first group of the sensors of the balancing robot;

determine a second measure of the particular property of the balancing robot for the first time,

wherein determining the second measure comprises determining an additional solution for the unknown variable and using the additional solution as the second measure, and

wherein determining the additional solution is based on second sensor data from a second group of the sensors of the balancing robot;

determine a final measure of the property of the balancing robot for the first time based on the first measure and the second measure;

generate one or more control commands based on the final measure of the property; and

provide the one or more control commands to one or more of the actuators of the balancing robot to control the one or more of the actuators based on the one or more control commands.