Tagged robot sensor data

• US 10,377,039 B1
• Filed: 09/19/2018
• Issued: 08/13/2019
• Est. Priority Date: 09/10/2015
• Status: Active Grant

• Current Assignee: X Development LLC (Alphabet Inc.)
• Original Assignee: X Development LLC (Alphabet Inc.)
• Inventors: Gennis, Jamie Moline, Hentz, Andre, Lam, Daniel
• Primary Examiner(s): Abu Roumi, Mahran Y
• Application Number:
  US16/135,497
• Time in Patent Office: 328 Days
• Field of Search: 709223
• CPC Class Codes: B25J 9/1656 characterised by programming, planning systems for manipulators
• CPC Class Codes: H04L 41/08 Configuration management of networks or network elements address allocation H04L61/50
• CPC Class Codes: H04L 41/085 Retrieval of network configuration; Tracking network configuration history
• CPC Class Codes: H04L 41/0896 Bandwidth or capacity management, i.e. automatically increasing or decreasing capacities flow or congestion control using dynamic resource allocation, e.g. in-call renegotiation, H04L47/76
• CPC Class Codes: H04L 67/1029 using data related to the state of servers by a load balancer
• CPC Class Codes: Y10S 901/46 Sensing device

Abstract

Methods, apparatus, systems, and computer-readable media are provided for creating, storing, and/or offloading tagged robot sensor data. In various implementations, a first plurality of sensor data points that are sampled by one or more sensors associated with a robot and that share a first attribute may be identified. Each of the first plurality of sensor data points may be tagged with a first tag, which may be indicative of the first attribute. A context in which a robot is operating may be identified. A first transport rule that governs how sensor data points tagged with the first tag are treated when the robot operates in the context may then be identified. At least a subset of the first plurality of tagged sensor data points may then be offloaded from the robot and/or stored locally on the robot pursuant to the first transport rule.

Claims

1. A method comprising:

identifying, by a robot, first and second pluralities of sensor data points that are generated during operation of the robot while the robot performs one or more tasks, wherein the first plurality of sensor data points share a first attribute and the second plurality of sensor data points share a second attribute that is different than the first attribute, and wherein the sensor data points are generated by one or more cameras integral with the robot or one or more light sensors integral with the robot;

tagging, by the robot, each of the identified first plurality of sensor data points with a first tag, wherein the first tag is indicative of the first attribute shared by the first plurality of sensor data points;

tagging, by the robot, each of the identified second plurality of sensor data points with a second tag, wherein the second tag is indicative of the second attribute shared by the second plurality of sensor data points, and wherein the second tag also indicates a lower priority than the first tag;

identifying, by the robot, a context in which the robot is operating;

identifying, by the robot, a first transport rule that governs how sensor data points tagged with the first tag are treated when the robot operates in the context;

identifying, by the robot, a second transport rule that governs how sensor data points tagged with the second tag are treated when the robot operates in the context;

offloading, by the robot to a remote computing device via a network connection, at least a subset of the first plurality of tagged sensor data points pursuant to the first transport rule, wherein the offloading comprises transmitting at least the subset of the first plurality of tagged sensor data points to the remote computing device in a manner selected based on the context in which the robot is operating; and

overwriting, by the robot, at least a subset of the second plurality of tagged sensor data points pursuant to the second transport rule.

2. The method of claim 1, wherein the subset is selected from the first plurality of tagged sensor data points based on the context in which the robot is operating.

3. The method of claim 1, wherein the network connection is selected based on the context.

4. The method of claim 1, wherein the context comprises available communication bandwidth between the robot and the remote computing device.

5. The method of claim 1, wherein the context comprises reliability of the network connection.

6. The method of claim 1, wherein the context comprises power available to the robot.

7. The method of claim 1, wherein the context comprises available memory in the robot.

8. The method of claim 1, wherein the context comprises an activity level of the robot.

9. The method of claim 1, wherein the one or more vision sensors include a proximity sensor.

10. A method comprising:

identifying, by a robot, first and second pluralities of sensor data points that are generated during operation of the robot while the robot performs one or more tasks, wherein the first plurality of sensor data points share a first attribute and the second plurality of sensor data points share a second attribute that is different than the first attribute, and wherein the sensor data points are generated by one or more cameras integral with the robot or one or more pressure wave sensors integral with the robot;

tagging, by the robot, each of the identified first plurality of sensor data points with a first tag, wherein the first tag is indicative of the first attribute shared by the first plurality of sensor data points;

tagging, by the robot, each of the identified second plurality of sensor data points with a second tag, wherein the second tag is indicative of the second attribute shared by the second plurality of sensor data points, and wherein the second tag also indicates a lower priority than the first tag;

identifying, by the robot, a context in which the robot is operating;

identifying, by the robot, a first transport rule that governs how sensor data points tagged with the first tag are treated when the robot operates in the context;

identifying, by the robot, a second transport rule that governs how sensor data points tagged with the second tag are treated when the robot operates in the context;

offloading, by the robot to a remote computing device via a network connection, at least a subset of the first plurality of tagged sensor data points pursuant to the first transport rule, wherein the offloading comprises transmitting at least the subset of the first plurality of tagged sensor data points to the remote computing device in a manner selected based on the context in which the robot is operating; and

overwriting, by the robot, at least a subset of the second plurality of tagged sensor data points pursuant to the second transport rule.

11. The method of claim 10, wherein the subset is selected from the first plurality of tagged sensor data points based on the context in which the robot is operating.

12. The method of claim 10, wherein the network connection is selected based on the context.

13. The method of claim 10, wherein the context comprises available communication bandwidth between the robot and the remote computing device.

14. The method of claim 10, wherein the context comprises reliability of the network connection.

15. The method of claim 10, wherein the context comprises power available to the robot.

16. The method of claim 10, wherein the context comprises available memory in the robot.

17. The method of claim 10, wherein the context comprises an activity level of the robot.

18. A method comprising:

identifying, by a robot, first and second pluralities of sensor data points that are generated during operation of the robot while the robot performs one or more tasks, wherein the first plurality of sensor data points share a first attribute and the second plurality of sensor data points share a second attribute that is different than the first attribute, and wherein the sensor data points are generated by two or more of an accelerometer integral with the robot, a gyroscope integral with the robot, a force sensor integral with the robot, and a torque sensor integral with the robot;

tagging, by the robot, each of the identified first plurality of sensor data points with a first tag, wherein the first tag is indicative of the first attribute shared by the first plurality of sensor data points;

tagging, by the robot, each of the identified second plurality of sensor data points with a second tag, wherein the second tag is indicative of the second attribute shared by the second plurality of sensor data points, and wherein the second tag also indicates a lower priority than the first tag;

identifying, by the robot, a context in which the robot is operating;

identifying, by the robot, a first transport rule that governs how sensor data points tagged with the first tag are treated when the robot operates in the context;

identifying, by the robot, a second transport rule that governs how sensor data points tagged with the second tag are treated when the robot operates in the context;

offloading, by the robot to a remote computing device via a network connection, at least a subset of the first plurality of tagged sensor data points pursuant to the first transport rule, wherein the offloading comprises transmitting at least the subset of the first plurality of tagged sensor data points to the remote computing device in a manner selected based on the context in which the robot is operating; and

overwriting, by the robot, at least a subset of the second plurality of tagged sensor data points pursuant to the second transport rule.