Operating multiple testing robots based on robot instructions and/or environmental parameters received in a request

• US 10,058,995 B1
• Filed: 07/08/2016
• Issued: 08/28/2018
• Est. Priority Date: 07/08/2016
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: X Development LLC (Alphabet Inc.)
• Inventors: Sampedro, Peter Pastor, Kalakrishnan, Mrinal, Valdovinos, Ali Yahya, Li, Adrian, Konolige, Kurt, Dureau, Vincent
• Primary Examiner(s): Figueroa, Jaime
• Application Number:
  US15/205,760
• Time in Patent Office: 781 Days
• Field of Search: 700245, 700248
• CPC Class Codes: B25J 9/0084 comprising a plurality of manipulators
• CPC Class Codes: B25J 9/163 learning, adaptive, model based, rule based expert control
• CPC Class Codes: G05B 2219/39271 Ann artificial neural network, ffw-nn, feedforward neural network

Abstract

Methods and apparatus related to receiving a request that includes robot instructions and/or environmental parameters, operating each of a plurality of robots based on the robot instructions and/or in an environment configured based on the environmental parameters, and storing data generated by the robots during the operating. In some implementations, at least part of the stored data that is generated by the robots is provided in response to the request and/or additional data that is generated based on the stored data is provided in response to the request.

Claims

1. A method, comprising:

receiving, by one or more processors via one or more network interfaces, a request that includes robot instructions and environmental parameters,

wherein the robot instructions specify at least part of operation of the robot, and

wherein the environmental parameters indicate one or more properties of environmental objects that are external to the robot and that are to be interacted with by the robot during operation of the robot based on the robot instructions;

for each of a plurality of given robots, configuring an operating space of the given robot based on the environmental parameters so that at least one environmental object having at least one of the properties is present in an operating space of the robot;

operating each of the given robots based on the robot instructions after configuring the operating space based on the environmental parameters;

storing, by one or more of the processors in one or more computer readable media, data generated by the given robots during the operation of the given robots based on the robot instructions; and

providing, in response to the request and by one or more of the processors via one or more of the network interfaces, information based on the data.

2. The method of claim 1, wherein the properties indicated by the environmental parameters include one or more pose constraints for the environmental objects and wherein configuring the operating space of the given robot based on the environmental parameters comprises:

positioning the environmental object in the operating space based on at least one of the pose constraints.

3. The method of claim 1, wherein the properties indicated by the environmental parameters include a class having a plurality of the environmental objects as members and wherein configuring the operating space of the given robot based on the environmental parameters comprises:

selecting the environmental object based on the environmental object being a member of the class.

4. The method of claim 1, wherein configuring the operating space of the given robot based on the environmental parameters is performed by the given robot or by one or more additional robots.

5. The method of claim 1, wherein the robot instructions comprise object code that is executable by the one or more processors of the robot.

6. The method of claim 1, wherein the robot instructions comprise source code, and wherein a compilation of the source code is executable by the one or more processors of the robot.

7. The method of claim 1, wherein the robot instructions comprise a machine learning model and wherein operating the given robot of the given robots based on the robot instructions comprises:

applying input to the machine learning model;

generating an output over the machine learning model based on the input; and

using the output in operating the given robot.

8. The method of claim 1, wherein the data comprises sensor data generated by sensors of the given robots and further comprising:

receiving a machine learning model;

generating training examples based on the sensor data and based on input parameters and output parameters of the machine learning model;

training the machine learning model based on the training examples to generate an updated machine learning model;

wherein providing information based on the data comprises providing the updated machine learning model as at least some of the information based on the data.

9. The method of claim 1, wherein the data comprises sensor data generated by sensors of the testing robots and further comprising:

generating training examples based on the sensor data and based on input parameters and output parameters of a machine learning model;

wherein providing information based on the data comprises providing the training examples as at least some of the information based on the data.

10. The method of claim 1, further comprising:

configuring a simulated operating space of a robot simulator based on the environmental parameters;

operating the robot simulator based on the robot instructions with the simulated operating space configured based on the environmental parameters;

storing, by one or more of the processors in one or more of the computer readable media, simulated data generated by the robot simulator during the operation of the robot simulator based on the robot instructions;

wherein the information is further based on the simulated data.

11. A method, comprising:

receiving user interface input generated via a user interface input device of a client computing device;

generating, based on the user interface input, a request that includes robot instructions and environmental parameters,

wherein the robot instructions specify at least part of operation of the robot, and

wherein the environmental parameters indicate one or more properties of environmental objects that are external to the robot and that are to be interacted with by the robot during operation of the robot based on the robot instructions;

transmitting the request via one or more network interfaces of the client computing device; and

receiving, by the client computing device or an additional computing device in response to the request;

sensor data generated by sensors of a plurality of robots each operated based on the robot instructions of the request in an operating environment configured based on the environmental parameters of the request, or

additional data generated based on the sensor data.

12. The method of claim 11, wherein the receiving comprises receiving the additional data generated based on the sensor data.

13. The method of claim 12, wherein the request further includes a machine learning model, wherein the additional data comprises an updated machine learning model, and wherein the updated machine learning model is updated based on training of the machine learning model based on training examples generated based on the sensor data.

14. A method, comprising:

receiving, by one or more processors via one or more network interfaces, a request that includes robot instructions and a machine learning model,

wherein the robot instructions in combination with the machine learning specify at least part of operation of the robot;

operating each of a plurality of given robots based on the robot instructions and the machine learning model;

storing, by one or more of the processors in one or more computer readable media, sensor data generated by the given robots during the operation of the given robots based on the robot instructions and the machine learning model;

generating training examples for the machine learning model based on the sensor data generated by the given robots during the operation of the given robots based on the robot instructions and the machine learning model; and

training the machine learning model based on the training examples to create an updated machine learning model.

15. The method of claim 14, wherein the request further includes environmental parameters, and further comprising:

for each of the given robots prior to operating the given robots based on the robot instructions and the machine learning model, configuring an operating space of the given robot based on the environmental parameters.

16. The method of claim 15, wherein the environmental parameters indicate one or more pose constraints for the environmental objects and wherein configuring the operating space of the given robot based on the environmental parameters comprises:

positioning the environmental object in the operating space based on at least one of the pose constraints.

17. The method of claim 15, wherein the environmental parameters indicate a class having a plurality of the environmental objects as members and wherein configuring the operating space of the given robot based on the environmental parameters comprises:

selecting the environmental object based on the environmental object being a member of the class.

18. The method of claim 14, further comprising:

operating a robot simulator based on the robot instructions and the machine learning model;

storing, by one or more of the processors in one or more of the computer readable media, simulated sensor data generated by the robot simulator during the operation of the robot simulator;

generating initial training examples for the machine learning model based on the simulated sensor data;

training the machine learning model based on the initial training examples to create an initial updated machine learning model;

wherein operating the plurality of given robots based on the robot instructions and the machine learning model comprises operating one or more of the given robots based on the initial updated machine learning model.

19. The method of claim 14, further comprising providing the updated machine learning model in response to the request.

20. The method of claim 14, further comprising:

operating one or more of the given robots or additional given robots based on the robot instructions and the updated machine learning model;

storing, by one or more of the processors in one or more computer readable media, additional sensor data generated during the operation based on the robot instructions and the updated machine learning model;

generating additional training examples for the machine learning model based on the sensor data generated during the operation based on the robot instructions and the updated machine learning model;

training the updated machine learning model based on the training examples to create a further updated machine learning model.