Shared robot knowledge base for use with cloud computing system

• US 8,639,644 B1
• Filed: 05/04/2012
• Issued: 01/28/2014
• Est. Priority Date: 05/06/2011
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Hickman, Ryan, Kuffner, James J. Jr., Bruce, James R., Gharpure, Chaitanya, Kohler, Damon, Poursohi, Arshan, Francis, Anthony G. Jr., Lewis, Thor
• Primary Examiner(s): Chaki, Kakali
• Assistant Examiner(s): PELLETT, DANIEL T
• Application Number:
  US13/464,699
• Time in Patent Office: 634 Days
• Field of Search: None
• US Class Current: 706/14
• CPC Class Codes: G05B 19/042 using digital processors G05B19/05 takes precedence
• CPC Class Codes: G05B 2219/31205 Remote transmission of measured values from site, local to host
• CPC Class Codes: G05B 2219/39393 Camera detects projected image, compare with reference image, position end effector
• CPC Class Codes: G05B 2219/45084 Service robot
• CPC Class Codes: G06N 3/008 based on physical entities controlled by simulated intelligence so as to replicate intelligent life forms, e.g. based on robots replicating pets or humans in their appearance or behaviour

Abstract

The present application discloses shared robot knowledge bases for use with cloud computing systems. In one embodiment, the cloud computing system collects data from a robot about an object the robot has encountered in its environment, and stores the received data in the shared robot knowledge base. In another embodiment, the cloud computing system sends instructions for interacting with an object to a robot, receives feedback from the robot based on its interaction with the object, and updates data in the shared robot knowledge base based on the feedback. In yet another embodiment, the cloud computing system sends instructions to a robot for executing an application based on information stored in the shared robot knowledge base. In the disclosed embodiments, information in the shared robot knowledge bases is updated based on robot experiences so that any particular robot may benefit from prior experiences of other robots.

Claims

1. A method comprising:

receiving a first query from a first robot, wherein the first query includes identification information associated with an object, and wherein the first robot has a computing ability and is configured to interact with its surroundings via one or more sensors or electromechanical devices associated with the first robot;

identifying data in a shared robot knowledge base based at least in part on the first query, wherein the identified data corresponds to the object of the first query and comprises robot executable instructions for interacting with the object that include instructions for controlling the object using commands sent over a communications link between the first robot and the object, wherein the instructions for interacting with the object include instructions for physically controlling the object using at least one of the electromechanical devices associated with the first robot, wherein the shared robot knowledge base comprises data related to an inventory of objects, and wherein the data stored in the shared robot knowledge base is accessible by at least the first robot;

sending the identified data to the first robot, wherein the identified data is usable by the first robot to interact with the object;

receiving feedback from the first robot based on the first robot'"'"'s interaction with the object, wherein the feedback is usable to at least one of (i) revise the identified data and (ii) create new data corresponding to the object of the first query; and

updating the shared robot knowledge base based at least in part on the feedback received from the first robot.

2. The method of claim 1, wherein the identification information associated with the object includes at least one of (i) language data corresponding to the object, (ii) image data corresponding to the object, and (iii) data read from a tag or beacon corresponding to the object.

3. The method of claim 1, wherein the feedback received from the first robot is based on coaching received by the first robot from a human.

4. The method of claim 1, wherein the identified data corresponding to the object of the first query further comprises a physical attribute of the object, wherein the physical attribute includes at least one of (i) a dimension, (ii) weight, (iii) mass, (iv) color, (v) texture, and (vi) material composition.

5. The method of claim 4, wherein the feedback is based on a comparison between (i) the value of at least one physical attribute of the object specified in the identified data and (ii) the first robot'"'"'s determination of the value of the at least one physical attribute of the object.

6. The method of claim 1, wherein the feedback includes at least one revision to the robot executable instructions for interacting with the object.

7. The method of claim 1, wherein the robot executable instructions for interacting with the object include instructions for controlling the one or more sensors or electromechanical devices associated with the first robot.

8. The method of claim 4, wherein at least one of the physical attributes of the object and the robot executable instructions for interacting with the object is received from a manufacturer of the object and stored in the shared robot knowledge base.

9. The method of claim 1, further comprising:

receiving a second query from a second robot, wherein the second query includes identification information associated with an object that is of the same type as the object of the first query, wherein the second robot has a computing ability and is configured to interact with its surroundings via one or more sensors or electromechanical devices associated with the second robot; and

sending data associated with the object of the second query to the second robot in response to the second query received from the second robot, wherein the data associated with the object of the second query is based at least in part on the feedback received from the first robot based on the first robot'"'"'s interaction with the object of the first query.

10. A system comprising:

a shared robot knowledge base that stores an inventory of objects, wherein the data stored in the shared robot knowledge base is accessible by a plurality of robots, wherein individual robots have a computing ability and are configured to interact with their surroundings via one or more associated sensors or electromechanical devices;

a cloud processing engine that (i) receives a first query from a first robot, wherein the first query includes identification information associated with an object detected by the robot, and (ii) identifies data in the shared robot knowledge base based at least in part on the first query, wherein the identified data corresponds to the object of the first query and comprises robot executable instructions for interacting with the object that include instructions for controlling the object using commands sent over a communications link between the first robot and the object and wherein the instructions for interacting with the object include instructions for physically controlling the object using at least one or the electromechanical devices associated with the first robot;

a communications interface that (i) sends the identified data to the first robot, wherein the identified data is usable by the first robot to interact with the object, and (ii) receives feedback from the first robot based on the first robot'"'"'s interaction with the object; and

wherein the cloud processing engine is configured to update the shared robot knowledge base by at least one of (i) revising the identified data based at least in part on the feedback from the first robot and (ii) creating new data corresponding to the object of the first query based at least in part on the feedback from the first robot.

11. The system of claim 10, wherein the feedback received from the first robot is based on at least one of:

(i) coaching received by the first robot from a human;

(ii) a comparison between (a) the value of at least one physical attribute of the object specified in the identified data and (b) the first robot'"'"'s determination of the value of the at least one physical attribute of the object.

12. The system of claim 10, wherein the identified data corresponding to the object of the first query further comprises a physical attribute of the object.

13. The system of claim 10, wherein the robot executable instructions for interacting with the object include instructions for controlling the one or more sensors or electromechanical devices associated with the first robot.

14. The system of claim 10, wherein the cloud processing engine is further configured to (i) receive a second query from a second robot, wherein the second query includes identification information associated with an object that is of the same type as the object of the first query, and (ii) send data associated with the object of the second query to the second robot in response to the second query, wherein the data associated with the object of the second query is based at least in part on the feedback received from the first robot.

15. An article of manufacture comprising tangible non-transitory computer readable media having instructions stored thereon, the instructions comprising:

instructions for processing a first query received from a first robot, wherein the first query includes identification information associated with an object, and wherein the first robot has a computing ability and is configured to interact with its surroundings via one or more associated sensors or electromechanical devices;

instructions for identifying data in a shared robot knowledge base based at least in part on the first query, wherein the identified data corresponds to the object of the first query and comprises robot executable instructions for interacting with the object that include instructions for controlling the object via commands sent over a communications link between the first robot and the object, wherein the instructions for interacting with the object include instructions for physically controlling the object using at least one of the electromechanical devices associated with the first robot, wherein the shared robot knowledge base comprises data related to an inventory of objects, and wherein the data stored in the shared robot knowledge base is accessible by at least the first robot;

instructions for initiating the transmission of the identified data to the first robot, wherein the identified data is usable by the first robot to interact with the object;

instructions for processing feedback received from the first robot based on the first robot'"'"'s interaction with the object, wherein the feedback is usable for at least one of revising the identified data or creating new data corresponding to the object of the first query; and

instructions for updating the shared robot knowledge base based at least in part on the feedback received from the first robot.

16. The article of manufacture of claim 15, wherein the identified data corresponding to the object of the first query further comprises a physical attribute of the object.

17. The article of manufacture of claim 15, wherein the robot executable instructions for interacting with the object include instructions for controlling the one or more sensors or electromechanical devices associated with the first robot.

18. The article of manufacture of claim 15, wherein the feedback received from the first robot is based on at least one of:

(i) coaching received by the first robot from a human;

(ii) a comparison between (a) the value of at least one physical attribute of the object specified in the identified data and (b) the first robot'"'"'s determination of the value of the at least one physical attribute of the object.

19. The article of manufacture of claim 15, further comprising:

instructions for processing a second query received from a second robot, wherein the second query includes identification information associated with an object that is of the same type as the object of the first query, wherein the second robot has a computing ability and is configured to interact with its surroundings via one or more associated sensors or electromechanical devices; and

instructions for initiating the transmission of data associated with the object of the second query to the second robot in response to the second query received from the second robot, wherein the data associated with the object of the second query is based at least in part on the feedback received from the first robot based on the first robot'"'"'s interaction with the object of the first query.

20. A method comprising:

receiving a first query from a first robot, wherein the first query includes identification information associated with an object detected by one or more sensors associated with the first robot;

determining whether the identification information received in the first query corresponds to an object having an object identifier stored in a shared robot knowledge base;

creating a new object identifier in the shared robot knowledge base in response to determining that the identification information received in the first query does not correspond to an object having an object identifier stored in the shared robot knowledge base;

instructing the first robot to collect information about the object;

receiving information about the object from the first robot in response to the instructions to collect information about the object, wherein the information about the object includes (i) image data and (ii) location data;

storing data about the object in the shared robot knowledge base, wherein the stored data is based at least in part on the information received from the first robot; and

generating instructions for interacting with the object based on instructions for interacting with at least one other object stored in the shared robot knowledge base, wherein the instructions for interacting with the at least one other object include instructions for controlling the object using commands sent over a communication link between the first robot and the object, and wherein the instructions for interacting with the object include instructions for physically controlling the object using at least one electromechanical device associated with the first robot.

21. The method of claim 20, wherein the identification information associated with the object includes (i) language data associated with the object, (ii) image data associated with the object, and (iii) data read from a tag or beacon associated with the object.

22. The method of claim 20, wherein determining whether the identification information received in the first query corresponds to an object having an object identifier stored in the shared robot knowledge base comprises at least one of:

comparing language data in the first query with language data associated with a plurality of object identifiers stored in the shared robot knowledge base;

comparing image data in the first query with image data associated with the plurality of object identifiers stored in the shared robot knowledge base; and

comparing data read from a tag or beacon in the first query with identification data associated with the plurality of object identifiers stored in the shared robot knowledge base.

23. The method of claim 20, further comprising:

storing the generated instructions in the shared robot knowledge base; and

associating the generated instructions with the new object identifier.

24. The method of claim 20, further comprising:

receiving a second query from a second robot, wherein the second query includes identification information associated with an object detected by one or more sensors associated with the second robot; and

sending data associated with the new object identifier in response to determining that the identification information received in the second query corresponds to the new object identifier, wherein the data associated with the new object identifier comprises at least one of (i) a physical attribute of the object associated with the new object identifier and (ii) robot executable instructions for interacting with the object associated with the new object identifier.