Robot system using virtual world

• US 20050090933A1
• Filed: 10/24/2003
• Published: 04/28/2005
• Est. Priority Date: 10/24/2003
• Status: Active Grant

• Current Assignee: SAP SE
• Original Assignee: SAP AG (SAP SE)
• Inventors: Ebert, Peter S.
• Granted Patent: US 7,099,745 B2
• Time in Patent Office: Days
• US Class Current: 700/245
• CPC Class Codes: B25J 9/1694 characterised by use of sensors other than normal servo-feedback from position, speed or acceleration sensors, perception control, multi-sensor controlled systems, sensor fusion
• CPC Class Codes: G05B 19/41885 characterised by modeling, simulation of the manufacturing system
• CPC Class Codes: G05B 2219/31007 Floor plan, map stored in on-board computer of vehicle
• CPC Class Codes: G05B 2219/32082 Planing, material requiring planning MRP, request
• CPC Class Codes: G05B 2219/40131 Virtual reality control, programming of manipulator
• CPC Class Codes: G05B 2219/45045 Maintenance, automatic storage and retrieval system
• CPC Class Codes: G05D 1/0088 characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours using knowledge based models G06N5/00
• CPC Class Codes: G05D 2201/0216 Vehicle for transporting goods in a warehouse, factory or similar
• CPC Class Codes: Y02P 90/02 Total factory control, e.g. smart factories, flexible manufacturing systems [FMS] or integrated manufacturing systems [IMS]

Abstract

A robot system, operable to perform a task, is integrated on a robot with a sensor, a control module, and a virtual world. The virtual world represents a generic continuum including a collection of information replicating the spatial and temporal properties of a number of articles and related circumstances in an environment of the robot system. As the virtual world provides a common space-time reference for all of the articles, including the robot system itself, the robot system may perform its task in an efficient and robust manner. For example, the robot may move about its environment without colliding with the other articles. Further, the virtual world may include predicted information about the articles, including an anticipated position or condition of the article at a particular time. In this way, the robot system may perform its task in a timely manner, since its operations can be synchronized with the other articles and an idle time may be reduced.

Claims

1. A robot system comprising:

a robot operable to perform a task;

a sensor associated with the robot and operable to detect local status data related to an environment of the robot;

a receiver associated with the robot and operable to receive task data associated with performance of the task within the environment; and

a control module associated with the robot and operable to associate the local status data and the task data with a common time reference, such that the robot performs the task based on the local status data and the task data and in conjunction with the time reference.

2. The robot system of claim 1, further comprising a virtual world that is in communication with the control module and that is operable to aggregate the local status data and the task data such that spatial and temporal properties of articles within the environment are replicated in the virtual world in conjunction with the time reference.

3. The robot system of claim 2 wherein the robot is mobile, and wherein the control module directs movement of the robot in performing the task based on the virtual world.

4. The robot system of claim 2 wherein the task data includes actual status data collected by separate sensors from the sensor associated with the robot and transmitted to the virtual world.

5. The robot system of claim 2 wherein the task data includes predicted status data describing a future position of one of the articles within the environment, with respect to the time reference.

6. The robot system of claim 2 wherein the task data includes instructions for implementing the task in the environment.

7. The robot system of claim 2 wherein the local status data and the task data are continually collected and transmitted to the virtual world substantially in real-time.

8. The robot system of claim 2 wherein the robot system and each of the articles are assigned a unique identifier in the virtual world.

9. The robot system of claim 8 wherein the virtual world is operable to interface with a shared virtual world that aggregates data received from the articles.

10. The robot system of claim 9 wherein the unique identifiers are unique across the shared virtual world.

11. The robot system of claim 2 wherein the task data includes a priority assigned to the robot system relative to one of the articles with respect to performance of the task.

12. A method comprising:

collecting data at a sensor of a robot system to produce a first data stream;

receiving a second data stream at the robot system, the second data stream including instructions for the robot system to perform a task;

aggregating the first data stream and the second data stream with respect to a common time reference; and

performing the task with the robot system and in accordance with the first data stream, the second data stream, and the common time reference.

13. The method of claim 12 wherein receiving the second data stream includes receiving a third data stream compiled from separate sensors apart from the sensor of the robot system, the third data stream describing physical phenomena related to performance of the task.

14. The method of claim 13 wherein aggregating the first data stream and the second data stream comprises including the first data stream and the second data stream in a virtual world in which spatial and temporal properties of real-world articles, as sensed by the sensor and the separate sensors, are represented.

15. The method of claim 14 wherein receiving the second data stream at the robot system comprises receiving the second data stream via the virtual world.

16. The method of claim 13 comprising receiving telemetry data from the virtual world at at least one of an enterprise application and a process control application.

17. The method of claim 16 wherein performing the task comprises transmitting instructions to the robot system from the enterprise application or the process control application via the virtual world based on the telemetry data.

18. The method of claim 14 wherein the virtual world is a local virtual world that is stored locally at the robot system, and wherein including the first data stream and the second data stream in the virtual world comprises:

determining that a communications link is available for communicating with a non-local virtual world that is stored apart from the robot system; and

sharing information between the local virtual world and the non-local virtual world, using the communications link.

19. The method of claim 12 wherein the second data stream includes predicted data that describes a spatial position of an article or event with respect to the robot system at a future time, relative to the common time reference.

20. The method of claim 12 wherein receiving the second data stream includes receiving priority information that prioritizes operations of the robot system relative to articles encountered by the robot system during performance of the task.

21. The method of claim 12 wherein the robot system is mobile, and further wherein performing the task comprises directing movement of the robot system and movement of articles relative to one another.

22. A robot system comprising:

a robot operable to perform a task;

a sensor operable to detect physical phenomena in a vicinity of the robot and output sensor data;

a virtual world associated with a space-time continuum and operable to input the sensor data to replicate the physical phenomena with respect to the continuum; and

a control system operable to direct the robot in performance of the task, based on the space-time continuum.

23. The robot system of claim 22 wherein the virtual world is operable to receive task data that is related to performance of the task and incorporate the task data consistently with the space-time continuum.

24. The robot system of claim 23 wherein the virtual world receives the task data from an enterprise application.

25. The robot system of claim 23, wherein the robot is mobile, and further wherein the control module guides movement of the robot relative to articles in the vicinity of the robot, based on the virtual world.