Methods and apparatuses for responding to a detected event by a robot

• US 9,914,218 B2
• Filed: 01/30/2015
• Issued: 03/13/2018
• Est. Priority Date: 01/30/2015
• Status: Active Grant

• Current Assignee: Toyota Jidosha Kabushiki Kaisha (Toyota Motor Corporation)
• Original Assignee: Toyota Motor Engineering & Manufacturing North America Incorporated (Toyota Motor Corporation)
• Inventors: Ota, Yasuhiro
• Primary Examiner(s): Bendidi, Rachid
• Application Number:
  US14/609,854
• Publication Number:
  US 20160221191A1
• Time in Patent Office: 1,138 Days
• Field of Search: 700245, 700249-255, 700259, 31856811, 31856812, 31856813, 318587, 901 1, 901 47
• CPC Class Codes: B25J 9/1664 characterised by motion, path, trajectory planning
• CPC Class Codes: B25J 9/1674 characterised by safety, monitoring, diagnostic
• CPC Class Codes: B25J 9/1697 Vision controlled systems
• CPC Class Codes: G05D 1/0088 characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours using knowledge based models G06N5/00
• CPC Class Codes: G05D 1/0246 using a video camera in combination with image processing means
• CPC Class Codes: Y10S 901/01 Mobile robot
• CPC Class Codes: Y10S 901/47 Optical

Abstract

In one embodiment, a method for responding to a detected event by a robot is provided. The method includes using a sensor to detect an event within an operational space of a robot. The event includes a movement of an object or a person within the operational space. The method also includes using a processor to predict an action to occur within the operational space of the robot based upon the detected event. The method also identifies at least one correlated robot action to be taken in response to the detected event and compares the predicted action to a movement plan of the robot. The method further selects at least one of the correlated robot actions and modifies a movement plan of a robot to include at least one of the identified correlated robot actions in response to the detected event.

Claims

1. A method for responding to a detected event by a robot comprising:

executing a movement plan of the robot within an operational space of the robot;

monitoring the operational space of the robot for a movement of an object within the operational space while executing the movement plan, wherein the operation of monitoring the operational space of the robot is performed using a camera and using object recognition on an image captured by the camera;

detecting, with a sensor, the event within the operational space of the robot, wherein the event comprises a movement of the object caused by the person within the operational space;

predicting, using a processor, an action of the person to occur within the operational space of the robot based upon the detected event, wherein the predicted action comprises a predicted movement of the person within the operational space;

accessing a data structure using the event to identify at least one correlated robot action to be taken in response to the detected event;

comparing the predicted action to the movement plan of the robot and selecting at least one of the correlated robot actions;

modifying the movement plan of the robot to include at least one of the identified correlated robot actions in response to the detected event; and

controlling the robot to take the at least one correlated robot action in accordance with the modified movement plan,

wherein the operation of comparing the predicted action to the movement plan of the robot comprises determining whether the predicted action will interfere with the movement plan,

wherein the movement plan is adapted for use in controlling movement of the robot within the operational space and

wherein the operation of controlling the robot to take the at least one correlated robot action comprises moving the robot in accordance with the modified movement plan.

2. The method of claim 1 wherein the operation of predicting comprises accessing a data store.

3. The method of claim 1 wherein the data structure comprises at least one of the group of:

a look up table, a database, and a relational database.

4. The method of claim 1 wherein the detected event comprises at least one of the group of:

a chair moving, a person standing, a person approaching the robot, a person pointing, a person gesturing, a person sitting, a person moving away from the robot, a person opening a door, and a person closing a door.

5. The method of claim 1 wherein the modified movement plan comprises at least one of the group of:

warning the person, slowing a speed of the robot, stopping the robot, continuing a task, moving away from the person, retracting a component of the robot, reaching for an object, grasping an object;

retrieving an object, approaching a person, and accelerating a speed of the robot.

6. The method of claim 1 wherein the operation of detecting an event is performed using object recognition on images captured by the camera.

7. The method of claim 1 wherein the data structure comprises a plurality of events including a plurality of object changes within the operational space.

8. An apparatus comprising:

a robot locomotion device adapted to move a robot within an operational space of the robot;

a camera sensor adapted to monitor the operational space for movement of an object; and

a controller adapted to control the robot locomotion device in accordance with a movement plan,

wherein the controller is further adapted to;

execute the movement plan of the robot within the operational space;

detect, using the camera sensor, an event within the operational space of the robot-comprising movement of the object while executing the movement plan, wherein the event comprises movement of the object caused by a person within the operational space, wherein the controller is adapted to monitor the operational space using object recognition on an image captured by the camera sensor;

predict an action of the person to occur within the operational space, wherein the predicted action comprises movement of the person within the operational space;

access a data structure using the event to identify at least one correlated robot action to be taken in response to the detected event;

compare the predicted action to the movement plan of the robot to determine if the predicted action will interfere with the movement plan and, when the predicted action is determined to interfere with the movement plan, select at least one of the correlated robot actions; and

modify the movement plan to include at least one of the identified correlated robot actions in response to the detected event; and

control the robot to take the at least one correlated robot action in accordance with the modified movement plan to move the robot in accordance with the modified movement plan,

wherein the movement plan is adapted for use in controlling movement of the robot within the operational space.

9. The apparatus of claim 8 wherein the modified movement plan comprises at least one of the group of:

warning the person, slowing a speed of the robot, stopping the robot, continuing a task, moving away from the person, retracting a component of the robot, reaching for the object, grasping the object;

retrieving the object, approaching the person, and accelerating the speed of the robot.

10. The apparatus of claim 8 wherein the camera sensor comprises a video camera.

11. The apparatus of claim 8 wherein the movement plan comprises locomotion of the robot within the operational space.

12. The apparatus of claim 8 wherein the movement plan comprises motion of one or more components of the robot.

13. The apparatus of claim 8 wherein the data structure comprises at least one of the group of:

a look up table, a database, and a relational database.

14. The apparatus of claim 8 wherein the data structure comprises a plurality of events and correlated robot actions including historical event data previously correlated with predicted actions.

15. The apparatus of claim 8 wherein the data structure comprises a plurality of events including a plurality of object changes within the operational space.