System and method for cooperative remote vehicle behavior

• US 8,577,126 B2
• Filed: 04/11/2008
• Issued: 11/05/2013
• Est. Priority Date: 04/11/2007
• Status: Active Grant

• Current Assignee: iRobot Corporation
• Original Assignee: iRobot Corporation
• Inventors: Jones, Christopher Vernon, Jenkins, Odest Chadwicke, Loper, Matthew M.
• Primary Examiner(s): Carter, Aaron W
• Application Number:
  US12/101,949
• Publication Number:
  US 20080253613A1
• Time in Patent Office: 2,034 Days
• Field of Search: 382/154, 382/103, 382/106, 700/245, 700/259, 704/E13.001, 704/E15.001, 704/E21.019, 704/275, 715/863, 348/77, 348/114
• US Class Current: 382/154
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: H04N 7/185 from a mobile camera, e.g. for remote control

Abstract

A method for facilitating cooperation between humans and remote vehicles comprises creating image data, detecting humans within the image data, extracting gesture information from the image data, mapping the gesture information to a remote vehicle behavior, and activating the remote vehicle behavior. Alternatively, voice commands can by used to activate the remote vehicle behavior.

Claims

1. A method for facilitating cooperation between humans and remote vehicles, the method comprising:

creating a scene image including 3D point cloud data;

connecting components within the scene image to extract all large solid objects from the scene image using the point cloud data;

executing a support vector machine trained on a shape of a human to identify a human shape among the large solid objects within the scene image;

extracting gesture information of observed poses of human arms from the human shape using a trained statistical model trained on poses of human arms in scene images, the gesture information corresponding to at least one remote vehicle motion command to control a remote vehicle behavior;

scoring the gesture information relative to a set of known gestures;

when a sequence of gesture information matches a complete sequence associated with a known gesture, mapping the at least one remote vehicle motion command to one or more remote vehicle behaviors; and

activating the one or more remote vehicle behaviors.

2. The method of claim 1, wherein the remote vehicle behavior gathers data from sensors of the remote vehicle and outputs one or more motion commands.

3. The method of claim 1, wherein the remote vehicle behavior includes one of person-following, obstacle-avoidance, door-breaching, u-turn, start/stop following and manual forward drive.

4. The method of claim 3, wherein conflicts between remote vehicle behaviors are resolved by assigning unique priorities to each remote vehicle behavior.

5. The method of claim 4, wherein commands from a low priority remote vehicle behavior are overridden by those from a high priority remote vehicle behavior.

6. The method of claim 1, wherein voice commands are issued into a wireless headset worn by a human operator.

7. The method of claim 1, further comprising utilizing speech synthesis to allow the remote vehicle to communicate with an operator in a natural way.

8. The method of claim 1, wherein the remote vehicle behavior includes person following.

9. The method of claim 1, wherein the trained statistical model is a trained Hidden Markov Model.

10. The method of claim 1, comprising analyzing voice commands with speech recognition software.

11. The method of claim 1, comprising creating image data from a camera.

12. The method of claim 11, wherein the camera comprises a time-of-flight camera.

13. The method of claim 11, wherein the camera comprises a 3D time-of-flight camera.

14. The method of claim 11, wherein the camera comprises part of a stereo vision system.