PROVIDING REMOTE GESTURAL AND VOICE INPUT TO A MOBILE ROBOT

• US 20120316679A1
• Filed: 06/07/2011
• Published: 12/13/2012
• Est. Priority Date: 06/07/2011
• Status: Abandoned Application

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Papaefstathiou, Efstathios, Clinton, Nathaniel T., Sanchez, Russell
• Application Number:
  US13/154,468
• Publication Number:
  US 20120316679A1
• Time in Patent Office: Days
• US Class Current: 700/258
• CPC Class Codes: B25J 13/003 by means of an audio-responsive input audible safety signals B25J19/061
• CPC Class Codes: B25J 13/006 by means of a wireless system for controlling one or several manipulators

Abstract

A system, such as a robot, which responds to voice, gesture and other natural inputs from a user, is controllable when the user is out of range through use of a wireless controller. The wireless controller provides inputs that allow the user to enter commands that are a proxy for the voice and gesture inputs the robot otherwise recognizes. The controller can include, for example, a microphone for voice input, a pad for directional control, and a speaker and display devices to provide responses from the robot.

Claims

1. A robot system, comprising:

a robot having a plurality of sensors for detecting actions of a user and providing sensory data as outputs, and a sensory input processing system having an input for receiving the sensory data and an output providing commands generated from the sensory data, and a control system responsive to the commands to control operations of the robot, and outputs providing feedback to the user;

a remote device in wireless communication with the robot, and providing at least a portion of the commands to the robot, and having outputs for providing at least a portion of the feedback from the robot to the user.

2. The robot system of claim 1, wherein the commands include commands derived from speech recognition, and wherein the sensory data provided by the remote device includes audio data.

3. The robot system of claim 1, wherein the feedback includes audio data, and the remote device outputs the audio data.

4. The robot system of claim 1, wherein the commands include directional commands derived from gesture recognition applied to the sensory data, and the inputs from the remote device include information indicating whether user presses one or more buttons, and the inputs are processed as the commands derived from gesture recognition.

5. The robot system of claim 1, wherein in response to an indication of a direction from the remote device, the robot system starts moving in the indicated direction while avoiding obstacles.

6. A robot comprising:

a plurality of sensors for detecting actions of a user and providing sensory data as outputs;

a sensory input processing system having an input for receiving the sensory data and an output providing commands generated from the sensory data;

a control system responsive to the commands to control operations of the robot;

outputs providing feedback to the user;

a wireless input that receives data from a remote device, wherein the data from the remote device is at least a subset of the sensory data and commands, and wherein the robot sends feedback to the remote device that includes at least a subset of feedback provided on the robot.

7. The robot of claim 6, wherein the commands include commands derived from speech recognition, and wherein the sensory data provided by the remote device includes audio data.

8. The robot of claim 6, wherein the feedback includes audio data, and the remote device outputs the audio data.

9. The robot of claim 6, wherein the commands include directional commands derived from gesture recognition applied to the sensory data, and the inputs from the remote device include information indicating whether user presses one or more buttons, and the inputs are processed as the commands derived from gesture recognition.

10. The robot of claim 6, wherein in response to an indication of a direction from the remote device, the robot starts moving in the indicated direction while avoiding obstacles.

11. A device for a wireless connection to a robot, comprising:

input devices for receiving inputs from a user;

a wireless transceiver for transmitting information about the user inputs to the robot,

such that the user inputs are translated into commands which are at least a subset of commands performed by the robot generated in response to processing sensory data obtained by sensing actions of the user; and

the wireless transceiver receiving feedback about performance of commands from the robot;

display devices for displaying the feedback from the robot.

12. The device of claim 11, wherein the commands include commands derived from speech recognition, and wherein the sensory data provided by the remote device includes audio data.

13. The device of claim 11, wherein the feedback includes audio data, and the remote device outputs the audio data.

14. The device of claim 11, wherein the commands include directional commands derived from gesture recognition applied to the sensory data, and the inputs from the remote device include information indicating whether user presses one or more buttons, and the inputs are processed as the commands derived from gesture recognition.

15. The robot of claim 11, wherein in response to an indication of a direction from the remote device, the robot starts moving in the indicated direction while avoiding obstacles.

16. A process for controlling a robot, comprising;

receiving data from a mobile device remote from the robot, wherein the data includes user input;

deriving commands from the user input, wherein the commands include at least a subset of commands that the robot derives from processing sensory data on the robot in response to a user;

generating feedback to the remote device about the command; and

causing the remote device to display the feedback to a user.

17. The process of claim 16, wherein the commands include commands derived from speech recognition, and wherein the sensory data provided by the remote device includes audio data.

18. The process of claim 16, wherein the feedback includes audio data, and the remote device outputs the audio data.

19. The process of claim 16, wherein the commands include directional commands derived from gesture recognition applied to the sensory data, and the inputs from the remote device include information indicating whether user presses one or more buttons, and the inputs are processed as the commands derived from gesture recognition.

20. The process of claim 16, wherein in response to an indication of a direction from the remote device, the robot starts moving in the indicated direction while avoiding obstacles.