Teleoperation of unmanned ground vehicle

• US 8,918,230 B2
• Filed: 01/21/2011
• Issued: 12/23/2014
• Est. Priority Date: 01/21/2011
• Status: Active Grant

• Current Assignee: Remote Imaging Solutions LLC (Endpoint IP LLC)
• Original Assignee: Mitre Corporation
• Inventors: Bushmitch, Dennis, Chen, Peter David
• Primary Examiner(s): To, Tuan C.
• Application Number:
  US13/011,178
• Publication Number:
  US 20120191269A1
• Time in Patent Office: 1,432 Days
• Field of Search: 701/2, 701/23, 700/259, 901/1, 901/47
• US Class Current: 701/2
• CPC Class Codes: G05D 1/0016 characterised by the operator's input device input arrangements for computing systems in general G06F3/00
• CPC Class Codes: G05D 1/0038 by providing the operator with simple or augmented images from one or more cameras located onboard the vehicle, e.g. tele-operation images analyzed by a computer and used for automatic navigation G05D1/0246
• CPC Class Codes: Y10S 901/01 Mobile robot
• CPC Class Codes: Y10S 901/47 Optical

Abstract

Described are systems and methods, including computer program products for controlling an unmanned vehicle. A user controls one or more unmanned vehicles with a smart phone. The smart phone receives video stream from the unmanned vehicles, and the smart phone displays the controls from the unmanned vehicle over the video. The smart phone and the unmanned vehicle communicate wirelessly.

Claims

1. A computerized method for controlling an unmanned vehicle, comprising:

receiving, by a smart phone, a request to control the unmanned vehicle;

establishing, by the smart phone, wireless communication between the smart phone and the unmanned vehicle;

receiving, by the smart phone, a video stream generated by the unmanned vehicle;

displaying the video stream and vehicle controls on a touch-sensitive display screen within the smart phone;

manipulating the vehicle controls to direct the unmanned vehicle;

transmitting, by the smart phone, an initiation request via a cable to a rf transmitter;

transmitting, by the rf transmitter, the initiation request to the unmanned vehicle;

receiving, by the rf transmitter, a message from the unmanned vehicle; and

transmitting, by the rf transmitter, the message from the unmanned vehicle to the smart phone via the cable to the smart phone.

2. The computerized method of claim 1, further comprising:

receiving, by the smart phone, information to direct the unmanned vehicle;

converting, by the smart phone, the information into a protocol recognizable by the unmanned vehicle; and

transmitting, by the smart phone, the converted information to the unmanned vehicle.

3. The computerized method of claim 1, further comprising establishing communication with the unmanned vehicle via a wireless local area network.

4. The computerized method of claim 1, further comprising providing a joystick for governing the acceleration, deceleration, and direction of travel of the unmanned vehicle.

5. The computerized method of claim 4, further comprising providing a controller to govern movement of one or more robot arms of the unmanned ground vehicle.

6. The computerized method of claim 1, further comprising providing a control to set a maximum speed of the unmanned vehicle.

7. A computerized method for controlling an unmanned vehicle, comprising:

receiving, by a smart phone, a request to control the unmanned vehicle;

establishing, by the smart phone, wireless communication between the smart phone and the unmanned vehicle;

receiving, by the smart phone, a video stream generated by the unmanned vehicle;

displaying the video stream and vehicle controls on a touch-sensitive display screen within the smart phone;

manipulating the vehicle controls to direct the unmanned vehicle; and

providing a control to store a frame of the video stream as a picture on the smart phone.

8. A computerized method for controlling an unmanned vehicle, comprising:

receiving, by a smart phone, a request to control the unmanned vehicle;

establishing, by the smart phone, wireless communication between the smart phone and the unmanned vehicle;

receiving, by the smart phone, a video stream generated by the unmanned vehicle;

displaying the video stream and vehicle controls on a touch-sensitive display screen within the smart phone;

manipulating the vehicle controls to direct the unmanned vehicle; and

providing a control to command one or more cameras on the unmanned vehicle to zoom in or out.

9. A computerized method for controlling an unmanned vehicle, comprising:

receiving, by a smart phone, a request to control the unmanned vehicle;

establishing, by the smart phone, wireless communication between the smart phone and the unmanned vehicle;

receiving, by the smart phone, a video stream generated by the unmanned vehicle;

displaying the video stream and vehicle controls on a touch-sensitive display screen within the smart phone;

manipulating the vehicle controls to direct the unmanned vehicle; and

providing one or more controls to command the unmanned vehicle to one or more predetermined poses.

10. The computerized method of claim 9, further comprising displaying the controls and a geographic map on the display screen within the smart phone.

11. The computerized method of claim 9, further comprising displaying one or more arrows, the video stream, and the controls on the display within the smart phone, wherein the one or more arrows indicate a heading for the unmanned vehicle such that the unmanned vehicle can reach a specified location.

12. The computerized method of claim 9, further comprising receiving information to rotate the unmanned vehicle.

13. The computerized method of claim 9, further comprising receiving information to move a robotic arm on the vehicle.

14. A computerized method for controlling an unmanned vehicle, comprising:

receiving, by a smart phone, a request to control the unmanned vehicle;

establishing, by the smart phone, wireless communication between the smart phone and the unmanned vehicle;

receiving, by the smart phone, a video stream generated by the unmanned vehicle;

displaying the video stream and vehicle controls on a touch-sensitive display screen within the smart phone;

manipulating the vehicle controls to direct the unmanned vehicle;

receiving, by the smart phone, information of objects within a distance of the unmanned vehicle; and

displaying, on the smart phone, the unmanned vehicle with the objects positioned relative to the unmanned vehicle based the information.

15. A computerized method for controlling a plurality of unmanned vehicles, comprising:

receiving, by a smart phone, a request to control the unmanned vehicles;

establishing, by the smart phone, wireless communication between the smart phone and each of the unmanned vehicles;

receiving, by the smart phone, a video stream from each of the unmanned vehicles;

selecting, by the smart phone, one of the plurality of unmanned vehicles;

displaying the video stream and vehicle controls of the selected vehicle on a display screen within the smart phone;

manipulating the displayed vehicle controls to direct the selected unmanned vehicle; and

displaying controls that correspond to the particular unmanned vehicle from which the video stream is received.

16. The computerized method of claim 15, wherein establishing a wireless communication further comprises transmitting, by the smart phone, a signal to each of the plurality of unmanned vehicles, such that the signal changes to a protocol of the particular unmanned vehicle the signal is transmitting to.

17. A computerized method for controlling a plurality of unmanned vehicles, comprising:

receiving, by a smart phone, a request to control the unmanned vehicles;

establishing, by the smart phone, wireless communication between the smart phone and each of the unmanned vehicles;

receiving, by the smart phone, a video stream from each of the unmanned vehicles;

selecting, by the smart phone, one of the plurality of unmanned vehicles;

displaying the video stream and vehicle controls of the selected vehicle on a display screen within the smart phone;

manipulating the displayed vehicle controls to direct the selected unmanned vehicle; and

toggling between displaying the video stream and controls of each of the plurality of unmanned vehicles based on operator input.

18. A computerized method for controlling a plurality of unmanned vehicles, comprising:

receiving, by a smart phone, a request to control the unmanned vehicles;

establishing, by the smart phone, wireless communication between the smart phone and each of the unmanned vehicles;

receiving, by the smart phone, a video stream from each of the unmanned vehicles;

selecting, by the smart phone, one of the plurality of unmanned vehicles;

displaying the video stream and vehicle controls of the selected vehicle on a display screen within the smart phone;

manipulating the displayed vehicle controls to direct the selected unmanned vehicle; and

selecting, by the smart phone, two of the plurality of the unmanned vehicles; and

displaying the video stream and controls of the two selected vehicles on a display screen within the smart phone; and

manipulating the displayed controls to direct the two selected unmanned vehicles.