Remote vehicle control system and method

• US 8,954,194 B2
• Filed: 05/12/2011
• Issued: 02/10/2015
• Est. Priority Date: 05/12/2010
• Status: Active Grant

• Current Assignee: Flir Detection, Inc. (Teledyne Technologies Incorporated)
• Original Assignee: iRobot Corporation
• Inventors: Allis, Daniel, Pack, Robert Todd
• Primary Examiner(s): Tran, Khoi
• Assistant Examiner(s): NGUYEN, ROBERT T
• Application Number:
  US13/106,792
• Publication Number:
  US 20110301786A1
• Time in Patent Office: 1,370 Days
• Field of Search: 701/2, 700/245, 700/249, 700/257, 700/258, 700/259, 700/264, 715700-866
• US Class Current: 700/259
• CPC Class Codes: G01C 21/26 specially adapted for navigation in a road network
• CPC Class Codes: G05D 1/0011 associated with a remote control arrangement
• CPC Class Codes: G05D 1/0038 by providing the operator with simple or augmented images from one or more cameras located onboard the vehicle, e.g. tele-operation images analyzed by a computer and used for automatic navigation G05D1/0246

Abstract

A system increases an operator'"'"'s situational awareness while the operator controls a remote vehicle. The system comprises an operator control unit having a point-and-click interface configured to allow the operator to view an environment surrounding the remote vehicle and control the remote vehicle, and a payload attached to the remote vehicle and in communication with at least one of the remote vehicle and the operator control unit. The payload comprises an integrated sensor suite including GPS, an inertial measurement unit, a stereo vision camera, and a range sensor, and a computational module receiving data from the GPS, the inertial measurement unit, the stereo vision camera, and the range sensor and providing data to a CPU including at least one of an autonomous behavior and a semi-autonomous behavior that utilize data from the integrated sensor suite.

Claims

1. A method for controlling a remote vehicle by interacting with a display, the method comprising:

displaying on the display a point-and-click interface configured to allow an operator to view an environment surrounding the remote vehicle and control the remote vehicle by inputting one or more commands via the point-and-click interface;

displaying on the display a 3D local perceptual space comprising an egocentric coordinate system encompassing a predetermined distance centered on the remote vehicle;

displaying on the display a remote vehicle representation having selectable portions actuatable through the point-and-click interface;

receiving a selection of a selectable portion of the remote vehicle representation corresponding to the portion of the remote vehicle to be commanded;

displaying on the display an indication of operator intent to command the selected part of the remote vehicle representation; and

selecting one or more commands for the remote vehicle, the one or more commands comprising a command for;

selecting a location in the 3D local perceptual space;

displaying a distance between the location corresponding to the selected point in the 3D local perceptual space and the remote vehicle; and

displaying an icon at the point selected in the 3D local perceptual space and at a corresponding location in an alternative view of a map having an identified current location of the remote vehicle.

2. The method of claim 1, wherein the part of the remote vehicle representation comprises a chassis, and selecting a command for the remote vehicle comprises selecting a stair climbing icon.

3. The method of claim 1, wherein the part of the remote vehicle representation comprises a chassis, and selecting a command for the remote vehicle comprises selecting a location in the video feed to which the remote vehicle will drive.

4. The method of claim 1, wherein the part of the remote vehicle representation comprises a chassis, and selecting a command for the remote vehicle comprises selecting a location in the map to which the remote vehicle will drive.

5. The method of claim 1, wherein the map comprises a 3D local perceptual space display.

6. The method of claim 1, wherein the part of the remote vehicle representation comprises a manipulator arm, and selecting a command for the remote vehicle comprises selecting an object in the video feed for the manipulator arm to manipulate.

7. The method of claim 1, wherein the part of the remote vehicle representation comprises a camera mounted on the remote vehicle, and selecting a command for the remote vehicle comprises selecting a location in a video feed of the camera that the camera should zoom to.

8. The method of claim 1, wherein the part of the remote vehicle representation comprises a camera mounted on the remote vehicle, and selecting a command for the remote vehicle comprises selecting a location on the map that the camera should pan to.

9. The method of claim 8, wherein the map comprises a 3D local perceptual space display.

10. The method of claim 1, further comprising looking at the map from more than one perspective by dragging on a displayed widget that changes the map between at least a top view and a forward view.

11. The method of claim 1, wherein the part of the remote vehicle representation comprises a manipulator arm, and selecting a command for the remote vehicle comprises selecting an object in the map for the manipulator arm to manipulate.

13. The system of claim 1, comprises at least one smart camera module, illuminators, and supporting electronics.

14. The system of claim 1, wherein an interface between the operator control unit and the remote vehicle includes an Ethernet link from the remote vehicle to the payload and a networked radio link from the payload to the operator control unit.

15. The system of claim 1, wherein the 3D local perceptual space data is stored in a high-performance database that fuses localization sensor data and ranging sensor data using fast geometric indexing and Bayesian evidence accumulation and scan registration functionality.

16. The system of claim 1, further comprising a behavior engine configured to provide kinodynamic, real-time motion planning that accounts for the dynamics and kinematics of the remote vehicle.

17. The system of claim 16, further comprising a Joint Architecture for Unmanned Systems (JAUS) gateway configured to interface with the behavior engine.

18. The system of claim 15, further comprising a Joint Architecture for Unmanned Systems (JAUS) gateway configured to interface with the 3D local perceptual space data.

12. A system for increasing an operator'"'"'s situational awareness while the operator controls a remote vehicle, the system comprising:

an operator control unit having a point-and-click interface configured to allow the operator to view an environment surrounding the remote vehicle and control the remote vehicle by inputting one or more commands via the point-and-click interface, the operator control unit displaying;

a 3D local perceptual space comprising an egocentric coordinate system encompassing a predetermined distance centered on the remote vehicle;

a remote vehicle representation having selectable portions, the selectable portions actuatable by the operator control unit;

an indication to command a portion of the remote vehicle corresponding to a selected portion of the remote vehicle representation;

a distance between a location corresponding to a selected point in the 3D local perceptual space and the remote vehicle; and

an icon at the point selected in the 3D local perceptual space and at a corresponding location in an alternative view of a map having an identified current location of the remote vehicle;

a payload attached to the remote vehicle and in communication with at least one of the remote vehicle and the operator control unit, the payload comprising;

an integrated sensor suite including a global positioning system, an inertial measurement unit, and a stereo vision camera or a range sensor, and

a computational module receiving data from the integrated sensor suite and determining the 3D local perceptual space based on the received data, the computational module determining a resolved motion of the remote vehicle corresponding to the one or more inputted commands.