Remote vehicle control system and method

• US 9,658,615 B2
• Filed: 12/31/2014
• Issued: 05/23/2017
• Est. Priority Date: 05/12/2010
• Status: Active Grant

• Current Assignee: Flir Detection, Inc. (Teledyne Technologies Incorporated)
• Original Assignee: iRobot Defense Holdings, Inc. (Teledyne Technologies Incorporated)
• Inventors: Pack, Robert Todd, Allis, Daniel P.
• Primary Examiner(s): Tran, Khoi
• Assistant Examiner(s): NGUYEN, ROBERT T
• Application Number:
  US14/587,396
• Publication Number:
  US 20150134146A1
• Time in Patent Office: 874 Days
• Field of Search: 701 2
• CPC Class Codes: G01C 21/26 specially adapted for navigation in a road network
• CPC Class Codes: G05D 1/0011 associated with a remote control arrangement
• CPC Class Codes: G05D 1/0038 by providing the operator with simple or augmented images from one or more cameras located onboard the vehicle, e.g. tele-operation images analyzed by a computer and used for automatic navigation G05D1/0246

Abstract

A system includes an operator control unit having a point-and-click interface configured to allow the operator to control the remote vehicle by inputting one or more commands via the point-and-click interface. The operator control unit displays a 3D local perceptual space comprising an egocentric coordinate system encompassing a predetermined distance centered on the remote vehicle, a remote vehicle representation having selectable portions, and an icon at a point selected in the 3D local perceptual space and at a corresponding location in an alternative view of a map having an identified current location of the remote vehicle. The system also includes a payload attached to the remote vehicle. The payload includes a computational module and an integrated sensor suite including a global positioning system, an inertial measurement unit, and a stereo vision camera.

Claims

1. A method for controlling a remote vehicle by interacting with a display, the method comprising:

displaying on the display an interface configured to allow an operator to view an environment surrounding the remote vehicle and control the remote vehicle;

displaying on the display a 3D local perceptual space comprising a coordinate system encompassing the remote vehicle;

displaying on the display a remote vehicle representation;

selecting a location in the 3D local perceptual space;

displaying an icon at the point selected in the 3D local perceptual space and at a corresponding location in an alternative view of a map having an identified current location of the remote vehicle; and

controlling the remote vehicle by selecting and transmitting one or more commands to the remote vehicle.

2. The method of claim 1, wherein displaying the remote vehicle representation comprises displaying selectable portions of the remote vehicle, and wherein at least one selectable portion of the remote vehicle representation comprises a chassis, and selecting one or more commands for the remote vehicle further comprises selecting a stair climbing icon.

3. The method of claim 1, wherein displaying the remote vehicle representation comprises displaying selectable portions of the remote vehicle, and wherein at least one selectable portion of the remote vehicle representation comprises a chassis, and selecting one or more commands for the remote vehicle further comprises selecting a location in a video feed of the display to which the remote vehicle will drive.

4. The method of claim 1, wherein displaying the remote vehicle representation comprises displaying selectable portions of the remote vehicle, and wherein at least one selectable portion of the remote vehicle representation comprises a chassis, and selecting one or more commands for the remote vehicle further comprises selecting a location in a map to which the remote vehicle will drive.

5. The method of claim 4, wherein the map displays an environment of the remote vehicle.

6. The method of claim 4, wherein the map comprises a 3D local perceptual space display.

7. The method of claim 1, wherein displaying the remote vehicle representation comprises displaying selectable portions of the remote vehicle, and wherein at least one selectable portion of the remote vehicle representation comprises a manipulator arm, and selecting one or more commands for the remote vehicle further comprises selecting an object in a video feed of the display for the manipulator arm to manipulate.

8. The method of claim 1, wherein displaying the remote vehicle representation comprises displaying selectable portions of the remote vehicle, and wherein at least one selectable portion of the remote vehicle representation comprises a camera, and selecting one or more commands for the remote vehicle further comprises selecting a location in a video feed of the camera that the camera should zoom to.

9. The method of claim 1, wherein displaying the remote vehicle representation comprises displaying selectable portions of the remote vehicle, and wherein at least one selectable portion of the remote vehicle representation comprises a camera, and selecting one or more commands for the remote vehicle further comprises selecting a location on a map that the camera should pan to.

10. The method of claim 9, wherein wherein displaying the interface comprises displaying a point-and-click interface.

11. The method of claim 9, wherein the the coordinate system of the 3D local perceptual space is an egocentric coordinate system.

12. The method of claim 1, further comprising displaying the alternate view of the map from more than one perspective by dragging on a displayed widget that changes the map between at least a top view and a forward view.

13. The method of claim 1, wherein at least one selectable portion of the remote vehicle representation comprises a manipulator arm, and selecting one or more commands for the remote vehicle further comprises selecting an object in the alternate view of the map for the manipulator arm to manipulate.

14. A system for increasing an operator'"'"'s situational awareness while the operator controls a remote vehicle, the system comprising:

an operator control unit configured to allow the operator to view an environment surrounding the remote vehicle and control the remote vehicle, the operator control unit displaying;

a 3D local perceptual space comprising a coordinate system encompassing the remote vehicle;

a remote vehicle representation; and

an icon at a point selected in the 3D local perceptual space and at a corresponding location in an alternative view of a map having an identified current location of the remote vehicle; and

a payload attached to the remote vehicle and in communication with the operator control unit, the payload comprising;

an integrated sensor suite, and

a computational module receiving data from the integrated sensor suite and determining the 3D local perceptual space based on the received data.

15. The system of claim 14, wherein the integrated sensor suite comprises a global positioning system, an inertial measurement unit, and a stereo vision camera or a range sensor, and wherein the integrated sensor suite further comprises at least one smart camera module, illuminators, and supporting electronics.

16. The system of claim 14, wherein the interface between the operator control unit and the remote vehicle comprises an Ethernet link from the remote vehicle to the payload and a networked radio link from the payload to the operator control unit.

17. The system of claim 14, wherein the computational module determines a resolved motion of the remote vehicle corresponding to the one or more inputted commands, and wherein the computational module correlates localization sensor data and ranging sensor data to determine the 3D local perceptual space.

18. The system of claim 17, further comprising a Joint Architecture for Unmanned Systems gateway configured to interface with the 3D local perceptual space.

19. The system of claim 14, further comprising a behavior engine configured to provide kinodynamic, real-time motion planning that accounts for the dynamics and kinematics of the remote vehicle.

20. The system of claim 19, further comprising a Joint Architecture for Unmanned Systems gateway configured to interface with the behavior engine.