Remote Vehicle Control System and Method

• US 20150134146A1
• Filed: 12/31/2014
• Published: 05/14/2015
• Est. Priority Date: 05/12/2010
• Status: Active Grant

• Current Assignee: Flir Detection, Inc. (Teledyne Technologies Incorporated)
• Original Assignee: iRobot Corporation
• Inventors: Pack, Robert Todd, Allis, Daniel P.
• Granted Patent: US 9,658,615 B2
• Time in Patent Office: Days
• US Class Current: 701/2
• CPC Class Codes: G01C 21/26 specially adapted for navigation in a road network
• CPC Class Codes: G05D 1/0011 associated with a remote control arrangement
• CPC Class Codes: G05D 1/0038 by providing the operator with simple or augmented images from one or more cameras located onboard the vehicle, e.g. tele-operation images analyzed by a computer and used for automatic navigation G05D1/0246

Abstract

A system includes an operator control unit having a point-and-click interface configured to allow the operator to control the remote vehicle by inputting one or more commands via the point-and-click interface. The operator control unit displays a 3D local perceptual space comprising an egocentric coordinate system encompassing a predetermined distance centered on the remote vehicle, a remote vehicle representation having selectable portions, and an icon at a point selected in the 3D local perceptual space and at a corresponding location in an alternative view of a map having an identified current location of the remote vehicle. The system also includes a payload attached to the remote vehicle. The payload includes a computational module and an integrated sensor suite including a global positioning system, an inertial measurement unit, and a stereo vision camera.

Claims

1. A system for increasing an operator'"'"'s situational awareness while the operator controls a remote vehicle, the system comprising:

an operator control unit having a point-and-click interface configured to allow the operator to view an environment surrounding the remote vehicle and control the remote vehicle by inputting one or more commands via the point-and-click interface, the operator control unit displaying;

a 3D local perceptual space comprising an egocentric coordinate system encompassing a predetermined distance centered on the remote vehicle;

a remote vehicle representation having selectable portions, the selectable portions actuatable by the operator control unit; and

an icon at a point selected in the 3D local perceptual space and at a corresponding location in an alternative view of a map having an identified current location of the remote vehicle; and

a payload attached to the remote vehicle and in communication with at least one of the remote vehicle and the operator control unit, the payload comprising;

an integrated sensor suite including a global positioning system, an inertial measurement unit, and a stereo vision camera or a range sensor, and

a computational module receiving data from the integrated sensor suite and determining the 3D local perceptual space based on the received data, the computational module determining a resolved motion of the remote vehicle corresponding to the one or more inputted commands.

2. The system of claim 1, wherein the integrated sensor suite further comprises at least one smart camera module, illuminators, and supporting electronics.

3. The system of claim 1, wherein an interface between the operator control unit and the remote vehicle comprises an Ethernet link from the remote vehicle to the payload and a networked radio link from the payload to the operator control unit.

4. The system of claim 1, wherein the computational module correlates localization sensor data and ranging sensor data to determine the 3D local perceptual space.

5. The system of claim 4, further comprising a Joint Architecture for Unmanned Systems gateway configured to interface with the 3D local perceptual space.

6. The system of claim 1, further comprising a behavior engine configured to provide kinodynamic, real-time motion planning that accounts for the dynamics and kinematics of the remote vehicle.

7. The system of claim 6, further comprising a Joint Architecture for Unmanned Systems gateway configured to interface with the behavior engine.

8. A method for controlling a remote vehicle by interacting with a display, the method comprising:

displaying on the display a point-and-click interface configured to allow an operator to view an environment surrounding the remote vehicle and control the remote vehicle by inputting one or more commands via the point-and-click interface;

displaying on the display a 3D local perceptual space comprising an egocentric coordinate system encompassing a predetermined distance centered on the remote vehicle;

displaying on the display a remote vehicle representation having selectable portions actuatable through the point-and-click interface;

selecting one or more commands for the remote vehicle, the one or more commands comprising a command for;

selecting a location in the 3D local perceptual space; and

displaying an icon at the point selected in the 3D local perceptual space and at a corresponding location in an alternative view of a map having an identified current location of the remote vehicle.

9. The method of claim 8, wherein at least one selectable portion of the remote vehicle representation comprises a chassis, and selecting one or more commands for the remote vehicle further comprises selecting a stair climbing icon.

10. The method of claim 8, wherein at least one selectable portion of the remote vehicle representation comprises a chassis, and selecting one or more commands for the remote vehicle further comprises selecting a location in a video feed of the display to which the remote vehicle will drive.

11. The method of claim 8, wherein at least one selectable portion of the remote vehicle representation comprises a chassis, and selecting one or more commands for the remote vehicle further comprises selecting a location in a map to which the remote vehicle will drive.

12. The method of claim 11, wherein the map displays an environment of the remote vehicle.

13. The method of claim 11, wherein the map comprises a 3D local perceptual space display.

14. The method of claim 8, wherein at least one selectable portion of the remote vehicle representation comprises a manipulator arm, and selecting one or more commands for the remote vehicle further comprises selecting an object in a video feed of the display for the manipulator arm to manipulate.

15. The method of claim 8, wherein at least one selectable portion of the remote vehicle representation comprises a camera, and selecting one or more commands for the remote vehicle further comprises selecting a location in a video feed of the camera that the camera should zoom to.

16. The method of claim 8, wherein at least one selectable portion of the remote vehicle representation comprises a camera, and selecting one or more commands for the remote vehicle further comprises selecting a location on a map that the camera should pan to.

17. The method of claim 16, wherein the map comprises a 3D local perceptual space display.

18. The method of claim 16, wherein the map displays an environment of the remote vehicle.

19. The method of claim 8, further comprising looking at the alternate view of the map from more than one perspective by dragging on a displayed widget that changes the map between at least a top view and a forward view.

20. The method of claim 8, wherein at least one selectable portion of the remote vehicle representation comprises a manipulator arm, and selecting one or more commands for the remote vehicle further comprises selecting an object in the alternate view of the map for the manipulator arm to manipulate.