Remotely controlled self-balancing robot including kinematic image stabilization

• US 8,160,747 B1
• Filed: 10/24/2008
• Issued: 04/17/2012
• Est. Priority Date: 10/24/2008
• Status: Active Grant

• Current Assignee: Anybots 2.0, Inc.
• Original Assignee: Anybots, Inc.
• Inventors: Blackwell, Trevor, Casner, Daniel, Wiley, Scott
• Primary Examiner(s): Tran, Khoi
• Assistant Examiner(s): Nguyen, Bao Long T
• Application Number:
  US12/258,268
• Time in Patent Office: 1,271 Days
• Field of Search: None
• US Class Current: 700/259
• CPC Class Codes: B25J 9/162 Mobile manipulator, movable base with manipulator arm mounted on it
• CPC Class Codes: G05B 2219/39215 Adaptive control with stabilizing compensation
• CPC Class Codes: G05B 2219/40298 Manipulator on vehicle, wheels, mobile
• CPC Class Codes: G05D 1/0038 by providing the operator with simple or augmented images from one or more cameras located onboard the vehicle, e.g. tele-operation images analyzed by a computer and used for automatic navigation G05D1/0246
• CPC Class Codes: G05D 1/0891 specially adapted for land vehicles

Abstract

Systems and methods as described for providing visual telepresence to an operator of a remotely controlled robot. The robot includes both video cameras and pose sensors. The system can also comprise a head-tracking sensor to monitor the orientation of the operator'"'"'s head. These signals can be used to aim the video cameras. The controller receives both the video signals and the pose sensor signals from the robot, and optionally receives head-tracking signals from the head-tracking sensor. The controller stitches together the various video signals to form a composite video signal that maps to a robot view. The controller renders an image to a display from that portion of the composite video signal that maps to an operator view. The relationship of the operator view to the robot view is varied according to the signals from the pose sensors and the head-tracking sensor.

Claims

1. A robotic system comprising:

a robot including a plurality of pose sensors;

a plurality of video cameras attached to the robot and including

a pair of stereo cameras aimed approximately in the same direction, each of the pair of stereo cameras having a field of view, and

a peripheral camera aimed differently than the pair of stereo cameras and having a field of view that at least partially overlaps the field of view of at least one of the pair of stereo cameras;

a display; and

a controller in communication between the robot and the display and configured to

receive video signals from the plurality of video cameras,

receive signals from the pose sensors, and

render an image to the display by

stitching together the video signal from the peripheral camera with the video signals from the pair of stereo cameras to form a composite video signal that maps to a robot view,

repositioning an operator view relative to the robot view to correct for motion of the robot using the signals received from the pose sensors, and

rendering to the display a portion of the composite video signal that maps to the operator view as the rendered image.

2. The robotic system of claim 1 wherein the robot is self-balancing.

3. The robotic system of claim 1 wherein the plurality of pose sensors are selected from the group consisting of an inertial measurement unit, an optical encoder, and a linear potentiometer.

4. The robotic system of claim 1 wherein the plurality of video cameras are attached to a head of the robot.

5. The robotic system of claim 1 wherein the plurality of cameras are arranged to be approximately confocal.

6. The robotic system of claim 1 wherein the display comprises a head-mounted display.

7. The robotic system of claim 1 wherein the controller is further configured to apply lens corrections to the video signals received from the plurality of cameras.

8. The robotic system of claim 1 wherein the controller is further configured to generate a heads up display and add the heads up display to the rendered image.

9. The robotic system of claim 8 wherein the robot further includes a data source and the controller is further configured to generate the heads up display from data received from the data source.

10. The robotic system of claim 8 further comprising a data source that is external to the robot, wherein the controller is further configured to generate the heads up display from data received from the data source.

11. The robotic system of claim 1 wherein the robot and the controller are in communication across a network.

12. The robotic system of claim 1 further comprising a head-tracking sensor configured to track an orientation of a head of an operator, and wherein the controller is further configured to additionally reposition the operator view relative to the robot view to correct for motion of the head of the operator using the signals received from the head-tracking sensor.

13. The robotic system of claim 12 wherein the head-tracking sensor and the display are both integrated within a head-mounted display.

14. The robotic system of claim 12 further comprising a second controller configured to control the orientations of the plurality of video cameras responsive to the head-tracking sensor.