Robot control information
First Claim
Patent Images
1. A method comprising:
- moving a mobile device with a desired motion for a robot that is remote from the mobile device, the mobile device having a camera;
determining a trajectory of the mobile device using images captured by the camera;
converting the trajectory of the mobile device into the desired motion of the robot;
generating control information for the robot to move with the desired motion;
the method further comprises calibrating a transformation that converts the trajectory of the mobile device to the desired motion of the robot, wherein converting the trajectory of the mobile device into the desired motion of the robot comprises using the transformation, wherein calibrating the transformation comprises;
coupling the mobile device to the robot;
moving the robot while determining a training trajectory of the mobile device;
using movement of the robot and the training trajectory of the mobile device to generate a calibration transformation of the movement of the robot to the training trajectory of the mobile device; and
inverting the calibration transformation to produce the transformation of the trajectory of the mobile device to the desired motion of the robot.
1 Assignment
0 Petitions
Accused Products
Abstract
Vision based tracking of a mobile device is used to remotely control a robot. For example, images captured by a mobile device, e.g., in a video stream, are used for vision based tracking of the pose of the mobile device with respect to the imaged environment. Changes in the pose of the mobile device, i.e., the trajectory of the mobile device, are determined and converted to a desired motion of a robot that is remote from the mobile device. The robot is then controlled to move with the desired motion. The trajectory of the mobile device is converted to the desired motion of the robot using a transformation generated by inverting a hand-eye calibration transformation.
14 Citations
23 Claims
-
1. A method comprising:
-
moving a mobile device with a desired motion for a robot that is remote from the mobile device, the mobile device having a camera; determining a trajectory of the mobile device using images captured by the camera; converting the trajectory of the mobile device into the desired motion of the robot; generating control information for the robot to move with the desired motion; the method further comprises calibrating a transformation that converts the trajectory of the mobile device to the desired motion of the robot, wherein converting the trajectory of the mobile device into the desired motion of the robot comprises using the transformation, wherein calibrating the transformation comprises; coupling the mobile device to the robot; moving the robot while determining a training trajectory of the mobile device; using movement of the robot and the training trajectory of the mobile device to generate a calibration transformation of the movement of the robot to the training trajectory of the mobile device; and inverting the calibration transformation to produce the transformation of the trajectory of the mobile device to the desired motion of the robot. - View Dependent Claims (2, 3, 4, 5, 6)
-
-
7. A mobile device comprising:
-
a camera for capturing images of an environment while the mobile device is moved; a wireless interface for communicating with a remote robot controller; and a processor coupled to receive captured images from the camera and coupled to the wireless interface, the processor configured to determine a trajectory of the mobile device using the captured images, convert the trajectory of the mobile device into a desired motion of a remote robot, and to cause the wireless interface to transmit the desired motion of the remote robot to the remote robot controller, the processor being further configured to calibrate a transformation that converts the trajectory of the mobile device to the desired motion of the remote robot, wherein the processor is configured to convert the trajectory of the mobile device into the desired motion of the remote robot using the transformation, wherein the processor is configured to calibrate the transformation that converts the trajectory of the mobile device to the desired motion of the remote robot by being configured to; determine a training trajectory of the mobile device while the remote robot holds the mobile device and during movement of the remote robot; use the movement of the remote robot and the training trajectory to generate a calibration transformation of the movement of the remote robot to the training trajectory of the mobile device; and invert the calibration transformation to produce the transformation of the trajectory of the mobile device to the desired motion of the remote robot. - View Dependent Claims (8)
-
-
9. A controller for controlling a robot, the controller comprising:
-
an external interface for communicating with a remote mobile device that has a camera for capturing images of an environment while moving with a desired motion for the robot; a robot interface for communicating and controlling the robot; and a processor coupled to the external interface and the robot interface, the processor being configured to convert a trajectory of the remote mobile device determined using captured images from the camera to the desired motion of the robot, and control the robot to move with the desired motion through the robot interface, wherein the processor is further configured to calibrate a transformation that converts the trajectory of the remote mobile device to the desired motion of the robot, wherein the processor is configured to convert the trajectory of the remote mobile device into the desired motion of the robot using the transformation, wherein the processor is configured to calibrate the transformation that converts the trajectory of the remote mobile device to the desired motion of the robot by being configured to; cause movement of the robot while the robot holds the remote mobile device; use the movement of the robot and a training trajectory of the remote mobile device determined from the movement of the robot while holding the remote mobile device to generate a calibration transformation of the movement of the robot to the training trajectory of the remote mobile device; and invert the calibration transformation to produce the transformation of the trajectory of the remote mobile device to the desired motion of the robot. - View Dependent Claims (10, 11, 12, 13, 14, 15, 16, 17)
-
-
18. A system comprising:
-
means for determining a trajectory of a mobile device using images captured by the mobile device while the mobile device is moved with a desired motion for a robot that is remote from the mobile device; means for converting the trajectory of the mobile device into the desired motion of the robot; means for generating control information for the robot to move with the desired motion; and means for calibrating a transformation that converts the trajectory of the mobile device to the desired motion of the robot, wherein the means for converting the trajectory of the mobile device into the desired motion of the robot uses the transformation, wherein the means for calibrating the transformation determines a training trajectory of the mobile device while the robot holds the mobile device and during movement of the robot, uses the movement of the robot and the training trajectory to generate a calibration transformation of the movement of the robot to the training trajectory of the mobile device; and
inverts the calibration transformation to produce the transformation of the trajectory of the mobile device to the desired motion of the robot. - View Dependent Claims (19, 20, 21, 22)
-
-
23. A storage medium including program code stored thereon, comprising:
-
program code to determine a trajectory of a mobile device using images captured by the mobile device while the mobile device is moved with a desired motion for a robot that is remote from the mobile device; program code to convert the trajectory of the mobile device into the desired motion of the robot; program code to generate control information for the robot to move with the desired motion; and program code to calibrate a transformation that converts the trajectory of the mobile device to the desired motion of the robot, wherein the program code to convert the trajectory of the mobile device into the desired motion of the robot uses the transformation, the program code to calibrate the transformation comprises; program code to determine a training trajectory of the mobile device while the robot holds the mobile device and during movement of the robot; program code to use the movement of the robot and the training trajectory to generate a calibration transformation of the movement of the robot to the training trajectory of the mobile device; and program code to invert the calibration transformation to produce the transformation of the trajectory of the mobile device to the desired motion of the robot.
-
Specification