Remote robot control method using three-dimensional pointing procedure and robot control system using the remote robot control method

• US 7,526,362 B2
• Filed: 10/27/2004
• Issued: 04/28/2009
• Est. Priority Date: 03/12/2004
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Choi, Eun-sook, Cho, Joon-kee, Chang, Wook, Bang, Won-chul, Cho, Sung-jung, Kim, Dong-yoon, Oh, Jong-koo, Kang, Kyoung-ho
• Primary Examiner(s): Black; Thomas G
• Assistant Examiner(s): LOUIE, WAE LENNY
• Application Number:
  US10/973,911
• Publication Number:
  US 20050200325A1
• Time in Patent Office: 1,644 Days
• Field of Search: 700/245, 700/231, 318/568.12, 318/567, 318/675, 701/28, 706/45
• US Class Current: 700/245
• CPC Class Codes: G05D 1/0016 characterised by the operator's input device input arrangements for computing systems in general G06F3/00
• CPC Class Codes: G05D 1/0033 by having the operator tracking the vehicle either by direct line of sight or via one or more cameras located remotely from the vehicle
• CPC Class Codes: G08C 2201/91 Remote control based on location and proximity
• CPC Class Codes: G08C 23/04 using light waves, e.g. infrared

Abstract

A remote robot control method using a three-dimensional pointing procedure and a robot control system using the remote robot control method. The robot control system controls a robot in response to a user'"'"'s gesture or sign without modifying the structure of the robot. The remote robot control method includes measuring a distance between a remote controller and a robot, calculating an initial position of the remote controller in an inertial navigation frame of the remote controller, calculating an initial position of the robot in the navigation frame of the remote controller, calculating an origin of the inertial navigation frame of the remote controller shown in an inertial navigation frame of the robot, calculating a new position of the remote controller in the inertial navigation frame of the remote controller, and calculating a new position of the robot in the navigation frame of the robot.

Claims

1. A remote robot control method using a three-dimensional pointing procedure, the remote control method comprising:

(a) measuring a distance between a remote controller and a robot;

(b) calculating an initial position of the remote controller in an inertial navigation frame of the remote controller;

(c) calculating an initial position of the robot in the inertial navigation frame of the remote controller;

(d) calculating an origin of the inertial navigation frame of the remote controller in an inertial navigation frame of the robot;

(e) calculating a new position of the remote controller in the inertial navigation frame of the remote controller;

(f) calculating a new position of the robot in the inertial navigation frame of the robot, and

(g) controlling said robot based on said operations (a)-(f).

2. The method of claim 1, wherein measuring the distance between the remote controller and the robot comprises:

pointing the remote controller at the robot;

transmitting a first infrared signal from the remote controller to the robot;

transmitting a second infrared signal from the robot to the remote controller to notify the remote controller that the robot has detected and received the first infrared signal;

transmitting an ultrasonic signal from the remote controller to the robot at timing T1;

receiving an ultrasonic signal, at the remote controller, reflected by the robot at timing T2; and

calculating an initial distance between the remote controller and the robot by using a difference between the timing T2 and the timing T1.

3. The method of claim 2, further comprising storing the timing T1 and the timing T2 in memory.

4. The method of claim 2, wherein an initial distance D(O) between the remote controller and the robot is given by D(O)=(T2-T1) * a speed of an ultrasonic wave.

5. The method of claim 4, wherein the speed of the ultrasonic wave is approximately 340m/s.

6. The method of claim 1, wherein, when calculating the initial position of the remote controller in the inertial navigation frame of the remote controller the initial position of the remote controller is given by P_u(0)=[0, 0, Z_u(0)]^T when a position coordinate of the remote controller is P_u(t)=[X_u(t), Y_u(t), Z_u(t)]^T.

7. The method of claim 6, wherein, when calculating the initial position of the robot in the inertial navigation frame of the remote controller, the initial position of the robot is given by P_r(0)=[X_r(0), Y_r(0), Z_r]^T when a position coordinate of the robot is P_r(t)=[X_r(t), Y_r(t), Z_r(t)]^T, where Z_r denotes a height of the robot according to its size.

8. The method of claim 7, wherein a positional relationship between the remote controller and the robot is given by C_b^ui_bzD(0)=P_r(0)−

P_u(0)=[X_r(0), Y_r(0), Z_r−

Z_u(0)]^T , where i_bz=[0 0 1]^T is a unit vector in a Z-axis direction of a body frame of the remote controller, D(0) denotes the initial distance between the remote controller and the robot, and C_b^u denotes a direction cosine matrix.

9. The method of claim 8, wherein the direction cosine matrix C_b^u is given by

$C_b^n=[\begin{array}{c}{\theta }_{}\end{array}$

c ⁢

Ψ

c - Φ

c ⁢

Ψ

s + Φ

s ⁢

θ

s ⁢

Ψ

c ⁢

Φ

s ⁢

Ψ

s + Φ

c ⁢

θ

s ⁢

Ψ

c θ

c ⁢

Ψ

s ⁢

Φ

c ⁢

Ψ

s + Φ

s ⁢

θ

s ⁢

Ψ

s - Φ

s ⁢

Ψ

c + Φ

c ⁢

θ

s ⁢

Ψ

s - θ

s ⁢

⁢

Φ

s ⁢

θ

c ⁢

⁢

Φ

c ⁢

θ

c ⁢

] ,

where a subscript c represents a cosine function, a subscript s represents a sine function, and (Φ

,θ

,Ψ

) denotes roll, pitch, and yaw angles, respectively.

10. The method of claim 9, wherein the roll angle Φ

and the pitch angle θ

are given by

$\left[\begin{array}{c}\mathrm{\Phi <br><br>}\\ \mathrm{\theta <br><br>}\end{array}\right]=\left[\begin{array}{c}\arctan <br><br><br><br>2<br><br>\left(-A_{b<br><br><br><br>y},-A_{b<br><br><br><br>z}\right)\\ \arctan <br><br><br><br>2<br><br>(A_{\mathrm{bx}},\sqrt{\left(A_{\mathrm{by}}^2+A_{\mathrm{bz}}^2\right)}\end{array}\right],$

where A_bx, A_by and A_bz denote acceleration measurements on X_b, Y_b, and Z_b axes of the body frame of the remote controller, respectively, obtained from a three-axis accelerometer included in the remote controller.

11. The method of claim 9, wherein the yaw angle Ψ

is given by

$\mathrm{\Psi <br><br>}={\tan }^{-1}<br><br>\left(\frac{-M_{u<br><br><br><br>y}}{M_{u<br><br><br><br>x}}\right),$

where M_ux, M_uy and M_uz denote magnetic measurements in the inertial navigation frame of the remote controller and are given by

$\left[\begin{array}{c}{M}_{\mathrm{ux}}\\ M_{\mathrm{uy}}\\ M_{\mathrm{uz}}\end{array}\right]=\left[\begin{array}{ccc}\cos <br><br><br><br>\mathrm{\theta <br><br>}& \sin <br><br><br><br>\mathrm{\theta <br><br>sin}<br><br><br><br>\mathrm{\Phi <br><br>}& \sin <br><br><br><br>\mathrm{\theta <br><br>cos}<br><br><br><br>\mathrm{\Phi <br><br>}\\ 0& \cos <br><br><br><br>\mathrm{\Phi <br><br>}& -\sin <br><br><br><br>\mathrm{\Phi <br><br>}\\ -\sin <br><br><br><br>\mathrm{\theta <br><br>}& \cos <br><br><br><br>\mathrm{\theta <br><br>sin}<br><br><br><br>\mathrm{\Phi <br><br>}& \cos <br><br><br><br>\mathrm{\theta <br><br>cos}<br><br><br><br>\mathrm{\Phi <br><br>}\end{array}\right]<br><br>\left[\begin{array}{c}{M}_{\mathrm{bx}}\\ M_{\mathrm{by}}\\ M_{\mathrm{bz}}\end{array}\right],$

where M_bx, M_by and M_bz denote magnetic measurements on the X_b, Y_b, and Z_b axes of the body frame of the remote controller, respectively, obtained from a three-axis magnetic sensor included in the remote controller.

12. The method of claim 1, wherein, when calculating the initial position of the robot in the inertial navigation frame of the remote controller, the initial position of the robot is given by P_r(0)=[X_r(0), Y_r(0), Z_r]^T when a position coordinate of the robot is P_r(t)=[X_r(t), Y_r(t), Z_r(t)]^T, where Z_r denotes a height of the robot according to its size.

13. The method of claim 1, wherein calculating the origin of the inertial navigation frame of the remote controller in the inertial navigation frame of the robot comprises:

calculating a difference between an absolute yaw angle of the remote controller in the navigation frame of the remote controller and an absolute yaw angle of the robot in the navigation frame of the robot; and

calculating the direction cosine matrix C_U^R specifying a relationship between the inertial navigation frame of the remote controller and the inertial navigation frame of the robot,

wherein the origin [0]_R of the inertial navigation frame of the remote controller in the inertial navigation frame of the robot being given by [0]_R=[P_r(0)]_R−

C_U^R[P_r(0)]_U, where [P_r(0)]_U denotes the initial position of the robot in the inertial navigation frame of the remote controller and [P_r(0)]_R denotes the initial position of the robot in the navigation frame of the remote controller.

14. The method of claim 1, wherein calculating a new position of the remote controller in the inertial navigation frame of the remote controller and calculating a new position of the robot in the inertial navigation frame of the robot are performed by increasing a k sampling time by one for every T sampling cycle, where a sampling time t is given by t=t+kT.

15. The method of claim 1, wherein calculating a new position of the remote controller in the inertial navigation frame of the remote controller comprises:

calculating the positional relationship between a starting position at which the robot is located and a target position to which the robot should move by C_b^u(t)i_bzD(t)=P_r(t)−

P_u(t)=[X_r(t)−

X_u(t),Y_r(t)−

Y_u(t),Z_r−

Z_u(t)]^T wherein the position coordinate of the remote controller in the navigation frame of the remote controller is P_u(t)=[X_u(t), Y_u(t), Z_u(t)]^T, the remote controller points at the target position, and the position coordinate of the robot in the inertial navigation frame of the remote controller is P_r(t)=[X_r(t), Y_r(t), Z_r(t)]^T when the robot arrives a the target position;

calculating {dot over (P)}_u=C_b^uV_u, {dot over (V)}_u=C_b^uA_b−

G, {dot over (Φ

)}=ω

_bx+(ω

_bysinΦ

+ω

_bzcosΦ

)tanΘ

,

$\stackrel{.}{\mathrm{\theta <br><br>}}={\mathrm{\omega <br><br>}}_{b<br><br><br><br>y}<br><br>\cos <br><br><br><br>\mathrm{\Phi <br><br>}-{\mathrm{\omega <br><br>}}_{b<br><br><br><br>z}<br><br>\sin <br><br><br><br>\mathrm{\Phi <br><br>},\stackrel{.}{\mathrm{\Psi <br><br>}}=\frac{{\mathrm{\omega <br><br>}}_{b<br><br><br><br>y}<br><br>\sin <br><br><br><br>\mathrm{\Phi <br><br>}+{\mathrm{\omega <br><br>}}_{b<br><br><br><br>z}<br><br>\cos <br><br><br><br>\mathrm{\Phi <br><br>}}{\cos <br><br><br><br>\mathrm{\Phi <br><br>}};<br><br>\mathrm{and}$

calculating X_r(t), Y_r(t), and D(t) by calculating C_b^u and P_u(t), wherein D(t) denotes the distance between the remote controller and a top surface of the robot measured when the robot arrives at the target position, Z_r denotes a height of the robot, A_b=[A_bxA_byA_bz]^T denotes a measurement output from the three-axis accelerometer, (Φ

,Θ

,Ψ

) denotes the roll, pitch, and yaw angles, respectively, which are known as the Euler angles, each of ω

_bx, ω

_by, and ω

_bz denotes a measurement output from a three-axis gyroscope of the remote controller, and G=[0098]^T denotes the acceleration of gravity.

16. The method of claim 1, wherein, when calculating a new position of the robot in the inertial navigation frame of the robot, the position [P_r(t)]_R of the robot is given by [P_r(t)]_R=[0]_R+C_U^R[P_r(t)]_U, where [0]_R denotes the origin of the inertial navigation frame of the remote controller in the inertial navigation frame of the robot and C_U^R denotes the direction cosine matrix specifying the relationship between the inertial navigation frame of the remote controller and the inertial navigation frame of the robot.

17. The method of claim 1, further comprising moving the robot in response to the calculated position information.

18. A robot control system comprising:

a remote controller operable by a user; and

a robot movable in accordance with a movement of the remote controller,

wherein the remote controller comprises;

a sensor to provide measurements necessary for calculating orientation and position information of the remote controller;

an orientation calculator to calculate an orientation of the remote controller based on the measurements provided by the sensor;

a position calculator to calculate a position of the remote controller based on the measurements provided by the sensor;

a target position calculator to calculate a target position indicated by the remote controller based on the orientation and position information calculated by the orientation calculator and the position calculator;

a coordinate converter to convert the target position calculated by the target position calculator in an inertial navigation frame of the remote controller into a target position shown in an inertial navigation frame of the robot;

a command generator to command the robot to move to the converted target position; and

a distance sensor to measure a distance between the remote controller and the robot, and

wherein the robot comprises;

a navigation controller to drive a navigation motor in response to a command from the command generator; and

a sensor to exchange signals with the distance sensor to measure the distance between the remote controller and the robot.

19. The robot control system of claim 18, wherein the sensor of the remote controller comprises an accelerometer, a gyroscope, a magnetic sensor, or a combination thereof.

20. The robot control system of claim 18, wherein the distance sensor comprises:

an infrared receiver/transmitter to transmit a first infrared signal to the sensor of the robot and to receive a second infrared signal transmitted from the sensor of the robot; and

an ultrasonic wave receiver/transmitter to transmit an ultrasonic signal to the robot and to receive the ultrasonic signal reflected by the robot.

21. The robot control system of claim 20, wherein the distance sensor further comprises a memory to store timing T1 when the ultrasonic wave receiver/transmitter transmits the ultrasonic signal and timing T2 when the ultrasonic wave receiver/transmitter receives the ultrasonic signal.

22. The robot control system of claim 18, wherein the sensor of the robot comprises an infrared receiver/transmitter to receive a first infrared signal transmitted from the distance sensor and to transmit a second infrared signal to the distance sensor.

23. The robot control system of claim 18, wherein:

the target position calculator calculates an initial position of the remote controller by P_u(0)=[0, 0, Z_u(0)]^T, when a position coordinate of the remote controller is P_u(t)=[X_u(t), Y_u(t), Z_u(t)]^T;

the target position calculator calculates an initial position of the robot by P_r(0)=[X_r(0), Y_r(0), Z_r]^T when a position coordinate of the robot is P_r(t)=[X_r(t), Y_r(t), Z_r(t)]^T; and

an initial positional relationship between the remote controller and the robot is given by C_b^ui_bzD(0)=P_r(0)−

P_u(0)=[X_r(0), Y_r(0), Z_r−

Z_u(0)]^T, where Z_r denotes a height of the robot, I_bz=[0 0 1]^T is a unit vector in a Z-axis direction of a body frame of the remote controller, D(0) denotes the initial distance between the remote controller and the robot, and C_b^u denotes a direction cosine matrix.

24. The robot control system of claim 23, wherein the direction cosine matrix C_b^u is given by

$C_b^n=[\begin{array}{c}{\theta }_{}\end{array}$

c ⁢

Ψ

c - Φ

c ⁢

Ψ

s + Φ

s ⁢

θ

s ⁢

Ψ

c Φ

s ⁢

Ψ

s + Φ

c ⁢

θ

s ⁢

Ψ

c θ

c ⁢

Ψ

s Φ

c ⁢

Ψ

s + Φ

s ⁢

θ

s ⁢

Ψ

s - Φ

s ⁢

Ψ

c + Φ

c ⁢

θ

s ⁢

Ψ

s - θ

s Φ

s ⁢

θ

c Φ

c ⁢

θ

c ] ,

where a subscript c represents a cosine function, a subscript s represents a sine function, and (Φ

,θ

,Ψ

) denotes roll, pitch, and yaw angles, respectively.

25. The robot control system of claim 24, wherein the roll angle Φ

and the pitch angle θ

are given by

$\left[\begin{array}{c}\mathrm{\Phi <br><br>}\\ \mathrm{\theta <br><br>}\end{array}\right]=\left[\begin{array}{c}\arctan <br><br><br><br>2<br><br>\left(-A_{\mathrm{by}},-A_{\mathrm{bz}}\right)\\ \arctan <br><br><br><br>2<br><br>(A_{\mathrm{bx},}<br><br>\sqrt{\left(A_{\mathrm{by}}^2+A_{\mathrm{bz}}^2\right)}\end{array}\right],$

where A_bx, A_by and A_bz denote acceleration measurements on X_b, Y_b, and Z_b axes of the body frame of the remote controller, respectively, obtained from a three-axis accelerometer included in the remote controller.

26. The robot control system of claim 24, wherein the yaw angle Ψ

is given by

$\mathrm{\Psi <br><br>}={\tan }^{-1}<br><br>\left(\frac{-M_{\mathrm{uy}}}{M_{\mathrm{ux}}}\right),$

where M_ux, M_uy and M_uz denote magnetic measurements shown in the inertial navigation frame of the remote controller and are given by

$\left[\begin{array}{c}{M}_{\mathrm{ux}}\\ M_{\mathrm{uy}}\\ M_{\mathrm{uz}}\end{array}\right]=\left[\begin{array}{ccc}\cos <br><br><br><br>\mathrm{\theta <br><br>}& \sin <br><br><br><br>\mathrm{\theta <br><br>}<br><br><br><br>\sin <br><br><br><br>\mathrm{\Phi <br><br>}& \sin <br><br><br><br>\mathrm{\theta <br><br>}<br><br><br><br>\cos <br><br><br><br>\mathrm{\Phi <br><br>}\\ 0& \cos <br><br><br><br>\mathrm{\Phi <br><br>}& -\sin <br><br><br><br>\mathrm{\Phi <br><br>}\\ -\sin <br><br><br><br>\mathrm{\theta <br><br>}& \cos <br><br><br><br>\mathrm{\theta <br><br>}<br><br><br><br>\sin <br><br><br><br>\mathrm{\Phi <br><br>}& \cos <br><br><br><br>\mathrm{\theta <br><br>}<br><br><br><br>\cos <br><br><br><br>\mathrm{\Phi <br><br>}\end{array}\right]<br><br>\left[\begin{array}{c}{M}_{\mathrm{bx}}\\ M_{\mathrm{by}}\\ M_{\mathrm{bz}}\end{array}\right],$

where M_bx, M_by and M_bz denote magnetic measurements on the X_b, Y_b, and Z_b axes of the body frame of the remote controller, respectively, obtained from a three-axis magnetic sensor included in the remote controller.

27. The robot control system of claim 18, wherein

a position coordinate of the remote controller in the inertial navigation frame of the remote controller is P_u(t)=[X_u(t), Y_u(t), Z_u(t)]^T, when the remote controller points at the target position,

a position coordinate of the robot in the inertial navigation frame of the remote controller is P_r(t)=[X_r(t),Y_r(t),Z_r(t)]^T when the robot arrives at the target position,

and the target position calculator calculates the positional relationship between a starting position at which the robot is located and a target position to which the robot should move by C_b^u(t)i_bzD(t)=P_r)t)−

P_u(t)=[xr(t)−

x_u(t),y_r(t)−

y_u(t),z_r−

z_u(t)]^T and by {dot over (P)}_u=C_b^uV_u, {dot over (V)}_u=C_b^uA_b−

G, {dot over (Φ

)}=ω

_bx+(ω

_bysin Φ

+ω

_bzcos Φ

)tan θ

, {dot over (θ

)}=ω

_bycos Φ

−

ω

_bzsin Φ

, and

$\stackrel{.}{\mathrm{\Psi <br><br>}}=\frac{{\mathrm{\omega <br><br>}}_{\mathrm{by}}<br><br><br><br>\sin <br><br><br><br>\mathrm{\Phi <br><br>}+{\mathrm{\omega <br><br>}}_{\mathrm{bz}}<br><br><br><br>\cos <br><br><br><br>\mathrm{\Phi <br><br>}}{\cos <br><br><br><br>\mathrm{\Phi <br><br>}},$

where D(t) denotes the distance between the remote controller and a top surface of the robot measured when the robot arrives at the target position, Z_r denotes a height of the robot according to its size, A_b=[A_bxA_byA_bz]^T denotes a measurement output from the three-axis accelerometer, (Φ

,θ

,Ψ

) denotes the roll, pitch, and yaw angles, respectively, each of ω

_bx, ω

_by, and ω

_bz denotes a measurement output from a three-axis gyroscope of the remote controller, and G=[0098]^T denotes the acceleration of gravity.

28. The robot control system of claim 18, wherein the coordinate converter calculates a direction cosine matrix C_U^R specifying a relationship between the inertial navigation frame of the remote controller and the inertia navigation frame of the robot by calculating a difference between an absolute yaw angle of the navigation frame of the remote controller and an absolute yaw angle of the navigation frame of the robot, and an origin [0]_R of the inertial navigation frame of the remote controller in the navigation frame of the robot is given by [0]_R=[P_r(0)]_R−

C_U^R[P_r(0)]_U, where [P_r(0)]_U denotes the initial position of the robot in the inertial navigation frame of the remote controller, [P_r(t)]_R denotes the initial position of the robot in the inertial navigation frame of the remote controller, and the position [P_r(t)]_R of the robot in the inertial navigation frame of the robot is given by [P_r(t)]_R=[0]_R=C_U^R[P_r(t)]_U.