System and Method for Real-Time Calculating Location

• US 20080310682A1
• Filed: 12/14/2005
• Published: 12/18/2008
• Est. Priority Date: 10/31/2005
• Status: Active Grant

• Current Assignee: Electronics and Telecommunications Research Institute
• Original Assignee: Electronics and Telecommunications Research Institute
• Inventors: Yu, Won Pil, Lee, Jae Yeong, Chae, Heesung, Cho, Young Jo
• Granted Patent: US 8,027,515 B2
• Time in Patent Office: Days
• US Class Current: 382/106
• CPC Class Codes: G01S 5/16 using electromagnetic waves other than radio waves
• CPC Class Codes: G05D 1/0246 using a video camera in combination with image processing means

Abstract

Provided is a system and method for providing location information of robot in real time using an artificial mark. The system includes: an image processing module for obtaining an image signal by photographing artificial marks installed at a predetermined space with a space coordinate and detecting an image coordinate of artificial mark from the obtaining the image signal; a location calculating module for calculating a current location by comparing the image coordinate of detected artificial mark and a pre-stored space coordinate of artificial mark; and an artificial mark identifying module for updating current location information by selectively using one of an artificial mark tracing process and an image coordinate estimating process.

Claims

1. A system of calculating a location based on an artificial mark in real time, the system comprising:

an image processing module for obtaining an image signal by photographing artificial marks installed at a predetermined space with a space coordinate and detecting an image coordinate of artificial mark from the obtaining the image signal,

a location calculating module for calculating a current location by comparing the image coordinate of detected artificial mark and a pre-stored space coordinate of artificial mark, and

an artificial mark identifying module for updating current location information by selectively using one of an artificial mark tracing process and an image coordinate estimating process, where the artificial mark tracing process identifies an artificial mark detected at a current time as one of artificial marks detected at a previous time which is closest to the currently detected artificial mark, and the image coordinate estimating process identifies an artificial mark by estimating an image coordinate of artificial mark using current location information and a pre-stored space coordinate of artificial mark and comparing the estimated image coordinate and the detected image coordinate.

2. The system of claim 1, wherein the artificial mark has a unique ID and is a light source that emits a predetermined wavelength of light.

3. The system of claim 2, further comprising a mark control module for controlling the artificial marks to be turned on and off.

4. The system of claim 2, wherein the camera includes an optical filter that filters the light emitted from a light emitting unit of the artificial mark to pass a predetermined wavelength of light.

5. The system of claim 1, further comprising an image estimating module for performing the image coordinate estimating process using the calculated current location information and the pre-stored space coordinate of artificial mark.

6. The system of claim 1, wherein the camera is installed at a robot and the system includes a remote server that receives and processes a signal from the camera through a wired link and a wireless link.

7. A method of calculating a location in real time based on an artificial mark, comprising the steps of:

a) calculating an current initial location using an image coordinate recognized through turning on and off an artificial mark or assigning the current initial location as a predetermined location;

b) identifying detected artificial marks through an artificial mark tracing process while traveling at a mark identifying module, where the artificial mark tracing process identifies an artificial mark detected at a current time as one of artificial marks detected at a previous time which is closest to the currently detected artificial mark; and

c) updating a current location at a location calculating module based on the image coordinate of the detected and identified artificial mark and a pre-stored space coordinate in a main control module.

10. The method of claim 7, wherein in the step a), the current location is calculated by calculating an initial current location at a location calculating module using an image coordinate detected at an image processing module through turning on and off a predetermined artificial mark at a mark control module and a space coordinate pre-stored in a main control module.

11. The method of claim 10, wherein the step a) includes the steps of:

a-1) turning on/off a predetermined artificial mark at a mark control module;

a-2) photographing images of the predetermined artificial mark which is turned on/off using a camera included in an image processing module;

a-3) detecting a predetermined artificial mark through processing an image signal obtained through photographing the images;

a-4) calculating an image coordinate of the detected artificial mark;

a-5) determining whether the number of detected artificial marks is larger than a predetermined number; and

a-6) calculating an image coordinate through turning on/off other artificial marks until the number of detected artificial marks is larger than the predetermined number when the number of detected artificial mark is smaller than the predetermined number, or deciding an initial current location from an image coordinate of detected artificial mark and the pre-stored space coordinate at a main control module when the number of detected artificial marks is larger than the predetermined number.

12. The method of claim 11, wherein the number of detected artificial marks is two.

13. The method of claim 12, wherein an error corrected image coordinate (u, v) is calculated from the image coordinate (x, y) of detected artificial mark through correcting error due to a camera lens based on following equations, and the calculated error corrected image coordinate (u, v) is used to calculate a current location, wherein the equations are:

$\begin{array}{cc}\left[\begin{array}{c}{x}_d\\ y_d\\ 1\end{array}\right]={\left[\begin{array}{ccc}{f}_x& 0& c_x\\ 0& f_y& c_y\\ 0& 0& 1\end{array}\right]}^{-1}<br><br>\left[\begin{array}{c}x\\ y\\ 1\end{array}\right]\Rightarrow <br><br>\left\{\begin{array}{c}{x}_d=\left[x-c_x\right]/f_x\\ y_d=\left[y-c_y\right]/f_y\end{array}\right\}& \mathrm{Eq}.<br><br>1\\ \left[\begin{array}{c}{x}_u\\ y_u\end{array}\right]=\frac{1}{1+k_1<br><br>r^2+k_2<br><br>r^4+k_3<br><br>r^6}<br><br>\left[\begin{array}{c}{x}_d\\ y_d\end{array}\right],r^2=x_u^2+y_u^2& \mathrm{Eq}.<br><br>2\\ \left[\begin{array}{c}u\\ v\\ 1\end{array}\right]=\left[\begin{array}{ccc}{f}_x& 0& c_x\\ 0& f_y& c_y\\ 0& 0& 1\end{array}\right]<br><br>\left[\begin{array}{c}{x}_u\\ y_u\\ 1\end{array}\right]& \mathrm{Eq}.<br><br>3\end{array}$

where f_x and f_y denote a focus distance, C_x and C_y denote internal parameters of camera representing an image coordinate at a center of lens, and k₁, k₂ and k₃ denote constants representing the lens error coefficients obtained through calibration.

14. The method of claim 13, wherein the initial current location (r_x, r_y, θ

) is calculated from the error corrected image coordinate (u_i, v_i) and (u_j and v_j) and the stored space coordinate (X_i, Y_i) and (X_j, Y_j) based on equations;

$\begin{array}{cc}s=\frac{D}{d}=\frac{\sqrt{{\left(X_i-X_j\right)}^2+{\left(Y_i-Y_j\right)}^2}}{\sqrt{{\left(u_i-u_j\right)}^2+{\left(v_i-v_j\right)}^2}}& \mathrm{Eq}.<br><br>4\\ \cos <br><br><br><br>\mathrm{\theta <br><br>}=\frac{\left(u_j-u_i\right)<br><br>\left(X_j-X_i\right)+\left(v_j-v_i\right)}{\mathrm{dD}}& \mathrm{Eq}.<br><br>5\\ \sin <br><br><br><br>\mathrm{\theta <br><br>}=\frac{\left(u_j-u_i\right)<br><br>\left(Y_j-Y_i\right)-\left(v_j-v_i\right)<br><br>\left(X_j-X_i\right)}{\mathrm{dD}}& \mathrm{Eq}.<br><br>6\\ \left[\begin{array}{c}{r}_x\\ r_y\end{array}\right]=s<br><br>\left[\begin{array}{cc}\cos <br><br><br><br>\mathrm{\theta <br><br>}& -\sin <br><br><br><br>\mathrm{\theta <br><br>}\\ \sin <br><br><br><br>\mathrm{\theta <br><br>}& \cos <br><br><br><br>\mathrm{\theta <br><br>}\end{array}\right]<br><br>\left(\left[\begin{array}{c}{c}_x\\ c_y\end{array}\right]-\left[\begin{array}{c}{u}_i\\ v_i\end{array}\right]\right)+\left[\begin{array}{c}{X}_i\\ Y_i\end{array}\right]& \mathrm{Eq}.<br><br>7\end{array}$

where θ

denotes a heading angle based on a Y axis of space coordinate.

15. The method of claim 7, wherein the predetermined location in the step a) is a predetermined driving staring location.

16. The method of claim 7, wherein the artificial mark tracing process detects artificial marks from a predetermined rectangular region based on an image coordinate of artificial marks detected at a predetermined time stage.

8. A method of calculating a location in real time based on an artificial mark, comprising the steps of:

a) calculating an current initial location using an image coordinate recognized through turning on and off an artificial mark or assigning the current initial location as a predetermined location;

b) identifying artificial marks detected while traveling at a mark identifying module by estimating an image coordinate of artificial mark at an image estimating module based on current location information of artificial mark calculated at a location calculating module and a space coordinate of artificial mark stored in the main control module, and comparing the estimated image coordinate and newly detected image coordinate; and

c) updating a current location at a location calculating module based on the image coordinate of the detected and identified artificial mark and a pre-stored space coordinate in a main control module.

17. The method of claim 8, wherein the step b) includes the steps of:

b-1) detecting an artificial mark through an image processing,

b-2) determining whether the number of detected artificial marks is larger than a predetermined number,

b-3) performing an artificial mark detection when the number of detected artificial marks is smaller than the predetermined number, or estimating an image coordinate of artificial marks through location information at a time t−

1 when the number of detected artificial marks is larger than the predetermined number; and

b-4) identifying an artificial mark having the estimated image coordinate closest to the image coordinate of detected artificial mark.

19. The method of claim 17, wherein an error corrected image coordinate is calculated through correcting the error of camera lens from the image coordinate of new detected artificial mark and the error corrected image coordinate is reflected to calculate a current location.

20. The method of claim 19, wherein the new detected artificial mark is identified as an artificial mark having an image coordinate (û

_k, {circumflex over (v)}_k) closest to an error corrected image coordinate (u, v) using following equation;

$\begin{array}{cc}\left[\begin{array}{c}{\hat{u}}_k\\ {\hat{v}}_k\end{array}\right]=\left[\begin{array}{c}{c}_x\\ c_y\end{array}\right]-\frac{1}{s}<br><br>\left[\begin{array}{cc}\cos <br><br><br><br>{\mathrm{\theta <br><br>}}^{t-1}& \sin <br><br><br><br>{\mathrm{\theta <br><br>}}^{t-1}\\ -\sin <br><br><br><br>{\mathrm{\theta <br><br>}}^{t-1}& \cos <br><br><br><br>{\mathrm{\theta <br><br>}}^{t-1}\end{array}\right]<br><br>\left(\left[\begin{array}{c}{r}_x^{t-1}\\ r_y^{t-1}\end{array}\right]-\left[\begin{array}{c}{X}_k\\ Y_k\end{array}\right]\right),<br><br>k=1,\mathrm{\dots <br><br>}<br><br>,n& \mathrm{Eq}.<br><br>8\end{array}$

where c_x and c_y denotes internal parameters of camera representing an image coordinate of center of lens, θ

denotes a heading angle based on a Y axis of space coordinate, X and Y denote the space coordinate and

$s=\frac{D}{d}=\frac{\sqrt{{\left(X_i-X_j\right)}^2+{\left(Y_i-Y_j\right)}^2}}{\sqrt{{\left(u_i-u_j\right)}^2+{\left(v_i-v_j\right)}^2}}$

21. The method of claim 20, wherein a current location information (r_x, r_y, θ

) is updated from the error corrected coordinate (u_i, v_i) and (u_j, v_j) and a stored space coordinate (X_i, Y_i) and (X_j, Y_j) using following equations;

$\begin{array}{cc}{s}^t=\frac{D}{d}=\frac{\sqrt{{\left(X_p-X_q\right)}^2+{\left(Y_p-Y_q\right)}^2}}{\sqrt{{\left(u_p^t-u_q^t\right)}^2+{\left(v_p^t-v_q^t\right)}^2}}& \mathrm{Eq}.<br><br>9\\ \cos <br><br><br><br>{\mathrm{\theta <br><br>}}^t=\frac{\left(u_q^t-u_p^t\right)<br><br>\left(X_q-X_p\right)+\left(v_q^t-v_p^t\right)<br><br>\left(Y_q-Y_p\right)}{\mathrm{dD}}& \mathrm{Eq}.<br><br>10\\ \sin <br><br><br><br>{\mathrm{\theta <br><br>}}^t=\frac{\left(u_q^t-u_p^t\right)<br><br>\left(Y_q-Y_p\right)-\left(v_q^t-v_p^t\right)<br><br>\left(X_q-X_p\right)}{\mathrm{dD}}& \mathrm{Eq}.<br><br>11\\ \left[\begin{array}{c}{r}_x^t\\ r_y^t\end{array}\right]=s^t<br><br>\left[\begin{array}{cc}\cos <br><br><br><br>{\mathrm{\theta <br><br>}}^t& -\sin <br><br><br><br>{\mathrm{\theta <br><br>}}^t\\ \sin <br><br><br><br>{\mathrm{\theta <br><br>}}^t& \cos <br><br><br><br>{\mathrm{\theta <br><br>}}^t\end{array}\right]<br><br>\left(\left[\begin{array}{c}{c}_x\\ c_y\end{array}\right]-\left[\begin{array}{c}{u}_p^t\\ v_p^t\end{array}\right]\right)+\left[\begin{array}{c}{X}_p\\ Y_p\end{array}\right]& \mathrm{Eq}.<br><br>12\end{array}$

9. A method of calculating a location in real time based on an artificial mark, comprising the steps of:

a) calculating an current initial location using an image coordinate recognized through turning on and off an artificial mark or assigning the current initial location as a predetermined location;

b) identifying artificial marks detected while traveling at a mark identifying module by performing an artificial mark tracing process on an image where the closest one of artificial marks detected a previous time stage is identified as an identical artificial mark or by estimating an image coordinate of artificial mark at an image estimating module based on current location information of artificial mark calculated at a location calculating module and a space coordinate of artificial mark stored in the main control module and comparing the estimated image coordinate and newly detected image coordinate when the artificial marks detected at the previous state are disappeared; and

c) updating a current location at a location calculating module based on the image coordinate of the detected and identified artificial mark and a pre-stored space coordinate in a main control module.

18. The method of claim 9, wherein the step b) includes the steps of:

b-1) performing an artificial mark tracing,

b-2) detecting a new artificial mark through an image process when the traced artificial mark is disappeared,

b-3) determining whether the number of detected artificial marks including the new artificial marks is larger than a predetermined number or not;

b-4) performing an artificial mark detection when the number of detected artificial marks is smaller than the predetermined number or estimating an image coordinate through location information at a previous time t−

1 for new artificial mark when the number of detected artificial marks is larger than the predetermined number; and

b-5) identifying an artificial mark having the estimated image coordinate closest to the image coordinate of the new detected artificial mark as an newly detected artificial mark.