SYSTEM, METHOD, AND MEDIUM FOR DETECTING MOVING OBJECT USING STRUCTURED LIGHT, AND MOBILE ROBOT INCLUDING SYSTEM THEREOF

• US 20080008353A1
• Filed: 07/02/2007
• Published: 01/10/2008
• Est. Priority Date: 07/05/2006
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd. (Samsung Group)
• Original Assignee: Samsung Electronics Co. Ltd. (Samsung Group)
• Inventors: Park, Jun-ho, Bang, Seok-won, Park, Dong-ryeol
• Granted Patent: US 7,983,449 B2
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 7/254 involving subtraction of images
• CPC Class Codes: G06V 10/145 Illumination specially adapted for pattern recognition, e.g. using gratings
• CPC Class Codes: G06V 10/24 Aligning, centring, orientation detection or correction of the image

Abstract

A system, a method, and medium for detecting a moving object using a structured light, and a mobile robot including the system are provided. The system includes an image obtaining unit to obtain a first image at a predetermined position and a second image at a position after a movement of the system by using a light source; an image predicting unit to predict the second image at the position after the movement of the system by using the first image and information with respect to the movement of the system; and a comparing/analyzing unit to compare and analyze the second image obtained by the image obtaining unit and the second image predicted by the image predicting unit.

Claims

1. A system for detecting a moving object, the system comprising:

an image obtaining unit to obtain a first image at a predetermined position and a second image at a position after a movement of the system by using a light source;

an image predicting unit to predict the second image at the position after the movement of the system by using the first image and information pertaining to the movement of the system; and

a comparing/analyzing unit to compare and analyze the second image obtained by the image obtaining unit and the second image predicted by the image predicting unit to detect the moving object.

2. The system of claim 1, wherein the image obtaining unit obtains an image of an obstacle by receiving the light source, which is generated in a plane and reflected from the obstacle, through a camera sensor.

3. The system of claim 2, wherein the light source comprises a near-infrared laser beam.

4. The system of claim 1t wherein:

a moving vector T indicating a movement of the system after the first image is obtained, a rotational matrix R from a rotational angle θ

of the system after the first image is obtained, and a distance vector T1 between the camera sensor and the rotational axis of the system are represented as follows;

$T=\lfloor <br><br>\begin{array}{cc}{T}_x& T_y\rfloor <br><br>,\end{array}<br><br><br><br>R=\left[\begin{array}{cc}\cos <br><br>\left(\mathrm{\theta <br><br>}\right)& \sin <br><br>\left(\mathrm{\theta <br><br>}\right)\\ -\sin <br><br>\left(\mathrm{\theta <br><br>}\right)& \cos <br><br>\left(\mathrm{\theta <br><br>}\right)\end{array}\right],\mathrm{and}<br><br><br><br>T<br><br><br><br>1=\lfloor <br><br>\begin{array}{cc}{T}_{x<br><br><br><br>1}& T_{y<br><br><br><br>1}\rfloor <br><br>;<br><br>\end{array}$

the light source is defined as an origin, an x-axis is defined as a data axis having a negative that is a left side of the origin and a positive that is a right side of the origin, and a y-axis is defined as a distance axis, a distance data A calculated from the obtained first image and a distance data A′

that is predicted after the movement of the system are represented as follows;

$A\equiv <br><br>\left[\begin{array}{ccccc}{x}_1& x_2& \mathrm{\cdots <br><br>}& x_{n-1}& x_n\\ y_1& y_2& \mathrm{\cdots <br><br>}& y_{n-1}& y_n\end{array}\right]\equiv <br><br>\left[\begin{array}{c}X\\ Y\end{array}\right]\equiv <br><br>\lfloor <br><br>\begin{array}{ccccc}{A}_1& A_2& \mathrm{\cdots <br><br>}& A_{n-1}& A_n\rfloor <br><br>\end{array},<br><br>A^{\mathrm{\prime <br><br>}}\equiv <br><br>\left[\begin{array}{ccccc}{x}_1^{\mathrm{\prime <br><br>}}& x_2^{\mathrm{\prime <br><br>}}& \mathrm{\cdots <br><br>}& x_{n-1}^{\mathrm{\prime <br><br>}}& x_n^{\mathrm{\prime <br><br>}}\\ y_1^{\mathrm{\prime <br><br>}}& y_2^{\mathrm{\prime <br><br>}}& \mathrm{\cdots <br><br>}& y_{n-1}^{\mathrm{\prime <br><br>}}& y_n^{\mathrm{\prime <br><br>}}\end{array}\right]\equiv <br><br>\left[\begin{array}{c}{X}^{\mathrm{\prime <br><br>}}\\ Y^{\mathrm{\prime <br><br>}}\end{array}\right]\equiv <br><br>\lfloor <br><br>\begin{array}{ccccc}{A}_1^{\mathrm{\prime <br><br>}}& A_2^{\mathrm{\prime <br><br>}}& \mathrm{\cdots <br><br>}& A_{n-1}^{\mathrm{\prime <br><br>}}& A_n^{\mathrm{\prime <br><br>}}\rfloor <br><br>\end{array};<br><br>$

and

the image predicting unit comprises a data converting unit to convert the distance data A calculated from the first image into the predicted distance data A′

by using an equation A′

=R(A−

T−

T1)+T1, and a data extracting unit to extract the predicted distance data A′

calculated in the data converting unit by using a predetermined algorithm.

5. The system of claim 4, wherein the predetermined algorithm satisfies a following condition when representing a visual angle of the light source as an a:

$\mathrm{if},<br><br>\arctan <br><br>\left(y_n^{\mathrm{\prime <br><br>}},x_n^{\mathrm{\prime <br><br>}}\right)<br><br>\ge <br><br>\frac{a}{2},\mathrm{then}<br><br><br><br>A_n^{\mathrm{\prime <br><br>}}=0$

6. The system of claim 4, wherein when a maximum value and a minimum value in a vector X′

of the predicted distance data A′

extracted by the data extracting unit are indicated as x′

_max and x′

_min, respectively, the comparing/analyzing unit compares the distance data between x′

_max and x′

_min in the distance data calculated from the obtained second image with the predicted distance data extracted by the data extracting unit.

7. The system of claim 4, wherein the distance data calculated from the obtained image is a distance data from the light source to the obstacle calculated by applying a triangular method to a coordinate value of the obtained image.

22. The system of claim 5, wherein the distance data calculated from the obtained image is a distance data from the light source to the obstacle calculated by applying a triangular method to a coordinate value of the obtained image.

8. A method for detecting a moving object, the method comprising:

obtaining a first image at a predetermined position by using a light source;

predicting a second image at a position after the movement of the system by using the first image and information pertaining to the movement of the system;

obtaining the second image at the position after the movement of the system; and

comparing the predicted second image with the obtained second image to detect the moving object.

9. The method of claim 8, wherein an image obtaining unit obtains first and second images of an obstacle by receiving the light source, which is generated in a plane and reflected from the obstacle, through a camera sensor.

10. The method of claim 9, wherein the light source comprises a near-infrared laser beam.

11. The method of claim 9, wherein:

a moving vector T indicating a movement of the system after the first image is obtained, a rotational matrix R from a rotational angle θ

of the system after the first image is obtained, and a distance vector T1 between the camera sensor and the rotational axis of the system are represented as follows;

$T=\lfloor <br><br>\begin{array}{cc}{T}_x& T_y\rfloor <br><br>,\end{array}<br><br><br><br>R=\left[\begin{array}{cc}\cos <br><br>\left(\mathrm{\theta <br><br>}\right)& \sin <br><br>\left(\mathrm{\theta <br><br>}\right)\\ -\sin <br><br>\left(\mathrm{\theta <br><br>}\right)& \cos <br><br>\left(\mathrm{\theta <br><br>}\right)\end{array}\right],\mathrm{and}<br><br><br><br>T<br><br><br><br>1=\lfloor <br><br>\begin{array}{cc}{T}_{x<br><br><br><br>1}& T_{y<br><br><br><br>1}\rfloor <br><br>;<br><br>\end{array}$

the light source is defined as an origin, an x-axis is defined as a data axis having a negative that is a left side of the origin and a positive that is a right side of the origin, and a y-axis is defined as a distance axis, a distance data A calculated from the obtained first image and a distance data A′

that is predicted after the movement of the system are represented as follows;

$A\equiv <br><br>\left[\begin{array}{ccccc}{x}_1& x_2& \mathrm{\cdots <br><br>}& x_{n-1}& x_n\\ y_1& y_2& \mathrm{\cdots <br><br>}& y_{n-1}& y_n\end{array}\right]\equiv <br><br>\left[\begin{array}{c}X\\ Y\end{array}\right]\equiv <br><br>\lfloor <br><br>\begin{array}{ccccc}{A}_1& A_2& \mathrm{\cdots <br><br>}& A_{n-1}& A_n\rfloor <br><br>\end{array}<br><br><br><br>A^{\mathrm{\prime <br><br>}}\equiv <br><br>\left[\begin{array}{ccccc}{x}_1^{\mathrm{\prime <br><br>}}& x_2^{\mathrm{\prime <br><br>}}& \mathrm{\cdots <br><br>}& x_{n-1}^{\mathrm{\prime <br><br>}}& x_n^{\mathrm{\prime <br><br>}}\\ y_1^{\mathrm{\prime <br><br>}}& y_2^{\mathrm{\prime <br><br>}}& \mathrm{\cdots <br><br>}& y_{n-1}^{\mathrm{\prime <br><br>}}& y_n^{\mathrm{\prime <br><br>}}\end{array}\right]\equiv <br><br>\left[\begin{array}{c}{X}^{\mathrm{\prime <br><br>}}\\ Y^{\mathrm{\prime <br><br>}}\end{array}\right]\equiv <br><br>\lfloor <br><br>\begin{array}{ccccc}{A}_1^{\mathrm{\prime <br><br>}}& A_2^{\mathrm{\prime <br><br>}}& \mathrm{\cdots <br><br>}& A_{n-1}^{\mathrm{\prime <br><br>}}& A_n^{\mathrm{\prime <br><br>}}\rfloor <br><br>\end{array};<br><br>$

and

the predicting a second image comprises converting the distance data A calculated from the first image into the predicted distance data A′

by using an equation A′

=R(A−

T−

T1)+T1, and extracting the predicted distance data A′

calculated in the converting the distance data A by using a predetermined algorithm.

12. The method of claim 11, wherein the predetermined algorithm satisfies a following condition when representing a visual angle of the light source as an a:

$\mathrm{if},<br><br>\arctan <br><br>\left(y_n^{\mathrm{\prime <br><br>}},x_n^{\mathrm{\prime <br><br>}}\right)<br><br>\ge <br><br>\frac{a}{2},\mathrm{then}<br><br><br><br>A_n^{\mathrm{\prime <br><br>}}=0$

13. The method of claim 11, wherein when a maximum value and a minimum value in a vector X′

of the predicted distance data extracted in the extracting the predicted distance data A (are indicated as x(max and x(min, respectively, in the comparing the second image predicted in the predicting a second image with the second image obtained in the obtaining the second image and detecting the moving object, the distance data between x(max and x(min in the distance data calculated from the obtained second image is compared with the predicted distance data extracted in the extracting the predicted distance data A′

.

14. The method of claim 11, wherein the distance data calculated from the obtained image is a distance data from the light source to the obstacle calculated by applying a triangular method to a coordinate value of the obtained image.

25. The method of claim 13, wherein the distance data calculated from the obtained image is a distance data from the light source to the obstacle calculated by applying a triangular method to a coordinate value of the obtained image.

28. At least one computer readable medium storing computer readable instructions that control at least one processor to implement the method of claim 8.

15. A mobile robot for detecting a moving object, the robot comprising:

a system to detect a moving object; and

a driving system to drive the mobile robot which calculates position and recognizes the moving object,

wherein the system to detect the moving object includes

an image obtaining unit to obtain a first image at a predetermined position and a second image at a position after a movement of the system by a light source;

an image predicting unit to predict the second image at the position after the movement of the system by using the first image and information pertaining to the movement of the system; and

a comparing/analyzing unit to compare and analyze the second image obtained by the image obtaining unit and the second image predicted by the image predicting unit to detect the moving object.

16. The mobile robot of claim 15, wherein the image obtaining unit obtains an image of an obstacle by receiving the light source, which is generated in a plane and reflected from the obstacle, through a camera sensor.

17. The mobile robot of claim 16, wherein the light source comprises a near-infrared laser beam.

18. The mobile robot of claim 15, wherein:

a moving vector T indicating a movement of the system after the first image is obtained, a rotational matrix R from a rotational angle θ

of the system after the first image is obtained, and a distance vector T between the camera sensor and the rotational axis of the system are represented as follows;

$T=\lfloor <br><br>\begin{array}{cc}{T}_x& T_y\rfloor <br><br>,\end{array}<br><br><br><br>R=\left[\begin{array}{cc}\cos <br><br>\left(\mathrm{\theta <br><br>}\right)& \sin <br><br>\left(\mathrm{\theta <br><br>}\right)\\ -\sin <br><br>\left(\mathrm{\theta <br><br>}\right)& \cos <br><br>\left(\mathrm{\theta <br><br>}\right)\end{array}\right],\mathrm{and}<br><br><br><br>T<br><br><br><br>1=\lfloor <br><br>\begin{array}{cc}{T}_{x<br><br><br><br>1}& T_{y<br><br><br><br>1}\rfloor <br><br>;<br><br>\end{array}$

the light source is defined as an origin, an x-axis is defined as a data axis having a negative that is a left side of the origin and a positive that is a right side of the origin, and a y-axis is defined as a distance axis, a distance data A calculated from the obtained first image and a distance data A′

that is predicted after the movement of the system are represented as follows;

$A\equiv <br><br>\left[\begin{array}{ccccc}{x}_1& x_2& \mathrm{\cdots <br><br>}& x_{n-1}& x_n\\ y_1& y_2& \mathrm{\cdots <br><br>}& y_{n-1}& y_n\end{array}\right]\equiv <br><br>\left[\begin{array}{c}X\\ Y\end{array}\right]\equiv <br><br>\lfloor <br><br>\begin{array}{ccccc}{A}_1& A_2& \mathrm{\cdots <br><br>}& A_{n-1}& A_n\rfloor <br><br>\end{array}<br><br><br><br>A^{\mathrm{\prime <br><br>}}\equiv <br><br>\left[\begin{array}{ccccc}{x}_1^{\mathrm{\prime <br><br>}}& x_2^{\mathrm{\prime <br><br>}}& \mathrm{\cdots <br><br>}& x_{n-1}^{\mathrm{\prime <br><br>}}& x_n^{\mathrm{\prime <br><br>}}\\ y_1^{\mathrm{\prime <br><br>}}& y_2^{\mathrm{\prime <br><br>}}& \mathrm{\cdots <br><br>}& y_{n-1}^{\mathrm{\prime <br><br>}}& y_n^{\mathrm{\prime <br><br>}}\end{array}\right]\equiv <br><br>\left[\begin{array}{c}{X}^{\mathrm{\prime <br><br>}}\\ Y^{\mathrm{\prime <br><br>}}\end{array}\right]\equiv <br><br>\lfloor <br><br>\begin{array}{ccccc}{A}_1^{\mathrm{\prime <br><br>}}& A_2^{\mathrm{\prime <br><br>}}& \mathrm{\cdots <br><br>}& A_{n-1}^{\mathrm{\prime <br><br>}}& A_n^{\mathrm{\prime <br><br>}}\rfloor <br><br>\end{array};<br><br>$

and

the image predicting unit comprises a data converting unit converting the distance data A calculated from the first image into the predicted distance data A′

by using an equation A′

=R(A−

T−

T1)+T1, and a data extracting unit extracting the predicted distance data A′

calculated in the data converting unit by using a predetermined algorithm.

19. The mobile robot of claim 18, wherein the predetermined algorithm satisfies a following condition when representing a visual angle of the light source as a:

$\mathrm{if},<br><br>\arctan <br><br>\left(y_n^{\mathrm{\prime <br><br>}},x_n^{\mathrm{\prime <br><br>}}\right)<br><br>\ge <br><br>\frac{a}{2},\mathrm{then}<br><br><br><br>A_n^{\mathrm{\prime <br><br>}}=0$

20. The mobile robot of claim 18, wherein when a maximum value and a minimum value in a vector X′

of the predicted distance data extracted by the data extracting unit are indicated as x′

_max and x′

_min, respectively, the comparing/analyzing unit compares the distance data between x′

_max and x′

_min in the distance data calculated from the obtained second image with the predicted distance data extracted by the data extracting unit.

21. The mobile robot of claim 18, wherein the distance data calculated from the obtained image is a distance data from the light source to the obstacle calculated by applying a triangular method to a coordinate value of the obtained image.

26. The mobile robot of claim 19, wherein the distance data calculated from the obtained image is a distance data from the light source to the obstacle calculated by applying a triangular method to a coordinate value of the obtained image.

27. The mobile robot of claim 20, wherein the distance data calculated from the obtained image is a distance data from the light source to the obstacle calculated by applying a triangular method to a coordinate value of the obtained image.

23. The system of 6, wherein the distance data calculated from the obtained image is a distance data from the light source to the obstacle calculated by applying a triangular method to a coordinate value of the obtained image.

24. The method of 12, wherein the distance data calculated from the obtained image is a distance data from the light source to the obstacle calculated by applying a triangular method to a coordinate value of the obtained image.