Control method for localization and navigation of mobile robot and mobile robot using the same

• US 8,744,665 B2
• Filed: 07/28/2009
• Issued: 06/03/2014
• Est. Priority Date: 07/28/2009
• Status: Active Grant

• Current Assignee: Yujin Robot Co., Ltd. (The Miele Group)
• Original Assignee: Yujin Robot Co., Ltd. (The Miele Group)
• Inventors: Shin, Kyung Chul, Park, Seong Ju, Lee, Hee Kong, Lee, Jae Young, Kim, Hyung O
• Primary Examiner(s): Jabr, Fadey
• Assistant Examiner(s): Mahne, Kevin P
• Application Number:
  US13/387,715
• Publication Number:
  US 20120191287A1
• Time in Patent Office: 1,771 Days
• Field of Search: None
• US Class Current: 701/28
• CPC Class Codes: G05D 1/0246 using a video camera in combination with image processing means
• CPC Class Codes: G05D 1/027 comprising intertial navigation means, e.g. azimuth detector inertial navigation G01C21/16; inertial navigation combined with non-inertial navigation instruments G01C21/165
• CPC Class Codes: G05D 1/0272 comprising means for registering the travel distance, e.g. revolutions of wheels measuring distance traversed on the ground by vehicles, e.g. using odometers G01C22/00
• CPC Class Codes: G05D 1/0274 using mapping information stored in a memory device navigation using map-matching G01C21/30
• CPC Class Codes: G05D 2201/0203 Cleaning or polishing vehicle

Abstract

The present invention relates to a control method for the localization and navigation of a mobile robot and a mobile robot using the same. More specifically, the localization and navigation of a mobile robot are controlled using inertial sensors and images, wherein local direction descriptors are employed, the mobile robot is changed in the driving mode thereof according to the conditions of the mobile robot, and errors in localization may be minimized.

Claims

1. A mobile robot, comprising:

an image sensor configured to obtain an image;

a second sensor configured to obtain driving information including at least a driving direction and a driving distance of the mobile robot;

a processor configured to process the image obtained by the image obtaining unit to extract a feature point and generate a descriptor for the feature point; and

a main controller configured to generate a spatial map in a driving range based on the feature point and the driving information, and control localization and navigation of the mobile robot,

wherein when the localization and the navigation of the mobile robot are performed, the main controller selectively applies a first driving mode that simultaneously uses the image and the driving information when the matching degree of the obtained image and a reference image meet a threshold, and the main controller selectively applies a second driving mode that uses only the image information when the matching degree of the obtained image and a reference image do not meet the threshold;

wherein the processor generates change in brightness and direction information for n×

n sub regions divided from the obtained image and selects direction information of different two sub regions in the n×

n sub regions to generate a pair-wise combination to describe the angle difference in local direction information, and thereby generates the descriptor based on the pair-wise combination.

2. The mobile robot of claim 1, further comprising:

an electronic memory configured to store the driving information including the driving direction and the driving distance of the mobile robot obtained by the second sensor and image information including the feature point obtained by the image processing in the processor and descriptor information.

3. The mobile robot of claim 1, wherein the processor is configured to:

correct the distortion of the image obtained by the image sensor;

determine whether the image is available;

extract the feature point from the image; and

generate a descriptor for the feature point.

4. The mobile robot of claim 3, wherein the processor determines the availability of the obtained image by comparing a brightness of the obtained image and a brightness of the reference image.

5. The mobile robot of claim 4, wherein when the image sensor obtains an abnormal image in the first driving mode and a time when the abnormal image is obtained exceeds a predetermined time, the processor determines that the mobile robot is blocked in a block area.

6. The mobile robot of claim 5, wherein the processor determines whether the image sensor is blocked, based on an average and a variation of the brightness of the obtained image.

7. The mobile robot of claim 3, wherein the processor divides the obtained image into four sections with reference to a moving direction of the mobile robot and determines whether the mobile robot is blocked based on an average and a variation of the obtained image based on the brightness average in the divided sections.

8. The mobile robot of claim 4, wherein if it is determined that the mobile robot is blocked in the block area, the main controller returns the mobile robot to an entering point of the block area.

9. The mobile robot of claim 1, wherein the main controller is configured to:

select the first driving mode and the second driving mode;

manage the feature point extracted from the image and match to the registered feature point;

generate and update the map; and

control the localization and navigation of the mobile robot.

10. The mobile robot of claim 9, wherein when the mobile robot is kidnapped while driving in the first driving mode or fails to establish a metric map or localize during a predetermined image frame, the main controller switches the driving mode to the second driving mode to restore the position in accordance with a topological map.

11. A control method of localization and navigation of a mobile robot, comprising:

(a) allowing the mobile robot to obtain image information via an image sensor and driving information of a target region via a second sensor;

(b) determining, via a processor, whether the obtained image is available by comparing a quality of the obtained image information with a reference value; and

(c) if it is determined that the image is available, extracting a feature point from the obtained image information, via a processor, and generating a descriptor of the extracted feature point via a main controller to create a driving map and drive in the target region;

wherein when the localization and the navigation of the mobile robot are performed, the driving in the target region includes;

selecting any one of a first driving mode that simultaneously uses the image and the driving information when the matching degree of the obtained image and a reference image meet a threshold, and a second driving mode that uses only the image information when the matching degree of the obtained image and the reference image do not meet the threshold; and

wherein the processor generates change in brightness and direction information for n×

n sub regions divided from the obtained image and selects direction information of different two sub regions in the n×

n sub regions to generate a pair-wise combination to describe the angle difference in local direction information and thereby generates the descriptor based on the pair-wise combination.

12. The control method of claim 11, before (b), further comprising:

correcting distortion of the obtained image information.

13. The control method of claim 11, wherein the generating of the descriptor includes:

generating a descriptor including at least direction information and brightness information for the feature point;

registering the feature point if the feature point is determined as a new feature point;

generating a descriptor for a key area feature point when the feature point is the key area feature point that represents a feature of a predetermined region; and

registering the key area feature point if the key area feature point is determined as a new key area feature point.

14. The control method of claim 13, wherein the key area feature point is spatially uniformly generated in the obtained image and set in a region where the sum of values of a corner response function calculated from an image in one frame is larger than a reference value.

15. The control method of claim 13, wherein the mobile robot corrects a positional error accumulated by the image matching of the feature point obtained in the key area and the key area feature point and a direction angle error.

16. The control method of claim 11, wherein when the brightness of the image obtained by the mobile robot in the first driving mode is below a reference and a time when the image below the reference is obtained exceeds a predetermined time, it is determined that the mobile robot is blocked in a block area.

17. The control method of claim 16, wherein if it is determined that the mobile robot is blocked in the block area, the mobile robot is returned to an entering point of the block area.

18. The control method of claim 16, wherein it is determined whether the mobile robot is blocked based on an average and a variation of the obtained image based on the brightness average in sections divided by dividing the obtained image into four sections with reference to a moving direction of the mobile robot.