Apparatus and method for estimating a position and an orientation of a mobile robot

• US 7,765,027 B2
• Filed: 09/16/2004
• Issued: 07/27/2010
• Est. Priority Date: 09/16/2003
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Park, Dong Ryeol, Hong, Sun-gi, Bang, Seokwon
• Primary Examiner(s): Tran; Khoi
• Assistant Examiner(s): Peche; Jorge O
• Application Number:
  US10/941,846
• Publication Number:
  US 20050065655A1
• Time in Patent Office: 2,140 Days
• Field of Search: 700/245, 700/59, 700/259, 701/23, 701/300, 901/45, 901/20, 901/44, 340/988, 340/901, 340/933, 340/937, 340/938, 180/167, 180/168, 180/169, 356/399, 356/400
• US Class Current: 700/245
• CPC Class Codes: G05D 1/0234 using optical markers or beacons optical beacons per se G01S1/70
• CPC Class Codes: G05D 1/0253 extracting relative motion information from a plurality of images taken successively, e.g. visual odometry, optical flow determining position or orientation from images G06T7/70

Abstract

A method and apparatus for estimating a position and an orientation of a mobile robot. The apparatus includes: a ceiling image grabber for obtaining a ceiling image of an area where the mobile device travels; a mark detector for detecting a retro-reflective artificial mark from the ceiling image, the retro-reflective artificial mark including a first mark and a second mark, each including a non-reflective portion and an infrared reflective portion; and a position & orientation estimator for estimating a position and an orientation of the mobile device using a position of the artificial mark or encoder information according to whether detection of the artificial mark is successful.

Claims

1. An apparatus for estimating a position and an orientation of a mobile device, the apparatus comprising:

an image grabber obtaining an image of an area where the mobile device travels;

an image processor processing the obtained image;

a mark detector detecting a position of an artificial mark from the obtained and processed image, the artificial mark including a first mark and a second mark; and

a position &

orientation estimator estimating a position and an orientation of the mobile device using the detected position of the artificial mark,

wherein the first and second marks are positioned apart from each other at a predetermined distance, an outer portion of the first mark and an inner portion of the second mark being non-reflective portions, and an inner portion of the first mark and an outer portion of the second mark being infrared reflective portions.

2. The apparatus of claim 1, wherein the image processor determines an environment from the obtained image and performs a predetermined image processing procedure according to the determined environment.

3. The apparatus of claim 2, wherein the image processor compares an average intensity of the obtained image to a reference value, and when the light of the current environment is in a dark state according to the comparison result, the image processor reverses light and dark portions of the obtained image to dark and light portions, respectively, to obtain a reversed image, and performs the predetermined image processing procedure with respect to the reversed image, the reversing of the light and dark portions to dark and light portions is providing a negative of the obtained image.

4. The apparatus of claim 1, wherein the mark detector detects at least one candidate region with respect to the obtained image, selects a largest reliability value as a template matching result with respect to the detected at least one candidate region, compares the selected reliability value to a predetermined reference value, and determines whether detection of the artificial mark is successful according to a comparison result.

5. A mobile robot comprising:

an image grabber obtaining an image of an area in which the mobile robot travels;

an image intensity comparator determining whether a light of a current environment is dark or bright by calculating an average intensity of the obtained image and comparing the calculated average intensity to a predetermined reference value;

an image processor selectively reversing the obtained image and not reversing the obtained image according to a determination result of the image intensity comparator;

a light controller controlling a light device attached to the mobile robot according to the determination result of the image intensity comparator;

a mark detector detecting a position of an artificial mark including a first mark and a second mark from the obtained image;

a position &

orientation estimator estimating a position and an orientation of the robot using the detected position of the artificial mark; and

a motion controller controlling traveling of the mobile robot according to the position and the orientation estimated by the position &

orientation estimator,

wherein the first and second marks are positioned apart from each other at a predetermined distance, an outer portion of the first mark and an inner portion of the second mark being non-reflective portions, and an inner portion of the first mark and an outer portion of the second mark being infrared reflective portions.

6. The mobile robot of claim 5, wherein the image processor determines a current environment from the obtained image and performs a predetermined image processing procedure according to the determined environment.

7. The mobile robot of claim 6, wherein the image processor compares an average intensity of the obtained image to a reference value, and when the light of the current environment is in a dark state according to the comparison result, the image processor reverses light and dark portions of the obtained image to dark and light portions, respectively, to obtain a reversed image, and performs distortion compensation processing and pre-processing with respect to the reversed image, the reversing of the light and dark portions to dark and light portions is providing a negative of the obtained image.

8. The mobile robot of claim 5, wherein the mark detector detects at least one candidate region, selects a largest reliability value as a template matching result with respect to the detected at least one candidate region, compares the selected reliability value to a predetermined reference value, determines whether detection of the artificial mark is successful according to a comparison result.

9. An apparatus for estimating a position and an orientation of a mobile device, the apparatus comprising:

an image grabber to obtain an image of an area where the mobile device operates;

an image intensity comparator to calculate an average intensity of the image and determine whether an illumination level in the area is dark according to a predetermined illumination level;

an image processor to reverse light and dark portions of the image to dark and light portions, respectively, to obtain a reversed image, when the light in the area is determined to be dark, and perform at least one of distortion compensation processing and pre-processing procedures on a non-dark image or the reversed image, the reversing of the light and dark portions to dark and light portions is providing a negative of the obtained image;

a mark detector to detect an artificial mark from the image, the artificial mark includes a first mark and a second mark;

an encoder information capturer to obtain position information and orientation information of the mobile device from at least one encoder sensor attached to the mobile device;

a position and orientation estimator to selectively estimate the position and the orientation of the mobile device using the position of the artificial mark that is detected by the mark detector when the mark detector is able to positionally detect the artificial mark or selectively estimate the position and the orientation of the mobile device using the information provided by the encoder information capturer when the mark detector is unable to positionally detect the artificial mark; and

a travel controller to control the mobile device to travel along a path using the position and orientation information obtained by the position and orientation estimator,

wherein the first and second marks are positioned apart from each other at a predetermined distance, an outer portion of the first mark and an inner portion of the second mark being non-reflective portions, and an inner portion of the first mark and an outer portion of the second mark being infrared reflective portions.

10. The apparatus for estimating the position and the orientation of the mobile device as claimed in claim 9, wherein the image processing device alters a lightness or darkness of the artificial mark so that it is different from an area surrounding the artificial mark, thereby enabling the mark detector to more easily detect the artificial mark from the image.

11. The apparatus for estimating the position and the orientation of the mobile device as claimed in claim 9, wherein the image processor performs the pre-processing procedure of the image for which the distortion compensation processing procedure is performed in order to adjust a contrast and/or an exposure of the image such that a resolution of a specific region of the image is increased.

12. The apparatus for estimating the position and the orientation of the mobile device as claimed in claim 11, wherein the image processing device adjusts the contrast and/or the exposure of the image by adjusting the contrast so that a dark portion of the image is darkened and a bright portion of the image is brightened, and adjusting the exposure by multiplying each pixel of the image by a constant, thereby increasing the resolution of a specific region of the image.

13. The apparatus for estimating the position and the orientation of the mobile device as claimed in claim 9, further comprising:

a light controller to control the light in the area such that when the image intensity comparator determines that the area is in a dark state according to a predetermined lightness value, the light controller increases the amount of light in the area, and when the image intensity comparator determines that the area is in a bright state according to the predetermined lightness value, the light controller decreases the amount of light in the area.

14. The apparatus for estimating the position and the orientation of the mobile device as claimed in claim 13, wherein the light controller is attached to the mobile device.

15. The apparatus for estimating the position and the orientation of the mobile device as claimed in claim 9, wherein the image intensity comparator calculates the average intensity of the image by dividing a value of summing intensity of all pixels of the image by the number of all pixels, and compares the calculated average intensity to a predetermined reference value.

16. An apparatus for estimating a location of a mobile device in an environment having changes in illumination, the apparatus comprising:

a device to obtain an image of the area where the mobile device is operating;

an image processor to determine a level of illumination according to an amount of light measured in the environment and to reverse the image when the amount of light is not greater than a predetermined level;

a detector to detect an artificial mark from the image obtained by the image processor; and

a location determination device to selectively estimate the location of the mobile device according to the detected artificial mark or information from at least one encoder sensor provided on the mobile device,

wherein the location of the mobile device is calculated using information provided from the at least one encoder sensor when the artificial mark is not detected by the detector, and the location of the mobile device is calculated using the artificial mark when the artificial mark is detected by the detector, and

wherein the artificial mark includes a first mark having a non-reflective portion and an infrared reflective portion, and a second mark having a non-reflective portion and an infrared reflective portion and located a predetermined distance apart from the first mark.

17. The apparatus for estimating the location of the mobile as claimed in claim 16, wherein the location of the mobile device includes a position and an orientation of the mobile device.

18. The apparatus for estimating the location of the mobile as claimed in claim 16, wherein the image processor performs at least one of distortion compensation processing and pre-processing procedures on a non-processed or reversed image.

19. The apparatus for estimating the location of the mobile as claimed in claim 16, wherein the device to obtain the image includes a wide-angle lens and obtains the image of the area where the mobile device operates.

20. The apparatus for estimating the location of the mobile as claimed in claim 16, wherein the location determination device estimates the location of the mobile device with respect to a location of the artificial mark according to a calculated distance between the first mark and the second mark and a calculated angle between a line obtained by expanding a center of the artificial mark toward the x-axis and a line connecting the first mark and the second mark.

21. The apparatus for estimating the location of the mobile as claimed in claim 16, wherein the location determination device estimates the location of the mobile device by performing a shift process, a rotation process, and a flip process with respect to a location of the artificial mark.

22. The apparatus for estimating the location of the mobile as claimed in claim 16, wherein the image processor compares an average intensity of the image to a reference level, and reverses light and dark portions of the ceiling image to dark and light portions, respectively, to obtain a reversed image, and performs distortion compensation processing and pre-processing with respect to the reversed image when the light of the current environment is not greater than the reference level according to the comparison result, the reversing of the light and dark portions to dark and light portions is providing a negative of the obtained image.

23. The apparatus for estimating the location of the mobile as claimed in claim 16, wherein the at least one encoder sensor is attached to a wheel of the mobile device and detects the location of the mobile device when the artificial mark on the image is not detected by the detector.

24. The apparatus for estimating the location of the mobile as claimed in claim 16, wherein the at least one encoder sensor comprises:

a first encoder sensor attached to a first wheel located on a first side of the mobile device;

a second encoder sensor attached to a second wheel located on a second side of the mobile device that is parallel with the first side; and

wherein a number of encoder pulses output from the first encoder sensor and the second encoder sensor is used to calculate the location of the mobile device according to a distance moved by the wheels of the mobile device.