Apparatus and method for estimating a position and an orientation of a mobile robot

• US 20050065655A1
• Filed: 09/16/2004
• Published: 03/24/2005
• Est. Priority Date: 09/16/2003
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Hong, Sun-gi, Bang, Seokwon, Park, Dong Ryeol
• Granted Patent: US 7,765,027 B2
• Time in Patent Office: Days
• US Class Current: 700/245
• CPC Class Codes: G05D 1/0234 using optical markers or beacons optical beacons per se G01S1/70
• CPC Class Codes: G05D 1/0253 extracting relative motion information from a plurality of images taken successively, e.g. visual odometry, optical flow determining position or orientation from images G06T7/70

Abstract

A method and apparatus for estimating a position and an orientation of a mobile robot. The apparatus includes: a ceiling image grabber for obtaining a ceiling image of an area where the mobile device travels; a mark detector for detecting a retro-reflective artificial mark from the ceiling image, the retro-reflective artificial mark including a first mark and a second mark, each including a non-reflective portion and an infrared reflective portion; and a position & orientation estimator for estimating a position and an orientation of the mobile device using a position of the artificial mark or encoder information according to whether detection of the artificial mark is successful.

Claims

1. A method of constructing an artificial mark, comprising:

forming a first mark having a non-reflective portion and an infrared reflective portion; and

forming a second mark having the non-reflective portion and the infrared reflective portion and locating the second mark a predetermined distance apart from the first mark,

wherein the first and second marks are a retro-reflective artificial mark.

2. The method of claim 1, wherein an outer portion of the first mark and an inner portion of the second mark are non-reflective, and an inner portion of the first mark and an outer portion of the second mark are infrared reflective portions.

3. A method of estimating a position and an orientation of a mobile device, the method comprising:

obtaining a ceiling image of an area in which the mobile device travels;

detecting a retro-reflective artificial mark from the ceiling image, the retro-reflective artificial mark including a first mark and a second mark, each of the first mark and the second mark including a non-reflective portion and an infrared reflective portion; and

estimating the position and the orientation of the robot using a position of the artificial mark or encoder information according to whether the detection of the artificial mark is successful.

4. The method of claim 3, wherein an outer portion of the first mark and an inner portion of the second mark are non-reflective portions, and an inner portion of the first mark and an outer portion of the second mark are infrared reflective portions.

5. The method of claim 3 further comprising:

determining an environment from the obtained ceiling image and performing a predetermined image processing procedure according to the determined environment.

6. The method of claim 5, wherein the determining the environment comprises:

calculating an average intensity of the obtained ceiling image;

determining whether a light of the environment is in a dark state or in a bright state by comparing the calculated average intensity to a reference value;

performing at least one of distortion compensation processing and pre-processing with respect to the ceiling image when determined that the light of the environment is in a bright state; and

reversing the ceiling image and performing the image processing procedure with respect to the reversed ceiling image when determined that the light of the environment is in a dark state.

7. The method of claim 3, wherein the detecting the retro-reflective artificial mark comprises:

extracting at least one candidate region with respect to the ceiling image using outline matching geometrical information of the artificial mark;

performing template matching with respect to the extracted at least one candidate region, selecting the largest reliability value as a template matching result, and comparing the selected reliability value to a predetermined reference value; and

determining that detection of the artificial mark is successful when the selected reliability value is greater than the predetermined reference value, and determining that detection of the artificial mark is not successful when the selected reliability value is not greater than the predetermined reference value.

8. The method of claim 7, wherein with regard to the ceiling image used in the extracting the at least one candidate region, at least one of distortion-compensation-processing, pre-processing, and binary-processing is sequentially performed in advance.

9. The method of claim 3, further comprising:

estimating a relative position and orientation of the mobile device from the artificial mark by sequentially performing a shift process, a rotation process, and a flip process.

10. An apparatus for estimating a position and an orientation of a mobile device, the apparatus comprising:

a ceiling image grabber for obtaining a ceiling image of an area where the mobile device travels;

a mark detector for detecting a retro-reflective artificial mark from the ceiling image, the retro-reflective artificial mark includes a first mark and a second mark, each including a non-reflective portion and an infrared reflective portion; and

a position &

orientation estimator for estimating a position and an orientation of the mobile device using a position of the artificial mark or encoder information according to whether detection of the artificial mark is successful.

11. The apparatus of claim 10, wherein an outer portion of the first mark and an inner portion of the second mark are non-reflective portions, and an inner portion of the first mark and an outer portion of the second mark are infrared reflective portions.

12. The apparatus of claim 10, further comprising:

an image processor determining an environment from the obtained ceiling image and performing a predetermined image processing procedure according to the determined environment.

13. The apparatus of claim 11, wherein the image processor compares an average intensity of the obtained ceiling image to a reference value, and when the light of the current environment is in a dark state according to the comparison result, the image processor reverses the ceiling image and performs the predetermined image processing procedure with respect to the reversed ceiling image

14. The apparatus of claim 10, wherein the mark detector detects at least one candidate region with respect to the ceiling image, selects a largest reliability value as a template matching result with respect to the detected at lease one candidate region, compares the selected reliability value to a predetermined reference value, and determines whether detection of the artificial mark is successful according to a comparison result.

15. A mobile robot comprising:

a ceiling image grabber obtaining a ceiling image of an area in which the mobile robot travels;

an image intensity comparator determining whether a light of a current environment is dark or bright by calculating an average intensity of the ceiling image and comparing the calculated average intensity to a predetermined reference value;

a light controller controlling a light device attached to the mobile robot according to a determination result of the image intensity comparator;

a mark detector detecting a retro-reflective artificial mark including a first mark and a second mark each having a non-reflective portion and an infrared reflective portion from the ceiling image;

a position &

orientation estimator estimating a position and an orientation of the robot using a position of the artificial mark or encoder information according to whether detection of the artificial mark is successful; and

a motion controller controlling traveling of the mobile robot according to the position and the orientation estimated by the position &

orientation estimator.

16. The mobile robot of claim 15, wherein the first and second marks include circles positioned apart from each other at a predetermined distance, such that an outer portion of the first mark and an inner portion of the second mark are non-reflective, and an inner portion of the first mark and an outer portion of the second mark are infrared reflective.

17. The mobile robot of claim 15, further comprising:

an image processor determining a current environment from the obtained ceiling image and performing a predetermined image processing procedure according to the determined environment.

18. The mobile robot of claim 17, wherein the image processor compares an average intensity of the obtained ceiling image to a reference value, and when the light of the current environment is in a dark state according to the comparison result, the image processor reverses the ceiling image and performs distortion compensation processing and pre-processing with respect to the reversed ceiling image.

19. The mobile robot of claim 15, wherein the mark detector detects at least one candidate region, selects a largest reliability value as a template matching result with respect to the detected at least one candidate region, compares the selected reliability value to a predetermined reference value, determines whether detection of the artificial mark is successful according to a comparison result.

20. A computer readable medium having recorded thereon a computer readable program to be read by at least one computer for performing a method of estimating a position and an orientation of a mobile device, the method comprising:

obtaining a ceiling image of an area where the mobile device travels;

detecting a retro-reflective artificial mark from the ceiling image, the retro-reflective artificial mark includes a first mark and a second mark, each of the first mark and the second mark includes a non-reflective portion and an infrared reflective portion; and

estimating the position and the orientation of the mobile device using a position of the artificial mark or encoder information according to whether the detection of the retro-reflective artificial mark is successful.

21. An apparatus for estimating a position and an orientation of a mobile device, the apparatus comprising:

a ceiling image grabber to obtain a ceiling image of an area where the mobile device operates;

an image intensity comparator to calculate an average intensity of the ceiling image and determine whether an illumination level in the area is dark according to a predetermined illumination level;

an image processor to reverse the ceiling image when the light in the area is determined to be dark, and perform at least one of distortion compensation processing and pre-processing procedures on a non-dark image or the reversed image;

a mark detector to detect an artificial mark from the ceiling image, the artificial mark includes a first mark and a second mark each having a non-reflective portion and an infrared reflective portion;

an encoder information capturer to obtain position information and orientation information of the mobile device from at least one encoder sensor attached to the mobile device;

a position and orientation estimator to estimate the position and the orientation of the mobile device using the position of the artificial mark that is detected by the mark detector when the mark detector is able to positionally detect the artificial mark or using the information provided by the encoder information capturer when the mark detector is unable to positionally detect the artificial mark; and

a travel controller to control the mobile device to travel along a path using the position and orientation information obtained by the position and orientation estimator,

wherein the artificial mark is a retro-reflective artificial mark.

22. The apparatus for estimating the position and the orientation of the mobile device as claimed in claim 21, wherein the image processing device alters a lightness or darkness of the artificial mark so that it is different from an area surrounding the artificial mark, thereby enabling the mark detector to more easily detect the artificial mark from the ceiling image.

23. The apparatus for estimating the position and the orientation of the mobile device as claimed in claim 21, wherein the image processor performs the pre-processing procedure of the ceiling image for which the distortion compensation processing procedure is performed in order to adjust a contrast and/or an exposure of the ceiling image such that a resolution of a specific region of the ceiling image is increased.

24. The apparatus for estimating the position and the orientation of the mobile device as claimed in claim 23, wherein the image processing device adjusts the contrast and/or the exposure of the ceiling image by adjusting the contrast so that a dark portion of the ceiling image is darkened and a bright portion of the ceiling image is brightened, and adjusting the exposure by multiplying each pixel of the ceiling image by a constant, thereby increasing the resolution of a specific region of the ceiling image.

25. The apparatus for estimating the position and the orientation of the mobile device as claimed in claim 21, further comprising:

a light controller to control the light in the area such that when the image intensity comparator determines that the area is in a dark state according to a predetermined lightness value, the light controller increases the amount of light in the area, and when the image intensity comparator determines that the area is in a bright state according to the predetermined lightness value, the light controller decreases the amount of light in the area.

26. The apparatus for estimating the position and the orientation of the mobile device as claimed in claim 25, wherein the light controller is attached to the mobile device.

27. The apparatus for estimating the position and the orientation of the mobile device as claimed in claim 21, wherein the image intensity comparator calculates the average intensity of the ceiling image by dividing a value of summing intensity of all pixels of the ceiling image by the number of all pixels, and compares the calculated average intensity to a predetermined reference value.

28. An apparatus for estimating a location of a mobile device in an environment having changes in illumination, the apparatus comprising:

a device to obtain an image of a ceiling of the area where the mobile device is operating;

an image processor to determine a level of illumination according to an amount of light measured in the environment and to reverse the ceiling image when the amount of light is not greater than a predetermined level;

a detector to detect an artificial mark from the ceiling image obtained by the image processor; and

a location determination device to estimate the location of the mobile device according to the detected artificial mark or information from at least one encoder sensor provided on the mobile device, wherein the location of the mobile device is calculated using information provided from the at least one encoder sensor when the artificial mark is not detected by the detector, and the location of the mobile device is calculated using the artificial mark when the artificial mark is detected by the detector.

29. The apparatus for estimating the location of the mobile as claimed in claim 28, wherein the location of the mobile device includes a position and an orientation of the mobile device.

30. The apparatus for estimating the location of the mobile as claimed in claim 28, wherein the image processor performs at least one of distortion compensation processing and pre-processing procedures on a non-processed or reversed image.

31. The apparatus for estimating the location of the mobile as claimed in claim 28, the artificial mark comprises:

a first mark having a non-reflective portion and an infrared reflective portion; and

a second mark having the non-reflective portion and the infrared reflective portion and located a predetermined distance apart from the first mark,

wherein each of the first and second marks is a retro-reflective artificial mark.

32. The apparatus for estimating the location of the mobile as claimed in claim 31, wherein an outer portion of the first mark and an inner portion of the second mark are non-reflective portions, and an inner portion of the first mark and an outer portion of the second mark are infrared reflective portions.

33. The apparatus for estimating the location of the mobile as claimed in claim 31, wherein an outer portion of the first mark and an inner portion of the second mark are infrared reflective portions, and an inner portion of the first mark and an outer portion of the second mark are non-reflective portions.

34. The apparatus for estimating the location of the mobile as claimed in claim 31, wherein the device to obtain the ceiling image includes a wide-angle lens and obtains the ceiling image of the area where the mobile device operates.

35. The apparatus for estimating the location of the mobile as claimed in claim 31, wherein the location determination device estimates the location of the mobile device with respect to a location of the artificial mark according to a calculated distance between the first mark and the second mark and a calculated angle between a line obtained by expanding a center of the artificial mark toward the x-axis and a line connecting the first mark and the second mark.

36. The apparatus for estimating the location of the mobile as claimed in claim 31, wherein the location determination device estimates the location of the mobile device by performing a shift process, a rotation process, and a flip process with respect to a location of the artificial mark.

37. The apparatus for estimating the location of the mobile as claimed in claim 31, wherein the image processor compares an average intensity of the ceiling image to a reference level, and reverses the ceiling image and performs distortion compensation processing and pre-processing with respect to the reversed ceiling image when the light of the current environment is not greater than the reference level according to the comparison result.

38. The apparatus for estimating the location of the mobile as claimed in claim 28, wherein the at least one encoder sensor is attached to a wheel of the mobile device and detects the location of the mobile device when the artificial mark on the ceiling image is not detected by the detector.

39. The apparatus for estimating the location of the mobile as claimed in claim 28, wherein the at least one encoder sensor comprises:

a first encoder sensor attached to a first wheel located on a first side of the mobile device;

a second encoder sensor attached to a second wheel located on a second side of the mobile device that is parallel with the first side; and

wherein a number of encoder pulses output from the first encoder sensor and the second encoder sensor is used to calculate the location of the mobile device according to a distance moved by the wheels of the mobile device.

40. A method of estimating a location of a mobile device in an environment subject to illuminations changes, the method comprising:

obtaining an image of an area where the mobile device operates;

detecting an artificial mark from the obtained image, the artificial mark includes a first mark and a second mark, each of the first mark and the second mark includes a non-reflective portion and an infrared reflective portion; and

estimating the position and the orientation of the mobile device using a position of the artificial mark.

41. The method of estimating the location of the mobile device as claimed in claim 40, further comprising:

extracting at least one candidate region with respect to the ceiling image according to geometrical information of the artificial mark; and

performing template matching with respect to the detected at least one candidate region, selecting the largest reliability value as a template matching result, and comparing the selected reliability value to a predetermined reference value,

wherein the artificial mark is determined to be detected when the selected reliability value is greater than the predetermined reference value.

42. The method of estimating the location of the mobile device as claimed in claim 41, further comprising performing at least one of distortion-compensation-processing, pre-processing, and binary-processing with respect to the ceiling image before extracting the at least one candidate region.

43. The method of estimating the location of the mobile device as claimed in claim 41, wherein a relative position and an orientation of the mobile device are estimated from the detected artificial mark by sequentially performing a shift process, a rotation process, and a flip process.