Localization system and method of mobile robot based on camera and landmarks
First Claim
1. A localization system of a mobile robot, comprising:
- n number of landmarks attached at a ceiling and having wireless transmitting and receiving functions,a camera photographing the n number of landmarks;
a landmark detection part which flashes the landmarks attached at the ceiling and obtains positions and IDs of the landmarks from an image photographed by the camera to detect at least two landmarks;
a robot position detection part calculating the position of the mobile robot using the detected landmarks, the robot position detection part comprising a first module calculating current position and orientation of the mobile robot using an image of the detected two landmarks when the mobile robot is in a stop state such that the first module calculates the current position ‘
Pw’
of the mobile robot using the equation Pw=Rwe·
s(Rie·
(Pi−
Tie))+Twe, wherePw is a real position of the mobile robot on a world coordinate that is the calculated final position of the mobile robot,Pi is a position of the mobile robot on an image coordinate,Rwe is a rotational vector informing how much an extra coordinate rotationally moves with respect to the world coordinate,Rie is a rotational vector informing how much the image coordinate rotationally moves with respect to the extra coordinate,Tie is a parallel movement vector informing how much the image coordinate moves in parallel with respect to the extra coordinate,Twe is a parallel movement vector informing how much the extra coordinate moves in parallel with respect to the world coordinate, ands is a distance ratio between the two landmarks on the image and the real two landmarks;
a landmark position prediction part which, when a new landmark is attached within a working zone, calculates a position of the new landmark on an absolute coordinate;
a topology map building part which builds a topology map of the mobile robot using the calculated position of the new landmark; and
a robot controller controlling a navigation of the mobile robot using the built topology map.
1 Assignment
0 Petitions
Accused Products
Abstract
A localization system and method of a mobile robot using a camera and artificial landmarks in a home and a general office environment (or working zone) is provided. The localization system includes artificial landmarks having an LED flash function in an invisible wavelength band, a camera with a wide-angle lens, a module flashing landmarks attached at the ceiling and identifying positions and IDs of the landmarks from an image photographed by the camera having a filter, a module calculating position and orientation of the robot using two landmarks of the image in a stop state, a module, when a ceiling to which the landmarks are attached has different heights, a position of the robot, and a module, when a new landmark is attached in the working zone, calculating a position of the new landmark on an absolute coordinate.
19 Citations
19 Claims
-
1. A localization system of a mobile robot, comprising:
-
n number of landmarks attached at a ceiling and having wireless transmitting and receiving functions, a camera photographing the n number of landmarks; a landmark detection part which flashes the landmarks attached at the ceiling and obtains positions and IDs of the landmarks from an image photographed by the camera to detect at least two landmarks; a robot position detection part calculating the position of the mobile robot using the detected landmarks, the robot position detection part comprising a first module calculating current position and orientation of the mobile robot using an image of the detected two landmarks when the mobile robot is in a stop state such that the first module calculates the current position ‘
Pw’
of the mobile robot using the equation Pw=Rwe·
s(Rie·
(Pi−
Tie))+Twe, wherePw is a real position of the mobile robot on a world coordinate that is the calculated final position of the mobile robot, Pi is a position of the mobile robot on an image coordinate, Rwe is a rotational vector informing how much an extra coordinate rotationally moves with respect to the world coordinate, Rie is a rotational vector informing how much the image coordinate rotationally moves with respect to the extra coordinate, Tie is a parallel movement vector informing how much the image coordinate moves in parallel with respect to the extra coordinate, Twe is a parallel movement vector informing how much the extra coordinate moves in parallel with respect to the world coordinate, and s is a distance ratio between the two landmarks on the image and the real two landmarks; a landmark position prediction part which, when a new landmark is attached within a working zone, calculates a position of the new landmark on an absolute coordinate; a topology map building part which builds a topology map of the mobile robot using the calculated position of the new landmark; and a robot controller controlling a navigation of the mobile robot using the built topology map. - View Dependent Claims (2, 3, 4, 5, 6, 7)
-
-
8. A localization system of a mobile robot, comprising:
-
n number of landmarks attached at a ceiling and having wireless transmitting and receiving functions, a camera photographing the n number of landmarks; a landmark detection part which flashes the landmarks attached at the ceiling and obtains positions and IDs of the landmarks from an image photographed by the camera to detect at least two landmarks; a robot position detection part calculating the position of the mobile robot using the detected landmarks a landmark position prediction part which, when a new landmark is attached within a working zone, calculates a position of the new landmark on an absolute coordinate; wherein when a new landmark is attached in the working zone, the landmark position prediction part identifies an ID of the new landmark, calculates a real position ‘
Pw’
of the mobile robot on a world coordinate that is the calculated final position of the mobile robot in accordance to
Pw=Rwe·
s(Rie·
(Pi−
Tie))+Twewhere Pi is a position of the mobile robot on an image coordinate, Rwe is a rotational vector informing how much an extra coordinate rotationally moves with respect to the world coordinate, Rie is a rotational vector informing how much the image coordinate rotationally moves with respect to the extra coordinate, Tie is a parallel movement vector informing how much the image coordinate moves in parallel with respect to the extra coordinate, Twe is a parallel movement vector informing how much the extra coordinate moves in parallel with respect to the world coordinate, s is a distance ratio between the two landmarks on the image and the real two landmarks, and calculates a real position Gw of the new landmark in accordance to
Gw=Rwe·
s(Rie·
(Gi−
Tie))+Twe whereGi is a position of the new landmark on the image coordinate, and Gw is a real position of the new landmark; a topology map building part which builds a topology map of the mobile robot using the calculated position of the new landmark; and a robot controller controlling a navigation of the mobile robot using the built topology map. - View Dependent Claims (9, 10, 11, 12, 13, 14, 15)
-
-
16. A localization method of a mobile robot using a camera and a landmark, the localization method comprising the steps of
detecting the landmark from an image of the landmark photographed by the camera to calculate a position of a mobile robot, such that the detecting step comprises the steps of: -
acquiring an ID of the landmark using a wireless transmitting and receiving module of the mobile robot, flashing the landmark and photographing an image of the landmark of a ceiling; performing a binary search for the photographed image of the landmark to detect at least two landmarks; and calculating position and orientation of the mobile robot using the detected at least two landmarks wherein when the mobile robot is in a stop state, calculating a final position ‘
Pw’
of the mobile robot using the detected at least two landmarks in accordance to
Pw=Rwe·
s(Rie·
(Pi−
Tie))+Twe,when the mobile robot is in a moving state, calculating the position of the mobile robot by setting a mask with a predetermined size based on image coordinates of the at least two landmarks and searching an area of the mask set for the image coordinates of the landmarks after a moving of the mobile robot, wherein Pw is a real position of the mobile robot on a world coordinate that is the calculated final position of the mobile robot, Pi is a position of the mobile robot on an image coordinate, Rwe is a rotational vector informing how much an extra coordinate rotationally moves with respect to the world coordinate, s is a distance ratio between two landmarks on the image coordinate and on the world coordinate Rie is a rotational vector informing how much the image coordinate rotationally moves with respect to the extra coordinate, Tie is a parallel movement vector informing how much the image coordinate moves in parallel with respect to the extra coordinate, Twe is a parallel movement vector informing how much the extra coordinate moves in parallel with respect to the world coordinate, and s is a distance ratio between the two landmarks on the image and the real two landmarks; obtaining a position of the added landmark using the calculated position of the mobile robot when a new landmark is added to a working zone; setting the added landmark as a node to build a topology map; and controlling a navigation of the mobile robot using the built topology map and the calculated position of the mobile robot. - View Dependent Claims (17, 18, 19)
-
Specification