Apparatus for localization using image and range data and method thereof

• US 8,818,016 B2
• Filed: 03/08/2011
• Issued: 08/26/2014
• Est. Priority Date: 06/08/2010
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Choi, Hyun-Do, Park, Jun-Ho
• Primary Examiner(s): Lu, Tom Y
• Application Number:
  US13/043,046
• Publication Number:
  US 20110299736A1
• Time in Patent Office: 1,267 Days
• Field of Search: 382/100, 382/103, 382/106
• US Class Current: 382/100
• CPC Class Codes: G06V 20/10 Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59

Abstract

A localization apparatus and method for localizing by use of image information and range data are provided. The localization apparatus includes an image sensor which captures an image and outputs image data, a range sensor which senses a distance and outputs a range sensing signal, an image feature point information processing unit which extracts image feature points from the sensed image and generates image feature point information about the extracted image feature points, a range feature point information processing unit which generates range feature point information about range feature points from the range sensing signal, and a state estimation unit which estimates a state variable including a position of the localization apparatus by using the image feature point information and the range feature point information as an observation value.

Claims

1. A localization apparatus comprising:

an image sensor which captures an image and outputs image data;

a range sensor which receives a range sensing signal transmitted from a transmitter which is located in a range feature point and outputs the received range sensing signal;

an image feature point information processing unit configured to extract image feature points from the image data output by the image sensor and generates image feature point information about the extracted image feature points;

a range feature point information processing unit which generates range feature point information about the range feature point from the range sensing signal; and

a state estimation unit which estimates a state variable comprising a position of the localization apparatus by using the image feature point information and the range feature point information as an observation value,

wherein the state estimation unit increases an uncertainty of the range feature point information in proportion to an average height of the image feature points and estimates the state variable by use of the image feature point information and the range feature point information, to which the increased uncertainty is applied.

2. The localization apparatus of claim 1, wherein if the range feature point information comprises a position of the range feature point, the state variable further comprises a posture of the localization apparatus and positions of a plurality of image feature points.

3. The localization apparatus of claim 1, wherein if the range feature point information does not comprise a position of the range feature point, the state variable further comprises a posture of the localization apparatus, position of a plurality of image feature points and the position of the range feature point.

4. The localization apparatus of claim 1, wherein the state estimation unit increases an uncertainty of the range feature point information in proportion to an average angle of the image feature points with respect to a horizontal plane and estimates the state variable by use of the image feature point information and the range feature point information, to which the increased uncertainty is applied.

5. A localization apparatus comprising:

an image sensor which captures an image and outputs image data;

a range sensor which receives a range sensing signal transmitted from a transmitter which is located in a range feature point and outputs the received range sensing signal;

an image feature point information processing unit configured to extract image feature points from the image data output by the image sensor and generates image feature point information about the extracted image feature points;

a range feature point information processing unit which generates range feature point information about the range feature point from the range sensing signal; and

a state estimation unit which estimates a state variable comprising a position of the localization apparatus by using the image feature point information and the range feature point information as an observation value,

wherein the state estimation unit determines an uncertain part of the range feature point information by use of the image feature point information, and if the uncertain part of the range feature point information is determined, the state estimation unit estimates the state variable in a condition that the uncertain part of the range feature point information is excluded from the observation value.

6. The localization apparatus of claim 5, wherein, if the number of the image feature points located at a region between the position of the localization apparatus and a position of the range feature point obtained from the range feature point information exceeds a value, the state estimation unit estimates the state variable in a condition that range feature point information corresponding to the position of the range feature point having the image feature points exceeding the value is excluded from the observation value.

7. The localization apparatus of claim 6, wherein the state estimation unit defines a plane perpendicular to a line connecting the localization apparatus and the range feature point, projects the image feature points onto the plane, defines an intersection point between the line connecting the localization apparatus and the range feature point and the plane, defines a circle having the intersection point as a center and having a radius, and if the number of the image feature points projected onto the defined circle exceeds a number, the state estimation unit estimates the state variable in a condition that the range feature point information is excluded from the observation value.

8. A localization method of a localization apparatus, the localization method comprising:

obtaining image data by an image sensor and range data by a range sensor;

extracting an image feature point from the image data;

generating image feature point information about the image feature point;

generating range feature point information about a range feature point from the range data; and

estimating a state variable comprising a position of a localization apparatus by using the image feature point information and the range feature point information as an observation value,

wherein the range data is received from a transmitter which is located in the range feature point, and

wherein the estimating the state variable comprises;

increasing uncertainty of the range feature point information in proportion to at least one of an average height of the image feature point and an average angle of the image feature points with respect to a horizontal plane; and

estimating the state variable by use of the image feature information and range feature point information, to which the increased uncertainty is applied.

9. The localization method of claim 8, wherein if the range feature point information comprises a position of the range feature point, the state variable further comprises a posture of the localization apparatus and positions of a plurality of image feature points.

10. The localization method of claim 8, wherein if the range feature point information does not comprise a position of the range feature point, the state variable comprises a posture of the localization apparatus, position of a plurality of image feature point and the position of the range feature point.

11. A localization method of a localization apparatus, the localization method comprising:

obtaining image data by an image sensor and range data by a range sensor;

extracting an image feature point from the image data;

generating image feature point information about the image feature point;

generating range feature point information about a range feature point from the range data; and

estimating a state variable comprising a position of a localization apparatus by using the image feature point information and the range feature point information as an observation value,

wherein the range data is received from a transmitter which is located in the range feature point, and

wherein the estimating the state variable further comprises, if the number of the image feature points located at a region between the position of the localization apparatus and a position of the range feature point obtained from the range feature point information exceeds a value, excluding range feature point information corresponding to the position of the range feature point having the image feature point exceeding the value, from the observation value.

12. The localization method of claim 11, wherein the excluding the range feature point information from the observation value comprises:

defining a plane perpendicular to a line connecting the localization apparatus and the range feature point;

projecting the image feature point onto the plane;

defining an intersection point between the line connecting the localization apparatus and the range feature point and the plane;

defining a circle having the intersection point as a center and having a radius of Rr; and

if a number of the image feature points projected onto the defined circle exceeds a number, excluding the range feature point information from the observation value.