METHOD AND DEVICE OF MAPPING AND LOCALIZATION METHOD USING THE SAME

• US 20110123135A1
• Filed: 02/04/2010
• Published: 05/26/2011
• Est. Priority Date: 11/24/2009
• Status: Active Grant

• Current Assignee: Industrial Technology Research Institute
• Original Assignee: Industrial Technology Research Institute
• Inventors: Wu, Chin-Chia, Yu, Hung-Hsiu, Wang, Wei-Han, Hsieh, Hsiang-Wen, Tsai, Yu-Kuen
• Granted Patent: US 8,588,471 B2
• Time in Patent Office: Days
• US Class Current: 382/285
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06V 20/10 Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59
• CPC Class Codes: G06V 20/64 Three-dimensional objects

Abstract

A mapping method is provided. The environment is scanned to obtain depth information of environmental obstacles. The image of the environment is captured to generate an image plane. The depth information of environmental obstacles is projected onto the image plane, so as to obtain projection positions. At least one feature vector is calculated from a predetermined range around each projection position. The environmental obstacle depth information and the environmental feature vector are merged to generate a sub-map at a certain time point. Sub-maps at all time points are combined to generate a map. In addition, a localization method using the map is also provided.

Claims

1. A mapping method, comprising:

scanning an environment to obtain depth information of environmental obstacles;

capturing an image of the environment to generate an image plane;

projecting the environmental obstacle depth information onto the image plane, so as to obtain a plurality of projection positions;

calculating at least one feature vector from a predetermined range around each of the projection positions in the image plane;

combining the depth information of environmental obstacles and each of the feature vectors to generate a sub-map for a certain time point; and

combining the sub-maps at all time points to generate a map.

2. The mapping method as claimed in claim 1, wherein when the feature vector cannot be calculated from the predetermined range around each of the projection positions, at least an upper feature vector or at least a lower feature vector is calculated from upper or lower image data of each of the projection positions.

3. The mapping method as claimed in claim 2, wherein the upper feature vector or the lower feature vector is calculated according to upper or lower image data in the image plane relative to a vertical axis of each of the projection positions.

4. The mapping method as claimed in claim 1, wherein the feature vector is determined according to corner points, edges and regional extremum in the predetermined range of each of the projection positions on the image plane.

5. The mapping method as claimed in claim 1, wherein the step of combining the depth information of environmental obstacles and each of the feature vectors comprises:

converting the depth information of environmental obstacles to a world coordinate system; and

combining each of the feature vectors with the depth information of environmental obstacles represented by the world coordinate system.

6. The mapping method as claimed in claim 1, wherein the step of calculating the feature vectors according to the projection positions in the image plane further comprises abandoning the feature vectors with high similarities.

7. The mapping method as claimed in claim 1, wherein the step of scanning the environment is performed with a laser range finder.

8. The mapping method as claimed in claim 1, wherein the step of capturing the image of the environment is performed with a digital camera.

9. A localization method, comprising:

obtaining a map comprising the depth information of environmental obstacles and at least one feature vector of an image corresponding to the depth information of environmental obstacles;

using a visual sensing device to obtain an environmental image;

extracting at least one image feature point from the environmental image; and

performing a localization according to the image feature point and the map.

10. The localization method as claimed in claim 9, wherein the step of performing the localization according to the image feature point and the map is to calculate a world coordinates of the visual sensing device by using an image coordinates and a world coordinates of the image feature point of the environmental image, so as to implement the localization.

11. The localization method as claimed in claim 9, wherein the step of obtaining the map comprises:

scanning an environment to obtain the environmental obstacle depth information;

capturing an image of the environment to generate an image plane;

projecting the depth information of environmental obstacles onto the image plane, so as to obtain a plurality of projection positions;

calculating at least one feature vector from a predetermined range around each of the projection positions in the image plane;

combining the depth information of environmental obstacles and each of the feature vectors to generate a sub-map for a certain time point; and

combining the sub-maps at all time points to generate the map.

12. The localization method as claimed in claim 11, wherein when the feature vector cannot be calculated from the predetermined range around each of the projection positions, at least an upper feature vector or at least a lower feature vector is calculated from upper or lower image data of each of the projection positions.

13. The localization method as claimed in claim 12, wherein the upper feature vector or the lower feature vector is calculated according to upper or lower image data in the image plane relative to a vertical axis of each of the projection positions.

14. The localization method as claimed in claim 11, wherein the feature vector is determined according to corner points, edges and regional extremum in the predetermined range of each of the projection positions on the image plane.

15. The localization method as claimed in claim 11, wherein the step of combining the depth information of environmental obstacles and each of the feature vectors comprises:

converting the depth information of environmental obstacles to a world coordinate system; and

combining each of the feature vectors with the depth information of environmental obstacles represented by the world coordinate system.

16. A mapping device, comprising:

at least one range sensing device, for scanning an environment to generate the depth information of environmental obstacles;

at least one visual sensing device, for capturing an image of the environment to generate an image plane; and

an image processing device, coupled to the range sensing device and the visual sensing device, for projecting the depth information of environmental obstacles onto the image plane to obtain a plurality of projection positions, calculating at least one feature vector from a predetermined range around each of the projection positions in the image plane, and combining the depth information of environmental obstacles and the feature vectors to generated a map.

17. The mapping device as claimed in claim 16, wherein the range sensing device is a laser range finder.

18. The mapping device as claimed in claim 16, wherein the visual sensing device is a digital camera.

19. The mapping device as claimed in claim 16, wherein the range sensing device, the visual sensing device and the image processing device are deployed on a mobile platform.

20. The mapping device as claimed in claim 16, wherein the range sensing device and the visual sensing device are integrated as an embedded handheld device.