MOBILE ROBOT WITH SINGLE CAMERA AND METHOD FOR RECOGNIZING 3D SURROUNDINGS OF THE SAME

• US 20120019627A1
• Filed: 03/12/2010
• Published: 01/26/2012
• Est. Priority Date: 03/31/2009
• Status: Active Grant

• Current Assignee: LG Electronics, Inc. (LG Corporation)
• Original Assignee: LG Electronics, Inc. (LG Corporation)
• Inventors: Kim, Young-Gie, Yoon, Jeong-Suk, Yi, Dong-Hoon, Choi, Yoo-Jin, Lee, Seong-Soo, Kim, Yie-Bin, Baek, Seung-Min, Na, Sang-Ik, Choe, Su-Uk, Lee, Jei-Hun
• Granted Patent: US 9,060,658 B2
• Time in Patent Office: Days
• US Class Current: 348/50
• CPC Class Codes: A47L 2201/04 Automatic control of the travelling movement; Automatic obstacle detection
• CPC Class Codes: A47L 9/009 Carrying-vehicles; Arrangements of trollies or wheels; Means for avoiding mechanical obstacles
• CPC Class Codes: G01S 15/931 of land vehicles
• CPC Class Codes: G01S 2015/938 in the bumper area radar in bumper area G01S2013/93275; bumper with obstacle sensor of electric or electronic type B60R19/483
• CPC Class Codes: G05D 1/0246 using a video camera in combination with image processing means
• CPC Class Codes: G05D 1/0253 extracting relative motion information from a plurality of images taken successively, e.g. visual odometry, optical flow determining position or orientation from images G06T7/70
• CPC Class Codes: G05D 1/0274 using mapping information stored in a memory device navigation using map-matching G01C21/30

Abstract

Disclosed are a mobile robot with a single camera capable of performing a cleaning process with respect to surroundings, and capable of more precisely making a 3D map of the surroundings including a plurality of feature points, and a method for recognizing 3D surroundings of the same. According to the method, images of the surroundings are captured, and a preset number of particles with respect to feature points of a first image are projected to a second image based on matching information of feature points extracted from the two images sequentially captured, thereby extracting 3D information of the surroundings.

Claims

1. A mobile robot with a single camera, comprising:

a detection unit having a single camera, and configured to sequentially capture images including a ceiling with using the single camera in a moving or stopped state on a cleaning region; and

a control unit configured to extract feature points from the sequentially-captured images, to extract 3D information of a substantial feature point of surroundings, and thereby to make a 3D map.

2. The mobile robot with a single camera of claim 1, wherein the control unit is configured to generate a preset number of particles on a virtual line based on matching information of the same feature points of two neighboring images, the virtual line passing through a feature point of a first image of the two neighboring images and a center of the camera, and is configured to extract 3D information of a substantial feature point of the surroundings by projecting the generated particles to a second image of the two neighboring images.

3. The mobile robot with a single camera of claim 2, wherein the control unit is configured to estimate a position of the mobile robot in the surroundings and to make a 3D map by repeatedly performing, on the cleaning region, the step of generating a preset number of particles on a virtual line based on matching information of the same feature points of two neighboring images, the virtual line passing through a feature point of a first image of the two neighboring images and a center of the camera, and the step of extracting 3D information of a substantial feature point of the surroundings by projecting the generated particles to a second image of the two neighboring images.

4. The mobile robot with a single camera of claim 3, wherein the control unit comprises:

a feature point extractor configured to extract feature points from the captured images;

a particle generator configured to generate feature point particles from a first image based on feature points extracted from two neighboring images sequentially captured; and

a 3D information processor configured to make a 3D map by extracting 3D information of a feature point of a first image by projecting the generated particles to a second image.

5. The mobile robot with a single camera of claim 4, further comprising:

a position detection unit configured to detect position information varied according to movement of the mobile robot; and

a storage unit configured to store therein an operation program for driving the mobile robot, and configured to store at least one of image information sensed by the detection unit, feature points extracted from the image information, position information of the mobile robot detected by the position detection unit, estimation information, and the 3D map.

6. The mobile robot with a single camera of claim 1, wherein the detection unit comprises:

a sensor unit configured to sense an obstacle by using a signal, or configured to determine a movement path; and

a camera unit configured to capture images including a ceiling in a moving or stopped state on the cleaning region.

7. The mobile robot with a single camera of claim 1, wherein the camera is fixedly-installed to an upper side of the mobile robot to obtain a ceiling image.

8. A method for recognizing 3D surroundings of a mobile robot with a single camera, the method comprising:

a feature point processing step of calculating matching information of the same feature points of two neighboring images, by extracting a plurality of feature points from each surrounding image sequentially captured with using the single camera;

generating a preset number of particles on a virtual line based on the matching information, the virtual line passing through a feature point of a first image of the two neighboring images and a center of the camera;

extracting 3D information of a substantial feature point of the surroundings by projecting the generated particles to a second image of the two neighboring images; and

estimating a position of the mobile robot in the surroundings and making a 3D map based on the 3D information.

9. The method of claim 8, wherein in the step of making a 3D map, a position of the mobile robot in the surroundings is estimated, and a 3D map is made by repeatedly performing, on a cleaning region, the feature point processing step, the particle generating step, and the 3D information extracting step.

10. The method of claim 8, wherein the preset number of particles are initially set to have the same interval and the same weight on the virtual line.

11. The method of claim 10, wherein the preset number of particles comprise a substantial feature point of the surroundings, the substantial feature point corresponding to the feature point of each of the first and second images.

12. The method of claim 8, wherein the step of making a 3D map comprises:

determining whether a bottom map of the created 3D map forms a closed curved line; and

connecting a starting point and an ending point of the bottom map to each other if the bottom map does not form a closed curved line.

13. The method of claim 8, wherein the step of extracting 3D information comprises:

calculating each matching probability of the preset number of particles projected to the second image with the substantial feature point; and

selecting a particle having a highest probability among the calculated probabilities thereby obtaining a height of the substantial feature point.

14. The method of claim 8, wherein the feature point processing step comprises:

obtaining a first image of the surroundings while performing a cleaning process with respect to the surroundings, thereby extracting feature points of the first image;

if the number of the feature points of the extracted first image is more than a reference value, moving the mobile robot to the surroundings for a predetermined time, then obtaining a second image thereby extracting feature points from the second image, and calculating movement information of the mobile robot; and

if there exist two or more images from which feature points having the number more than the reference value have been extracted, matching the feature points of the first and second images with each other based on the movement information of the mobile robot.

15. The method of claim 14, wherein in a case that the camera is fixedly-installed to the mobile robot, movement information of the camera is calculated from movement information of the mobile robot.

16. The method of claim 8, further comprising storing said one or more images captured from the surroundings, the extracted feature points, the matching information of the feature points, and the 3D information of the extracted feature points.

17. The method of claim 8, wherein the feature points comprise ceiling edges, corners, and a lamp.