Mobile robot with single camera and method for recognizing 3D surroundings of the same

• US 9,060,658 B2
• Filed: 03/12/2010
• Issued: 06/23/2015
• Est. Priority Date: 03/31/2009
• Status: Active Grant

• Current Assignee: LG Electronics, Inc. (LG Corporation)
• Original Assignee: LG Electronics, Inc. (LG Corporation)
• Inventors: Choi, Yoo-Jin, Kim, Young-Gie, Yoon, Jeong-Suk, Lee, Seong-Soo, Kim, Yie-Bin, Baek, Seung-Min, Na, Sang-Ik, Choe, Su-Uk, Yi, Dong-Hoon, Lee, Jei-Hun
• Primary Examiner(s): Anyikire, Chikaodili E
• Application Number:
  US13/262,192
• Publication Number:
  US 20120019627A1
• Time in Patent Office: 1,929 Days
• Field of Search: 700/253, 382/154, 382/104, 382/103, 455/456.1, 701/28
• US Class Current: 1/1
• CPC Class Codes: A47L 2201/04 Automatic control of the travelling movement; Automatic obstacle detection
• CPC Class Codes: A47L 9/009 Carrying-vehicles; Arrangements of trollies or wheels; Means for avoiding mechanical obstacles
• CPC Class Codes: G01S 15/931 of land vehicles
• CPC Class Codes: G01S 2015/938 in the bumper area radar in bumper area G01S2013/93275; bumper with obstacle sensor of electric or electronic type B60R19/483
• CPC Class Codes: G05D 1/0246 using a video camera in combination with image processing means
• CPC Class Codes: G05D 1/0253 extracting relative motion information from a plurality of images taken successively, e.g. visual odometry, optical flow determining position or orientation from images G06T7/70
• CPC Class Codes: G05D 1/0274 using mapping information stored in a memory device navigation using map-matching G01C21/30

Abstract

Disclosed are a mobile robot with a single camera capable of performing a cleaning process with respect to surroundings, and capable of more precisely making a 3D map of the surroundings including a plurality of feature points, and a method for recognizing 3D surroundings of the same. According to the method, images of the surroundings are captured, and a preset number of particles with respect to feature points of a first image are projected to a second image based on matching information of feature points extracted from the two images sequentially captured, thereby extracting 3D information of the surroundings.

Claims

1. A mobile robot with a single camera, comprising:

a detection unit having a single camera, and configured to sequentially capture images of different locations in a cleaning region using the single camera; and

a control unit configured to extract feature points from the sequentially-captured images, to match at least one same feature point of two neighboring images, and to extract 3D information of at least one real feature point, wherein the at least one real feature point is represented by the at least one same feature point,

wherein the control unit generates a plurality of particles on a virtual line, projects the generated particles to a second image of the two neighboring images, and extracts 3D information of the at least one real feature point based on a correlation between the projected particles and the at least one same feature point of the second image, and

wherein the virtual line passes through both a center of the camera and the at least one real feature point of a first image of the two neighboring images.

2. The mobile robot with a single camera of claim 1, wherein the control unit is configured to make a 3D map by extracted 3D information.

3. The mobile robot with a single camera of claim 1, wherein the control unit estimates a position of the mobile robot in the surroundings and make a 3D map by repeatedly performing, on the cleaning region, the step of generating the plurality of particles on the virtual line, and the step of extracting 3D information of the at least one real feature point of the surroundings.

4. The mobile robot with a single camera of claim 3, wherein the control unit comprises:

a feature point extractor configured to extract feature points from the captured images;

a particle generator configured to generate the plurality of particles on the virtual line based on feature points extracted from two neighboring images sequentially captured; and

a 3D information processor configured to make a 3D map by extracting 3D information of the at least one same feature point by projecting the generated particles to the second image.

5. The mobile robot with a single camera of claim 4, further comprising:

a position detection unit configured to detect position information varied according to movement of the mobile robot; and

a storage unit configured to store therein an operation program for driving the mobile robot, and configured to store at least one of image information sensed by the detection unit, feature points extracted from the image information, position information of the mobile robot detected by the position detection unit, estimation information, and the 3D map.

6. The mobile robot with a single camera of claim 1, wherein the detection unit comprises:

a sensor unit configured to sense an obstacle by using a signal, or configured to determine a movement path; and

a camera unit configured to capture images including a ceiling in a moving or stopped state on the cleaning region.

7. The mobile robot with a single camera of claim 1, wherein the camera is fixedly-installed to an upper side of the mobile robot to obtain a ceiling image.

8. The mobile robot with a single camera of claim 1, wherein the plurality of particles have the same interval on the virtual line.

9. A method for recognizing 3D surroundings of a mobile robot with a single camera, the method comprising:

a feature point processing step of calculating matching information of at least one same feature points of two neighboring images, by extracting a plurality of feature points from each surrounding image sequentially captured from different locations in a cleaning region using the single camera;

generating a plurality of particles on a virtual line based on the matching information, wherein the virtual line passes through both a center of the camera and at least one real feature point of a first image of the two neighboring images, wherein the at least one real feature point is represented by the at least one same feature point;

projecting the generated particles to a second image of the two neighboring images; and

extracting 3D information of the at least one real feature point based on a correlation between the projected particles and the at least one same feature point of the second images.

10. The method of claim 9, wherein the plurality of particles are to have the same interval and the same weight on the virtual line.

11. The method of claim 10, wherein the particles comprise the at least one real feature point of the surroundings.

12. The method of claim 9, wherein the step of extracting 3D information comprises:

calculating each matching probability of the plurality of particles projected to the second image with the real feature point; and

selecting a particle having a highest probability among the calculated probabilities thereby obtaining a height of the real feature point.

13. The method of claim 9, wherein the feature point processing step comprises:

obtaining a first image of the surroundings while performing a cleaning process with respect to the surroundings, thereby extracting feature points of the first image;

if the number of the feature points of the extracted first image is more than a reference value, moving the mobile robot to the surroundings for a predetermined time, then obtaining a second image thereby extracting feature points from the second image, and calculating movement information of the mobile robot; and

if there exist two or more images from which feature points having the number more than the reference value have been extracted, matching the feature points of the first and second images with each other based on the movement information of the mobile robot.

14. The method of claim 13, wherein in a case that the camera is fixedly-installed to the mobile robot, movement information of the camera is calculated from movement information of the mobile robot.

15. The method of claim 9, further comprising storing said one or more images captured from the surroundings, the extracted feature points, the matching information of the feature points, and the 3D information of the extracted feature points.

16. The method of claim 9, wherein the feature points comprise at least one of ceiling edges, corners, and a lamp.

17. The method of claim 9, further comprising:

estimating a position of the mobile robot in the surroundings and making a 3D map based on the 3D information.

18. The method of claim 17, wherein in the step of making the 3D map, the position of the mobile robot in the surroundings is estimated, and the 3D map is made by repeatedly performing, on a cleaning region, the feature point processing step, the particle generating step, and the 3D information extracting step.

19. The method of claim 17, wherein the step of making a 3D map comprises:

determining whether a bottom map of the created 3D map forms a closed curved line; and

connecting a starting point and an ending point of the bottom map to each other if the bottom map does not form a closed curved line.