Methods and systems for obstacle detection using structured light

• US 9,002,511 B1
• Filed: 10/20/2006
• Issued: 04/07/2015
• Est. Priority Date: 10/21/2005
• Status: Active Grant

• Current Assignee: iRobot Corporation
• Original Assignee: iRobot Corporation
• Inventors: Hickerson, Kevin, Goncalves, Luis, Pirjanian, Paolo, Romanov, Nikolai
• Primary Examiner(s): Nguyen, John Q
• Assistant Examiner(s): KIM, KYUNG J
• Application Number:
  US11/584,355
• Time in Patent Office: 3,091 Days
• Field of Search: 700/245
• US Class Current: 700/245
• CPC Class Codes: G01S 17/04 Systems determining the presence of a target
• CPC Class Codes: G01S 17/86 Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders
• CPC Class Codes: G01S 17/89 for mapping or imaging
• CPC Class Codes: G01S 17/931 of land vehicles
• CPC Class Codes: G01S 7/483 Details of pulse systems
• CPC Class Codes: G05D 1/0038 by providing the operator with simple or augmented images from one or more cameras located onboard the vehicle, e.g. tele-operation images analyzed by a computer and used for automatic navigation G05D1/0246
• CPC Class Codes: G05D 1/0246 using a video camera in combination with image processing means
• CPC Class Codes: G05D 1/0248 in combination with a laser lasers per se H01S
• CPC Class Codes: G05D 1/0253 extracting relative motion information from a plurality of images taken successively, e.g. visual odometry, optical flow determining position or orientation from images G06T7/70
• CPC Class Codes: G05D 1/0274 using mapping information stored in a memory device navigation using map-matching G01C21/30
• CPC Class Codes: Y10S 901/01 Mobile robot

Abstract

An obstacle detector for a mobile robot while the robot is in motion is disclosed. The detector preferably includes at least one light source configured to project pulsed light in the path of the robot; a visual sensor for capturing a plurality of images of light reflected from the path of the robot; a processing unit configured to extract the reflections from the images; and an obstacle detection unit configured to detect an obstacle in the path of the robot based on the extracted reflections. In the preferred embodiment, the reflections of the projected light are extracted by subtracting pairs of images in which each pair includes a first image captured with the at least one light source on and a second image captured with the at least one light source off, and then combining images of two or more extracted reflections to suppress the background.

Claims

1. An obstacle detector for a mobile robot, the detector comprising:

at least one light source configured to project light in a path of the robot wherein light projected in the path of the robot has structure, and

wherein the light source is pulsed in a determined sequence;

a visual sensor for capturing a plurality of images comprising reflections of the projected light, wherein at least a portion of the plurality of images are captured by the visual sensor while the robot is in motion;

a processing unit for detecting reflections, wherein the processing unit is configured to;

generate at least two difference images from a first plurality of image pairs, a given image pair in the first plurality of images pairs comprising a first image captured with the at least one light source on and a second image captured with the at least one light source off; and

mathematically generate at least one filtered image to suppress movement artifacts, based at least in part on at least two difference images generated from respective pairs of images captured by the visual sensor;

compare pixels of the at least one filtered image to a threshold value to detect reflections from the light source;

a tilt sensor, included in the robot, configured to sense a tilt of the robot caused by a floor surface over which wheels of the robot are traversing;

wherein the tilt sensor included in the robot is configured to determine an orientation of the robot with respect to at least the floor surface the wheels of robot are traversing; and

an obstacle detection unit configured to detect an object in the filtered image and determine whether the object is an obstacle or is the surface the robot is traversing based on the detected reflections from the at least one filtered image, wherein the obstacle detection unit is configured to apply tilt compensation with respect to one or more images, comprising one or more images of reflected light, based at least in part on information from the tilt sensor to facilitate determination as to whether the object is an obstacle or is the surface the robot is traversing.

2. The obstacle detector of claim 1, wherein the at least one light source comprises at least one laser.

3. The obstacle detector of claim 1, wherein the at least one light source comprises at least one light emitting diode (LED).

4. The obstacle detector of claim 1, wherein the light projected in the path of the robot comprises one or more line segments.

5. The obstacle detector of claim 1, wherein the light projected in the path of the robot comprises one or more points of focused light.

6. The obstacle detector of claim 1, wherein the obstacle detection unit is configured to detect an obstacle in the path of the robot further based on the orientation of the robot.

7. The obstacle detector of claim 1, wherein, the determined sequence is a pseudo-random sequence.

8. The obstacle detector of claim 1, wherein the visual sensor is a video camera.

9. The obstacle detector of claim 8, wherein the video camera and the at least one light source are synchronized.

10. The obstacle detector of claim 1, wherein the plurality of images are captured by the visual sensor while the robot is in motion.

11. The obstacle detector of claim 1, wherein the processing unit is further configured to weight each of the plurality of images with a frame difference persistence scalar prior to generating associated difference images.

12. The obstacle detector of claim 11, wherein a value of each frame difference persistence scalar is selected based on an exponential decay model.

13. The obstacle detector of claim 1, wherein the threshold is based at least in part on one or more of:

robot velocity information,

ambient lighting conditions, or

reflectivity of observed surfaces.

14. The obstacle detector of claim 1, wherein the tilt sensor is configured to sense an uneven condition on the surface the robot is traversing.

15. The obstacle detector of claim 1, wherein the obstacle detector is configured to apply tilt compensation, with respect to the one or more images, to adjust a region in at least one of the one or more images corresponding to a floor plane based at least in part on information from the tilt sensor.

16. The obstacle detector of claim 1, wherein the plurality of images are captured by the visual sensor while the robot is in motion.

17. The obstacle detector of claim 1, wherein the tilt sensor, included in the robot, is configured to sense a tilt of the robot caused by the robot traversing floor surface warp or objects on the floor surface.

18. A method of detecting obstacles in a path of a robot, the method comprising:

repeatedly pulsing at least one light source on and off in accordance with a determined sequence;

capturing a plurality of images of light reflected from the path of the robot, wherein at least a portion of the plurality of images are captured by a sensor while the robot is in motion;

generating, by at least one processing unit, two or more difference images by subtracting pairs of the plurality of images, wherein a given pair of images in the plurality of images comprises a first image with the at least one light source on and a second image with the light source off;

mathematically generating, by at least one processing unit, at least one filtered image to suppress movement artifacts, based at least in part on at least two difference images generated from respective pairs of images captured by the visual sensor;

comparing pixels of the at least one filtered image to a threshold value to detect reflections from the light source;

detecting an object based on a location of the detected reflections from the light source in the filtered images;

receiving tilt information from a tilt sensor included in the robot, the tilt information indicating an orientation of the robot with respect to at least a surface the robot is traversing, the tilt sensor configured to sense a tilt of the robot caused by a floor surface over which wheels of the robot are traversing;

applying tilt compensation with respect to one or more images, comprising one or more images of reflected light, based at least in part on information from the tilt sensor to facilitate determination as to whether the object is an obstacle or is the surface the robot is traversing.

19. The method of claim 18, wherein detecting an object comprises determining a position of a surface from which the reflected light is received.

20. The method of claim 19, wherein determining the position of the surface from which the reflected light is received comprises triangulating the position based on the location of the reflected light in the at least one filtered image.

21. The method of claim 18, wherein the threshold is based at least in part on one or more of:

robot velocity information,

ambient lighting conditions, or

reflectivity of observed surfaces.

22. The method of claim 18, wherein the tilt sensor is configured to sense an uneven condition on the surface the robot is traversing.

23. The method of claim 18, the method further comprising adjusting a region in at least one of the one or more images corresponding to a floor plane based at least in part on information from the tilt sensor.

24. The method of claim 18, wherein the tilt sensor, included in the robot, is configured to sense a tilt of the robot caused by the robot traversing floor surface warp or objects on the floor surface.