Methods and systems for obstacle detection using structured light

• US 9,632,505 B2
• Filed: 03/02/2015
• Issued: 04/25/2017
• Est. Priority Date: 10/21/2005
• Status: Active Grant

• Current Assignee: iRobot Corporation
• Original Assignee: iRobot Corporation
• Inventors: Hickerson, Kevin, Goncalves, Luis, Pirjanian, Paolo, Romanov, Nikolai
• Primary Examiner(s): Shaawat, Mussa A
• Assistant Examiner(s): KIM, KYUNG J
• Application Number:
  US14/635,800
• Publication Number:
  US 20150168954A1
• Time in Patent Office: 785 Days
• Field of Search: None
• CPC Class Codes: G01S 17/04 Systems determining the presence of a target
• CPC Class Codes: G01S 17/86 Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders
• CPC Class Codes: G01S 17/89 for mapping or imaging
• CPC Class Codes: G01S 17/931 of land vehicles
• CPC Class Codes: G01S 7/483 Details of pulse systems
• CPC Class Codes: G05D 1/0038 by providing the operator with simple or augmented images from one or more cameras located onboard the vehicle, e.g. tele-operation images analyzed by a computer and used for automatic navigation G05D1/0246
• CPC Class Codes: G05D 1/0246 using a video camera in combination with image processing means
• CPC Class Codes: G05D 1/0248 in combination with a laser lasers per se H01S
• CPC Class Codes: G05D 1/0253 extracting relative motion information from a plurality of images taken successively, e.g. visual odometry, optical flow determining position or orientation from images G06T7/70
• CPC Class Codes: G05D 1/0274 using mapping information stored in a memory device navigation using map-matching G01C21/30
• CPC Class Codes: Y10S 901/01 Mobile robot

Abstract

An obstacle detector for a mobile robot while the robot is in motion is disclosed. The detector preferably includes at least one light source configured to project pulsed light in the path of the robot; a visual sensor for capturing a plurality of images of light reflected from the path of the robot; a processing unit configured to extract the reflections from the images; and an obstacle detection unit configured to detect an obstacle in the path of the robot based on the extracted reflections. In the preferred embodiment, the reflections of the projected light are extracted by subtracting pairs of images in which each pair includes a first image captured with the at least one light source on and a second image captured with the at least one light source off, and then combining images of two or more extracted reflections to suppress the background.

Claims

1. A mobile robot, the mobile robot comprising:

at least one laser light source configured to at least;

project a first laser stripe ahead of the mobile robot, and

project a second laser stripe ahead of the mobile robot,

wherein the first laser stripe and the second laser stripe cover an entire profile of the mobile robot;

a visual sensor configured to detect the first laser stripe and the second laser stripe;

an obstacle detection unit configured to detect, from images captured by the visual sensor, an obstacle based at least in part on detection of an intersection of the first laser stripe, or the second laser stripe, or both the first laser stripe and the second laser stripe, with the obstacle; and

a processing unit configured to maintain a two dimensional map of detected obstacles, wherein the visual sensor comprises a single camera utilized for obstacle detection and the at least one laser light source is configured to upwardly project the first laser stripe, and the obstacle detection unit is configured to detect an overhanging obstacle based at least in part on an intersection of the upwardly projected first laser stripe with the overhanging obstacle.

2. The mobile robot as defined in claim 1, wherein:

the processing unit is further configured to maintain a three dimensional map of an area traversed by the mobile robot, the three dimensional map including a three dimensional mapping in space of one or more of the detected obstacles;

the at least one laser light source comprises a first laser light source and a second laser light source, and the first laser light source is configured to upwardly project the first laser stripe, and the second laser light source is configured to downwardly project the second laser stripe,

wherein the first laser stripe and the second laser stripe cover a full forward facing profile of the mobile robot, and the obstacle detection unit is configured to detect an overhanging obstacle based at least in part on an intersection of the upwardly projected first laser stripe with the overhanging obstacle; and

the visual sensor comprises a bandpass interference filter.

3. The mobile robot as defined in claim 1, wherein the processing unit is further configured to maintain a three dimensional map of an area traversed by the mobile robot, the three dimensional map including a three dimensional mapping in space of the detected obstacles.

4. The mobile robot as defined in claim 1, wherein the at least one laser light source is configured to:

downwardly project the first laser stripe, and

upwardly project the second laser stripe,

wherein the first laser stripe and the second laser stripe cover a full forward facing profile of the mobile robot.

5. The mobile robot as defined in claim 1, wherein the visual sensor is configured to cover the full forward facing profile of the mobile robot.

6. The mobile robot as defined in claim 1, wherein the obstacle detection unit is configured to detect a size of the obstacle based at least in part on an intersection of both the first laser stripe and the second laser stripe with the obstacle.

7. The mobile robot as defined in claim 1, wherein the at least one laser light source is configured to:

project the first laser stripe onto a floor surface a first distance ahead of a current position of the mobile robot; and

project the second laser stripe onto the floor surface a second distance ahead of the current position of the mobile robot, the second distance greater than the first distance.

8. The mobile robot as defined in claim 1, wherein the visual sensor comprises at least one of object recognition, object tracking, video streaming for surveillance, tele-operation, or video-conferencing.

9. The mobile robot as defined in claim 1, wherein the obstacle detection unit configured to detect, from images captured by the visual sensor, an obstacle based at least in part on a determination that a position of the first stripe in an image captured by the visual sensor is at a height higher than a floor plane of a floor on which the mobile robot is supported.

10. The mobile robot as defined in claim 1, wherein the obstacle detection unit is configured to mathematically generate at least one filtered image to suppress movement artifacts based at least in part on at least two difference images generated from respective pairs of images captured by the visual sensor.

11. The mobile robot as defined in claim 10, further comprising a tilt sensor configured to determine an orientation of the robot with respect to at least the floor surface the wheels of the robot are traversing, wherein the obstacle detection unit is configured to apply tilt compensation with respect to one or more images.

12. A method of detecting obstacles in the path of a mobile robot, the method comprising:

projecting, by a light source affixed to the mobile robot, a first stripe ahead of the mobile robot, and

projecting, by the light source affixed to the mobile robot, a second stripe ahead of the mobile robot,

wherein the first stripe and the second stripe cover an entire profile of the mobile robot;

capturing, by a sensor, one or more images of the first stripe and the second stripe;

detecting, by a processing unit, from one or more images captured by the visual sensor, an obstacle based at least in part on detection of an intersection of the first stripe, or the second stripe, or both the first stripe and the second stripe, with the obstacle; and

maintaining a map of detected obstacles, wherein the visual sensor comprises a single camera utilized for obstacle detection and the at least one laser light source is configured to upwardly project the first laser stripe, and the obstacle detection unit is configured to detect an overhanging obstacle based at least in part on an intersection of the upwardly projected first laser stripe with the overhanging obstacle.

13. The method as defined in claim 12,

wherein projecting, by a light source affixed to the mobile robot, the second stripe ahead of the mobile robot, further comprises downwardly projecting the second stripe using a second device;

the method further comprising;

detecting, by the processing unit, an overhanging obstacle based at least in part on an intersection of the upwardly projected first stripe with the overhanging obstacle; and

maintaining, by the processing unit, a three dimensional map of an area traversed by the mobile robot, the three dimensional map including a three dimensional mapping in space of one or more of the detected obstacles.

14. The method as defined in claim 12, the method further comprising maintaining a three dimensional map of an area traversed by the mobile robot, the three dimensional map including a three dimensional mapping in space of the detected obstacles.

15. The method as defined in claim 12, the method further comprising:

downwardly projecting the first stripe, and

upwardly projecting the second stripe,

wherein the first stripe and the second stripe cover a full forward facing profile of the mobile robot.

16. The method as defined in claim 12, wherein the visual sensor is configured to cover the full forward facing profile of the mobile robot.

17. The method as defined in claim 12, the method further comprising:

upwardly projecting the first stripe; and

detecting an overhanging obstacle based at least in part on an intersection of the upwardly projected first stripe with the overhanging obstacle.

18. The method as defined in claim 12, the method further comprising detecting a size of the obstacle based at least in part on an intersection of both the first stripe and the second stripe with the obstacle.

19. The method as defined in claim 12, the method further comprising:

projecting the first stripe onto a floor surface a first distance ahead of a current position of the mobile robot; and

projecting the second stripe onto the floor surface a second distance ahead of the current position of the mobile robot, the second distance greater than the first distance.

20. The method as defined in claim 12, further comprising utilizing the single camera for at least one of video streaming for surveillance, tele-operation, or video-conferencing, in addition to utilizing the visual sensor for obstacle detection.

21. The method as defined in claim 12, the method further comprising detecting, from images captured by the visual sensor, an obstacle based at least in part on a determination that a position of the first stripe in an image captured by the visual sensor is at a height higher than a floor plane of a floor on which the mobile robot is supported.

22. The method as defined in claim 12, further comprising mathematically generating at least one filtered image to suppress movement artifacts, based at least in part on at least two different images generated from respective pairs of images captures by the visual sensor.

23. The method as defined in claim 22, further comprising:

receiving tilt information from a tilt sensor included in the robot, the tilt information indicating an orientation of the robot with respect to at least a surface the robot is traversing, the tilt sensor configured to sense a tilt of the robot caused by a floor surface over which wheels of the robot are traversing; and

applying tilt compensation with respect to one or more images, comprises one or more images of reflected light, based at least in part on information from the tilt sensor to facilitate determination as to whether the object is an obstacle or is the surface the robot is traversing.