Systems and Methods for Performing Occlusion Detection

• US 20160144505A1
• Filed: 09/16/2015
• Published: 05/26/2016
• Est. Priority Date: 11/26/2014
• Status: Active Grant

• Current Assignee: iRobot Corporation
• Original Assignee: iRobot Corporation
• Inventors: Fong, Phillip, Meltzer, Jason, Barnada Rius, Marc
• Granted Patent: US 9,751,210 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: B25J 9/1602 characterised by the control system, structure, architecture
• CPC Class Codes: B25J 9/1697 Vision controlled systems
• CPC Class Codes: G05B 2219/40572 Camera combined with position sensor
• CPC Class Codes: G05D 1/0253 extracting relative motion information from a plurality of images taken successively, e.g. visual odometry, optical flow determining position or orientation from images G06T7/70
• CPC Class Codes: G05D 1/0274 using mapping information stored in a memory device navigation using map-matching G01C21/30
• CPC Class Codes: Y10S 901/01 Mobile robot
• CPC Class Codes: Y10S 901/47 Optical

Abstract

The present invention provides a mobile robot configured to navigate an operating environment, that includes a machine vision system comprising a camera that captures images of the operating environment using a machine vision system; detects the presence of an occlusion obstructing a portion of the field of view of a camera based on the captured images, and generate a notification when an occlusion obstructing the portion of the field of view of the camera is detected, and maintain occlusion detection data describing occluded and unobstructed portions of images being used by the SLAM application.

Claims

1. A mobile robot configured to navigate an operating environment, comprising:

a body containing;

a drive configured to translate the robot in a direction of motion;

a machine vision system comprising a camera that captures images of the operating environment of the mobile robot;

a processor;

memory containing a simultaneous localization and mapping (SLAM) application and a behavioral control application;

wherein the behavioral control application directs the processor to;

capture images using the machine vision system;

detect the presence of an occlusion obstructing a portion of the field of view of the camera based on the captured images; and

generate a notification when an occlusion obstructing the portion of the field of view of the camera is detected; and

wherein the behavioral control application further directs the processor to maintain occlusion detection data describing occluded and unobstructed portions of images being used by the SLAM application.

2. The mobile robot of claim 1, wherein the occlusion detection data describes portions of images that correspond to portions of the field of view of the camera.

3. The mobile robot of claim 1, wherein the occlusion detection data comprises a histogram that identifies different portions of the camera field of view and provides a corresponding frequency with which each portion of the field of view is used by the SLAM application to generate or identify features.

4. The mobile robot of claim 1, wherein detecting the presence of an occlusion further comprises identifying a portion of the camera field of view in the captured images that is not utilized by the SLAM application for a threshold number of images.

5. The mobile robot of claim 4, wherein detecting the threshold number of images is between at least 1-10 images per 300 mm of robot travel for a mobile robot traveling at a rate of speed between 220 mm per second and 450 mm per second.

6. The mobile robot of claim of 1, wherein the behavioral control application further directs a processor to identify a certain percentage of the camera field of view is occluded prior to generating the notification.

7. The mobile robot of claim of 6, wherein the occluded percentage of the camera field of view is between 30 and 75 percent.

8. The mobile robot of claim 1, wherein the notification is one of a displayed message, text message, voicemail, or electronic mail message.

9. The mobile robot of claim 1, wherein the notification is one of an alert, sound, or indicator light provided by the mobile robot.

10. The mobile robot of claim 1, wherein the behavioral control application further directs a processor to send the notification of the occlusion to a server.

11. The mobile robot of claim 1, wherein the occlusion is detected within a first portion of a set of captured images.

12. The mobile robot of claim 11, wherein the occlusion is a semi-transparent occlusion that blurs the first portion of the set of captured images that corresponds to the obstructed portion of the field of view.

13. The mobile robot of claim 11, wherein the occlusion is an opaque occlusion that completely occludes the first portion of the set of captured images that corresponds the obstructed portion of the field of view.

14. The mobile robot of claim 1, wherein the camera is positioned within a recessed structure under the top surface of the mobile robot and is tilted so that the camera optical axis is aligned at an acute angle of 30-40 degrees above a horizontal plane in line with the top surface and is aimed in a forward drive direction of the robot body, and wherein the field of view of the camera spans a frustum of 45-65 degrees in the vertical direction.

15. The mobile robot of claim 14, wherein the camera is protected by a lens cover aligned at an acute angle with respect to the optical axis of the camera.

16. The mobile robot of claim 15, wherein the lens cover is set back relative to an opening of the recessed structure and an acute angle with respect to the optical axis of the camera that is closer to perpendicular than an angle formed between a plane defined by the top surface and the optical axis of the camera.

17. The mobile robot of claim 15, wherein the acute angle is between 15 and 70 degrees.

18. The mobile robot of claim 15, wherein the angle formed between a plane defined by the opening in the recessed structure and the optical axis of the camera ranges between 10 and 60 degrees.

19. A mobile robot configured to navigate an operating environment, comprising:

a body containing;

a drive mechanism configured to translate the robot in a direction of motion;

an odometry sensor system;

a machine vision system comprising a camera that captures images of the operating environment of the mobile robot; and

at least one processor;

memory containing a behavioral control application and a visual simultaneous localization and mapping (VSLAM) application, a pre-loaded map of the operating environment, and an estimated robot pose;

wherein the behavioral control application directs a processor to;

actuate the drive and capture;

odometry data using the odometry sensor system; and

images using the machine vision system;

determine updated robot poses and update the map of the operating environment by providing at least the captured images and the odometry data as inputs to the VSLAM application;

update occlusion detection data describing portions of images being used by the VSLAM application to determine updated robot poses and update the map of the operating environment;

detect the presence of an occlusion obstructing the camera based on the occlusion detection data; and

provide a notification when the occlusion detection data indicates the presence of the occlusion obstructing the camera,

wherein the VSLAM application directs a processor to;

estimate the location of the mobile robot within the map of the environment based upon images captured by the machine vision sensor system and odometry data captured by the odometry sensor system, and

update the map based upon the images captured by the machine vision sensor system.

20. The mobile robot of claim 19, wherein the occlusion detection data describes portions of images that correspond to portions of a field of view of the camera.