Systems and methods for automatic detection of spills

• US 9,987,752 B2
• Filed: 06/10/2016
• Issued: 06/05/2018
• Est. Priority Date: 06/10/2016
• Status: Active Grant

• Current Assignee: Brain Corporation
• Original Assignee: Brain Corporation
• Inventors: Fisher, Dimitry, Griffin, Cody, Richert, Micah, Piekniewski, Filip, Izhikevich, Eugene, Nageswaran, Jayram Moorkanikara, Black, John
• Primary Examiner(s): Dang, Duy M
• Application Number:
  US15/179,851
• Publication Number:
  US 20170355081A1
• Time in Patent Office: 725 Days
• CPC Class Codes: B25J 9/1697 Vision controlled systems
• CPC Class Codes: G05D 1/0016 characterised by the operator's input device input arrangements for computing systems in general G06F3/00
• CPC Class Codes: G05D 1/0246 using a video camera in combination with image processing means
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/10048 Infrared image
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G08B 21/20 responsive to moisture
• CPC Class Codes: H04N 5/33 Transforming infrared radiation cameras or camera modules for generating image signals from infrared radiation H04N23/20; circuitry of SSIS for transforming infrared radiation into image signals H04N25/20

Abstract

Systems and methods for automatic detection of spills are disclosed. In some exemplary implementations, a robot can have a spill detector comprising at least one optical imaging device configured to capture at least one image of a scene containing a spill while the robot moves between locations. The robot can process the at least one image by segmentation. Once the spill has been identified, the robot can then generate an alert indicative at least in part of a recognition of the spill.

Claims

1. A robot comprising:

an actuator configured to move the robot between a plurality of locations;

a spill detector comprising at least one optical imaging device configured to capture a plurality of images of a scene containing a spill while the robot moves between the plurality of locations; and

a processor apparatus configured to identify the spill in the plurality of images, and generate an alert indicative at least in part of a recognition of the spill;

wherein;

the processor is further configured to determine a confidence measure in the identification of the spill;

the generation of the alert indicative at least in part of the recognition of the spill is based on the confidence measure meeting or exceeding a predetermined threshold, the confidence measure being determined based at least on a quantity of images containing at least a portion of the identified spill relative to a quantity of images captured over a predetermined time interval; and

the generation of the alert comprises a presentation of a plurality of user-selectable options associated with actions of the robot with respect to the identified spill.

2. The robot of claim 1, wherein the at least one optical imaging device comprises an infrared camera and the plurality of images comprises thermal images.

3. The robot of claim 1, further comprising a temperature adjuster configured to change a temperature value of the scene containing the spill.

4. The robot of claim 3, wherein the temperature adjuster comprises at least one of an exhaust and a fan.

5. The robot of claim 1, wherein the robot further comprises a sensor configured to detect at least one of reflectance properties, emission properties, electrical properties, noises, and friction of the scene.

6. The robot of claim 5, wherein the confidence measure is based at least in part on information from both the sensor and the at least one image.

7. The robot of claim 1, wherein the processor is further configured to generate a color image having a plurality of colors based at least in part on a plurality of thermal values of a segment of the plurality of images, and the plurality of colors are indicative at least in part of the spill.

8. The robot of claim 1, wherein the robot further comprises a floor cleaning system.

9. The robot of claim 8, wherein the processor is further configured to receive an action command in response to the alert and to turn off the floor cleaning system based at least in part on the received action command.

10. The robot of claim 1, wherein the processor is further configured to utilize a learning-based visual classification on a library of spill images, and identify the spill in the at least one image based at least in part on the learning-based visual classification.

11. A method for detecting a spill comprising:

generating a first image of a first scene at a first location that contains a spill;

generating a second image of a second scene at a second location that contains no spills;

segmenting the first image to detect the spill from at least thermal values in a segment of the first image, the segmenting of the first image comprising identifying a difference between (i) a known thermal value associated with the second scene and (ii) the at least thermal values in the segment of the first image;

identifying the spill based on the difference; and

generating an alert indicative at least in part of the identification of the spill.

12. The method of claim 11, further comprising adjusting a temperature value of the first scene while generating the first image.

13. The method of claim 11, further comprising determining a confidence value in the identified spill, wherein the generated alert is further indicative of the confidence value.

14. The method of claim 11, further comprising sensing at least one of reflectance properties, emission properties, electrical properties, noises, and friction of the first scene.

15. The method of claim 14, further comprising determining a confidence value in the identified spill based at least in part on the segmentation of the first image and the sensed at least one of reflectance properties, emission properties, electrical properties, noises, and friction of the first scene, wherein the generated alert is further indicative of the confidence value.

16. The method of claim 11, further comprising receiving an action command in response to the generated alert and performing an action in response to the action command.

17. A robot comprising:

an actuator configured to move the robot between a plurality of locations;

a spill detector comprising at least one optical imaging device configured to capture at least one image of a scene containing a spill while the robot moves between the plurality of locations; and

a processor apparatus configured to;

generate an action command based at least in part on the at least one image;

receive feedback from an operator, the feedback comprising at least a confirmation of the generated action command; and

based at least on the confirmation;

perform an autonomous physical action with respect to the spill;

adjust a confidence parameter associated with a detection of the spill by the robot; and

associate the autonomous physical action performed by the robot with detection of a subsequent spill.

18. The robot of claim 17, further comprising a temperature adjuster configured to change a temperature value of the scene containing the spill.

19. The robot of claim 17, wherein the action command is a stop command that is configured to stop the robot from moving between the plurality of locations.

20. The robot of claim 17, wherein:

the robot is in data communication with a user interface configured to receive the feedback;

the action command comprises a query by the robot, the query being related to at least one of (i) the detection of the spill and (ii) the autonomous physical action with respect to the spill.