Systems and methods for automatic detection of spills

• US 10,464,213 B2
• Filed: 06/04/2018
• Issued: 11/05/2019
• Est. Priority Date: 06/10/2016
• Status: Active Grant

• Current Assignee: Brain Corporation
• Original Assignee: Brain Corporation
• Inventors: Fisher, Dimitry, Griffin, Cody, Richert, Micah, Piekniewski, Filip, Izhikevich, Eugene, Nageswaran, Jayram Moorkanikara, Black, John
• Primary Examiner(s): Dang, Duy M
• Application Number:
  US15/997,397
• Publication Number:
  US 20190061160A1
• Time in Patent Office: 519 Days
• CPC Class Codes: B25J 9/1697 Vision controlled systems
• CPC Class Codes: G05D 1/0016 characterised by the operator's input device input arrangements for computing systems in general G06F3/00
• CPC Class Codes: G05D 1/0246 using a video camera in combination with image processing means
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/10048 Infrared image
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G08B 21/20 responsive to moisture
• CPC Class Codes: H04N 23/23 from thermal infrared radiation
• CPC Class Codes: H04N 5/33 Transforming infrared radiation cameras or camera modules for generating image signals from infrared radiation H04N23/20; circuitry of SSIS for transforming infrared radiation into image signals H04N25/20

Abstract

Systems and methods for automatic detection of spills are disclosed. In some exemplary implementations, a robot can have a spill detector comprising at least one optical imaging device configured to capture at least one image of a scene containing a spill while the robot moves between locations. The robot can process the at least one image by segmentation. Once the spill has been identified, the robot can then generate an alert indicative at least in part of a recognition of the spill.

Claims

1. A robot for detecting at least one item, comprising:

a memory having computer readable instructions stored thereon; and

at least one processor configured to execute the computer readable instructions to,

identify the at least one item in a plurality of images captured by the robot between a plurality of locations;

determine a confidence measure in the identification of the at least one item, the confidence measure being determined based on at least a quantity of images containing at least a portion of the identified at least one item relative to a quantity of images captured over a predetermined time interval;

recognize the at least one item based on the confidence measure meeting or exceeding a predetermined threshold; and

generate an alert notification in response to the recognizing of the at least one item, the alert notification includes plurality of user-selectable options corresponding to actions of the robot with respect to the at least one item.

2. The robot of claim 1, further comprising:

at least one optical imaging device including an infrared camera, and

wherein,

the plurality of images includes thermal images, and

the at least one item corresponds to a spill on a floor.

3. The robot of claim 1, further comprising:

a temperature adjuster configured to change a temperature value of a scene containing the at least one item, the temperature adjuster includes at least one of an exhaust and a fan.

4. The robot of claim 1, further comprising:

a sensor configured to detect at least one of a reflectance property of a scene, emission property of the scene, electrical property of the scene, and friction property of the scene.

5. The robot of claim 1, wherein the at least one processor is further configured to execute the computer readable instructions to,

receive an action command in response to the alert notification;

and

transmit a signal to turn off a floor cleaning system based on the receipt of the action command.

6. The robot of claim 1, wherein the at least one processor is further configured to execute the computer readable instructions to,

utilize a learning-based visual classification on a library of images corresponding to the at least one item;

and

identify the at least one item based on the learning-based visual classification.

7. The robot of claim 1, wherein the at least one processor is further configured to execute the computer readable instructions to,

generate a color image based on the at least thermal values in the segment of the first image;

and

determine whether colors in the color image correspond to the at least one item.

8. A method for detecting at least one item, comprising:

segmenting a first image of a first scene to detect the at least one item from at least thermal values in a segment of the first image, the segmenting of the first image including identifying a difference between;

(i) a known thermal value associated with a different second scene, and (ii) the at least thermal values in the segment of the first image;

identifying the at least one item based on the difference; and

generating an alert notification corresponding to the identification of the at least one item.

9. The method of claim 8, further comprising:

adjusting a temperature value of the first scene while generating the first image.

10. The method of claim 8, further comprising:

determining a confidence value in the identified at least one item; and

sensing at least one of reflectance properties of the first scene, emission properties of the first scene, electrical properties of the first scene, and friction properties of the first scene,

wherein the alert notification is indicative of the confidence value.

11. The method of claim 10, wherein the confidence value is determined based at least in part on the segmentation of the first image and detection of at least one of the reflectance properties of the first scene, the emission properties of the first scene, the electrical properties of the first scene, and friction properties of the first scene.

12. The method of claim 8, further comprising:

receiving an action command in response to the generated alert and performing an action in response to the action command.

13. The method of claim 8, further comprising:

generating a color image based on the at least thermal values in the segment of the first image; and

determining whether colors in the color image correspond to the at least one item.

14. A non-transitory computer readable medium having computer readable instructions stored thereon that when executed by at least one processor configure the at least one processor to,

capture at least one image of a scene containing at least one item while a robot moves between a plurality of locations;

and

generate an action command based on at least in part on the at least one image;

receive feedback from an operator, the feedback comprising at least a confirmation of the generated action command; and

transmit a signal, based on the at least one confirmation, to do at least one of the following;

(i) perform an autonomous physical action with respect to the at least one item, (ii) adjust a confidence parameter associated with a detection of the at least one item, and (iii) associate the autonomous physical action performed by the robot with detection of a subsequent at least one item.

15. The non-transitory computer readable medium of claim 14, wherein the at least one processor is further configured to execute the computer readable instructions to change a temperature value of the scene including the at least one item.

16. The non-transitory computer readable medium of claim 14, wherein the non-transitory computer-readable medium coupled to a robot, the robot is in data communication with a user interface to receive feedback.

17. The non-transitory computer readable medium of claim 16, wherein the robot is configured to receive at least one action command, via the user interface, from a user.

18. The non-transitory computer readable medium of claim 14, wherein the at least one processor is further configured to,

generate a color image based on the at least thermal values in the segment of the first image;

and

determine whether colors in the color image correspond to the at least one item.