Systems, Devices, and Methods for Detecting Spills Using Audio Sensors

• US 20180091909A1
• Filed: 09/28/2017
• Published: 03/29/2018
• Est. Priority Date: 09/29/2016
• Status: Active Grant

• Current Assignee: Walmart Apollo, LLC (WalMart Inc.)
• Original Assignee: Wal Mart Stores Inc (WalMart Inc.)
• Inventors: Lewis, Steven Jackson, Charles, Kevin Matthew, Biermann, Matthew Dwain
• Granted Patent: US 10,531,210 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 3/16 Sound input; Sound output speech processing G10L
• CPC Class Codes: G06F 3/162 Interface to dedicated audio devices, e.g. audio drivers, interface to CODECs
• CPC Class Codes: G10L 2021/02166 Microphone arrays; Beamforming
• CPC Class Codes: G10L 25/51 for comparison or discrimination
• CPC Class Codes: H04R 1/406 microphones
• CPC Class Codes: H04R 2201/021 Transducers or their casings adapted for mounting in or to a wall or ceiling
• CPC Class Codes: H04R 2201/025 Transducer mountings or cabinet supports enabling variable orientation of transducer of cabinet
• CPC Class Codes: H04R 29/00 Monitoring arrangements; Testing arrangements for hearing aids H04R25/30; detection of loudspeaker connection H04R5/04; sound-field adaptation dependent on speaker detection H04S7/308
• CPC Class Codes: H04R 3/00 Circuits for transducers , loudspeakers or microphones

Abstract

A technique for detecting spills is described. A number of audio sensors detect a sound potentially associated with an object spilling on a surface and the audio data from the sensors is analyzed in order to generate an acoustical analysis of the sound. The analysis of the sound detected by the audio sensors is compared against a database of known sounds corresponding to a number of types of spill incidents. Based on a match detected between the analysis of the sound and a type of spill incident in the database of known sounds, the identity of the object spilled can be determined. The location of the spilled object can also be calculated by triangulating the sensor data from the audio sensors.

Claims

1. A system for detecting spills comprising:

a plurality of audio sensors configured to detect a sound potentially associated with an object spilling on a surface;

a sound processing module executed by a processor in a processing device, the sound processing module configured to;

perform an analysis of the sound detected by the plurality of audio sensors;

compare the analysis of the sound detected by the plurality of audio sensors against a database of known sounds corresponding to a plurality of types of spill incidents;

identify the object spilled based on a match between the analysis of the sound and a type of spill incident in the database of known sounds; and

a location analysis module executed by the processor, the location analysis module configured to;

triangulate sensor data from the plurality of audio sensors to determine a location associated with the sound detected by the plurality of audio sensors.

2. The system of claim 1, further comprising a sensor adjustment module executed by the processor, the sensor adjustment module configured to:

adjust an orientation of the plurality of audio sensors based on the determined location associated with the sound detected by the plurality of audio sensors.

3. The system of claim 1, further comprising a sensor adjustment module executed by the processor wherein the sound adjustment module is further configured to:

adjust a sensitivity of the plurality of audio sensors based on the analysis.

4. The system of claim 1, wherein the sound processing module is further configured to:

identify the surface on which the object spilled based on a match between the analysis of the sound detected by the plurality of audio sensors and a type of spill incident in the database of known sounds.

5. The system of claim 1, further comprising a notification module executed by the processor, the notification module configured to:

generate a task requesting that the surface upon which the object spilled be cleaned, a priority of the task determined based on the identity of the object spilled; and

transmit the task to an individual assigned to clean the spill.

6. The system of claim 1, further comprising:

one or more image devices configured to acquire one or more images of the determined location to confirm the identity of the object.

7. The system of claim 6 wherein the processing device is further configured to:

programmatically compare the one or more acquired images to a database of images of known objects that produce the plurality of known sounds so as to verify the identity of the object.

8. A method for detecting spills comprising:

detecting a sound potentially associated with an object spilling on a surface using a plurality of audio sensors;

performing an analysis of the sound detected by the plurality of audio sensors using a sound processing module executed by a processor in a processing device;

comparing, using the sound processing module, the analysis of the sound detected by the plurality of audio sensors against a database of known sounds corresponding to a plurality of types of spill incidents;

identifying, using the sound processing module, the object spilled based on a match between the analysis of the sound and a type of spill incident in the database of known sounds; and

triangulating sensor data from the plurality of audio sensors to determine a location associated with the detected sound.

9. The method of claim 8, further comprising:

adjusting an orientation of the plurality of audio sensors based on the determined location associated with the sound detected by the plurality of audio sensors.

10. The method of claim 9, further comprising:

detecting the sound potentially associated with the object spilling on the surface a second time following the adjustment of the orientation.

11. The method of claim 8, further comprising:

adjusting a sensitivity of the plurality of audio sensors based on the analysis of the sound detected by the plurality of audio sensors.

12. The method of claim 8, further comprising:

identifying the surface on which the object spilled, using the sound processing module, based on a match between the analysis of the sound detected by the plurality of audio sensors and a type of spill incident in the database of known sounds.

13. The method of claim 8, further comprising:

generating a task, using a notification module executed by the processor, requesting that the surface upon which the object spilled be cleaned, a priority of the task determined based on the identity of the object spilled; and

transmitting the task to an individual assigned to clean up the spill.

14. The method of claim 8 wherein one or more image devices are configured to acquire one or more images of the determined location to confirm the identity of the object, the method further comprising:

programmatically comparing the one or more images to a database of images of known objects that produce the plurality of known sounds so as to verify the identity of the object.

15. A non-transitory machine readable medium storing instructions executable by a processing device, wherein execution of the instructions causes the processing device to:

detect a sound potentially associated with an object spilling on a surface using a plurality of audio sensors;

perform an analysis of the sound detected by the plurality of audio sensors using a sound processing module executed by the processing device;

compare, using the sound processing module, the analysis of the sound detected by the plurality of audio sensors against a database of known sounds corresponding to a plurality of types of spill incidents;

identify the object spilled, using the sound processing module, based on a match between the analysis of the sound and a type of spill incident in the database of known sounds; and

triangulate sensor data from the plurality of audio sensors to determine a location associated with the sound.

16. The non-transitory machine readable medium of claim 15, wherein execution of the instructions further causes the processing device to:

identify the surface on which the object spilled, using the sound processing module, based on a match between the analysis of the sound detected by the plurality of audio sensors and a type of spill incident in the database of known sounds.

17. The non-transitory machine readable medium of claim 15, wherein execution of the instructions further causes the processing device to:

adjust a sensitivity of the plurality of audio sensors based on the analysis of the sound detected by the plurality of audio sensors.

18. The non-transitory machine readable medium of claim 15, wherein execution of the instructions further causes the processing device to:

execute a notification module to generate a task requesting that the surface upon which the object spilled be cleaned, wherein a priority of the task is determined based on the identity of the object spilled; and

transmit the task to an individual assigned to clean the spill.

19. The non-transitory machine readable medium of claim 15, wherein execution of the instructions further causes the processing device to:

adjust an orientation of the plurality of audio sensors based on the determined location associated with the sound detected by the plurality of audio sensors.

20. The non-transitory machine readable medium of claim 19, wherein execution of the instructions further causes the processing device to:

detect the sound potentially associated with the object spilling on the surface a second time following the adjustment of the orientation.