Collaboration of audio sensors for geo-location of events in an artificial intelligence (AI) environment

• US 9,710,753 B1
• Filed: 10/22/2014
• Issued: 07/18/2017
• Est. Priority Date: 10/22/2013
• Status: Active Grant

• Current Assignee: Lumin, LLC
• Original Assignee: Lumin, LLC
• Inventors: Shams, Nima Lahijani, Malekal, Vinay Venkatesh, Nourshahi, Narges Banou
• Primary Examiner(s): Hill, Stanley K
• Assistant Examiner(s): Lee, Tsu-Chang
• Application Number:
  US14/521,450
• Time in Patent Office: 1,000 Days
• Field of Search: 706 46
• CPC Class Codes: G01S 2205/01 specially adapted for specific applications
• CPC Class Codes: G01S 5/18 using ultrasonic, sonic, or infrasonic waves
• CPC Class Codes: G01S 5/30 Determining absolute distances from a plurality of spaced points of known location
• CPC Class Codes: G06N 5/04 Inference or reasoning models
• CPC Class Codes: H04L 12/2803 Home automation networks
• CPC Class Codes: H04L 12/2825 Reporting to a device located outside the home and the home network access arrangements H04L12/2856; for remote control or remote monitoring of applications H04L67/025; telephonic communication systems adapted for combination with telemetering systems H04M11/002
• CPC Class Codes: H04L 12/2827 Reporting to a device within the home network; wherein the reception of the information reported automatically triggers the execution of a home appliance functionality
• CPC Class Codes: H04L 67/10 in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46
• CPC Class Codes: H04L 67/52 specially adapted for the location of the user terminal
• CPC Class Codes: H04W 4/02 Services making use of location information
• CPC Class Codes: H04W 4/38 for collecting sensor information

Abstract

An artificial intelligence (AI) system leverages collaboration of multiple audio sensors for geo-location of an event. Location information is stored for each of the plurality of geographically-dispersed AI clients. Each of the AI clients can include an audio sensor, a network controller, and a feedback mechanism (e.g., a speaker or an LED) within a common enclosure. Responsive to detection of an event (e.g., involving a human, machine, or ambient condition) at two or more of the plurality of geographically-dispersed AI clients, audio event data is centrally received from audio sensors of the two or more of the plurality of geographically-dispersed AI sensors. A location is estimated based on the location information and the audio event data. Responsive to classification of the event in view of the estimated location, a command is received for a response to the event at the AI clients that detected the event, or others.

Claims

1. A computer-implemented method in a controller coupled to a plurality of audio sensors collaborating for geo-location of events in an artificial intelligence (AI) environment, the method comprising the steps of:

storing location information for each of a plurality of geographically-dispersed AI clients, each of the AI clients comprising an audio sensor, a network controller, and a feedback mechanism within a common enclosure;

responsive to detection of an event at two or more of the plurality of geographically-dispersed AI clients, centrally receiving audio event data from audio sensors of the two or more of the plurality of geographically-dispersed AI sensors;

estimating a location based on the location information and the audio event data;

responsive to classification of the audio event data in view of the estimated location, receiving a command for a response to the event; and

executing the command for the response to the event in order to produce an event response involving either;

at least one of the plurality of geographically-dispersed AI clients or an application.

2. The method of claim 1, wherein the audio event data comprises a sound file capturing audio from the event.

3. The method of claim 2, wherein the sound file capturing the audio event is formed by audio sensors of the two or more of the plurality of geographically-dispersed audio sensors, wherein the audio sensors continuously store ambient audio in a temporary memory and, upon detecting the event, define a beginning point and an end point of the event corresponding to the sound file.

4. The method of claim 1, further comprising:

classifying the audio event data by comparing the audio event data against a library of known audio data and corresponding identification of a human or machine event; and

mapping the event classification to a command response that causes a change in an electronic device connected to the one of the plurality of geographically-dispersed AI clients.

5. The method of claim 1, wherein executing the response command comprises outputting a sound file corresponding to the event through a speaker at the at least one of the plurality of geographically-dispersed AI appliances.

6. The method of claim 1 wherein the executing the response command adjusts a setting on an electronic device connected to the at least one of the plurality of geographically-dispersed AI appliances, comprising at least one of:

powering on, powering off, increasing an intensity, decreasing an intensity, initiating a script of commands, and ending a script of commands.

7. The method of claim 1, further comprising:

responsive to classification of the audio event data, sending a request to an Internet resource and receiving a response from the Internet resource that affects the command response.

8. The method of claim 1, further comprising:

determining a direction of movement for a subject of the event,

wherein the audio event data comprises audio event data at a first time corresponding to a first location of the event and event data at a second time corresponding to a second location, and

wherein the response command is based on the direction of movement of the subject.

9. The method of claim 8, further comprising:

predicting a future location for response to the event subject based on a difference between the audio event at the first time and the audio event at the second time,

wherein the response command is based on the predicted location.

10. The method of claim 9, wherein the executing the response comprises executing the response command to the event in order to produce the event response wherein the speaker of at least one of the plurality of geographically-dispersed AI appliances is proximate to the predicted future location.

11. The method of claim 1, wherein the event originates from natural human speech.

12. The method of claim 1, wherein the event originates from a human activity, an electronic device, an environmental activity, or an ambient condition.

13. The method of claim 1, wherein executing the response command comprises:

executing the response command to the event in order to produce an event response involving an application that is an application that controls at least one of the plurality of geographically-dispersed AI clients.

14. The method of claim 1, wherein executing the response command comprises:

executing the response command to the event in order to produce an event response involving an application that is a remote application.

15. The method of claim 1, wherein the AI environment comprises one or more of a home, business, an outdoor area, and an automobile.

16. A non-transitory computer-readable medium storing source code that, when executed by a processor, performs a method in a controller coupled to a plurality of audio sensors collaborating for geo-location of events in an artificial intelligence (AI) environment, the method comprising the steps of:

storing location information for each of a plurality of geographically-dispersed AI clients, each of the AI clients comprising an audio sensor, a network controller, and a feedback mechanism within a common enclosure;

responsive to detection of an event at two or more of the plurality of geographically-dispersed AI clients, centrally receiving audio event data from audio sensors of the two or more of the plurality of geographically-dispersed AI sensors;

estimating a location based on the location information and the audio event data;

responsive to classification of the audio event data in view of the estimated location, receiving a command for a response to the event; and

executing the command for the response to the event in order to produce an event response involving either at least one of the plurality of geographically-dispersed AI clients or an application.

17. A controller coupled to a plurality of audio sensors collaborating for geo-location of events in an artificial intelligence (AI) environment, the controller comprising:

a processor;

a memory, comprising;

a first module to store location information for each of a plurality of geographically-dispersed AI clients, each of the AI clients comprising an audio sensor, a network controller, and a feedback mechanism within a common enclosure;

a second module to, responsive to detection of an event at two or more of the plurality of geographically-dispersed AI clients, centrally receive audio event data from audio sensors of the two or more of the plurality of geographically-dispersed AI sensors;

a third module to estimate a location based on the location information and the audio event data;

a fourth module to, responsive to classification of the audio event data in view of the estimated location, receive a command for a response to the event; and

a fifth module to execute the command for the response to the event in order to produce an event response involving either at least one of the plurality of geographically-dispersed AI clients or an application.