Collaboration of audio sensors for geo-location and continuous tracking of multiple users in a device-independent artificial intelligence (AI) environment

• US 9,818,061 B1
• Filed: 09/15/2016
• Issued: 11/14/2017
• Est. Priority Date: 10/22/2013
• Status: Active Grant

• Current Assignee: Lumin, LLC
• Original Assignee: Lumin, LLC
• Inventors: Shams, Nima Lahijani, Maheshaiah, Suhas
• Primary Examiner(s): Hill, Stanley K
• Assistant Examiner(s): Lee, Tsu-Chang
• Application Number:
  US15/266,497
• Time in Patent Office: 425 Days
• Field of Search: 706 46
• CPC Class Codes: G06F 3/165 Management of the audio stream, e.g. setting of volume, audio stream path
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 5/022 Knowledge engineering; Knowledge acquisition
• CPC Class Codes: G06N 5/04 Inference or reasoning models
• CPC Class Codes: G10L 17/04 Training, enrolment or model building
• CPC Class Codes: G10L 17/06 Decision making techniques; Pattern matching strategies
• CPC Class Codes: H04L 67/01 Protocols
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]
• CPC Class Codes: H04L 67/04 specially adapted for terminals or networks with limited capabilities; specially adapted for terminal portability
• CPC Class Codes: H04L 67/12 specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
• CPC Class Codes: H04L 67/52 specially adapted for the location of the user terminal
• CPC Class Codes: H04L 67/535 Tracking the activity of the user network monitoring arrangements H04L43/00; recording of computer activity G06F11/34
• CPC Class Codes: H04L 67/56 Provisioning of proxy services store-and-forward switching systems in data switching networks H04L12/54

Abstract

Audio sensors collaborate for geo-location and tracking of multiple users. Different users can be independently geo-located and tracked within the AI environment. Location is determined from two or more AI clients of known locations that detect an event such as a human voice command to connect a call with a specific user. Responsive to classification of the event in view of the estimated location, a command for an AI action, such as connecting a call between users, is received for a response to the event at the AI clients that detected the event, or others.

Claims

1. A computer-implemented method in an AI server of an artificial intelligence (AI) system for continuously tracking multiple users that are identified by the AI server, the method comprising the steps of:

storing location information for each of a plurality of AI clients, each of the AI clients comprising an audio sensor, a network controller, and a feedback mechanism within a common enclosure, and the plurality of AI clients being geographically-dispersed;

responsive to an explicit identification of at least a first user and a second user of a plurality of users to any of the AI clients, collecting identity information associated with the individual user for an identification profile, the identity information comprising biometric data and non-biometric data;

detecting a first audio event and identity data at a first location that is received by at least a first AI client of the plurality of AI clients;

identifying the first user out of the plurality of users as being associated with the audio event based on probability calculations from comparing collected identity information against detected identity information;

associating the first location with the first user including estimating coordinates of the first location from at least an intensity level of the first audio event as detected by a first AI client and coordinates of the first AI client, and an intensity level of the first audio event as detected by a second AI client and coordinates of the second AI client, the first and second AI clients being geographically distinct;

confirming or changing the first user identity by continuously tracking events of the first user as locations dynamically change and updating probability calculations;

continuously tracking the second user by identifying the second user out of the plurality of users after detecting at least a second event; and

providing a first AI action specific to the first user based on the identification and the first location; and

providing a second AI action specific to the second user based on the identification and the second location.

2. The method of claim 1, wherein the first AI action and the second AI action involve a real-time interaction between the first and second users, wherein the first location is distinct from the second location.

3. The method of claim 1, wherein the estimated locations of the first user and the second user are determined without attached location devices.

4. The method of claim 1:

wherein collecting identification information comprises collecting biometric identification information associated with the individual user, and

wherein determining that that the audio event data originates from the first user comprises classifying the audio event data by comparing the audio event data against at least the biometric identification information.

5. The method of claim 1,

wherein collecting identification information comprises collecting non-biometric information originating from the individual user interacting with one or more physical objects, and

wherein determining that that the audio event data originates from the first user comprises classifying the audio event data by comparing the audio event data against at least the non-biometric information using a probabilistic model.

6. The method of claim 5, wherein collecting non-biometric audio originating from the individual user interacting with one or more physical objects comprises one or more of:

audio of the first user walking on a floor, audio of the first user adjusting a device, a time of arrival for the first user, and audio of a television program frequently watched by the first user.