Location based voice association system

• US 10,026,403 B2
• Filed: 08/12/2016
• Issued: 07/17/2018
• Est. Priority Date: 08/12/2016
• Status: Active Grant

• Current Assignee: PayPal, Inc. (PayPal Holdings, Inc.)
• Original Assignee: PayPal, Inc. (PayPal Holdings, Inc.)
• Inventors: Tian, Cheng, Narasimhan, Srivathsan
• Primary Examiner(s): Park, Chan S
• Assistant Examiner(s): Brinich, Stephen M
• Application Number:
  US15/236,120
• Publication Number:
  US 20180047394A1
• Time in Patent Office: 704 Days
• Field of Search: 704238-239, 704246, 704270, 704273-275, 345633
• CPC Class Codes: G10L 15/24 Speech recognition using non-acoustical features
• CPC Class Codes: G10L 17/06 Decision making techniques; Pattern matching strategies
• CPC Class Codes: G10L 17/22 Interactive procedures; Man-machine interfaces
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: G10L 25/51 for comparison or discrimination

Abstract

Systems and methods for associating audio signals in an environment surrounding a voice-controlled system include receiving by a voice-controlled system through a microphone, an audio signal from a user of a plurality of users within an environment surrounding the microphone. The voice-controlled system determines a source location of the audio signal. The voice-controlled system determines a first user location of a first user and a second user location of a second user. The voice-controlled system then determines that the first user location correlates with the source location such that the source location and the first user location are within a predetermined distance of each other. In response, the voice-controlled system performs at least one security action associated with the first user providing the audio signal.

Claims

1. A voice association system, comprising:

a microphone configured to capture an audio signal from an environment surrounding the microphone and convert the audio signal into an electrical signal;

a non-transitory memory; and

one or more hardware processors coupled to the non-transitory memory, and the microphone, wherein the one or more hardware processors are configured to read instructions from the non-transitory memory to cause the system to perform operations comprising;

receiving, through the microphone, a first audio signal from a first user of a plurality of users within the environment;

determining a first source location of the first audio signal;

determining a first user location of the first user and a second user location of a second user;

determining that the first user location correlates with the first source location such that the first source location and the first user location are within a predetermined distance of each other;

performing voice recognition of the first audio signal to determine an identity of the first user providing the first audio signal;

determining an identity of the first user at the first user location; and

performing, in response to the determining that the first user location correlates with the first source location and determining the identity of the first user providing the first audio signal correlates with the identity of the first user at the first user location, at least one security action associated with the first user providing the first audio signal.

2. The system of claim 1, wherein the operations further comprise:

receiving, by the microphone, a second audio signal from the second user of the plurality of users within the environment;

determining a second source location of the second audio signal;

determining that the second user location correlates with the second source location; and

performing, in response to determining that the second user location correlates with the second source location, at least one security action associated with the second user providing the second audio signal.

3. The system of claim 1, wherein the operations further comprise:

receiving, by the microphone, a second audio signal;

determining a second source location of the second audio signal;

determining the first user is located at the second source location; and

performing, in response to determining the first user is located at the second source location, at least one security action associated with the first user providing the second audio signal.

4. The system of claim 3, wherein the determining the first user is located at the second source location includes:

determining a third user location of the first user and a fourth location of the second user; and

determining the third user location correlates with the second source location and, in response, associating the second audio signal with the first user.

5. The system of claim 3, wherein the determining the first user is located at the second source location includes comparing a first voice print of the first audio signal associated with the first user to a second voice print of the second audio signal and, in response to the second voice print of the second audio signal satisfying a threshold similarity to the first voice print of the first audio signal, associating the second audio signal with the first user.

6. The system of claim 1, wherein the operations further comprise:

receiving, by the microphone, a second audio signal such that at least a portion of the second audio signal is received at the same time as the first audio signal such that a composite audio signal is received that includes the first audio signal and second audio signal;

determining a second source location of the second audio signal; and

separating the composite audio signal into the first audio signal and the second audio signal based on the first source location and the second source location.

7. The system of claim 1, wherein the determining the first source location of the first audio signal is based on time of arrival of arrival the first audio signal at the microphone, and wherein the determining the first user location of the first user and the second user location of the second user is independent from the first audio signal.

8. A method for voice association, comprising:

receiving, by a voice-controlled system through a microphone, a first audio signal from a first user of a plurality of users within an environment surrounding the microphone;

determining, by the voice-controlled system, a first source location of the first audio signal;

determining, by the voice-controlled system, a first user location of the first user and a second user location of a second user;

determining, by the voice-controlled system, that the first user location correlates with the first source location such that the first source location and the first user location are within a predetermined distance of each other;

performing, by the voice-controlled system, voice recognition of the first audio signal to determine an identity of the first user providing the first audio signal;

determining, by the voice-controlled system, an identity of the first user at the first user location; and

performing, by the voice-controlled system in response to the determining that the first user location correlates with the first source location and determining the identity of the first user providing the first audio signal correlates with the identity of the first user at the first user location, at least one security action associated with the first user providing the first audio signal.

9. The method of claim 8, further comprising:

receiving, by the voice-controlled system through the microphone, a second audio signal from the second user of the plurality of users within the environment;

determining, by the voice-controlled system, a second source location of the second audio signal;

determining, by the voice-controlled system, that the second user location correlates with the second source location; and

performing, by the voice-controlled system in response to determining that the second user location correlates with the second source location, at least one security action associated with the second user providing the second audio signal.

10. The method of claim 8, further comprising:

receiving, by the voice-controlled system through the microphone, a second audio signal;

determining, by the voice-controlled system, a second source location of the second audio signal;

determining, by the voice-controlled system, the first user is located at the second source location; and

performing, by the voice-controlled system in response to determining the first user is located at the second source location, at least one security action associated with the first user providing the second audio signal.

11. The method of claim 10, wherein the determining the first user is located at the second source location includes:

determining, by the voice-controlled system, a third user location of the first user and a fourth location of the second user; and

determining, by the voice-controlled system, the third user location correlates with the second source location and, in response, associating the second audio signal with the first user.

12. The method of claim 10, wherein the determining the first user is located at the second source location includes comparing, by the voice-controlled system, a first voice print of the first audio signal associated with the first user to a second voice print of the second audio signal and, in response to the second voice print of the second audio signal satisfying a threshold similarity to the first voice print of the first audio signal, associating the second audio signal with the first user.

13. The method of claim 8 further comprising:

receiving, by the voice-controlled system, a second audio signal such that at least a portion of the second audio signal is received at the same time as the first audio signal such that a composite audio signal is received that includes the first audio signal and second audio signal;

determining, by the voice-controlled system, a second source location of the second audio signal; and

separating, by the voice-controlled system, the composite audio signal into the first audio signal and the second audio signal based on the first source location and the second source location.

14. The method of claim 8, wherein the determining the first source location of the first audio signal is based on time of arrival of the first audio signal at the microphone, and wherein the determining the first user location of the first user and the second user location of the second user is independent from the first audio signal.

15. A non-transitory machine-readable medium having stored thereon machine-readable instructions executable to cause a machine to perform operations comprising:

receiving, through a microphone, a first audio signal from a first user of a plurality of users within an environment surrounding the microphone;

determining a first source location of the first audio signal;

determining a first user location of the first user and a second user location of a second user;

determining that the first user location correlates with the first source location such that the first source location and the first user location are within a predetermined distance of each other;

performing voice recognition of the first audio signal to determine an identity of the first user providing the first audio signal;

determining an identity of the first user at the first user location; and

performing, in response to determining that the first user location correlates with the first source location and the determining the identity of the first user providing the first audio signal correlates with the identity of the first user at the first user location, at least one security action associated with the first user providing the first audio signal.

16. The non-transitory machine-readable medium of claim 15, wherein the operations further comprise:

receiving, through the microphone, a second audio signal from the second user of the plurality of users within the environment surrounding the microphone;

determining a second source location of the second audio signal;

determining that the second user location correlates with the second source location; and

performing, in response to determining that the second user location correlates with the second source location, at least one security action associated with the second user providing the second audio signal.

17. The non-transitory machine-readable medium of claim 15, wherein the operations further comprise:

receiving, through the microphone, a second audio signal;

determining a second source location of the second audio signal;

determining the first user is located at the second source location; and

performing, in response to determining the first user is located at the second source location, at least one security action associated with the first user providing the second audio signal.

18. The non-transitory machine-readable medium of claim 17, wherein the determining the first user is located at the second source location includes:

determining a third user location of the first user and a fourth location of the second user; and

determining the third user location correlates with the second source location and, in response, associating the second audio signal with the first user.

19. The non-transitory machine-readable medium of claim 17, wherein the determining the first user is located at the second source location includes comparing a first voice print of the first audio signal associated with the first user to a second voice print of the second audio signal and, in response to the second voice print of the second audio signal satisfying a threshold similarity to the first voice print of the first audio signal, associating the second audio signal with the first user.

20. The non-transitory machine-readable medium of claim 15, wherein the determining the first source location of the first audio signal is based on time of arrival of the first audio signal at the microphone, and wherein the determining the first user location of the first user and the second user location of the second user is independent from the first audio signal.