Initiating device speech activity monitoring for communication sessions

• US 10,192,553 B1
• Filed: 12/20/2016
• Issued: 01/29/2019
• Est. Priority Date: 12/20/2016
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Chenier, Mario, Hardie, Tony Roy, Uppal, Nawdesh, Oliver, Brian, Mokady, Ran
• Primary Examiner(s): Godbold, Douglas
• Application Number:
  US15/385,410
• Time in Patent Office: 770 Days
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 17/00 Speaker identification or verification techniques
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: G10L 25/21 the extracted parameters being power information
• CPC Class Codes: G10L 25/84 for discriminating voice from noise
• CPC Class Codes: H04L 65/1069 Session establishment or de-establishment
• CPC Class Codes: H04L 65/4015 where at least one of the additional parallel sessions is real time or time sensitive, e.g. white board sharing, collaboration or spawning of a subconference
• CPC Class Codes: H04L 65/80 Responding to QoS
• CPC Class Codes: H04L 69/28 Timers or timing mechanisms used in protocols
• CPC Class Codes: H04N 21/00 Selective content distribution, e.g. interactive television or video on demand [VOD] real-time bi-directional transmission of motion video data H04N7/14 ; broadcast or conference over packet switching networks H04L12/18
• CPC Class Codes: H04N 7/147 Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals selecting H04Q

Abstract

Methods and devices for causing a communications session between a first device and a second device to end based on lack of speech activity are described herein. In some embodiments, a communications between a first device and a second device may be initiated by the first device, where a first user account associated with the first device is authorized to initiate communications session with the second device by a second user account. After the communications session is started, audio data is received by a speech activity detection system, which determines whether the audio data represents speech or non-speech. If, after the communications session begins, non-speech is detected by the first device for more than a predefined amount of time, then the communications session is caused to end so that the first device is not capable of receiving video and/or audio associated with the second device.

Claims

1. A method, comprising:

receiving, from an initiating device, a request to initiate a communications session between the initiating device and a recipient device;

establishing the communications session;

receiving first data indicating that first sounds were received by at least one microphone of the initiating device during a first amount of time after the communications session is initiated;

determining that the first sounds correspond to non-speech;

receiving second data indicating that second sounds were received by the at least one microphone during a second amount of time after an end of the first amount of time;

determining that the second sounds correspond to non-speech;

determining a third amount of time between the first amount of time and the second amount of time;

determining that the third amount of time is greater than a threshold amount of time; and

causing the communications session to end based, at least in part, on the first sounds and the second sounds corresponding to non-speech and the third amount of time being greater than the threshold amount of time.

2. The method of claim 1, further comprising:

receiving third data indicating that third sounds were received by the at least one microphone during a fourth amount of time, the fourth amount of time beginning after initiation of an additional communications session between the initiating device and the recipient device;

determining that the third sounds correspond to speech; and

causing the additional communications session to continue.

3. The method of claim 1, further comprising:

receiving, prior to causing the communications session to end, presence information from the initiating device, the presence information indicating that an individual is located proximate to the initiating device; and

causing the communications session to continue based at least in part on the presence information.

4. The method of claim 1, further comprising;

determining, prior to causing the communications session to end, that a mute function of the initiating device has been activated; and

causing the communications session to end further based at least in part on the mute function being activated.

5. A method, comprising:

receiving, from a first device, a request to establish a communications session with at least a second device;

causing the communications session between the first device and at least the second device to be established;

determining, during a first period of time of the communications session, a first absence of speech from the first device;

determining, during a second period of time of the communications session, a second absence of speech from the first device;

determining an amount of time between the first period of time and the second period of time; and

causing, based at least in part on the amount of time, the communications session to end.

6. The method of claim 5, further comprising:

receiving first data indicating that first sounds were received by the first device; and

determining that the first sounds correspond to non-speech.

7. The method of claim 5, further comprising:

determining, prior to causing the communications session to end, a user account associated with the first device;

determining voice biometric data associated with the user account;

receiving audio data representing speech; and

determining, based at least in part on the voice biometric data, that the speech is unassociated with the user account.

8. The method of claim 5, further comprising:

receiving a second request to establish a second communications session with at least the second device;

causing the second communications session between the first device and at least the second device to be established;

determining that, during the second communications session, audio data representing speech was received from the first device; and

causing the second communications session to continue.

9. The method of claim 5, further comprising:

receiving, prior to causing the communications session to end, image data representing at least a portion of a human body part; and

determining a threshold amount of time based at least in part on the image data.

10. The method of claim 5, further comprising:

receiving audio data;

determining an energy level of the audio data; and

determining that the energy level is less than or equal to a predefined energy threshold associated with speech.

11. The method of claim 5, further comprising:

receiving audio data;

generating first text data representing the audio data;

determining second text data representing a message to be uttered in response to the communications session being established; and

determining that the first text data and the second text data differ.

12. The method of claim 5, further comprising:

determining the first device has been authorized to establish communications sessions with the second device without the second device receiving indicators of acceptance of the communications sessions; and

after determining the first device has been authorized, causing the communications session between the first device and at least the second device to be established.

13. A system, comprising:

a first device;

a second device;

at least one processor; and

at least one memory including instructions that, when executed by the at least one processor, cause the system to;

receive, from the first device, a request to establish a communications session with at least the second device;

cause the communications session between the first device and at least the second device to be established;

determine, during a first period of time of the communications session, a first absence of speech associated with the first device;

determine, during a second period of time of the communications session, a second absence of speech from the first device;

determining an amount of time between the first period of time and the second period of time; and

cause, based at least in part on the amount of time, the communications session to end.

14. The system of claim 13, wherein the at least one memory further includes instructions that, when executed by the at least one processor, further cause the system to:

receive first data indicating that first sounds were received by the first device; and

determine that the first sounds correspond to non-speech.

15. The system of claim 13, wherein the at least one memory further includes instructions that, when executed by the at least one processor, further cause the system to:

determine, prior to causing the communications session to end, a user account associated with the first device;

determine voice biometric data associated with the user account;

receive audio data representing speech; and

determine, based at least in part on the voice biometric data, that the speech is unassociated with the user account.

16. The system of claim 13, wherein the at least one memory further includes instructions that, when executed by the at least one processor, further cause the system to:

receive a second request to establish a second communications session with at least the second device;

cause the second communications session between the first device and at least the second device to be established;

determine that, during the second communications session, audio data representing speech was received from the first device; and

cause the second communications session to continue.

17. The system of claim 13, wherein the at least one memory further includes instructions that, when executed by the at least one processor, further cause the system to:

receive, prior to causing the communications session to end, first image representing at least a portion of a human body part; and

determine a threshold amount of time based at least in part on the image data.

18. The system of claim 13, wherein the at least one memory further includes instructions that, when executed by the at least one processor, further cause the system to:

receive audio data;

determine an energy level of the audio data; and

determine that the energy level is less than or equal to predefined energy threshold associated with speech.

19. The system of claim 13, wherein the at least one memory further includes instructions that, when executed by the at least one processor, further cause the system to:

receive audio data;

generate first text data representing the audio data;

determine second text data representing a message to be uttered in response to the communications session being established; and

determine that the first text data and the second text data differ.

20. The system of claim 13, wherein the at least one memory further includes instructions that, when executed by the at least one processor, further cause the system to:

determine the first device has been authorized to establish communications sessions with the second device without the second device receiving indicators of acceptance of the communications sessions; and

after determining the first device has been authorized, cause the communications session between the first device and at least the second device to be established.