Systems and methods for recognizing and performing voice commands during advertisement

• US 10,522,146 B1
• Filed: 07/09/2019
• Issued: 12/31/2019
• Est. Priority Date: 07/09/2019
• Status: Active Grant

• Current Assignee: Instreamatic, Inc.
• Original Assignee: Instreamatic, Inc.
• Inventors: Tushinskiy, Stanislav
• Primary Examiner(s): Colucci, Michael C
• Application Number:
  US16/506,802
• Time in Patent Office: 175 Days
• Field of Search: 704251, 7042701, 705 1466, 705 1449, 715728, 715716, 4554563, 455567
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06Q 30/0271 Personalized advertisement
• CPC Class Codes: G10L 15/08 Speech classification or search
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/222 Barge in, i.e. overridable guidance for interrupting prompts
• CPC Class Codes: G10L 2015/088 Word spotting
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: G10L 2015/228 of application context
• CPC Class Codes: G10L 25/78 Detection of presence or absence of voice signals switching of direction of transmission by voice frequency in two-way loud-speaking telephone systems H04M9/10

Abstract

Methods and systems for recognizing and performing voice commands during advertisements are provided. An example method may include playing, by an audio device, a media stream to a user, the media stream including at least one advertisement; sensing, by an acoustic sensor, an ambient acoustic signal; determining, by processors communicatively coupled to the audio device and the acoustic sensor, that the audio device has started playing the advertisement; in response to the determination, monitoring, by the processors, the ambient acoustic signal to detect a presence of at least one command spoken by the user; and in response to the detection of the presence of the at least one command, determining data associated with the at least one advertisement; and causing, by the processors, the audio device to perform one or more actions associated with the command and the data associated with the advertisement.

Claims

1. A system for recognizing and performing voice commands during advertisements, the system comprising:

an audio device configured to play a media stream to a user, the media stream including at least one advertisement;

at least one acoustic sensor configured to sense an ambient acoustic signal; and

one or more processors communicatively coupled to the audio device and the at least one acoustic sensor, the one or more processors being configured to;

determine that the audio device has started playing the at least one advertisement;

in response to the determination, monitor the ambient acoustic signal to detect, based on the ambient acoustic signal, a presence of at least one command spoken by the user; and

in response to the detection of the presence of the at least one command, cause the audio device to perform one or more actions associated with the at least one command.

2. The system of claim 1, wherein:

the media stream includes metadata indicative of starting the at least one advertisement; and

the determination that the audio device has started playing the at least one advertisement is based on the metadata.

3. The system of claim 1, wherein the one or more processors are configured to analyze audio data of the media stream;

and

wherein the determination that the audio device has started playing the at least one advertisement is based on the analysis of the audio data.

4. The system of claim 1, wherein the detection of the presence of the at least one command includes:

analyzing the ambient acoustic signal to detect a presence of a pre-determined keyword in the ambient acoustic signal; and

upon detection of the presence of the at least one keyword, analyzing a portion of the ambient acoustic signal after the pre-determined keyword to recognize the at least one command.

5. The system of claim 4, wherein upon detection of the presence of the pre-determined keyword, the one or more processors are configured to cause the audio device to lower a level of sound of playing the media stream.

6. The system of claim 1, wherein the detection of the presence of the at least one command includes comparing a portion of the ambient acoustic signal to one or more phrases stored in a local vocabulary, the local vocabulary being stored in a memory of the audio device.

7. The system of claim 6, wherein the one or more processors are communicatively connected to at least one remote computing source, the at least one remote computing source being configured to store a remote vocabulary, wherein the remote vocabulary is larger than the local vocabulary;

and

wherein the one or more processors are configured to;

determine that the portion of the ambient acoustic signal does not match the one or more phrases stored in the local vocabulary; and

in response to the determination, comparing the portion of the ambient acoustic signal to one or more phrases stored in the remote vocabulary.

8. The system of claim 1, wherein the one or more processors are configured to:

determine data associated with the at least one advertisement; and

prior to causing the audio device to perform the actions, modify the at least one command based on the data associated with the at least one advertisement.

9. The system of claim 8, wherein the one or more processors are configured to:

perform speech recognition based on an audio data of the media stream, the audio data corresponding to the at least one advertisement; and

determine, based on results of the speech recognition, the data associated with the at least one advertisement.

10. The system of claim 8, wherein:

the media stream includes metadata including the data associated with the at least one advertisement; and

the one or more processors are configured to determine the data associated with the at least one advertisement based on the metadata.

11. The system of claim 8, wherein the data associated with the at least one advertisement include one or more of the following:

a phone number, an Internet address of a website, a name of product or a service, a type of the product or the service, a name of company, and a street address of the company.

12. The system of claim 11, wherein the performing one or more actions corresponding to the at least one command includes one or more of:

making a call to the phone number, opening a browser at the Internet address of a website, displaying a map showing the address of the company, displaying a driving direction to the street address of the company, and displaying an information concerning the product or the service in the browser.

13. The system of claim 1, wherein the at least one acoustic sensor and the one or more processors are integrated in the audio device.

14. The system of claim 1, wherein the audio device includes one of:

a smart phone, a smart television set, a tablet computer, a desktop computer, in-vehicle infotainment, and a smart home device.

15. The system of claim 1, wherein the media stream includes an Internet radio stream, an Internet television stream broadcast live online, or a podcast.

16. The system of claim 1, wherein after determining that the audio device has started playing the at least one advertisement, the one or more processors are configured to:

detect a presence of at least one clue in the at least one advertisement;

in response to the detection of the at least one clue, monitor the ambient acoustic signal to detect, based on the ambient acoustic signal, a presence of at least one keyword spoken by the user, the at least one keyword being associated with the at least one clue; and

in response to the detection of the presence of the at least one keyword, cause the audio device to perform one or more actions associated with the at least one clue.

17. A method for recognizing and performing voice commands during advertisement, the method comprising:

playing, by an audio device, a media stream to a user, the media stream including at least one advertisement;

sensing, by at least one acoustic sensor, an ambient acoustic signal;

determining, by one or more processors communicatively coupled to the audio device and the at least one acoustic sensor, that the audio device has started playing the at least one advertisement;

in response to the determination, monitoring, by the one or more processors, the ambient acoustic signal to detect, based on the ambient acoustic signal, a presence of at least one command spoken by the user; and

in response to the detection of the presence of the at least one command, causing, by the one or more processors, the audio device to perform one or more actions associated with the at least one command.

18. The method of claim 17, wherein the detection of the presence of the at least one command includes:

analyzing the ambient acoustic signal to detect a presence of a pre-determined keyword in the ambient acoustic signal; and

upon detection of the presence of the at least one keyword, analyzing a portion of the ambient acoustic signal after the pre-determined keyword to recognize the at least one command.

19. The method of claim 17, further comprising:

determining data associated with the at least one advertisement, wherein the data associated with the at least one advertisement are determined by at least one of;

performing speech recognition of audio data of the media stream, the audio data corresponding to the at least one advertisement; and

reading metadata of the media stream; and

prior to causing the audio device to perform the actions, modifying the at least one command based on the data associated with the at least one advertisement.

20. An audio device for recognizing and performing voice commands during advertisements, the audio device comprising:

an output device configured to play a media stream to a user, the media stream including at least one advertisement;

at least one acoustic sensor configured to sense an ambient acoustic signal; and

one or more processors communicatively coupled to the output device and the at least one acoustic sensor and configured to;

determine that the output device has started playing the at least one advertisement;

in response to the determination;

monitor the ambient acoustic signal to detect a presence of a predetermined keyword spoken by the user in the ambient acoustic signal;

in response to the detection of the presence of the pre-determined keyword;

cause the output device to lower a level of a sound of playing the media stream; and

analyze a portion of the ambient acoustic signal after the pre-determined keyword to recognize at least one command; and

upon recognition of the at least one command;

determine data associated with the at least one advertisement, wherein the data associated with the at least one advertisement are determined by at least one of;

performing speech recognition of audio data of the media stream, the audio data corresponding to the at least one advertisement; and

reading metadata of the media stream; and

perform one or more actions associated with the one or more command and the data associated with the at least one advertisement.