EVENT-BASED SPEECH INTERACTIVE MEDIA PLAYER

• US 20170103754A1
• Filed: 10/09/2015
• Published: 04/13/2017
• Est. Priority Date: 10/09/2015
• Status: Active Grant

• Current Assignee: XappMedia, Inc.
• Original Assignee: XappMedia, Inc.
• Inventors: HIGBIE, Patrick B., KELVIE, John P., MYERS, Michael M., RAINES, Franklin D.
• Granted Patent: US 9,978,366 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 3/165 Management of the audio stream, e.g. setting of volume, audio stream path
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 2015/088 Word spotting
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: G10L 2015/228 of application context
• CPC Class Codes: G10L 25/78 Detection of presence or absence of voice signals switching of direction of transmission by voice frequency in two-way loud-speaking telephone systems H04M9/10
• CPC Class Codes: H04N 21/23424 involving splicing one content stream with another content stream, e.g. for inserting or substituting an advertisement
• CPC Class Codes: H04N 21/42203 sound input device, e.g. microphone
• CPC Class Codes: H04N 21/44016 involving splicing one content stream with another content stream, e.g. for substituting a video clip
• CPC Class Codes: H04N 21/812 involving advertisement data advertising per se G06Q30/02

Abstract

Interactive content containing audio or video may be provided in conjunction with non-interactive content containing audio or video to enhance user engagement and interest with the contents and to increase the effectiveness of the distributed information. Interactive content may be directly inserted into the existing, non-interactive content. Additionally or alternatively, interactive content may be streamed in parallel to the existing content, with only minimal modification to the existing content. For example, the server may monitor content from a content provider; detect an event (e.g., detecting a marker embedded in the content stream, or in a data source external to the content stream); upon detection of the event, play interactive content at one or more designated times while silencing the content stream of the content provider (e.g., by muting, pausing, playing silence.) The marker may be a sub-audible tone or metadata associated with the content stream. The user may respond to the interactive content by voice, touch or gesture.

Claims

1. A device comprising:

one or more processors;

a memory storing one or more computer programs that include computer instructions, which when executed by the one or more processors, cause the device to;

monitor media content provided by a source, the media content comprising both non-interactive content and interactive content in at least an audio or video format;

detect a first event in the monitored media content;

in response to detecting the first event in the media content;

begin using a speech recognizer associated with the media content;

in accordance with a determination that a user has spoken an action phrase, perform an action according to the user'"'"'s spoken action phrase; and

in accordance with a determination that the user has not spoken the action phrase, cause the media content to continue to be played.

2. The device of claim 1, wherein the first event in the media content is in the form of metadata.

3. The device of claim 2, wherein the first event in the media content is in the form of at least ID3 metadata or HLS metadata.

4. The device of claim 1, wherein the first event in the media content is a sub-audible tone embedded in the media content, wherein the sub-audible tone is not detectable by an unaided human ear.

5. The device of claim 4, wherein the sub-audible tone has a frequency of 20 Hertz or less.

6. The device of claim 1, wherein the media content is associated with metadata comprising instructions for performing the action according to the action phrase, and the metadata is either supplied from the source along with the media content or supplied separately from an external source.

7. The device of claim 1, wherein the media content is silenced during a period for which the speech recognizer is used, and the media content is silenced by at least one of:

muting a playback mechanism used to play the media content,

pausing the playback mechanism used to play the media content, or

playing silence over the playback mechanism used to play the media content.

8. The device of claim 1, wherein the source is at least one of a remote media server, a local file system, and a local memory.

9. The device of claim 1, wherein the computer instructions, which when executed by the one or more processors, further cause the device to:

prior to activating the speech recognizer, play the interactive content followed by a prompt, the prompt comprising a message instructing the user to say the action phrase, wherein the first event is associated with an end of the prompt such that the speech recognizer is used after playing the prompt.

10. A method for providing speech interactive content, the method comprising:

monitoring media content provided by a source, the media content comprising both non-interactive content and interactive content in at least an audio or video format;

detecting a first event in the monitored media content;

in response to detecting the first event in the media content;

begin using a speech recognizer associated with the media content;

in accordance with a determination that a user has spoken an action phrase, performing an action according to the user'"'"'s spoken action phrase; and

in accordance with a determination that the user has not spoken the action phrase, causing the media content to continue to be played.

11. The method of claim 10, wherein the first event in the media content is in the form of metadata.

12. The method of claim 11, wherein the first event in the media content is in the form of at least ID3 metadata or HLS metadata.

13. The method of claim 10, wherein the first event in the media content is a sub-audible tone embedded in the media content, wherein the sub-audible tone is not detectable by an unaided human ear.

14. The method of claim 13, wherein the sub-audible tone has a frequency of 20 Hertz or less.

15. The method of claim 10, wherein the media content is associated with metadata comprising instructions for performing the action according to the action phrase, and the metadata is either supplied from the source along with the media content or supplied separately from an external source.

16. The method of claim 10, wherein the media content is silenced during a period for which the speech recognizer is used, and the media content is silenced by at least one of:

muting a playback mechanism used to play the media content,

pausing the playback mechanism used to play the media content, or

playing silence over the playback mechanism used to play the media content.

17. The method of claim 10, wherein the source is at least one of a remote media server, a local file system, and a local memory.

18. The method of claim 10, further comprising:

prior to activating the speech recognizer, play the interactive content followed by a prompt, the prompt comprising a message instructing the user to say the action phrase, wherein the first event is associated with an end of the prompt such that the speech recognizer is activated after playing the prompt.

19. A non-transitory computer readable medium comprising one or more computer programs, which when executed by a device with one or more processors, cause the device to:

monitor media content provided by a source, the media content comprising both non-interactive content and interactive content in at least an audio or video format;

detect a first event in the monitored media content;

in response to detecting the first event in the media content;

begin using a speech recognizer associated with the media content;

in accordance with a determination that a user has spoken an action phrase, perform an action according to the user'"'"'s spoken action phrase; and

in accordance with a determination that the user has not spoken the action phrase, cause the media content to continue to be played.

20. The medium of claim 19, wherein the media content is silenced during a period for which the speech recognizer is used, and the media content is silenced by at least one of:

muting a playback mechanism used to play the media content,

pausing the playback mechanism used to play the media content, or

playing silence over the playback mechanism used to play the media content.

21. A device comprising:

one or more processors;

a memory storing one or more computer programs that include computer instructions, which when executed by the one or more processors, cause the device to;

cause non-interactive media content in at least an audio or video format to be played by a first source;

detect a first event in the non-interactive media content, the first event indicating a start time of a playback of interactive media content;

in response to detecting the first event in the non-interactive media content, cause the interactive media content in at least an audio or video format to be played by a second source at the indicated start time; and

begin using a speech recognizer;

in accordance with a determination that a user has spoken an action phrase associated with the interactive media content, perform an action according to the action phrase; and

in accordance with a determination that the user has not spoken the action phrase associated with the interactive media content, cause the non-interactive media content to continue to be played.

22. The device of claim 21, wherein the first event in the non-interactive media content is in the form of metadata.

23. The device of claim 22, wherein the first event in the non-interactive media content is in the form of at least ID3 metadata or HLS metadata.

24. The device of claim 21, wherein the first event in the non-interactive media content is a sub-audible tone embedded in the media content, wherein the sub-audible tone is not detectable by an unaided human ear.

25. The device of claim 24, wherein the sub-audible tone has a frequency of 20 Hertz or less.

26. The device of claim 21, wherein the first event is associated with metadata comprising instructions for selecting the interactive media content and the associated action phrase, and the metadata is either supplied from the source along with the non-interactive media content or supplied separately from an external source.

27. The device of claim 21, wherein the non-interactive media content is silenced during at least a period for which the speech recognizer is used, and the non-interactive media content is silenced by at least one of:

muting a first playback mechanism of the first source used to play the non-interactive media content,

pausing the first playback mechanism of the first source used to play the non-interactive media content, or

playing silence over the first playback mechanism of the first source used to play the non-interactive media content.

28. The device of claim 21, wherein the interactive media content is silenced during at least a period for which the speech recognizer is used, and the interactive media content is silenced by at least one of:

muting a second playback mechanism of the second source used to play the interactive media content,

pausing the second playback mechanism of the second source used to play the interactive media content, or

playing silence over the second playback mechanism of the second source used to play the interactive media content.

29. The device of claim 21, wherein the first source and the second source are each at least one of a remote media server, a local file system, and a local memory.

30. The device of claim 21, wherein the computer instructions, which when executed by the one or more processors, further cause the device to:

prior to activating the speech recognizer, play the interactive media content followed by a prompt, the prompt comprising a message instructing the user to say the action phrase, wherein the second event is associated with an end of the prompt such that the speech recognizer is used after playing the prompt.

31. A method for providing speech interactive content, the method comprising:

causing non-interactive media content in at least an audio or video format to be played by a first source;

detecting a first event in the non-interactive media content, the first event indicating a start time of a playback of interactive media content;

in response to detecting the first event in the non-interactive media content, causing the interactive media content in at least an audio or video format to be played by a second source at the indicated start time;

begin using a speech recognizer;

in accordance with a determination that a user has spoken an action phrase associated with the interactive media content, performing an action according to the action phrase; and

in accordance with a determination that the user has not spoken the action phrase associated with the interactive media content, causing the non-interactive media content to continue to be played.

32. The method of claim 31, wherein the first event in the non-interactive media content is in the form of metadata.

33. The method of claim 32, wherein the first event in the non-interactive media content is in the form of at least ID3 metadata or HLS metadata.

34. The method of claim 31, wherein the first event in the non-interactive media content is a sub-audible tone embedded in the media content, wherein the sub-audible tone is not detectable by an unaided human ear.

35. The method of claim 34, wherein the sub-audible tone has a frequency of 20 Hertz or less.

36. The method of claim 31, wherein the first event is associated with metadata comprising instructions for selecting the interactive media content and the associated action phrase, and the metadata is either supplied from the source along with the non-interactive media content or supplied separately from an external source.

37. The method of claim 31, wherein the non-interactive media content is silenced during at least a period for which the speech recognizer is used, and the non-interactive media content is silenced by at least one of:

muting a first playback mechanism of the first source used to play the non-interactive media content,

pausing the first playback mechanism of the first source used to play the non-interactive media content, or

playing silence over the first playback mechanism of the first source used to play the non-interactive media content.

38. The method of claim 31, wherein the interactive media content is silenced during at least a period for which the speech recognizer is used, and the interactive media content is silenced by at least one of:

muting a second playback mechanism of the second source used to play the interactive media content,

pausing the second playback mechanism of the second source used to play the interactive media content, or

playing silence over the second playback mechanism of the second source used to play the interactive media content.

39. The method of claim 31, wherein the first source and the second source are each at least one of a remote media server, a local file system, and a local memory.

40. The method of claim 31, further comprising:

prior to activating the speech recognizer, play the interactive media content followed by a prompt, the prompt comprising a message instructing the user to say the action phrase, wherein the second event is associated with an end of the prompt such that the speech recognizer is used after playing the prompt.

41. A non-transitory computer readable medium comprising one or more computer programs, which when executed by a device with one or more processors, cause the device to:

cause non-interactive media content in at least an audio or video format to be played by a first source;

detect a first event in the non-interactive media content, the first event indicating a start time of a playback of interactive media content;

in response to detecting the first event in the non-interactive media content, cause the interactive media content in at least an audio or video format to be played by a second source at the indicated start time;

begin using a speech recognizer;

in accordance with a determination that a user has spoken an action phrase associated with the interactive media content, perform an action according to the action phrase; and

in accordance with a determination that the user has not spoken the action phrase associated with the interactive media content, cause the non-interactive media content to continue to be played.