Delivery of synchronised soundtracks for electronic media content

• US 9,836,271 B2
• Filed: 07/16/2014
• Issued: 12/05/2017
• Est. Priority Date: 07/17/2013
• Status: Active Grant

• Current Assignee: Booktrack Holdings Limited
• Original Assignee: Booktrack Holdings Limited
• Inventors: Cameron, Paul Charles, Cameron, Mark Steven, Wilson, Craig Andrew, Buer, Mark Anthony
• Primary Examiner(s): Tsang, Fan
• Assistant Examiner(s): Siegel, David
• Application Number:
  US14/332,521
• Publication Number:
  US 20150025663A1
• Time in Patent Office: 1,238 Days
• CPC Class Codes: G06F 16/685 using automatically derived transcript of audio data, e.g. lyrics speech recognition G10L15/00
• CPC Class Codes: G06F 3/165 Management of the audio stream, e.g. setting of volume, audio stream path
• CPC Class Codes: G11B 27/005 Reproducing at a different information rate from the information rate of recording for television signals H04N5/783
• CPC Class Codes: G11B 27/11 by using information not detectable on the record carrier
• CPC Class Codes: H04L 65/60 Network streaming of media packets
• CPC Class Codes: H04N 21/2335 involving reformatting operations of audio signals, e.g. by converting from one coding standard to another details of audio signal transcoding G10L19/173
• CPC Class Codes: H04N 21/2343 involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements video transcoding H04N19/40; media packet handling at the source H04L65/762
• CPC Class Codes: H04N 21/43072 of multiple content streams on the same device
• CPC Class Codes: H04N 21/44218 Detecting physical presence or behaviour of the user, e.g. using sensors to detect if the user is leaving the room or changes his face expression during a TV program methods or arrangements for recognising human body or animal bodies or body parts G06V40/10; methods or arrangements for acquiring or recognising human faces, facial parts, facial sketches, facial expressions G06V40/16; methods or arrangements for recognising movements or behaviour G06V40/20; arrangements for identifying users in broadcast systems H04H60/45
• CPC Class Codes: H04N 21/6582 Data stored in the client, e.g. viewing habits, hardware capabilities, credit card number arrangements where receivers interact with the broadcast H04H20/38
• CPC Class Codes: H04N 21/6587 Control parameters, e.g. trick play commands, viewpoint selection
• CPC Class Codes: H04N 21/81 Monomedia components thereof
• CPC Class Codes: H04N 21/8106 involving special audio data, e.g. different tracks for different languages
• CPC Class Codes: H04N 21/8456 by decomposing the content in the time domain, e.g. in time segments
• CPC Class Codes: H04N 21/858 Linking data to content, e.g. by linking an URL to a video object, by creating a hotspot

Abstract

A method and system for streaming a soundtrack from a server to a remote user device for a reader of electronic media content. The soundtrack is defined by multiple audio regions. Each audio region defined by an audio track for playback in the audio region, a start position in the electronic media content corresponding to where the playback of the audio region is to begin, and a stop position in the electronic media content corresponding to where the playback of the audio region is to cease. The streaming of the soundtrack is based on control data generated by the remote user device.

Claims

1. A method of playing an uncompiled soundtrack on a user device for a reader of electronic media content, the uncompiled soundtrack representing a soundtrack defined by multiple audio regions, each audio region defined by:

an audio track for playback in the audio region, a start position in the electronic media content corresponding to where the playback of the audio region is to begin, and a stop position in the electronic media content corresponding to where the playback of the audio region is to cease, the method comprising;

sending control data to a remote server over a network requesting the soundtrack for playback;

receiving a soundtrack data file from the remote server in response to the sent control data, the soundtrack data file defining the audio regions of the soundtrack;

processing the soundtrack data file to determine at least the next audio region due for playback based on a user reading position variable indicative of the user'"'"'s reading position in the electronic media content;

sending periodic or arbitrary control data over a network to one or more remote audio track file sources requesting at least an audio track file corresponding to the next audio region due for playback;

receiving and storing requested audio track files from the one or more remote audio track file sources in memory on the user device; and

co-ordinating synchronized playback of the stored audio track files in memory as defined by their associated audio regions in the soundtrack data file based on the user reading position variable.

2. The method according to claim 1 wherein the soundtrack data file comprises source data indicative of the one or more audio track file sources to request each audio track file from.

3. The method according to claim 2 wherein the soundtrack data file defines a single audio track file source from which to request all the audio track files.

4. The method according to claim 2 wherein the soundtrack data file defines two or more different audio track file sources from which to request the audio track files.

5. The method according to claim 3 wherein the audio track file source is the remote server, and the method comprises requesting all audio track files from the remote server.

6. The method according to claim 4 wherein the audio track file sources comprise the remote server and one or more other audio track file sources, and the method comprises requesting the audio track files from the remote server or the one or more other audio track file sources based on the source data in the soundtrack data file.

7. The method according to claim 6 wherein some audio track files are sourced from the remote server, while other audio track files are sourced from one or more other audio track file sources.

8. The method according to claim 2 wherein the one or more audio track file sources are separate to the remote server, and the method comprises requesting all audio track files from the one or more separate audio track file sources such that all audio track files are sourced from somewhere other than the remote server.

9. The method according to claim 1 wherein the method comprises sending control data to the one or more audio track file sources requesting individual audio track files one by one.

10. The method according to claim 1 wherein the method comprises sending control data to the one or more audio track file sources comprising a single request for all audio track files associated with the audio regions of the soundtrack.

11. The method according to claim 1 wherein the method comprises sending control data to the one or more audio track file sources requesting batches of audio track files.

12. The method according to claim 1 wherein the method comprises maintaining a sliding download window relative to the user reading position variable, and sending control data to the one or more audio track file sources requesting all audio track files associated with audio regions falling within the sliding download window such that the all audio track files associated with audio regions in the sliding window are received and stored ready for future playback based on the user reading position variable.

13. The method according to claim 12 wherein size of the sliding download window is configurable.

14. The method according to claim 1 wherein the method further comprises deleting from memory the audio track files after their corresponding audio regions have completed playback either immediately or after a configurable predetermined delay.

15. The method according to claim 1 wherein the method further comprises analyzing the soundtrack data file for audio track files that are repeated in more than one audio region of the soundtrack, and tagging those audio track files as repeated audio tracks upon receipt and storage, and further retaining each such repeated audio tracks in memory until the last audio region associated with the audio track has completed playback.

16. The method according to claim 1 wherein the soundtrack data file comprises, for each audio region, data indicative of:

an audio track for playback in the audio region;

a start position in the electronic media content corresponding to a word in the electronic media content at which the playback of the audio region is to begin; and

a stop position in the electronic media content corresponding to a word in the electronic media content at which the playback of the audio region is to cease, and wherein the user reading position variable is indicative of the word in the electronic media content the user is estimated to be reading.

17. The method according to claim 16 wherein the method further comprises maintaining a user reading position variable indicative of the word in the electronic media content the user is estimated to be reading.

18. The method according to claim 17 wherein the method further comprises displaying the electronic media content on a display associated with the user device.

19. A user device for playing an uncompiled soundtrack for a reader of electronic media content, the uncompiled soundtrack representing a soundtrack defined by multiple audio regions, each audio region defined by:

an audio track for playback in the audio region, a start position in the electronic media content corresponding to where the playback of the audio region is to begin, and a stop position in the electronic media content corresponding to where the playback of the audio region is to cease, the user device comprising;

a user interface configured to receive user input to control the device;

an audio output system that is operable to generate audio output for the user to hear; and

a processor configured to;

send control data to a remote server over a network requesting the soundtrack for playback;

receive a soundtrack data file from the remote server in response to the sent control data, the soundtrack data file defining the audio regions of the soundtrack;

process the soundtrack data file to determine at least the next audio region due for playback based on a user reading position variable indicative of the user'"'"'s reading position in the electronic media content;

send periodic or arbitrary control data over a network to one or more remote audio track file sources requesting at least an audio track file corresponding to the next audio region due for playback;

receive and store requested audio track files from the one or more remote audio track file sources in memory on the user device; and

co-ordinate synchronized playback of the stored audio track files in memory as defined by their associated audio regions in the soundtrack data file based on the user reading position variable.

20. The user device according to claim 19 wherein the soundtrack data file comprises, for each audio region, data indicative of:

an audio track for playback in the audio region;

a start position in the electronic media content corresponding to a word in the electronic media content at which the playback of the audio region is to begin; and

a stop position in the electronic media content corresponding to a word in the electronic media content at which the playback of the audio region is to cease, and wherein the user reading position variable is indicative of the word in the electronic media content the user is estimated to be reading.

21. The user device according to claim 20 wherein the processor is further configured to maintain a user reading position variable indicative of the word in the electronic media content the user is estimated to be reading.

22. The user device according to claim 21 further comprising a display configured to display the electronic media content.