Soundtrack for electronic text

• US 9,613,654 B2
• Filed: 01/28/2014
• Issued: 04/04/2017
• Est. Priority Date: 07/26/2011
• Status: Active Grant

• Current Assignee: Booktrack Holdings Limited
• Original Assignee: Booktrack Holdings Limited
• Inventors: Cameron, Paul Charles, Cameron, Mark Steven, Zhang, Rui, Davenport, Andrew Russell, McGrath, Paul Anthony
• Primary Examiner(s): Saunders, Jr., Joseph
• Application Number:
  US14/166,436
• Publication Number:
  US 20140142954A1
• Time in Patent Office: 1,162 Days
• CPC Class Codes: G06F 3/0483 Interaction with page-structured environments, e.g. book metaphor
• CPC Class Codes: G06F 3/165 Management of the audio stream, e.g. setting of volume, audio stream path
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G09B 5/06 with both visual and audible presentation of the material to be studied
• CPC Class Codes: G09B 5/065 Combinations of audio and video presentations, e.g. videotapes, videodiscs, television systems
• CPC Class Codes: G10L 21/10 Transforming into visible information
• CPC Class Codes: G11B 20/10527 Audio or video recording; Data buffering arrangements G11B20/12 - G11B20/18 take precedence
• CPC Class Codes: G11B 27/005 Reproducing at a different information rate from the information rate of recording for television signals H04N5/783
• CPC Class Codes: G11B 27/10 Indexing; Addressing; Timing or synchronising; Measuring tape travel
• CPC Class Codes: G11B 27/11 by using information not detectable on the record carrier

Abstract

A soundtrack creation method and user playback system for soundtracks synchronized to electronic text. Synchronization is achieved by maintaining a reading speed variable indicative of the user'"'"'s reading speed. The system provides for multiple channels of audio to enable concurrent playback of two or more partially or entirely overlapping audio regions so as to create an audio output having, for example, sound effects, ambience, music or other audio features that are triggered to playback at specific portions in the electronic text to enhance the reading experience.

Claims

1. A method for displaying electronic text and synchronizing the playback of a soundtrack for the electronic text, the soundtrack comprising multiple audio regions configured for playback during corresponding text regions of the electronic text, each audio region being defined by audio data comprising:

an audio track for playback in the audio region, a start text position indicative of the word in the electronic text corresponding to where the playback of the audio track is to begin; and

a stop text position indicative of the word in the electronic text corresponding to where playback of the audio track is to cease, and at least some audio regions further each being defined by pre-defined playback effect data that defines one or more controllable audio playback effect settings for applying sound manipulation to the playback of the audio track of the audio region and which are to be varied across the audio region or a portion of the audio region based on the user'"'"'s real-time text position, comprising;

displaying at least a portion of the electronic text on a display;

maintaining a reading position counter corresponding to the estimated text position indicative of the word in the electronic text that the user is currently reading, the counter incrementing at a rate based on a configurable user reading speed variable representing the user'"'"'s reading speed;

controlling playback of the audio regions of the soundtrack over an audio output system based on the audio data and reading position counter to synchronise playback of the audio regions with the user'"'"'s estimated reading position; and

controlling audio playback effect settings of the audio output system in real-time based on the reading position counter for any audio regions having pre-defined playback effect data such that the audio playback effect settings are varied in real-time to apply sound manipulation to the playback of the audio tracks based on the user'"'"'s real-time text position to match the desired sound manipulation of the audio tracks of such audio regions according to their respective pre-defined playback effect data.

2. A method according to claim 1 wherein the audio output system comprises a plurality of channels such that any overlapping audio regions can playback concurrently on separate channels, and wherein the method comprises controlling independently the playback effect settings of each channel based on any pre-defined audio playback effect data associated with the overlapping audio regions.

3. A method according to claim 1 wherein the pre-configured playback effect data is defined by a programmable function the output of which represents a playback effect setting that is dependent on a variable corresponding to the reading position counter or a counter that is a product or function of the reading position counter.

4. A method according to claim 1 wherein the pre-configured playback effect data associated with an audio region is in the form of a look-up table which defines a discrete playback effect setting for a series of discrete text positions extending across at least a portion of the audio region.

5. A method according to claim 1 wherein the pre-configured playback effect data associated with an audio region is represented in the form of one or more envelopes that each define a playback effect setting that is variable across at least a portion of the audio region, and where the shape of each envelope across the audio region or portion of the audio region defines how its associated playback effect setting is varied across the audio region or portion of the audio region.

6. A method according to claim 5 wherein the pre-configured playback effect data comprises a volume envelope that defines a variable volume setting to be applied during playback of the audio region, and controlling audio playback effect settings of the audio output system comprises updating a volume setting of the audio output system in real-time to match the volume envelope based on the user'"'"'s real-time text position within the audio region.

7. A method according to claim 5 wherein the pre-configured playback effect data comprises a panning envelope that defines a variable balance setting to be applied during playback of the audio region, and controlling the audio playback effect settings of the audio output system comprises updating a balance setting of the audio output system in real-time to match the panning envelope based on the user'"'"'s real-time text position within the audio region.

8. A method according to claim 1 wherein the pre-configured playback effect data defines any one or more of the following controllable audio playback effect settings:

volume, balance, equalization, reverberation, distortion, and phase-shifting.

9. A method according to claim 1 wherein controlling audio playback effect settings of the audio output system in real-time comprises varying the audio playback effect settings in real-time at a frequency at least as high as the reading position counter rate.

10. A method according to claim 1 wherein controlling audio playback effect settings of the audio output system in real-time comprises varying the audio playback effect settings to match the predefined playback effect data at a frequency that is based on an update counter.

11. A system for displaying electronic text and controlling playback of a soundtrack synchronized to the reading speed of a user comprising:

a display configured to display the electronic text;

a user interface configured to receive user input to control the system;

an audio output system that is operable to generate audio output for the user to hear and which has controllable audio playback effect settings for applying audio playback effects to the audio output;

an electronic text display module configured to process electronic text data representing the electronic text and control display of the electronic text on the display in response to user input from the user interface;

a reading position module configured to generate a reading position counter representing the estimated current text position indicative of the word in the electronic text that the user is currently reading, the reading position counter incrementing at a rate based at least partly on a configurable user reading speed variable representing the user'"'"'s reading speed; and

a soundtrack playback module configured to process soundtrack data representing the soundtrack to control playback of the soundtrack over the audio output system, the soundtrack data defining multiple audio regions for the soundtrack, each audio region being defined by audio data comprising;

an audio track for playback in the audio region;

a start text position indicative of the word in the electronic text corresponding to where the playback of the audio region is to begin; and

a stop text position indicative of the word in the electronic text corresponding to where the playback of the audio region is to cease;

and at least some of the audio regions further each being defined by pre-defined playback effect data that defines one or more controllable audio playback effect settings for applying sound manipulation to the playback of the audio track of the audio region and which are to be varied across an audio region or a portion of the audio region based on the user'"'"'s real-time text position,

and wherein the soundtrack playback module is configured to coordinate playback of the audio regions based on the reading position counter to synchronize playback of the audio regions of the soundtrack with the user'"'"'s reading of the electronic text, and

the soundtrack playback module being further configured to control the audio playback effect settings of the audio output system in real-time based on the reading position counter for any audio regions having associated pre-defined playback effect data such that the audio playback effect settings for each such audio region are varied in real-time to apply sound manipulation to the playback of the audio tracks based on the user'"'"'s real-time text position to match the desired sound manipulation of the audio tracks of such audio regions according to their respective pre-defined playback effect data.

12. A system according to claim 11 wherein the audio output system comprises a plurality of channels such that any overlapping audio regions can playback concurrently on separate channels, and wherein the method comprises controlling independently the playback effect settings of each channel based on any pre-defined audio playback effect data associated with the overlapping audio regions.

13. A system according to claim 11 wherein the pre-configured playback effect data is defined by a programmable function the output of which represents a playback effect setting that is dependent on a variable corresponding to the reading position counter or a counter that is a product or function of the reading position counter.

14. A system according to claim 11 wherein the pre-configured playback effect data associated with an audio region is in the form of a look-up table which defines a discrete playback effect setting for a series of discrete text positions extending across at least a portion of the audio region.

15. A system according to claim 11 wherein the pre-configured playback effect data associated with an audio region is represented in the form of one or more envelopes that each define a playback effect setting that is variable across at least a portion of the audio region, and where the shape of each envelope across the audio region or portion of the audio region defines how its associated playback effect setting is varied across the audio region or portion of the audio region.

16. A system according to claim 15 wherein the pre-configured playback effect data comprises a volume envelope that defines a variable volume setting to be applied during playback of the audio region, and controlling audio playback effect settings of the audio output system comprises updating a volume setting of the audio output system in real-time to match the volume envelope based on the user'"'"'s real-time text position within the audio region.

17. A system according to claim 15 wherein the pre-configured playback effect data comprises a panning envelope that defines a variable balance setting to be applied during playback of the audio region, and controlling the audio playback effect settings of the audio output system comprises updating a balance setting of the audio output system in real-time to match the panning envelope based on the user'"'"'s real-time text position within the audio region.

18. A system according to claim 11 wherein the pre-configured playback effect data defines any one or more of the following controllable audio playback effect settings:

volume, balance, equalization, reverberation, distortion, and phase-shifting.

19. A system according to claim 11 wherein controlling audio playback effect settings of the audio output system in real-time comprises varying the audio playback effect settings in real-time at a frequency at least as high as the reading position counter rate.

20. A system according to claim 11 wherein controlling audio playback effect settings of the audio output system in real-time comprises varying the audio playback effect settings to match the predefined playback effect data at a frequency that is base on an update counter.