Soundtrack for electronic text

• US 9,666,227 B2
• Filed: 07/26/2012
• Issued: 05/30/2017
• Est. Priority Date: 07/26/2011
• Status: Active Grant

• Current Assignee: Booktrack Holdings Limited
• Original Assignee: Booktrack Holdings Limited
• Inventors: Cameron, Paul Charles, Cameron, Mark Steven, Zhang, Rui, Davenport, Andrew Russell, McGrath, Paul Anthony
• Primary Examiner(s): Saunders, Jr., Joseph
• Application Number:
  US14/234,555
• Publication Number:
  US 20140156046A1
• Time in Patent Office: 1,769 Days
• CPC Class Codes: G06F 3/0483 Interaction with page-structured environments, e.g. book metaphor
• CPC Class Codes: G06F 3/165 Management of the audio stream, e.g. setting of volume, audio stream path
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G09B 5/06 with both visual and audible presentation of the material to be studied
• CPC Class Codes: G09B 5/065 Combinations of audio and video presentations, e.g. videotapes, videodiscs, television systems
• CPC Class Codes: G10L 21/10 Transforming into visible information
• CPC Class Codes: G11B 20/10527 Audio or video recording; Data buffering arrangements G11B20/12 - G11B20/18 take precedence
• CPC Class Codes: G11B 27/005 Reproducing at a different information rate from the information rate of recording for television signals H04N5/783
• CPC Class Codes: G11B 27/10 Indexing; Addressing; Timing or synchronising; Measuring tape travel
• CPC Class Codes: G11B 27/11 by using information not detectable on the record carrier

Abstract

A soundtrack creation method and user playback system for soundtracks synchronized to electronic text. Synchronization is achieved by maintaining a reading speed variable indicative of the user'"'"'s reading speed. The system provides for multiple channels of audio to enable concurrent playback of two or more partially or entirely overlapping audio regions so as to create an audio output having, for example, sound effects, ambience, music or other audio features that are triggered to playback at specific portions in the electronic text to enhance the reading experience.

Claims

1. A method for displaying electronic text and synchronizing the playback of a soundtrack for the electronic text, the soundtrack comprising multiple audio regions configured for playback during corresponding text regions of the electronic text, each audio region being recited in audio data comprises:

an audio track for playback in the audio region, a start text position indicative of a word in the electronic text corresponding to where the playback of the audio track is to begin; and

a stop text position indicative of a word in the electronic text comprising to where playback of the audio track is to cease, comprising;

displaying at least a portion of the electronic text on an electronic display, the display having an associated operable user interface that is configured to receive user input to enable user interaction with the display;

sensing and tracking a tracing movement of user input relative to the displayed electronic text that is generated by the operable user interface as it is controlled by the user for a tracking period initiated by the user, the tracing movement of the user input representing the user'"'"'s reading position on the display as the user traces their reading word-by-word or line-by-line on the displayed electronic text using the operable user interface;

generating data indicative of the quantity of text traversed and time taken for the tracing movement;

calculating a measurement of reading speed based on the quantity of text read during the tracing movement and the time taken for the tracing movement;

updating a user reading speed variable representing the user'"'"'s reading speed based on the calculated measurement of reading speed;

maintaining a reading position counter corresponding to the estimated text position indicative of a word in the electronic text that the user is currently reading, the counter incrementing at a rate based at least partly on the updated user reading speed variable; and

controlling playback of the audio regions of the soundtrack over an audio output system based on the audio data and reading position counter to synchronize playback of the audio regions with the user'"'"'s estimated reading position.

2. A method according to claim 1 wherein sensing and tracking the tracing movement of a user input comprises sensing a start text position in the displayed electronic text at which the user initiates the tracing movement and an end text position in the displayed electronic text at which the user ceases the tracing movement, and generating data indicative of the quantity of the text traversed during the tracing movement comprises determining the number of words or characters between the detected start and end text positions associated with the tracing movement.

3. A method according to claim 2 wherein generating data indicative of the time taken for the tracing movement comprises initiating a timer upon initiation of the tracing movement at the start text position and halting the timer at the end of the tracing movement at the end text position, the output of the timer being indicative of the time taken for the tracing movement.

4. A method according to claim 2 wherein the electronic display is a touch screen display such that the operable user interface is configured to receive user input in the form of touch input upon the surface of the display which generates representative touch signals, and wherein the start text position in the displayed electronic text is detected at initiation of the tracing movement upon sensing user contact with the surface of the display and the end text position is sensed at the position in the displayed electronic text where the user disengages their contact from the surface of the display.

5. A method according to claim 1 wherein sensing and tracking the tracing movement of the user input comprises tracking the user input as it moves substantially horizontally relative to the displayed electronic text word-by-word.

6. A method according to claim 1 wherein sensing and tracking the tracing movement of the user input comprises tracking the user input as it moves substantially vertically relative to the displayed electronic text line-by-line.

7. A method according to claim 1 wherein calculating a measurement of reading speed comprises generating a measurement of reading speed at the end of the tracing movement based on the total quantity of text read and total time taken.

8. A method according to claim 1 wherein calculating a measurement of reading speed comprises continuously or periodically generating a measurement of reading speed in real-time based on quantity of text read since the initiation of the tracing movement and the time elapsed since the initiation of the tracing movement;

and continuously or periodically updating the user reading speed variable.

9. A method for displaying electronic text and synchronizing the playback of a soundtrack for the electronic text, the soundtrack comprising multiple audio regions configured for playback during corresponding text regions of the electronic text, each audio region being recited in audio data comprises:

an audio track for playback in the audio region, a start text position indicative of a word in the electronic text corresponding to where the playback of the audio track is to begin; and

a stop text position indicative of a word in the electronic text comprising to where playback of the audio track is to cease, comprising;

displaying at least a portion of the electronic text on a display;

detecting a first user reading position input indicative of a first reading text position identifying any arbitrary word in a portion of the displayed electronic text the user is reading at the time of the first user reading position input, and generating data indicative of the arbitrary first reading text position;

detecting a second subsequent user reading position input indicative of a second reading text position identifying any arbitrary word in a portion of the displayed electronic text the user is reading at the time of the second user reading position input, and generating data indicative of the arbitrary second reading text position;

calculating the quantity of text read between the arbitrary first and second reading text positions based on their respective generated indicative data;

calculating an estimate of the user'"'"'s reading speed based on the calculated quantity of text read and the time period between the arbitrary first and second user reading position inputs;

updating a user reading speed variable representing the user'"'"'s reading speed based on the calculated estimate of the user'"'"'s reading speed;

maintaining a reading position counter corresponding to the estimated text position indicative of a word in the electronic text that the user is currently reading, the counter incrementing at a rate based at least partly on the updated user reading speed variable; and

controlling playback of the audio regions of the soundtrack over an audio output system based on the audio data and reading position counter to synchronize playback of the audio regions with the user'"'"'s estimated reading position, and

wherein the detected arbitrary first and second user reading position inputs are not indicative of page turn events of the displayed electronic text.

10. A method according to claim 9 wherein the electronic text is indexed based on the number of words or characters in the electronic text, and the data indicative of the first reading text position comprises a first index value corresponding to the index value at the first reading text position, and the data indicative of the second reading text position comprises a second index value corresponding to the index value at the second reading text position, and wherein calculating the quantity of text read comprises calculating the absolute difference between the first and second index values.

11. A method according to claim 9 wherein the method further comprises generating a first time stamp and a second time stamp corresponding to the respective times of the first user reading position input and second user reading position input, and calculating the time period between the first and second user reading position inputs based on absolute difference between the first and second time stamps.

12. A method according to claim 9 wherein the method further comprises starting a timer on detection of the first user reading position input and stopping the timer on detection of the second user reading position input, and the output of the timer representing the time period between the first and second user reading position inputs.

13. A method according to claim 9 wherein the display is a touch screen display, and detecting the first and second user reading position inputs comprises sensing user selections of the arbitrary text positions in the displayed electronic text by sensing a user touch input on the surface of the touch screen display at locations corresponding to the arbitrary text positions.

14. A method according to claim 9 wherein detecting the first and second user reading position inputs comprises detecting user selection of the arbitrary text positions in the displayed electronic text by sensing user activation via a user input device of a pointer on the display situated at the location corresponding to the arbitrary text positions.

15. A method according to claim 9 wherein the method further comprises calculating a new estimate of the user'"'"'s reading speed for each new subsequent user reading position input indicative of the user'"'"'s current reading text position corresponding to any arbitrary word in the electronic text the user is reading at the time of the new user reading position input, each new estimate of the user'"'"'s reading speed being calculated based on the quantity of text and time period between the new user reading position input indicative of the user'"'"'s current arbitrary reading text position and the last user reading position input indicative of the user'"'"'s previously indicated arbitrary reading text position.

16. A method according to claim 15 wherein the method comprises updating the user reading speed variable by applying an update algorithm for each new calculated estimate of the user'"'"'s reading speed, the update algorithm comprising:

calculating a new moving average of the reading speed variable based on a predetermined number of previous reading speed variables;

calculating a difference value indicative of the percentage change between the new calculated estimate of the user'"'"'s reading speed and the moving average;

determining a probability weighting for the new calculated estimate of the user'"'"'s reading speed based on the difference value and a probability density function; and

calculating a new reading speed variable based on the new calculated estimate of the user'"'"'s reading speed, the moving average, and the probability weighting.

17. A method for displaying electronic text and synchronizing the playback of a soundtrack for the electronic text, the soundtrack comprising multiple audio regions configured for playback during corresponding text regions of the electronic text, each audio region being recited in audio data comprises:

an audio track for playback in the audio region, a start text position indicative of a word in the electronic text corresponding to where the playback of the audio track is to begin; and

a stop text position indicative of a word in the electronic text comprising to where playback of the audio track is to cease, comprising;

displaying at least a portion of the electronic text on a display;

displaying a visual moving reading position indicator on the display which identifies the current text position that is estimated as being read based on an initial user reading speed variable representing the user'"'"'s estimated reading speed, the visual moving reading position indicator moving relative to the text at a rate based on the user reading speed variable representing the user'"'"'s reading speed;

providing a user adjustment interface that is operable by a user to increase or decrease the user reading speed variable; and

updating the user reading speed variable according to any user adjustment to thereby modify the rate of movement of the visual moving reading position indicator to match the updated user reading speed variable,

wherein the user adjustment interface is operable simultaneously as the electronic text and visual moving reading position indicator are displayed during playback such that the user can view in real-time the effect of any adjustment of the reading speed variable on the rate of movement of the visual moving reading position indicator;

maintaining a reading position counter corresponding to the estimated text position indicative of a word in the electronic text that the user is currently reading, the counter incrementing at a rate at least partly based on the updated user reading speed variable; and

controlling playback of the audio regions of the soundtrack over an audio output system based on the audio data and reading position counter to synchronize playback of the audio regions with the user'"'"'s estimated reading position.

18. A method according to claim 17 wherein the user adjustment interface is in the form of activatable increase and decrease inputs that are configured to respectively increase or decrease the reading speed variable by a predetermined value upon activation by a user.

19. A method according to claim 17 wherein the user adjustment interface is in the form of an operable dial or slider that is moveable to increase or decrease the reading speed variable to a user selected value within a scale of reading speed values defined between upper and lower limits.

20. A method according to claim 17 wherein the visual moving reading position indicator is an inline indicator that is configured to traverse the lines of text word-by-word or character-by-character at a rate that is dependent on the reading speed variable.

21. A method according to claim 17 wherein the visual moving reading position indicator is a margin indicator that is configured to move down the margin of the display text line-by-line at a rate that is dependent on the reading speed variable.

22. A system for displaying electronic text and controlling playback of a soundtrack synchronized to the reading speed of a user comprising:

an electronic display configured to display the electronic text;

an operable user interface configured to receive user input to enable user interaction with the system;

an audio output system that is operable to generate audio output for the user to hear;

an electronic text display module configured to process electronic text data representing the electronic text and control display of the electronic text on the display in response to user input from the operable user interface;

a reading position module configured to generate a reading position counter corresponding to the estimated current text position indicative of a word in the electronic text that the user is currently reading, the reading position counter incrementing at a rate based at least partly on a configurable user reading speed variable representing the user'"'"'s reading speed;

a soundtrack playback module configured to process soundtrack data representing the soundtrack to control playback of the soundtrack over the audio output system, the soundtrack data defining multiple audio regions for the soundtrack, each audio region being recited in audio data comprising;

an audio track for playback in the audio region;

a start text position indicative of a word in the electronic text corresponding to where the playback of the audio region is to begin; and

a stop text position indicative of a word in the electronic text corresponding to where the playback of the audio region is to cease;

and wherein the soundtrack playback module is configured to coordinate playback of the audio regions based on the reading position counter to synchronize playback of the audio regions of the soundtrack with the user'"'"'s reading of the electronic text; and

wherein the system further comprises a reading speed adjustment module that is configured to sense and track a tracing movement of user input relative to the displayed electronic text that is generated by the operable user interface as it is controlled by the user for a tracking period initiated by the user, and which calculates and updates the user reading speed variable based on the quantity of text traversed during the tracing movement and the time taken for the tracing movement, the tracing movement of the user input representing the user'"'"'s reading position on the display as the user traces their reading word-by-word or line-by-line of the displayed electronic text using the operable user interface.

23. A system according to claim 22 wherein the reading speed adjustment module is configured to sense and track the tracing movement of the user input by sensing a start text position in the displayed electronic text at which the user initiates the tracing movement and an end text position in the displayed electronic text at which the user ceases the tracing movement, and generating data indicative of the quantity of the text traversed during the tracing movement comprises determining the number of words or characters between the detected start and end text positions associated with the tracing movement.

24. A system according to claim 23 wherein the reading speed adjustment module is configured to generate data indicative of the time taken for the tracing movement by initiating a timer upon initiation of the tracing movement at the start text position and halting the timer at the end of the tracing movement at the end text position, the output of the timer being indicative of the time taken for the tracing movement.

25. A system according to claim 23 wherein the electronic display is a touch screen display such that the operable user interface is configured to receive user input in the form of touch input upon the surface of the display which generates representative touch signals, and wherein the start text position in the displayed electronic text is detected at initiation of the tracing movement upon sensing user contact with the surface of the display and the end text position is sensed at the position in the displayed electronic text where the user disengages their contact from the surface of the display.

26. A system according to claim 22 wherein the reading speed adjustment module is configured to sense and track the tracing movement of the user input by tracking the user input as it moves substantially horizontally relative to the displayed electronic text word-by-word.

27. A system according to claim 22 wherein the reading speed adjustment module is configured to sense and track the tracing movement of the user input by tracking the user input as it moves substantially vertically relative to the displayed electronic text line-by-line.

28. A system according to claim 22 wherein the reading speed adjustment module is configured to calculate a measurement of reading speed by generating a measurement of reading speed at the end of the tracing movement based on the total quantity of text read and total time taken.

29. A system according to claim 22 wherein the reading speed adjustment module is configured to calculate a measurement of reading speed by continuously or periodically generating a measurement of reading speed in real-time based on quantity of text read since the initiation of the tracing movement and the time elapsed since the initiation of the tracing movement;

and continuously or periodically updating the user reading speed variable.

30. A system for displaying electronic text and controlling playback of a soundtrack synchronized to the reading speed of a user comprising:

an electronic display configured to display the electronic text;

a user interface configured to receive user input to control the system;

an audio output system that is operable to generate audio output for the user to hear;

an electronic text display module configured to process electronic text data representing the electronic text and control display of the electronic text on the display in response to user input from the user interface;

a reading position module configured to generate a reading position counter corresponding to the estimated current text position indicative of a word in the electronic text that the user is currently reading, the reading position counter incrementing at a rate based at least partly on a configurable user reading speed variable representing the user'"'"'s reading speed;

a soundtrack playback module configured to process soundtrack data representing the soundtrack to control playback of the soundtrack over the audio output system, the soundtrack data defining multiple audio regions for the soundtrack, each audio region being recited in audio data comprising;

an audio track for playback in the audio region;

a start text position indicative of a word in the electronic text corresponding to where the playback of the audio region is to begin; and

a stop text position indicative of a word in the electronic text corresponding to where the playback of the audio region is to cease;

and wherein the soundtrack playback module is configured to coordinate playback of the audio regions based on the reading position counter to synchronize playback of the audio regions of the soundtrack with the user'"'"'s reading of the electronic text; and

the system further comprising a reading speed adjustment module that is configured to;

detect a first user reading position input indicative of a first reading text position identifying any arbitrary word in a portion of the displayed electronic text the user is reading at the time of the first user reading position input, and generate data indicative of the arbitrary first reading text position;

detect a second subsequent user reading position input indicative of a second reading text position identifying any arbitrary word in a portion of the displayed electronic text the user is reading at the time of the second user reading position input, and generate data indicative of the arbitrary second reading text position;

calculate the quantity of text read between the arbitrary first and second reading text positions based on their respective generated indicative data;

calculate an estimate of the user'"'"'s reading speed based on the calculated quantity of text read and time period between the arbitrary first and second user reading position inputs; and

update the configurable user reading speed variable based on the calculated estimate of the user'"'"'s reading speed, and

wherein the detected arbitrary first and second user reading position inputs are not indicative of page turn events of the displayed electronic text.

31. A system according to claim 30 wherein the electronic text is indexed based on the number of words or characters in the electronic text, and the data indicative of the first reading text position comprises a first index value corresponding to the index value at the first reading text position, and the data indicative of the second reading text position comprises a second index value corresponding to the index value at the second reading text position, and wherein calculating the quantity of text read comprises calculating the absolute difference between the first and second index values.

32. A system according to claim 30 wherein the reading speed adjustment module is further configured to generate a first time stamp and a second time stamp corresponding to the respective times of the first user reading position input and second user reading position input, and calculating the time period between the first and second user reading position inputs based on absolute difference between the first and second time stamps.

33. A system according to claim 30 wherein the reading speed adjustment module is further configured to start a timer on detection of the first user reading position input and stop the timer on detection of the second user reading position input, and the output of the timer representing the time period between the first and second reading position user inputs.

34. A system according to claim 30 wherein the electronic display is a touch screen display, and wherein the reading speed adjustment module is configured to detect the first and second user reading position inputs by sensing user selections of the arbitrary text positions in the displayed electronic text by sensing a user touch input on the surface of the touch screen display at locations corresponding to the arbitrary text positions.

35. A system according to claim 30 wherein the reading speed adjustment module is configured to detect the first and second user reading position inputs by detecting user selection of the arbitrary text positions in the displayed electronic text by sensing user activation via a user input device of a pointer on the display situated at the location corresponding to the arbitrary text positions.

36. A system according to claim 30 wherein the reading speed adjustment module is further configured to calculate a new estimate of the user'"'"'s reading speed for each new subsequent user reading position input indicative of the user'"'"'s current reading text position corresponding to any arbitrary word in the electronic text the user is reading at the time of the new user reading position input, each new estimate of the user'"'"'s reading speed being calculated based on the quantity of text and time period between the new user reading position input indicative of the user'"'"'s current arbitrary reading text position and the last user reading position input indicative of the user'"'"'s previously indicated arbitrary reading text position.

37. A system according to claim 36 wherein the reading speed adjustment module is further configured to update the configurable user reading speed variable by applying an update algorithm for each new calculated estimate of the user'"'"'s reading speed, the update algorithm comprising:

calculating a new moving average of the reading speed variable based on a predetermined number of previous reading speed variables;

calculating a difference value indicative of the percentage change between the new calculated estimate of the user'"'"'s reading speed and the moving average;

determining a probability weighting for the new calculated estimate of the user'"'"'s reading speed based on the difference value and a probability density function; and

calculating a new reading speed variable based on the new calculated estimate of the user'"'"'s reading speed, the moving average, and the probability weighting.

38. A system for displaying electronic text and controlling playback of a soundtrack synchronized to the reading speed of a user comprising:

an electronic display configured to display the electronic text;

a user interface configured to receive user input to control the system;

an audio output system that is operable to generate audio output for the user to hear;

an electronic text display module configured to process electronic text data representing the electronic text and control display of the electronic text on the display in response to user input from the user interface;

a reading position module configured to generate a reading position counter corresponding to the estimated current text position indicative of a word in the electronic text that the user is currently reading, the reading position counter incrementing at a rate based at least partly on a configurable user reading speed variable representing the user'"'"'s reading speed;

a soundtrack playback module configured to process soundtrack data representing the soundtrack to control playback of the soundtrack over the audio output system, the soundtrack data defining multiple audio regions for the soundtrack, each audio region being recited in audio data comprising;

an audio track for playback in the audio region;

a start text position indicative of a word in the electronic text corresponding to where the playback of the audio region is to begin; and

a stop text position indicative of a word in the electronic text corresponding to where the playback of the audio region is to cease;

and wherein the soundtrack playback module is configured to coordinate playback of the audio regions based on the reading position counter to synchronize playback of the audio regions of the soundtrack with the user'"'"'s reading of the electronic text; and

the system further comprising a reading speed adjustment module that is configured to;

display a visual moving reading position indicator on the display which identifies the estimated current text position that is estimated as being read based on the reading position counter, the visual moving reading position indicator moving relative to the text at a rate based on the user reading speed variable representing the user'"'"'s reading speed;

provide a user adjustment interface that is operable by a user to increase or decrease the configurable user reading speed variable; and

update the user reading speed variable according to any user adjustment to thereby modify the rate of movement of the visual moving reading position indicator to match the updated user reading speed variable, and

wherein the user adjustment interface is operable simultaneously as the electronic text and visual moving reading position indicator are displayed during playback such that the user can view in real-time the effect of any adjustment of the user reading speed variable on the rate of movement of the visual moving reading position indicator.

39. A system according to claim 38 wherein the user adjustment interface is in the form of activatable increase and decrease inputs that are configured to respectively increase or decrease the reading speed variable by a predetermined value upon activation by a user.

40. A system according to claim 38 wherein the user adjustment interface is in the form of an operable dial or slider that is moveable to increase or decrease the reading speed variable to a user selected value within a scale of reading speed values defined between upper and lower limits.

41. A system according to claim 38 wherein the visual moving reading position indicator is an inline indicator that is configured to traverse the lines of text word-by-word or character-by-character at a rate that is dependent on the reading speed variable.

42. A system according to claim 38 wherein the visual moving reading position indicator is a margin indicator that is configured to move down the margin of the display text line-by-line at a rate that is dependent on the reading speed variable.