PLAYBACK SYSTEM FOR SYNCHRONISED SOUNDTRACKS FOR ELECTRONIC MEDIA CONTENT

• US 20150082136A1
• Filed: 09/16/2014
• Published: 03/19/2015
• Est. Priority Date: 09/18/2013
• Status: Active Grant

• Current Assignee: Booktrack Holdings Limited
• Original Assignee: Booktrack Holdings Limited
• Inventors: Cameron, Mark Steven, Cameron, Paul Charles, Wilson, Craig Andrew
• Granted Patent: US 9,898,077 B2
• Time in Patent Office: Days
• US Class Current: 715/203
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 40/12 Use of codes for handling textual entities
• CPC Class Codes: G09B 17/04 for increasing the rate of reading; Reading rate control
• CPC Class Codes: G11B 27/005 Reproducing at a different information rate from the information rate of recording for television signals H04N5/783
• CPC Class Codes: G11B 27/11 by using information not detectable on the record carrier

Abstract

A method for displaying electronic text and synchronizing the playback of a soundtrack for the electronic text. The soundtrack contains multiple audio regions configured for playback during corresponding text regions of the electronic text. Playback of the audio regions of the soundtrack over an audio output system is based on a reading position counter indicative of the user'"'"'s estimate reading position, and which increments based on a user reading speed variable. The user reading speed variable is updated by processing eye tracking signals from an eye tracker to determine the user'"'"'s reading scan rate.

Claims

1. A method for displaying electronic text and synchronizing the playback of a soundtrack for the electronic text, the soundtrack comprising multiple audio regions configured for playback during corresponding text regions of the electronic text, each audio region being defined by audio data comprising:

an audio track for playback in the audio region, a start text position indicative of the word in the electronic text corresponding to where the playback of the audio track is to begin; and

a stop text position indicative of the word in the electronic text comprising to where playback of the audio track is to cease, comprising;

displaying at least a portion of the electronic text on a display;

tracking the movement of at least one eye of the user with an eye tracker and generating representative eye tracking signals;

determining the user'"'"'s reading scan rate based on the eye tracking signals;

calculating an estimate of the user'"'"'s reading speed based on the determined reading scan rate and supplementary scan rate data relating to the electronic text and/or user;

updating a user reading speed variable representing the user'"'"'s reading speed based at least partly on the calculated estimate of the user'"'"'s reading speed;

maintaining a reading position counter corresponding to the estimated text position indicative of the word in the electronic text that the user is currently reading, the counter incrementing at a rate based on the user reading speed variable; and

controlling playback of the audio regions of the soundtrack over an audio output system based on the audio data and reading position counter to synchronize playback of the audio regions with the user'"'"'s estimated reading position.

2. The method according to claim 1 wherein the reading scan rate is a line scan rate and determining the line scan rate comprises:

processing the eye tracking signals from the eye tracker to detect line change eye movements;

determining the user'"'"'s line scan rate based on detected line change movements and a timer; and

calculating an estimate of the user'"'"'s reading speed based on the line scan rate and supplementary scan rate data indicative of the average or actual quantity of electronic text per line.

3. The method according to claim 2 wherein the method comprises determining the user'"'"'s line scan rate as an average line scan rate over multiple successive detected line change movements.

4. The method according to claim 3 wherein the line scan rate is a moving average.

5. The method according to claim 1 wherein the reading scan rate is a page turn rate and determining the page turn rate comprises:

processing the eye tracking signals from the eye tracker to detect page change eye movements;

determining the user'"'"'s page turn rate based on the time interval between detected page change movements; and

calculating an estimate of the user'"'"'s reading speed based on the page turn rate and supplementary scan rate data indicative of the average or actual quantity of electronic text per page.

6. The method according to claim 1 wherein the reading scan rate is a page turn rate and determining the page turn rate comprises:

initiating a timer when the user turns to a new page of the electronic text;

processing the eye tracking signals from the eye tracker to detect a page change eye movement; and

calculating an estimate of the user'"'"'s reading speed based on supplementary scan rate data indicative of the quantity of electronic text on the page and the value of the timer at the detected page change eye movement.

7. The method according to claim 1 further comprising processing the eye tracking signals to detect a page change eye movement, and generating a page turn signal in response to each detected page change eye movement to cause an electronic text display module to turn to the next page in the electronic text.

8. The method according to claim 1 wherein the reading scan rate is a saccade rate and determining the saccade rate comprises:

processing the eye tracking signals from the eye tracker to detect gaze fixation points between saccades;

determining the user'"'"'s saccade rate based on the time interval between detected gaze fixation points; and

calculating an estimate of the user'"'"'s reading speed based on the saccade rate and supplementary scan rate data indicative of the average or actual quantity of electronic text between gaze fixation points.

9. The method according to claim 8 wherein the supplementary scan data comprises data indicative of the average quantity of text traversed per saccade for the user.

10. A system for displaying electronic text and controlling playback of a soundtrack synchronized to the reading speed of a user comprising:

a display configured to display the electronic text;

a user interface configured to receive user input to control the system;

an audio output system that is operable to generate audio output for the user to hear;

an electronic text display module configured to process electronic text data representing the electronic text and control display of the electronic text on the display in response to user input from the user interface;

a soundtrack playback module configured to;

generate a reading position counter corresponding to the estimated current text position indicative of the word in the electronic text that the user is currently reading, the reading position counter incrementing at a rate based at least partly on a configurable user reading speed variable representing the user'"'"'s reading speed; and

process soundtrack data representing the soundtrack to control playback of the soundtrack over the audio output system, the soundtrack data defining multiple audio regions for the soundtrack, each audio region being defined by audio data comprising;

an audio track for playback in the audio region;

a start text position indicative of the word in the electronic text corresponding to where the playback of the audio region is to begin; and

a stop text position indicative of the word in the electronic text corresponding to where the playback of the audio region is to cease;

and wherein the soundtrack playback module is configured to coordinate playback of the audio regions based on the reading position counter to synchronize playback of the audio regions of the soundtrack with the user'"'"'s reading of the electronic text; and

a reading speed detector that is configured to;

track the movement of at least one eye of the user with an eye tracker to generate representative eye tracking signals;

determine the user'"'"'s reading scan rate based on the eye tracking signals; and

calculate an estimate of the user'"'"'s reading speed based on the determined reading scan rate and supplementary scan rate data relating to the electronic text and/or user,

wherein the soundtrack playback module is configured to update the user reading speed variable at least partly based on the estimate of the user'"'"'s reading speed as calculated by the reading speed detector.

11. The system according to claim 10 wherein the reading scan rate is a line scan rate and the reading speed detector is configured to:

process the eye tracking signals from the eye tracker to detect line change eye movements;

determine the user'"'"'s line scan rate based on the detected line change movements and a timer; and

calculate an estimate of the user'"'"'s reading speed based on the line scan rate and supplementary scan rate data indicative of the average or actual quantity of electronic text per line.

12. The system according to claim 11 wherein the reading speed detector is configured to:

determine the user'"'"'s line scan rate as an average line scan rate over multiple successive detected line change movements.

13. The system according to claim 12 wherein the line scan rate is a moving average.

14. The system according to claim 10 wherein the reading scan rate is page turn rate and the reading speed detector is configured to:

process the eye tracking signals from the eye tracker to detect page change eye movements;

determine the user'"'"'s page turn rate based on the time interval between detected page change movements; and

calculate an estimate of the user'"'"'s reading speed based on the page turn rate and supplementary scan rate data indicative of the average or actual quantity of electronic text per page.

15. The system according to claim 10 wherein the reading scan rate is page turn rate and the reading speed detector is configured to:

initiate a timer when the user turns to a new page of the electronic text;

process the eye tracking signals from the eye tracker to detect a page change eye movement; and

calculate an estimate of the user'"'"'s reading speed based on supplementary scan rate data indicative of the quantity of electronic text on the page and the value of the timer at the detected page change eye movement.

16. The system according to claim 10 wherein the reading scan rate is a saccade rate and the reading speed detector is configured to:

process the eye tracking signals from the eye tracker to detect gaze fixation points between saccades;

determine the user'"'"'s saccade rate based on the time interval between detected gaze fixations; and

calculate an estimate of the user'"'"'s reading speed based on the saccade rate and supplementary scan rate data indicative of the average or actual quantity of electronic text between successive gaze fixation points.

17. The system according to claim 16 wherein the supplementary scan rate data comprises data indicative of the average quantity of text traversed per saccade for the user.

18. The system according to claim 10 wherein the system is further configured to process the eye tracking signals to detect a page change eye movement, and generate a page turn signal in response to each detected page change eye movement to cause the electronic text display module to turn to the next page in the electronic text.

19. The system according to claim 10 wherein the system is implemented on a hardware system and the eye tracker comprises an optical sensor or sensors having a field of view that at least captures one or both of the user'"'"'s eyes.

20. A method of estimating a reader'"'"'s reading position in an electronic text comprising:

using a first means to locate a first reading position of a user at a first time;

detecting a reading scan rate by tracking a readers eye movement;

using the reading scan rate to estimate a reading speed; and

estimating the user'"'"'s reading position at a second time based on the incremental time from the first time, the estimated reading speed and the first position.

21. The method according to claim 20 wherein the first reading position is determined from any of the following:

a page turn signal, a touch input on a touch screen display showing the electronic text, and/or or a navigational input for a traversing electronic text.