Synchronizing the playing and displaying of digital content

• US 9,542,926 B2
• Filed: 03/12/2014
• Issued: 01/10/2017
• Est. Priority Date: 06/12/2009
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Nguyen, Laurent An Minh, Gayles, Edward J., Chu, Robert Wai-Chi, Fleming, Dennis Paul, Rachabathuni, Sailesh, Berbessou, David
• Primary Examiner(s): JACKSON, JAKIEDA R
• Application Number:
  US14/207,222
• Publication Number:
  US 20140195241A1
• Time in Patent Office: 1,035 Days
• Field of Search: 704/260
• US Class Current: 1/1
• CPC Class Codes: G10L 13/00 Speech synthesis; Text to speech systems
• CPC Class Codes: G10L 21/0356 for synchronising with other signals, e.g. video signals

Abstract

The techniques disclosed herein allow a user to synchronize the playing and displaying of digital content on an electronic device. The device may render a first portion of digital content so it may be displayed. The device may also play a segment of the digital content as audio using text to speech software. The device may also render a second portion of digital content for display depending on whether the position of the last word read is greater than the last position in the first portion of digital content.

Claims

1. One or more non-transitory computer-readable media that include a plurality of instructions executable by one or more processors of a computing device to:

obtain digital content comprising text data;

insert a plurality of markings into the digital content, wherein each marking is associated with a particular position in the digital content;

cause a speaker of the computing device to output speech audio corresponding to the digital content;

cause a display of the computing device to display a first portion of the text data while the speech audio is output by the speaker, wherein the display of the first portion of the text data is according to a first display property;

receive an input to change from the first display property to a second display property;

responsive to receipt of the input, determine a marking of the plurality of markings associated with a word within the first portion of the text data; and

based upon the marking associated with the first portion of the text data, cause a second portion of the text data to be displayed according to the second display property such that the second portion of the text data includes the word and corresponds to a currently playing portion of the speech audio.

2. The one or more non-transitory computer-readable media of claim 1, wherein the first display property and the second display property are one of:

a first font size and a second font size, respectively, or

a first screen orientation and a second screen orientation, respectively.

3. The non-transitory computer-readable media of claim 1, wherein the plurality of instructions are further executable by the one or more processors of the computing device to:

determine an initial portion of the second portion of the text data displayed according to the second display property based at least on an amount of time that the computing device takes to display a new portion of the text data and a play speed of the speech audio.

4. The one or more non-transitory computer-readable media of claim 1, wherein the text data includes a plurality of text units, ones of the text units including a plurality of words, and wherein the plurality of instructions are further executable by the one or more processors of the computing device to insert the plurality of markings between ones of the text units.

5. The one or more non-transitory computer-readable media of claim 1, wherein the one or more non-transitory computer-readable media are included within an electronic book (eBook) reader, and wherein the digital content comprises an eBook.

6. An electronic device comprising:

one or more processors;

memory;

instructions stored in the memory, the instructions executable by the one or more processors to;

insert a plurality of markings into digital content;

cause output of speech audio that corresponds to the digital content;

cause a first portion of the digital content to be displayed on the electronic device;

receive input to change a display property of the digital content;

responsive to receipt of the input, determine a marking of the plurality of markings associated with a word within the first portion of the digital content; and

based upon the marking, cause a second portion of the digital content to be displayed such that the second portion;

includes the word,

corresponds to a portion of the speech audio that is output at a time that the second portion of the digital content is displayed, and

is displayed according to the change in the display property.

7. The electronic device of claim 6, wherein the instructions are further executable by the one or more processors to synthesize the speech audio from the digital content.

8. The electronic device of claim 6, wherein the instructions are further executable by the one or more processors to determine the second portion such that a beginning of the second portion of the digital content corresponds to the speech audio that is output at the time that the second portion of the digital content is displayed.

9. The electronic device of claim 6, wherein the instructions are further executable by the one or more processors to estimate the portion of the speech audio that is output at the time that the second portion of the digital content is displayed based at least on a speed of the speech audio output.

10. The electronic device of claim 9, wherein the instructions are further executable by the one or more processors to estimate the portion of the speech audio that is output at the time that the second portion of the digital content is displayed further based at least on an amount of time that the electronic device takes to display portions of the digital content.

11. The electronic device of claim 6, wherein the instructions are further executable by the one or more processors to insert the markings between a plurality of display portions, the plurality of display portions including one or more of text units and images.

12. The electronic device of claim 6, wherein the display property is one of a font size or a screen orientation.

13. The electronic device of claim 6, wherein the electronic device comprises an electronic book (eBook) reader, and wherein the digital content comprises an eBook.

14. A method comprising:

causing an electronic device to insert a plurality of markings into digital content;

causing the electronic device to display a first portion of digital content on a display of the electronic device;

causing speech audio corresponding to the digital content to be output to a speaker associated with the electronic device;

based upon a changed display property, causing the electronic device to determine a marking of the plurality of markings associated with a word within the first portion of the digital content; and

based upon the marking, cause a second portion of the digital content to be displayed such that the second portion;

includes the word,

corresponds to a portion of the speech audio that is output at a time that the second portion of the digital content is displayed, and

is displayed according to the change in the display property.

15. The method of claim 14, wherein the digital content includes text data, and the method further comprises generating the speech audio using text-to-speech conversion of the text data.

16. The method of claim 14, further comprising estimating a location of the second portion of the digital content based at least on a play speed of the speech audio.

17. The method of claim 14, wherein the digital content includes text data, and wherein the display property change results in a change in a number of text characters displayable on the display at a single time.

18. The method of claim 17, wherein the display property is a font size of the text data.

19. The method of claim 14, wherein the electronic device comprises an electronic book (eBook) reader, and wherein the digital content comprises an eBook.