Synchronizing playback of digital content with physical content

• US 9,472,113 B1
• Filed: 02/05/2013
• Issued: 10/18/2016
• Est. Priority Date: 02/05/2013
• Status: Active Grant

• Current Assignee: Audible Incorporated (Amazon.com, Inc.)
• Original Assignee: Audible Incorporated (Amazon.com, Inc.)
• Inventors: Hwang, Douglas Cho, Story, Guy Ashley Jr.
• Primary Examiner(s): Frisby, Kesha
• Application Number:
  US13/759,901
• Time in Patent Office: 1,351 Days
• Field of Search: 434/167, 434/169, 434/317
• US Class Current: 1/1
• CPC Class Codes: G09B 5/062 Combinations of audio and printed presentations, e.g. magnetically striped cards, talking books, magnetic tapes with printed texts thereon

Abstract

A computing device may provide a visual cue to items of content (for example, words in a book) synchronized with the playback of companion content (for example, audio content corresponding to the book). For example, embodiments of the present disclosure are directed to a content playback synchronization system for use with physical books (or other physical media). In an embodiment, the computing device may display a visual cue (for example, an underline, box, dot, cursor, or the like) to identify a current location in textual content of the physical book corresponding to a current output position of companion audio content. As the audio content is presented (i.e., as it “plays back”), the highlight and/or visual cue may be advanced to maintain synchronization between the output position within the audio content and a corresponding position in the physical textual content.

Claims

1. A computer-implemented method comprising:

under control of one or more computing devices configured with specific computer executable instructions,

causing output of a first portion of a plurality of portions of digital content via a computing device, wherein the digital content corresponds to physical content;

causing a user perceptible cue source to output a user perceptible cue on the physical content at a first position in the physical content corresponding to the first portion of the digital content; and

maintaining synchronous output of the digital content and the user perceptible cue on the physical content based at least in part on content synchronization information that associates each portion of the plurality of portions of the digital content with a respective position in the physical content.

2. The computer-implemented method of claim 1, wherein maintaining synchronous output of the digital content and the user perceptible cue on the physical content comprises:

causing output of a second portion of the plurality of portions of the digital content; and

causing the user perceptible cue source to output the user perceptible cue on the physical content at the respective position in the physical content that is associated, by the content synchronization information, with the second portion of the digital content.

3. The computer-implemented method of claim 2, further comprising:

under control of the one or more computing devices configured with specific computer executable instructions,

detecting an end of a section of the physical content;

determining that the user perceptible cue is output on the physical content at a position in the physical content that corresponds to the end of the section of the physical content; and

suspending output of the digital content.

4. The computer-implemented method of claim 1, wherein the digital content comprises at least one of an audio book, a song, a video, a television program, a video game, or a movie.

5. The computer-implemented method of claim 1, wherein the physical content comprises at least one of a book, a newspaper, a magazine, a newsletter, or a computing device displaying content.

6. The computer-implemented method of claim 1, wherein the user perceptible cue is at least one of a visual cue or a tactile cue.

7. The computer-implemented method of claim 6, wherein the user perceptible cue is a visual cue comprising at least one of an underline, a box, a circle, a cursor, or a highlight.

8. The computer implemented method of claim 6, wherein the user perceptible cue is a tactile cue comprising at least one of electrical pulses or thermopneumatic actuators.

9. The computer-implemented method of claim 1, further comprising:

under control of the one or more computing devices configured with specific computer executable instructions,

identifying the physical content by at least one of a cover, a title page, an ISBN, a barcode, an embedded electronic device, a format, or a unique identifier; and

retrieving the digital content corresponding to the physical content based on the identification of the physical content.

10. The computer-implemented method of claim 1, wherein the corresponding digital content comprises captured audio of a user reading the physical content.

11. A device for synchronizing physical content with the output of corresponding content, the device comprising:

a user perceptible cue source configured to output a user perceptible cue on portions of physical content;

a content output device configured to output electronic content that corresponds to the physical content; and

a processor in communication with the content output device and the user perceptible cue source, the processor operative to;

cause the content output device to output a portion of the electronic content;

identify, based at least in part on content synchronization information, a portion of the physical content corresponding to the portion of the electronic content output by the content output device; and

cause the user perceptible cue source to output a user perceptible cue on the portion of the physical content corresponding to the portion of the electronic content output by the content output device.

12. The device of claim 11, wherein the processor is further operative to:

cause the content output device to output a next portion of the electronic content;

identify, based at least in part on the content synchronization information, a next portion of the physical content corresponding to the next portion of the electronic content output by the content output device; and

cause the user perceptible cue source to output the user perceptible cue on the next portion of the physical content corresponding to the next portion of the electronic content output by the content output device.

13. The device of claim 11, wherein the user perceptible cue source identifies the portion of the physical content by least one of underlining the portion of the physical content, boxing the portion of the physical content, circling the portion of the physical content, pointing to the portion of the physical content, illuminating the portion of the physical content, or obscuring another portion of the physical content.

14. The device of claim 13, wherein the portion of the physical content comprises at least one of a word, a syllable, a phoneme, a morpheme, a letter, a sentence, a line, a paragraph, a chapter, a stanza, a section, a column, a picture, a table, or an equation.

15. The device of claim 11, further comprising:

an image capture device configured to capture images of the physical content,

wherein the processor is in communication with the image capture device, and wherein the processor is further operative to;

cause the image capture device to capture an image of a section of the physical content; and

determine content detection information based at least in part on the captured image of the section of the physical content.

16. The device of claim 15, wherein the processor is further operative to determine at least one of word boundaries, word identities, or spatial coordinates of words.

17. The system of claim 15, wherein the processor is further operative to:

identify the physical content by at least one of a captured cover, a captured title page, a captured ISBN, a captured barcode, an embedded electronic device, a captured format, or a unique identifier; and

retrieve the electronic content based on the identification of the physical content.

18. The device of claim 11, wherein the electronic content comprises at least one of analog content and digital content.

19. A computer-readable, non-transitory storage medium having at least one computer-executable component for synchronizing output of digital content with corresponding physical content, the at least one computer-executable component comprising:

a content synchronization module operative to;

cause output of a first portion of digital content via a computing device, wherein the digital content corresponds to physical content;

cause a user perceptible cue source to output a user perceptible cue on the physical content at a first position in the physical content, wherein the first position in the physical content corresponds to the first portion of the digital content; and

maintain synchronous presentation output of the digital content and the user perceptible cue on the physical content based at least in part on content synchronization information that associates each portion of a plurality of portions of the digital content with a position in the physical content.

20. The computer-readable, non-transitory storage medium of claim 19, wherein the content synchronization module is operative to maintain synchronous output of the digital content and the user perceptible cue by:

causing output of a second portion of the digital content;

obtaining, from the content synchronization information, a second position in the physical content that is associated with the second portion of the digital content; and

causing the user perceptible source cue to output the user perceptible cue on the physical content at the second position in the physical content obtained from the content synchronization information.

21. The computer-readable, non-transitory storage medium of claim 20, wherein the content synchronization module is further operative to:

cause an image capture device to capture an image of the physical content;

detect an end of a section of the physical content based at least in part on the captured image;

determine that the second position in the physical content corresponds to the end of the section of the physical content; and

suspend output of the digital content.

22. The computer-readable, non-transitory storage medium of claim 21, wherein the content synchronization module is further operative to:

detect a page turn, wherein the content synchronization information is determined after the page turn is detected.

23. A system for providing content synchronization information, the system comprising:

a data store that stores content information and content synchronization information;

one or more configured computing devices in communication with the data store that are operative to;

receive, from a device, an image of a section of physical content, wherein the device is configured to;

capture images of physical content,

output digital content, and

output a user perceptible cue on the physical content using synchronization information associated with both the physical content and the digital content;

identify the physical content from the image of the section of the physical content;

retrieve digital content corresponding to the identified physical content from the content information in the data store;

retrieve synchronization information from the content synchronization information in the data store, the synchronization information associated with both the physical content and the digital content; and

cause transmission of the digital content and the synchronization information from the data store to the device via a network.

24. The system of claim 23, wherein the one or more configured computing devices are further operative to:

determine content detection information from the image of the section of the physical content; and

cause transmission of the content detection information to the device via the network,

wherein the content detection information comprises at least one of;

word boundaries, word identities, and spatial coordinates of words.