Auditory enhancement using word analysis

• US 9,575,960 B1
• Filed: 09/17/2012
• Issued: 02/21/2017
• Est. Priority Date: 09/17/2012
• Status: Expired due to Fees

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Lerner, David M., Smith, Brandon J., Ducrou, Jon Robert, Miller, Erik J., Barry, Marcus A., Sanders, Kenneth O. II
• Primary Examiner(s): Desir, Pierre-Louis
• Assistant Examiner(s): Thomas-Homescu, Anne
• Application Number:
  US13/621,806
• Time in Patent Office: 1,618 Days
• Field of Search: 707/706, 704/9, 709/204, 715/863, 715/256, 715/785, 715/201, 715/776, 715/202, 345/8, 345/173, 345/158, 345/684, 345/157, 345/619, 345/156, 345/168, 345/163, 345/661, 345/419, 345/473, 348/78, 348/222.1, 348/211.1
• US Class Current: 1/1
• CPC Class Codes: G06F 40/30 Semantic analysis

Abstract

One or more words at a specified location in an electronic document can be identified. The identified one or more words can be analyzed to determine one or more semantic meanings associated with the words. An audio clip (i.e., audio file, audio element) associated with or corresponding to (the semantic meaning(s) of) the one or more words can be searched for in an audio database. The search for the audio clip associated with the one or more words can utilize an index that specifies the associations between words and audio clips. In some embodiments, the audio clip can be played when an estimated location of where the user is reading is at or near the specified location of the one or more words. In some embodiments, the audio clip can be played when it is calculated that the user is reading the one or more words at the specified location.

Claims

1. A computer-implemented method comprising:

displaying an electronic document on a display of a computing device;

determining a semantic meaning associated with one or more words in the electronic document, the one or more words being located at a specific location of the electronic document being displayed on the computing device;

identifying an audio file corresponding to the semantic meaning;

detecting a gaze direction of a user of the computing device, the gaze direction being relative to the display of the computing device;

determining an estimated location on the display that corresponds to the gaze direction;

determining that the estimated location is within a threshold distance from the specific location of the one or more words; and

playing the audio file with a volume based at least in part upon the semantic meaning of the one or more words and determining that the estimated location is within the threshold distance from the specific location of the one or more words.

2. The computer-implemented method of claim 1, wherein the audio file comprises at least one of a song, a poem, a soundtrack, background music, an ambient sound, or a sound effect, and wherein the electronic document is at least one of an electronic book, a word processing file, a slideshow file, or an electronic message.

3. The computer-implemented method of claim 1, wherein determining the semantic meaning includes determining at least one of a definition, feeling, emotion, sound, or mood associated with the one or more words.

4. A computer-implemented method, comprising:

identifying a semantic meaning associated with one or more words in an electronic document being displayed on a display of a computing device, the one or more words being located at a specific location of the electronic document;

identifying an audio file associated with the semantic meaning;

determining a gaze direction of a user relative to the display;

determining an estimated location on the display corresponding to the gaze direction;

determining that the estimated location is within a threshold distance from the specific location of the one or more words; and

playing the audio file with a volume based at least in part upon the semantic meaning of the one or more words and determining that the estimated location is within the threshold distance from the specific location of the one or more words.

5. The computer-implemented method of claim 4, further comprising:

capturing an image including a representation of one or more eyes of the user using a camera on the computing device; and

analyzing the image including the representation of the one or more eyes.

6. The computer-implemented method of claim 4, wherein monitoring the user interaction includes:

estimating a reading pace associated with the user; and

detecting a page turn command input by the user,

wherein determining the estimated location is based at least in part upon the page turn command and the reading pace.

7. The computer-implemented method of claim 4, wherein the semantic meaning corresponds to a mood, and wherein the audio file is identified based on the mood.

8. The computer-implemented method of claim 4, further comprising:

receiving feedback relating to the audio file.

9. The computer-implemented method of claim 4, wherein the audio file is at least one of a song, a poem, a soundtrack, background music, an ambient sound, or a sound effect, and wherein the electronic document is at least one of an electronic book, a word processing file, a slideshow file, or an electronic message.

10. The computer-implemented method of claim 9, wherein the electronic document is an electronic book relating to a movie and wherein the audio file is a soundtrack relating to the movie.

11. The computer-implemented method of claim 4, further comprising:

audibly outputting content of the electronic document, including the one or more words, wherein the electronic document is an electronic audio book, and wherein playing the audio file is further based at least in part upon the one or more words being audibly outputted.

12. The computer-implemented method of claim 6, wherein estimating the reading pace of the user is based at least in part upon historical data regarding a time it took for the user to turn one or more pages in one or more previously read electronic documents.

13. The computer-implemented method of claim 6, wherein estimating the reading pace of the user is based at least in part upon a time it took for the user to turn one or more previous pages in the electronic document.

14. The computer-implemented method of claim 4, further comprising:

receiving a type of audio content for the audio file, wherein the audio file is identified from a plurality of audio files corresponding to the type of audio content.

15. A system comprising:

a display;

a processor; and

a memory device including instructions that, when executed by the processor, cause the system to;

identify a semantic meaning associated with one or more words in an electronic document being displayed on the display, the one or more words being located at a specific location of the electronic document;

identify an audio file associated with the semantic meaning;

determine a gaze direction of a user relative to the display;

determine an estimated location on the display corresponding to the gaze direction;

determine that the estimated location is within a threshold distance from the specific location of the one or more words; and

play the audio file with a volume based at least in part upon the semantic meaning of the one or more words and a determination that the estimated location is within the threshold distance from the specific location of the one or more words.

16. The system of claim 15, further comprising:

at least one camera, wherein the instructions when executed further cause the system to;

capture an image including a representation one or more eyes of the user using the at least one camera; and

analyze the image including the representation of the one or more eyes.

17. The system of claim 15, wherein the instructions when executed further cause the system to:

estimate a reading pace associated with the user; and

detect a page turn command input by the user,

wherein the estimated location is determined based at least in part upon the page turn command and the reading pace.

18. The system of claim 17, wherein the reading pace associated with the user is estimated based at least in part upon historical data regarding a time it took for the user to turn one or more pages in one or more previously read electronic documents.

19. The system of claim 17, wherein the reading pace associated with the user is estimated based at least in part upon a time it took for the user to turn one or more previous pages in the electronic document.

20. The system of claim 15, wherein the audio file is at least one of a song, a poem, a soundtrack, background music, an ambient sound, or a sound effect, and wherein the electronic document is at least one of an electronic book, a word processing file, a slideshow file, or an electronic message.

21. A non-transitory computer-readable storage medium including instructions for identifying elements, the instructions when executed by a processor of a computing device causing the computing device to:

identify a semantic meaning associated with one or more words in an electronic document being displayed on a display of the computing device, the one or more words being located at a specific location of the electronic document;

identify an audio file associated with the semantic meaning;

determine a gaze direction of a user relative to the display;

determine an estimated location on the display corresponding to the gaze direction;

determine that the estimated location is within a threshold distance from the specific location of the one or more words; and

play the audio file with a volume based at least in part upon the semantic meaning of the one or more words and a determination that the estimated location is within the threshold distance from the specific location of the one or more words.

22. The non-transitory computer-readable storage medium of claim 21, wherein the instructions when executed cause the computing device to further:

receive a type of audio content for the audio file, wherein the audio file is identified from a plurality of audio files corresponding to the type of audio content.

23. The non-transitory computer-readable storage medium of claim 21, wherein the instructions when executed cause the computing device to further:

receive feedback relating to the audio file.