Method and system for reading fluency training

• US 9,424,834 B2
• Filed: 09/06/2013
• Issued: 08/23/2016
• Est. Priority Date: 09/06/2012
• Status: Active Grant

• Current Assignee: Rosetta Stone Limited (Rosetta Stone Incorporated)
• Original Assignee: Rosetta Stone Limited (Rosetta Stone Incorporated)
• Inventors: Simmons, Aaron M., Pellom, Bryan, Ridgeway, Karl F.
• Primary Examiner(s): Saint Cyr, Leonard
• Application Number:
  US14/020,385
• Publication Number:
  US 20140067367A1
• Time in Patent Office: 1,082 Days
• Field of Search: 704 7- 9, 704270-271, 704246-252
• US Class Current: 1/1
• CPC Class Codes: G09B 17/003 electrically operated apparatus or devices
• CPC Class Codes: G09B 17/02 Line indicators or other guides or masks
• CPC Class Codes: G09B 19/04 Speaking with audible presentation of the material to be studied G09B5/04
• CPC Class Codes: G09B 19/06 Foreign languages with audible presentation of material to be studied G09B5/04
• CPC Class Codes: G10L 13/08 Text analysis or generation of parameters for speech synthesis out of text, e.g. grapheme to phoneme translation, prosody generation or stress or intonation determination
• CPC Class Codes: G10L 15/005 Language recognition
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/30 Distributed recognition, e.g. in client-server systems, for mobile phones or network applications

Abstract

A non-transitory processor-readable medium stores code representing instructions to be executed by a processor. The code causes the processor to receive a request from a user of a client device to initiate a speech recognition engine for a web page displayed at the client device. In response to the request, the code causes the processor to (1) download, from a server associated with a first party, the speech recognition engine into the client device; and then (2) analyze, using the speech recognition engine, content of the web page including text in an identified language to produce analyzed content based on the identified language, where the content of the web page is received from a server associated with a second party. The code further causes the processor to send a signal to cause the client device to present the analyzed content to the user at the client device.

Claims

1. A non-transitory processor-readable medium storing code representing instructions to be executed by a processor, the code comprising code to cause the processor to:

receive a first request from a user of a client device, the first request being to initiate a speech recognition engine for a first web page displayed at the client device;

download, in response to the first request and from a server associated with a first party, the speech recognition engine to the client device;

analyze, using the speech recognition engine and in response to the first request, content of the first web page including text in a first identified language to produce first analyzed content based on the first identified language, the content of the first web page being received from a server associated with a second party different from the first party;

send a first signal, the first signal to cause the client device to present the first analyzed content at the client device;

receive a second request, the second request being to initiate the speech recognition engine for a second web page displayed at the client device;

analyze, using the speech recognition engine and in response to the second request, content of the second web page including text in a second identified language to produce second analyzed content based on the second identified language, the content of the second web page being received from a server associated with a third party different from the first party and the second party; and

send a second signal, the second signal to cause the client device to present the second analyzed content at the client device.

2. The non-transitory processor-readable medium of claim 1, wherein the code to cause the processor to download includes code to cause the processor to:

download, via a network and into a browser module at the client device, the speech recognition engine using a bookmarklet, the browser module configured to present the first web page at the client device.

3. The non-transitory processor-readable medium of claim 1, wherein the code to cause the processor to download includes code to cause the processor to:

download, via a network and into a browser module at the client device, the speech recognition engine using a bookmarklet such that the first request includes an instruction to initiate the bookmarklet when the first web page is presented by the browser module.

4. The non-transitory processor-readable medium of claim 1, wherein the code to cause the processor to analyze content of the first web page is not executed outside the client device.

5. The non-transitory processor-readable medium of claim 1, wherein the speech recognition engine is not executed at the server associated with the first party.

6. The non-transitory processor-readable medium of claim 1, wherein the server associated with the first party is remote from the client device and the server associated with the second party.

7. The non-transitory processor-readable medium of claim 1, wherein the content of the first web page includes text associated with audio in the first identified language, the code to cause the processor to analyze content of the first web page including code to cause the processor to map a set of spoken words from the audio of the first web page to a set of textual words from the text of the first web page such that the first signal is configured to cause the client device to visually highlight, at a display screen of the client device, each word from the set of textual words when an associated spoken word from the set of spoken words is output at an audio output device associated with the client device.

8. The non-transitory processor-readable medium of claim 1, wherein the speech recognition engine utilizes an acoustic model and a pronunciation lexicon for the first identified language to analyze the content of the first web page.

9. The non-transitory processor-readable medium of claim 1, wherein the code to cause the processor to analyze content of the first web page is not executed prior to the first request.

10. A method, comprising:

receiving, at a speech recognition engine on a client device, content of a first web page in response to a first request at the client device to execute the speech recognition engine, the content of the first web page including text in a first identified language, the first web page being presented to the user on a display screen of the client device;

determining, via the speech recognition engine, a set of character indices related to the text of the first web page when the first web page is presented at the client device;

mapping, via the speech recognition engine, a first set of spoken words to a set of textual words in the text of the first web page based on the set of character indices related to the text of the first web page, the first set of spoken words being associated with the content of the first web page;

sending a signal to cause the client device to visually highlight, on the display screen presenting the first web page, each word from the set of textual words in the text of the first web page when an associated spoken word from the first set of spoken words is output at an audio device associated with the client device;

receiving, at the speech recognition engine, content of a second web page in response to a second request at the client device to execute the speech recognition engine, the content of the second web page including text in a second identified language, the second web page being presented on the display screen of the client device;

determining, via the speech recognition engine, a set of character indices related to the text of the second web page when the second web page is presented at the client device;

mapping, via the speech recognition engine, a second set of spoken words to a set of textual words in the text of the second web page based on the set of character indices related to the text of the second web page, the second set of spoken words being associated with the content of the second web page; and

sending a signal to cause the client device to visually highlight, on the display screen presenting the second web page, each word from the set of textual words in the text of the second web page when an associated spoken word from the second set of spoken words is output at the audio device.

11. The method of claim 10, wherein the mapping the first set of spoken words occurs when the first web page is presented at the client device.

12. The method of claim 10, wherein each character index from the set of character indices related to the text of the first web page is a plain text character index, the method further comprising:

defining a data structure configured to map the set of character indices related to the text of the first web page to a set of Document Object Model (DOM) element indices associated with the first web page.

13. The method of claim 10, wherein the mapping the first set of spoken words includes identifying, via the speech recognition engine, each spoken word from the first set of spoken words as that spoken word is output at or produced by the audio device.

14. The method of claim 10, wherein the audio device is an audio input device, the sending a signal to cause the client device to visually highlight each word from the set of textual words in the text of the first web page includes sending the signal to cause the client device to visually highlight, on the display screen presenting the first web page, each word from the set of textual words in the text of the first web page when the user produces an associated spoken word from the first set of spoken words using the audio device.

15. A method, comprising:

receiving a first request from a user of a client device, the first request being a request to execute, at the client device, a speech recognition engine for a first web page when the first web page is displayed at the client device, the first web page hosted by a server associated with a first party, the first web page including text in a first language;

receiving, at the speech recognition engine, a first audio file from a database maintained by a second party in response to the receiving the first request, the first audio file being linked to the first web page by the database and including a set of spoken words in the first language corresponding, at least in part, to the text of the first web page;

sending a signal to cause the client device to output an audio recording from the first audio file as the speech recognition engine processes, at the client device, the text of the first web page and the first audio file;

receiving a second request from the user, the second request being a request to execute, at the client device the speech recognition engine for a second web page when the second web page is displayed at the client device, the second web page hosted by a server associated with a third party, the second web page including text in a second language different from the first language;

receiving, at the speech recognition engine, a second audio file from the database and in response to receiving the second request, the second audio file being linked to the second web page by the database and including a set of spoken words in the second language corresponding, at least in part, to the text of the second web page; and

sending a signal to cause the client device to output an audio recording from the second audio file as the speech recognition engine processes, at the client device, the text of the second web page and the second audio file.

16. The method of claim 15, further comprising:

sending a signal to cause the client device to visually highlight each word of the text of the first web page that corresponds to a spoken word from the first audio file when that spoken word is output from an audio output device associated with the client device.

17. The method of claim 15, wherein the first party is different from and not affiliated with the second party or the third party.

18. The method of claim 15, wherein the user is a first user, the set of spoken words in the first audio file is spoken by a second user associated with the second party when the first web page is presented to the second user.

19. The method of claim 15, wherein the receiving the first request, the receiving the first audio file and the sending the signal to cause the client device to output the audio recording from the first audio file occur when the first web page is presented via a browser module on the client device.