Interactive multimedia book

• US 6,728,681 B2
• Filed: 01/05/2001
• Issued: 04/27/2004
• Est. Priority Date: 01/05/2001
• Status: Active Grant

• Current Assignee: Whitham Holdings LLC
• Original Assignee: Charles L. Whitham
• Inventors: Whitham, Charles L.
• Primary Examiner(s): Dorvil, Richemond
• Assistant Examiner(s): Nolan, Daniel A.
• Application Number:
  US09/754,053
• Publication Number:
  US 20020091529A1
• Time in Patent Office: 1,208 Days
• Field of Search: 704/3, 704/10, 704/271, 704/255, 704/270.1, 704/270, 704/233, 704/275, 704/260, 704/231, 704/251, 701/211, 434/317, 345/781, 345/784, 707/10
• US Class Current: 704/275
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: H04M 3/4938 comprising a voice browser which renders and interprets, e.g. VoiceXML

Abstract

An interactive multimedia book provides hands-on multimedia instruction to the user in response to voiced commands. The book is implemented on a computer system and includes both text and audio/video clips. The interactive multimedia book is accessed by voiced commands and natural language queries as the primary user input. The displayed text is written in a markup language and contains hyperlinks which link the current topic with other related topics. The user may command the book to read the text and, as the text is read by the voice synthesizer, a word which is also a hyperlink will change its attributes upon being spoken. The user will be able to observe or hear this and simply utter the word which is the hyperlink to navigate to the linked topic.

Claims

1. A method of presenting an interactive multimedia book on a computer system comprising the steps of:

recognizing voiced commands spoken by a user of the book;

responding to a voiced command to read text on a displayed page of the book by processing text on the displayed page of the book and synthesizing an audible output corresponding to the processed text;

identifying words which are active hyperlinks in the text on a displayed page of the book as the text is being read;

activating at least one attribute of a word identified as an active hyperlink to indicate to the user of the book that the word is a hyperlink;

recognizing a voiced word spoken by the user of the book as an active hyperlink; and

responding to a voiced word which is an active hyperlink by moving to another page of the book which contains an anchor for the hyperlink.

2. The method of presenting an interactive multimedia book on a computer system recited in claim 1, further comprising the steps of:

displaying an icon representative of a video clip on the page of the book which contains an anchor for the hyperlink;

recognizing a voiced command to play the video clip; and

responding to the voiced command to play the video clip by playing the video clip.

3. The method of presenting an interactive multimedia book on a computer system recited in claim 2, further comprising the step of:

providing a user interface which allows the user to pause, resume and stop the playing of the video clip;

responding to a user input to pause the video clip by halting the playing of the video clip;

responding to a user input to resume playing the video clip by playing the video clip from a point at which the video clip was halted; and

responding to a user input to stop playing the video clip by stopping the playing of the video clip and returning to a main program.

4. The method of presenting an interactive multimedia book on a computer system recited in claim 3, wherein the step of providing a user interface is by means of a single button which when pressed once pauses the playing of the video clip if the video clip is playing, when pressed again resumes the playing of the video clip if the video clip is halted, and when pressed twice in succession causes the playing of the video clip to be stopped.

5. The method of presenting an interactive multimedia book on a computer system recited in claim 1, wherein said at least one attribute of a word identified as an active hyperlink is a change in color, the word on the displayed page changing color when read.

6. The method of presenting an interactive multimedia book on a computer system recited in claim 1, wherein said at least one attribute of a word identified as an active hyperlink is a sound, the sound being emitted when the word on the displayed page is read.

7. The method of presenting an interactive multimedia book on a computer system recited in claim 1, wherein said at least one attribute of a word identified as an active hyperlink is a change in color and a sound, the sound being emitted and the word on the displayed page changing color when read.

8. The method of presenting an interactive multimedia book on a computer system recited in claim 1, wherein the voiced commands spoken by the user of the book further include a command to list active hyperlinks on a displayed page.

9. The method of presenting an interactive multimedia book on a computer system recited in claim 1, wherein the voiced commands spoken by the user of the book further include a command to display an image of a completed project described on a currently displayed page of the book.

10. A method of presenting an interactive multimedia book on a computer system comprising the steps of:

recognizing a voiced word spoken by the user of the book as an active hyperlink;

responding to a voiced word which is an active hyperlink by first storing a current page number in a last in, first out register before moving to a second page of the book which contains an anchor for the hyperlink, the current page being an origin page, and then moving to said second page of the book which contains the anchor for the hyperlink;

recognizing a voiced command spoken by the user of the book to return to the origin page;

responding to the voiced command to return to the origin page by moving back to the origin page;

recognizing a natural language query spoken by the user of the book;

analyzing the natural lananage query; and

responding to the natural language query, wherein the natural language query includes “

WHAT”

, the step of analyzing the natural language query analyzing a word or words following the word “

WHAT”

, and the step of responding to the natural language query includes the steps of displaying and synthesizing an audible output which is a definition of the word or words following the word “

WHAT”

.

11. A method of presenting an interactive multimedia book on a computer system comprising the steps of:

recognizing a voiced word spoken by the user of the book as an active hyperlink;

responding to a voiced word which is an active hyperlink by first storing a current page number in a last in, first out register before moving to a second page of the book which contains an anchor for the hyperlink, the current page being an origin page, and then moving to said second page of the book which contains the anchor for the hyperlink;

recognizing a voiced command spoken by the user of the book to return to the origin page;

responding to the voiced command to return to the origin page by moving back to the origin page;

recognizing a natural language query spoken by the user of the book;

analyzing the natural language query; and

responding to the natural language query, wherein the natural language query includes “

MAKE”

, the step of analyzing the natural language query analyzing a word or words following the word “

MAKE”

, and the step of responding to the natural language query includes the step of moving to a section of the book which covers a project or recipe corresponding to the word or words following the word “

MAKE”

.

12. A method of presenting an interactive multimedia book on a computer system comprising the steps of:

recognizing a voiced word spoken by the user of the book as an active hyperlink;

responding to a voiced word which is an active hyperlink by first storing a current page number in a last in, first out register before moving to a second page of the book which contains an anchor for the hyperlink, the current page being an origin page, and then moving to said second page of the book which contains the anchor for the hyperlink;

recognizing a voiced command spoken by the user of the book to return to the origin page;

responding to the voiced command to return to the origin page by moving back to the origin page;

recognizing a natural language query spoken by the user of the book;

analyzing the natural language query; and

responding to the natural language query, wherein the natural language query includes “

HOW”

, the step of analyzing the natural language query analyzing a word or words following the word “

HOW”

, and the step of responding to the natural language query includes the step of moving to a section of the book which covers a technique corresponding to the word or words following the word “

HOW”

.

13. A method of presenting an interactive multimedia book on a computer system comprising the steps of:

recognizing a voiced word spoken by the user of the book as an active hyperlink;

responding to a voiced word which is an active hyperlink by first storing a current page number in a last in, first out register before moving to a second page of the book which contains an anchor for the hyperlink, the current page being an origin page, and then moving to said second page of the book which contains the anchor for the hyperlink;

recognizing a voiced command spoken by the user of the book to return to the origin page;

responding to the voiced command to return to the origin page by moving back to the origin page;

recognizing a natural language query spoken by the user of the book;

analyzing the natural language query; and

responding to the natural language query, wherein the natural language query includes “

GO”

, the step of analyzing the natural language query analyzing a word or words following the word “

GO”

, and the step of responding to the natural language query includes the step of moving to a page, a section of the book which covers a project or recipe or a technique corresponding to the word or words following the word “

GO”

.

14. A machine readable medium containing computer code for presenting an interactive multimedia book on a computer, the computer code performing the steps of:

recognizing voiced commands spoken by a user of the book;

responding to a voiced command to read text on a displayed page of the book by text on the displayed page of the book and synthesizing an audible output of the text;

identifying words which are active hyperlinks in the text on a displayed page of the book as the text is being read;

activating at least one attribute of a word identified as an active hyperlink to indicate to the user of the book that the word is a hyperlink;

recognizing a voiced word spoken by the user of the book as an active hyperlink; and

responding to a voiced word which is an active hyperlink by moving to another page of the book which contains an anchor for the hyperlink.

15. The machine readable medium containing computer code for presenting an interactive multimedia book on a computer system recited in claim 14, the computer code further performing the steps of:

displaying an icon representative of a video clip on the page of the book which contains an anchor for the hyperlink;

recognizing a voiced command to play the video clip; and

responding to the voiced command to play the video clip by playing the video clip.

16. The machine readable medium containing computer code for presenting an interactive multimedia book on a computer system recited in claim 15, the computer code further performing the steps of:

providing a user interface which allows the user to pause, resume and stop the playing of the video clip;

responding to a user input to pause the video clip by halting the playing of the video clip;

responding to a user input to resume playing the video clip by playing the video clip from a point at which the video clip was halted; and

responding to a user input to stop playing the video clip by stopping the playing of the video clip and returning to a main program.

17. The machine readable medium containing computer code for presenting an interactive multimedia book on a computer system recited in claim 16, wherein the code that performs the step of providing a user interface responds to a single button which when pressed once pauses the playing of the video clip if the video clip is playing, when pressed again resumes the playing of the video clip if the video clip is halted, and when pressed twice in succession causes the playing of the video clip to be stopped.

18. The machine readable medium containing computer code for presenting an interactive multimedia book on a computer system recited in claim 14, wherein said at least one attribute of a word identified as an active hyperlink is implemented in code as a change in color, the word on the displayed page changing color when read.

19. The machine readable medium containing computer code for presenting an interactive multimedia book on a computer system recited in claim 14, wherein said at least one attribute of a word identified as an active hyperlink is implemented in code as a sound, the sound being emitted when the word on the displayed page is read.

20. The machine readable medium containing computer code for presenting an interactive multimedia book on a computer system recited in claim 14, wherein said at least one attribute of a word identified as an active hyperlink is implemented in code as a change in color and a sound, the sound being emitted and the word on the displayed page changing color when read.

21. The machine readable medium containing computer code for presenting an interactive multimedia book on a computer system recited in claim 14, wherein the code further implements the steps of:

storing a current page number in a last in, first out register before moving to another page of the book which contains an anchor for the hyperlink, the current page being the origin page;

recognizing a voiced command spoken by the user of the book to return to the origin page; and

responding to the voiced command to return to the origin page by moving back to the origin page.

22. The machine readable medium containing computer code for presenting an interactive multimedia book on a computer system recited in claim 14, wherein the code further implements the steps of:

recognizing a natural language query spoken by the user of the book;

analyzing the natural language query; and

responding to the natural language query.

23. The machine readable medium containing computer code for presenting an interactive multimedia book on a computer system recited in claim 22, wherein the natural language query includes “

WHAT”

, the code implementing the step of analyzing the natural language query analyzing a word or words following the word “

WHAT”

, and the code implementing the step of responding to the natural language query by displaying and synthesizing an audible output which is a definition of the word or words following the word “

WHAT”

.

24. The machine readable medium containing computer code for presenting an interactive multimedia book on a computer system recited in claim 22, wherein the natural language query includes “

MAKE”

, the code implementing the step of analyzing the natural language query analyzing a word or words following the word “

MAKE”

, and the code implementing the step of responding to the natural language query moves to a section of the book which covers a project or recipe corresponding to the word or words following the word “

MAKE”

.

25. The machine readable medium containing computer code for presenting an interactive multimedia book on a computer system recited in claim 22, wherein the natural language query includes “

HOW”

, the code implementing the step of analyzing the natural language query analyzing a word or words following the word “

HOW”

, and the code implementing the step of responding to the natural language query moves to a section of the book which covers a technique corresponding to the word or words following the word “

HOW”

.

26. The machine readable medium containing computer code for presenting an interactive multimedia book on a computer system recited in claim 22, wherein the natural language query includes “

GO”

, the code implementing the step of analyzing the natural language query analyzing a word or words following the word “

GO”

, and the code implementing the step of responding to the natural language query moves to a page or a section of the book which covers a project or recipe or a technique corresponding to the word or words following the word “

GO”

.

27. The machine readable medium containing computer code for presenting an interactive multimedia book on a computer system recited in claim 22, wherein the voiced commands spoken by the user of the book and recognized by the code further include a command to list active hyperlinks on a displayed page.

28. The machine readable medium containing computer code for presenting an interactive multimedia book on a computer system recited in claim 22, wherein the voiced commands spoken by the user of the book and recognized by the code further include a command to display an image of a completed project described on a currently displayed page of the book.