Interactive multimedia book

• US 20040111270A1
• Filed: 12/01/2003
• Published: 06/10/2004
• Est. Priority Date: 01/05/2001
• Status: Active Grant

• Current Assignee: Whitham Holdings LLC
• Original Assignee: Charles Lamont Whitham
• Inventors: Whitham, Charles Lamont
• Granted Patent: US 7,136,819 B2
• Time in Patent Office: Days
• US Class Current: 704/275
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: H04M 3/4938 comprising a voice browser which renders and interprets, e.g. VoiceXML

Abstract

An interactive multimedia book provides hands-on multimedia instruction to the user in response to voiced commands. The book is implemented on an easy to use computer system which is suitable to various environments in which the book might be used. The interactive multimedia book is published on a computer readable medium with the necessary software to support the interactive operation of the book. Alternatively, the book may be downloaded form a remote site using a network, such as the Internet, in which case the content of the book and the necessary software are copied to a local medium, such as a computer hard disk. The content includes both text and audio/video clips. The interactive multimedia book is accessed by a computer system which is equipped with a microphone and voice recognition software. Voiced commands and natural language queries are the primary user input to the computer system. The computer system is also equipped with a high resolution display, a voice synthesizer and a speaker or headphone system to provide output to the user. A combination headphone and directional microphone can be especially convenient in some environments as, for example, the wood shop where the headphones allow the user to better hear the instruction over the din of machine noise while at the same time protecting the user'"'"'s hearing. The displayed text is written in a markup language, such as HyperText Markup Language (HTML), and contains hyperlinks which link the current topic with other related topics. The user may command the book to read the text and, as the text is read by the voice synthesizer, a word which is also a hyperlink will change its attributes upon being spoken. The user will be able to observe or hear this and, without having to click a mouse button, simply utter the word which is the hyperlink to navigate to the linked topic.

Claims

1. A method of presenting an interactive multimedia book on a computer system comprising the steps of:

recognizing voiced commands spoken by a user of the book;

responding to a voiced command to read text on a displayed page of the book by reading text on the displayed page of the book;

identifying words which are active hyperlinks in the text on a displayed page of the book as the text is being read;

activating at least one attribute of a word identified as an active hyperlink to indicate to the user of the book that the word is a hyperlink;

recognizing a voiced word spoken by the user of the book as an active hyperlink; and

responding to a voiced word which is an active hyperlink by moving to another page of the book which contains an anchor for the hyperlink.

2. The method of presenting an interactive multimedia book on a computer system recited in claim 1, further comprising the steps of:

displaying an icon representative of a video clip on the page of the book which contains an anchor for the hyperlink;

recognizing a voiced command to play the video clip; and

responding to the voiced command to play the video clip by playing the video clip.

3. The method of presenting an interactive multimedia book on a computer system recited in claim 2, further comprising the step of:

providing a user interface which allows the user to pause, resume and stop the playing of the video clip;

responding to a user input to pause the video clip by temporarily halting the playing of the video clip;

responding to a user input to resume playing the video clip by playing the video clip from a point at which the video clip was temporarily halted; and

responding to a user input to stop playing the video clip by stopping the playing of the video clip and returning to a main program.

4. The method of presenting an interactive multimedia book on a computer system recited in claim 3, wherein the step of providing a user interface is by means of a single button which when pressed once pauses the playing of the video clip if the video clip is playing, when pressed again resumes the playing of the video clip if the video clip is temporarily halted, and when pressed twice in succession causes the playing of the video clip to be stopped.

5. The method of presenting an interactive multimedia book on a computer system recited in claim 1, wherein said at least one attribute of a word identified as an active hyperlink is a change in color, the word on the displayed page changing color when read.

6. The method of presenting an interactive multimedia book on a computer system recited in claim 1, wherein said at least one attribute of a word identified as an active hyperlink is a sound, the sound being emitted when the word on the displayed page is read.

7. The method of presenting an interactive multimedia book on a computer system recited in claim 1, wherein said at least one attribute of a word identified as an active hyperlink is a change in color and a sound, the sound being emitted and the word on the displayed page changing color when read.

8. The method of presenting an interactive multimedia book on a computer system recited in claim 1, wherein the voiced commands spoken by the user of the book further include a command to list active hyperlinks on a displayed page.

9. The method of presenting an interactive multimedia book on a computer system recited in claim 1, wherein the voiced commands spoken by the user of the book further include a command to display an image of a completed project described on a currently displayed page of the book.

10. A method of presenting an interactive multimedia book on a computer system comprising the steps of:

recognizing a voiced word spoken by the user of the book as an active hyperlink;

responding to a voiced word which is an active hyperlink by first storing a current page number in a last in, first out register before moving to another page of the book which contains an anchor for the hyperlink, the current page being an origin page, and then moving to another page of the book which contains an anchor for the hyperlink;

recognizing a voiced command spoken by the user of the book to return to the origin page; and

responding to the voiced command to return to the origin page by moving back to the origin page.

11. The method of presenting an interactive multimedia book on a computer system recited in claim 10, further comprising the steps of:

recognizing a natural language query spoken by the user of the book;

analyzing the natural language query; and

responding to the natural language query.

12. The method of presenting an interactive multimedia book on a computer system recited in claim 11, wherein the natural language query includes “

WHAT”

, the step of analyzing the natural language query analyzing a word or words following the word “

WHAT”

, and the step of responding to the natural language query includes the steps of displaying and speaking a definition of the word or words following the word “

WHAT”

.

13. The method of presenting an interactive multimedia book on a computer system recited in claim 11, wherein the natural language query includes “

MAKE”

, the step of analyzing the natural language query analyzing a word or words following the word “

MAKE”

, and the step of responding to the natural language query includes the step of moving to a section of the book which covers a project or recipe corresponding to the word or words following the word “

MAKE”

.

14. The method of presenting an interactive multimedia book on a computer system recited in claim 11, wherein the natural language query includes “

HOW”

, the step of analyzing the natural language query analyzing a word or words following the word “

HOW”

, and the step of responding to the natural language query includes the step of moving to a section of the book which covers a technique corresponding to the word or words following the word “

HOW”

.

15. The method of presenting an interactive multimedia book on a computer system recited in claim 11, wherein the natural language query includes “

GO”

, the step of analyzing the natural language query analyzing a word or words following the word “

GO”

, and the step of responding to the natural language query includes the step of moving to a page, a section of the book which covers a project or recipe or a technique corresponding to the word or words following the word “

GO”

.

16. A computer system comprising:

a system unit having a generally rectangular shape;

a display hinged at one edge of the system unit, said display being rotated away from the system unit when the computer system is in operation and rotated against the system unit when the computer system is not in operation;

detecting means associated with the display for detecting when a user wants to operate the computer system and when the user wants turn the computer system off; and

an automatic on and off function implemented on the computer system and responsive to said detecting means for turning on said computer system and performing an initial program load of an operating system installed on the computer system when the display is rotated away from the system unit and for initiating a shut down procedure of any open applications and the operating system when the display is rotated against the system unit.

17. The computer system recited in claim 16, wherein the computer system may optionally be placed on a horizontal work surface or mounted under a cabinet or shelf, the computer system further comprising orientation means—

for detecting an—

orientation of the computer system as either on a horizontal work surface or mounted under a cabinet or shelf, said automatic on and off function being further responsive to said orientation means for rotating display information to the display during the initial program load of the operating system so that a displayed image is oriented right side up.

18. The computer system recited in claim 16, wherein the detecting means detects a position of the display as either rotated away from the system unit, indicating that the computer system is to be turned on, or rotated against the system unit, indicating that the computer system is to be turned off.

19. The computer system recited in claim 16, wherein the display is hinged along a front edge of the system unit.

20. The computer system recited in claim 19, further comprising a protective cover hinged along a back edge of the computer system, the protective cover being rotated to cover and protect the display when the display is rotated against the system unit and rotated away from the system unit to allow the display to be rotated away from the system unit.

21. The computer system recited in claim 20, wherein the detecting means is actuated by movement of the protective cover.

22. The computer system recited in claim 21, wherein the computer system may optionally be placed on a horizontal work surface or mounted under a cabinet or shelf, the computer system further comprising orientation means for detecting an orientation of the computer system as either on a horizontal work surface or mounted under a cabinet or shelf, said automatic on and off function being further responsive to said orientation means for rotating display information to the flat panel display during the initial program load of the operating system so that a displayed image is oriented right side up.

23. The computer system recited in claim 16, further comprising a single user interface button on a front surface of the system unit, the user interface button allowing a user to control functions of software running on the computer system by a combination of single and double presses of the user interface button.

24. The computer system recited in claim 23, wherein the software includes a multimedia presentation and the single and double presses of the user interface button control pause, resume and stop functions of the multimedia presentation.

25. The computer system recited in claim 23, wherein the software presents a user with choices for making a selection in a displayed image on the display and the single and double presses of the user interface button control movement among the displayed choices and selection of one of the displayed choices.

26. The computer system recited in claim 23, further comprising a slot on the front face of the computer system for receiving computer readable media and an eject button for ejecting computer readable media currently in the slot.

27. The computer system recited in claim 16, further comprising:

an internal microphone and an internal speaker providing a audible interface with a user;

a wireless headphone and microphone set providing an alternative audible interface with the user;

a wireless transceiver attachable to the computer system and communicating with the wireless headphone and microphone set; and

means in the computer system for detecting attachment of the wireless transceiver to the computer system and inhibiting operation of the internal microphone and internal speaker.

28. A computer system for presenting an interactive multimedia book to a user comprising:

a display for displaying text and video clips of the interactive multimedia book, some words of the text being hyperlinks to anchors in text not currently displayed;

a voice recognition function installed on the computer system;

a speech synthesis function installed on the computer system;

a microphone for inputting voiced commands by the user to the voice recognition function;

at least one speaker for outputting synthesized speech from the speech synthesis function; and

an application program running on a central processing unit of the computer system for responding to a voiced command recognized by the voice recognition function to read text on a displayed page of the interactive multimedia book by causing the speech synthesis function to read the text, the application program identifying words which are active hyperlinks in the text on a displayed page as the text is being read and activating at least one attribute of a word identified as an active hyperlink to indicate to the user of the book that the word is a hyperlink, the application responding to a voiced hyperlink recognized by the voice recognition function by moving to another page of the book which contains an anchor for the hyperlink.

29. The computer system recited in claim 28, wherein the application program displays an icon representative of a video clip on the page of the book which contains an anchor for the hyperlink, the application program responding to a voiced command recognized by the voice recognition function to play the video clip by playing the video clip.

30. The computer system recited in claim 29, further comprising a user interface which allows a user to pause, resume and stop the playing of the video clip, the application program responding to a user input to pause the video clip by temporarily halting the playing of the video clip, responding to a user input to resume playing the video clip by playing the video clip from a point at which the video clip was temporarily halted, and responding to a user input to stop playing the video clip by stopping the playing of the video clip and returning to a main part of the application program.

31. The computer system recited in claim 30, wherein the user interface is a single button which when pressed once pauses the playing of the video clip if the video clip is playing, when pressed again resumes the playing of the video clip if the video clip is temporarily halted, and when pressed twice in succession causes the playing of the video clip to be stopped.

32. The computer system recited in claim 28, wherein said at least one attribute of a word identified as an active hyperlink is a change in color, the word on the displayed page changing color when read.

33. The computer system recited in claim 28, wherein said at least one attribute of a word identified as an active hyperlink is a sound, the sound being emitted when the word on the displayed page is read.

34. The computer system recited in claim 28, wherein said at least one attribute of a word identified as an active hyperlink is a change in color and a sound, the sound being emitted and the word on the displayed page changing color when read.

35. The computer system recited in claim 28, wherein the voiced commands which may be spoken by the user of the book and recognized by the voice recognition function further include a command to list active hyperlinks on a displayed page.

36. The computer system recited in claim 28, wherein the voiced commands which may be spoken by the user of the book and recognized by the voice recognition function further include a command to display an image of a completed project described on a currently displayed page of the book.

37. The computer system recited in claim 28, further comprising:

a system unit having a generally rectangular shape, said central processing unit being housed within the system unit, wherein the display is a flat panel display hinged at one edge of the system unit, said flat panel display being rotated away from the system unit when the computer system is in operation and rotated flat against the system unit when the computer system is not in operation;

detecting means associated with the display for detecting when a user desires to turn on the computer system and when the user wants to turn off the computer system; and

an automatic on and off function implemented on the computer system and responsive to said detecting means for turning on said computer system and performing an initial program load of an operating system installed on the computer system when it is detected that the computer systems is to be turned on and for initiating a shut down procedure of any open applications and the operating system when it that the computer system is to be turned off.

38. The computer system recited in claim 37, wherein the computer system may optionally be sat on a horizontal work surface or mounted under a cabinet or shelf, the computer system further comprising orientation means for detecting an orientation of the computer system as either on a horizontal work surface or mounted under a cabinet or shelf, said automatic on and off function being further responsive to said orientation means for rotating display information to the flat panel display during the initial program load of the operating system so that a display is oriented right side up.

39. The computer system recited in claim 37, wherein the detecting means detects a position of the display as either rotated away from the system unit or rotated against the system unit.

40. The computer system recited in claim 37, wherein the display is hinged along a front edge of the system unit.

41. The computer system recited in claim 40, further comprising a protective cover hinged along a back edge of the computer system, the protective cover being rotated to cover and protect the flat panel display when the flat panel display is rotated flat against the system unit and rotated away from the system unit to allow the flat panel display to be rotated away from the system unit.

42. The computer system recited in claim 41, wherein the detecting means is actuated by movement of the protective cover.

43. The computer system recited in claim 42, wherein the computer system may optionally be sat on a horizontal work surface or mounted under a cabinet or shelf, the computer system further comprising orientation means for detecting an orientation of the computer system as either on a horizontal work surface or mounted under a cabinet or shelf, said automatic on and off function being further responsive to said orientation means for rotating display information to the flat panel display during the initial program load of the operating system so that a display is oriented right side up.

44. The computer system recited in claim 37, further comprising a single user interface button on a front surface of the system unit, the user interface button allowing a user to control functions of software running on the computer system by a combination of single and double presses of the user interface button.

45. The computer system recited in claim 44, wherein the software includes a multimedia presentation and the single and double presses of the user interface button control pause, resume and stop functions of the multimedia presentation.

46. The computer system recited in claim 44, wherein the software presents a user with choices for making a selection in a display on the flat panel display and the single and double presses of the user interface button control movement among the displayed choices and selection of one of the displayed choices.

47. The computer system recited in claim 44, further comprising a slot on the front face of the computer system for receiving computer readable media and an eject button for ejecting computer readable media currently in the slot.

48. A computer system for presenting an interactive multimedia book to a user comprising:

a display for displaying text and video clips of the interactive multimedia book, some words of the text being hyperlinks to anchors in text not currently displayed;

a voice recognition function installed on the computer system;

a speech synthesis function installed on the computer system;

a microphone for inputting voiced commands by the user to the voice recognition function;

at least one speaker for outputting synthesized speech from the speech synthesis function;

a last in, first out register; and

an application program running on a central processing unit of the computer system for responding to a voiced hyperlink recognized by the voice recognition function by first storing a current page number in the last in, first out register before moving to another page of the book which contains an anchor for the hyperlink, the current page being an origin page, and then moving to another page of the book which contains the anchor for the hyperlink, said application program responding to a voiced command recognized by the voice recognition function to return to the origin page by moving back to the origin page.

49. The computer system recited in claim 48, wherein said voice recognition function recognizes a natural language query spoken by the user of the book by analyzing the natural language query and responding to the natural language query.

50. The computer system recited in claim 49, wherein the natural language query includes “

WHAT”

, the analysis of the natural language query by the voice recognition function analyzing a word or words following the word “

WHAT” and

responding to the natural language query by displaying on the display and causing the speech synthesis function to speak a definition of the word or words following the word “

WHAT”

.

51. The computer system recited in claim 49, wherein the natural language query includes “

MAKE”

, the analysis of the natural language query by the voice recognition function analyzing a word or words following the word “

MAKE” and

responding to the natural language query by moving to a section of the book which covers a project or recipe corresponding to the word or words following the word “

MAKE”

.

52. The computer system recited in claim 49, wherein the natural language query includes “

HOW”

, the analysis of the natural language query by the voice recognition function analyzing a word or words following the word “

HOW” and

responding to the natural language query includes by moving to a section of the book which covers a technique corresponding to the word or words following the word “

HOW”

.

53. The computer system recited in claim 49, wherein the natural language query includes “

GO”

, the analysis of the natural language query by the voice recognition function analyzing a word or words following the word “

GO” and

responding to the natural language query by moving to a page or a section of the book which covers a project or recipe or a technique corresponding to the word or words following the word “

GO”

.

54. A machine readable medium containing computer code for presenting an interactive multimedia book on a computer, the computer code performing the steps of:

recognizing voiced commands spoken by a user of the book;

responding to a voiced command to read text on a displayed page of the book by reading text on the displayed page of the book;

identifying words which are active hyperlinks in the text on a displayed page of the book as the text is being read;

activating at least one attribute of a word identified as an active hyperlink to indicate to the user of the book that the word is a hyperlink;

recognizing a voiced word spoken by the user of the book as an active hyperlink; and

responding to a voiced word which is an active hyperlink by moving to another page of the book which contains an anchor for the hyperlink.

55. The machine readable medium containing computer code for presenting an interactive multimedia book on a computer system recited in claim 54, the computer code further performing the steps of:

displaying an icon representative of a video clip on the page of the book which contains an anchor for the hyperlink;

recognizing a voiced command to play the video clip; and

responding to the voiced command to play the video clip by playing the video clip.

56. The machine readable medium containing computer code for presenting an interactive multimedia book on a computer system recited in claim 55, the computer code further performing the steps of:

providing a user interface which allows the user to pause, resume and stop the playing of the video clip;

responding to a user input to pause the video clip by temporarily halting the playing of the video clip;

responding to a user input to resume playing the video clip by playing the video clip from a point at which the video clip was temporarily halted; and

responding to a user input to stop playing the video clip by stopping the playing of the video clip and returning to a main program.

57. The machine readable medium containing computer code for presenting an interactive multimedia book on a computer system recited in claim 56, wherein the code that performs the step of providing a user interface responds to a single button which when pressed once pauses the playing of the video clip if the video clip is playing, when pressed again resumes the playing of the video clip if the video clip is temporarily halted, and when pressed twice in succession causes the playing of the video clip to be stopped.

58. The machine readable medium containing computer code for presenting an interactive multimedia book on a computer system recited in claim 54, wherein said at least one attribute of a word identified as an active hyperlink is implemented in code as a change in color, the word on the displayed page changing color when read.

59. The machine readable medium containing computer code for presenting an interactive multimedia book on a computer system recited in claim 54, wherein said at least one attribute of a word identified as an active hyperlink is implemented in code as a sound, the sound being emitted when the word on the displayed page is read.

60. The machine readable medium containing computer code for presenting an interactive multimedia book on a computer system recited in claim 54, wherein said at least one attribute of a word identified as an active hyperlink is implemented in code as a change in color and a sound, the sound being emitted and the word on the displayed page changing color when read.

61. The machine readable medium containing computer code for presenting an interactive multimedia book on a computer system recited in claim 54, wherein the code further implements the steps of:

storing a current page number in a last in, first out register before moving to another page of the book which contains an anchor for the hyperlink, the current page being the origin page;

recognizing a voiced command spoken by the user of the book to return to the origin page; and

responding to the voiced command to return to the origin page by moving back to the origin page.

62. The machine readable medium containing computer code for presenting an interactive multimedia book on a computer system recited in claim 54, wherein the code further implements the steps of:

recognizing a natural language query spoken by the user of the book;

analyzing the natural language query; and

responding to the natural language query.

63. The machine readable medium containing computer code for presenting an interactive multimedia book on a computer system recited in claim 62, wherein the natural language query includes “

WHAT”

, the code implementing the step of analyzing the natural language query analyzing a word or words following the word “

WHAT”

, and the code implementing the step of responding to the natural language query displays and speaks a definition of the word or words following the word “

WHAT”

.

64. The machine readable medium containing computer code for presenting an interactive multimedia book on a computer system recited in claim 62, wherein the natural language query includes “

MAKE”

, the code implementing the step of analyzing the natural language query analyzing a word or words following the word “

MAKE”

, and the code implementing the step of responding to the natural language query moves to a section of the book which covers a project or recipe corresponding to the word or words following the word “

MAKE”

.

65. The machine readable medium containing computer code for presenting an interactive multimedia book on a computer system recited in claim 62, wherein the natural language query includes “

HOW”

, the code implementing the step of analyzing the natural language query analyzing a word or words following the word “

HOW”

, and the code implementing the step of responding to the natural language query moves to a section of the book which covers a technique corresponding to the word or words following the word “

HOW”

.

66. The machine readable medium containing computer code for presenting an interactive multimedia book on a computer system recited in claim 62, wherein the natural language query includes “

GO”

, the code implementing the step of analyzing the natural language query analyzing a word or words following the word “

GO”

, and the code implementing the step of responding to the natural language query moves to a page or a section of the book which covers a project or recipe or a technique corresponding to the word or words following the word “

GO”

.

67. The machine readable medium containing computer code for presenting an interactive multimedia book on a computer system recited in claim 62, wherein the voiced commands spoken by the user of the book and recognized by the code further include a command to list active hyperlinks on a displayed page.

68. The machine readable medium containing computer code for presenting an interactive multimedia book on a computer system recited in claim 62, wherein the voiced commands spoken by the user of the book and recognized by the code further include a command to display an image of a completed project described on a currently displayed page of the book.