User created interactive interface

• US 20060033725A1
• Filed: 06/03/2004
• Published: 02/16/2006
• Est. Priority Date: 06/03/2004
• Status: Abandoned Application

• Current Assignee: LeapFrog Enterprises, Inc. (VTech Holdings Ltd.)
• Original Assignee: LeapFrog Enterprises, Inc. (VTech Holdings Ltd.)
• Inventors: Marggraff, James, Edgecomb, Tracy L., Chisholm, Alex, Fast, Nathaniel A.
• Application Number:
  US10/861,243
• Publication Number:
  US 20060033725A1
• Time in Patent Office: Days
• US Class Current: 345/179
• CPC Class Codes: G06F 3/03545 Pens or stylus
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 3/16 Sound input; Sound output speech processing G10L
• CPC Class Codes: G06V 30/387 using human interaction, e.g. selection of the best displayed recognition candidate

Abstract

An interactive apparatus is disclosed. The interactive apparatus includes a stylus housing, a processor coupled to the stylus housing, and a memory unit comprising (i) computer code for recognizing a plurality of graphic elements created using a stylus, (ii) computer code for recognizing the selection of at least two of the graphic elements in a user defined sequence using the stylus, and (iii) computer code for playing at least one audio output that relates to the formed graphic elements, and an audio output device.

Claims

1. A method comprising:

(a) a graphic element, the graphic element created by handheld device;

(b) generating an audio recitation of at least one menu item out of a plurality of menu items after recognition of the graphic element; and

(c) recognizing a selection of a menu item from the plurality of menu items upon a subsequent actuation of the handheld device, the actuation related to the graphic element.

2. The method of claim 1 wherein the handheld device is in the form of an interactive apparatus comprising a processor, an emitter, a detector, and a speaker, wherein the emitter, detector, and the speaker are operatively coupled to the processor.

3. The method of claim 1, wherein the graphic element is on a printable surface.

4. The method of claim 1 wherein the graphic element is a print element.

5. The method of claim 1 wherein the handheld device comprises an antenna.

6. The method of claims 3 wherein the printable surface is a sheet of paper.

7. The method of claim 1 wherein the graphic element includes a symbol.

8. The method of claim 1 wherein the graphic element includes a symbol and a line circumscribing the symbol.

9. The method of claim 1 wherein after recognition of the selection of the menu item, a speech synthesizer operatively associated with the handheld device audibly recites instructions for creating additional graphic elements.

10. The method of claim 1 wherein the plurality of menu items include at least one of calculator menu item, a reference menu item, and a games menu item.

11. An interactive apparatus comprising:

handheld device housing;

a processor coupled to the handheld device housing;

a memory unit comprising;

(i) computer code for recognizing a graphic element created by the handheld device;

(ii) computer code for playing an audio recitation of at least one menu item in a plurality of menu items after the graphic element is created by the user; and

(iii) computer code for recognizing a user selection of a menu item from the plurality of menu items; and

an audio output device, wherein the audio output device and the memory unit are operatively coupled to the processor.

12. The interactive apparatus of claim 11 wherein the processor, the audio output device, and the memory unit are in the handheld device housing.

13. The interactive apparatus of claim 11 wherein the processor, the audio output device and the memory are in a platform that is coupled to the handheld device housing.

14. The interactive apparatus of claim 11 wherein the processor, the memory unit, and the audio output device are all in the handheld device housing, and wherein the handheld device housing further comprises an optical emitter and an optical detector coupled to the processor.

15. The interactive apparatus of claim 11 wherein the graphic elements include letters or numbers.

16. The interactive apparatus of claim 11 wherein the memory unit comprises computer code for recognizing substantially invisible codes on an article.

17. The interactive apparatus of claim 11 wherein the memory unit comprises computer code for recognizing substantially invisible codes on an article, and wherein the substantially invisible codes are in the from of dot codes.

18. The interactive apparatus of claim 11 wherein the memory unit comprises computer code for recognizing substantially invisible codes on an article, and wherein the substantially invisible codes are in the from of dot codes that encode a relative or absolute position.

19. A system comprising:

(a) the interactive apparatus of claim 18;

and

(b) an article including the substantially invisible codes.

20. A system comprising:

(a) the interactive apparatus of claim 18;

and

(b) an article including the substantially invisible codes, wherein the article includes a sheet of paper.

21. A system comprising:

(a) the interactive apparatus of claim 18;

and

(b) an article including the substantially invisible codes, wherein the article includes a sheet of paper that is free of any pre-printing.

22. A system comprising:

(a) the interactive apparatus of claim 18;

and

(b) an article including the substantially invisible codes, wherein the article includes a sheet of paper that includes pre-printing.

23. A system comprising:

(a) an interactive apparatus comprising a stylus housing, a processor coupled to the stylus housing, a speech synthesizer, a memory unit comprising

(i) computer code for allowing a user to create a graphic element using the stylus,

(ii) computer code for playing an audio recitation of at least one menu item in a plurality of menu items after creating the graphic element,

(iii) computer code for allowing a user to select a menu item from the plurality of menu items, and

(iv) computer code for recognizing substantially invisible codes on an article, and wherein the substantially invisible codes are in the from of dot codes that encode a relative or absolute position, and an audio output device, wherein the speech synthesizer, the audio output device and the memory unit are operatively coupled to the processor, and wherein the speech synthesizer, the audio output device, the processor and the memory unit are in the stylus housing;

24. The system of claim 23 wherein the substantially invisible codes are dot codes.

25. The system of claim 23 wherein the memory unit comprises computer code for causing the interactive apparatus to recite the menu items after each sequential selection of the graphic element.

26. The system of claim 23 wherein the substantially invisible codes relate to the absolute positions on the article.

27. The system of claim 23 wherein the article includes a sheet of paper.

28. The system of claim 23 wherein the article includes a sheet of paper that is free of pre-printed print elements.

29. The system of claim 23 wherein the article includes a sheet of paper that includes pre-printed print elements.

30. The system of claim 23 wherein the graphic element includes one selected from the group consisting of at least one of indicium, and a combination of at least one indicium and a line circumscribing the at least one indicium.

31. A method comprising:

recognizing a plurality of created graphic elements on a surface;

recognizing a selection of at least one of the graphic elements, the selection implemented by a stylus upon an actuation of the stylus related to the at least one graphic element;

accessing a function related to the at least one graphic element;

providing at least one audio output in accordance with the function.

32. The method of claim 31 wherein the plurality of graphic elements comprise a plurality of numbers and mathematical operators, and wherein recognizing the selection comprises recognizing the selection of a first number, a first mathematical operator, a second number, and a second mathematical operator, wherein the first number, the first mathematical operator, and the second mathematical operator together form a math problem, and wherein the at least one audio output that relates to the selected first number, first mathematical operator, and the second mathematical operator comprises the answer to the math problem.

33. The method of claim 31 wherein the stylus comprises an emitter, a detector, a processor, and a speaker, wherein the emitter, detector, and the speaker are coupled to the processor.

34. The method of claim 31 wherein the stylus is coupled to a platform, which supports a sheet upon which the graphic elements are formed.

35. The method of claim 31 wherein the elements comprise letters.

36. The method of claim 31 wherein the elements comprise a first graphic element comprising a name of a language and a second graphic element comprising a word that is in a language that is different than the language, and wherein recognizing the selection includes recognizing the selection of the word and then recognizing the selection of the name of the language, and wherein to the at least one audio output includes a synthesized voice audibly rendering the word in the language.

37. The method of claim 31 wherein the graphic elements comprise a first graphic element comprising a name of a language and a second graphic element comprising a word that is in a language that is different than the language, and wherein recognizing the selection include recognizing the selection of the word and then recognizing the selection of the name of the language, and wherein the at least one audio output includes a synthesized voice audibly rendering the word in the language, wherein the language is a non-English language and wherein the word is in English.

38. The method of claim 31 wherein the stylus comprises a writing element, and wherein graphic elements are user created graphic elements on a sheet and are generated in conjunction with the stylus.

39. The method of claim 38 wherein the sheet includes a plurality of substantially invisible codes.

40. The method of claim 39 wherein the sheet is free of pre-printed print elements.

41. An interactive apparatus comprising:

a device housing;

a processor coupled to the device housing;

a memory unit comprising

(i) computer code for recognizing a plurality of graphic elements created using a device.

(ii) computer code for recognizing the selection of at least two of the graphic elements in a user defined sequence using the device, and

(iii) computer code for playing at least one audio output that relates to the formed graphic elements; and

an audio output device, wherein the audio output device and the memory unit are operatively coupled to the processor.

42. The interactive apparatus of claim 41 wherein the device comprises a writing element.

43. The interactive apparatus of claim 41 wherein the processor, the memory unit and the audio output device are in the device housing.

44. The interactive apparatus of claim 41 wherein the memory unit further comprises computer code for recognizing substantially invisible codes printed on an article.

45. The interactive apparatus of claim 41 wherein the memory unit further comprises computer code for recognizing substantially invisible codes printed on an article, wherein the substantially invisible codes comprise dot codes.

46. The interactive apparatus of claim 41 wherein the apparatus further comprises a platform and wherein the memory, the processor, and the audio output device are in the platform.

47. The interactive apparatus of claim 41 wherein the graphic elements comprise numbers and wherein the memory unit further comprises code for calculating numbers.

48. The interactive apparatus of claim 41 wherein the interactive apparatus comprises a writing element that is retractable.

49. The interactive apparatus of claim 41 wherein the memory unit further comprises computer code for teaching about at least one of letters, numbers, and phonics.

50. The interactive apparatus of claim 41 wherein the memory unit comprises computer code for causing a synthesized voice to recite a plurality of menu items.

51. A system comprising:

an interactive device comprising a device housing, a processor coupled to the device housing, a memory unit comprising

(i) computer code for recognizing a plurality of graphic elements created using the device.

(ii) computer code for recognizing the selection of at least two of the graphic elements in a user defined sequence using the device, and

(iii) computer code for playing at least one audio output that relates to the formed graphic elements, and an audio output device, wherein the audio output device and the memory unit are operatively coupled to the processors.

52. The system of claim 51 further comprising an article upon which the graphic elements are created.

53. The system of claim 52 wherein the article comprises a sheet of paper and wherein the sheet of paper includes a plurality of substantially invisible codes.

54. The system of claim 52 wherein the article comprises a sheet of paper and where[n the sheet of paper includes a plurality of substantially invisible codes comprising dot codes.

55. The system of claim 52 wherein the article comprises a sheet of paper and wherein the sheet of paper includes a plurality of substantially invisible codes wherein the substantially invisible codes include relative or absolute position information.

56. The system of claim 52 wherein the article comprises a sheet of paper and wherein the sheet of paper includes a plurality of substantially invisible codes, wherein the codes are dot codes, and wherein the sheet of paper is substantially free of pre-printed print elements.

57. The system of claim 51 wherein the processor, the audio output device, and the memory unit are in the device housing.

58. The system of claim 51 wherein the interactive device is in the form of a self-contained device.

59. The system of claim 51 wherein the memory unit comprises computer code for a plurality of menu items.

60. The system of claim 51 wherein the memory unit includes computer code for an English-foreign language dictionary.

61. A method for interpreting user commands, comprising:

recognizing a created graphical element on a surface;

accessing a function related to the graphical element;

providing an output in accordance with the function; and

associating the function with the graphical element.

62. The method of claim 61, wherein the output comprises an audio output related to the function.

63. The method of claim 61, further comprising:

enabling a subsequent access of the function in response to a subsequent selection of the graphical element by storing the association of the function with the graphical element.

64. The method of claim 63, wherein the storing of the association of the function with the graphical element implements a persistent availability of the function, for a predetermined amount of time, via interaction with the graphical element.

65. The method of claim 61, wherein the graphical element is created by a pen device on the surface.

66. The method of claim 65, wherein the surface comprises a sheet of paper.

67. The method of claim 61, further comprising:

accessing one of a plurality of functions related to the graphical element by interpreting at least one actuation of the graphical element, wherein the at least one actuation selects the one of the plurality of functions.

68. The method of claim 67, wherein the at least one actuation comprises recognizing at least one tap of the graphical element.

69. The method of claim 67, further comprising:

providing one of a plurality of audio outputs when the one of the plurality of functions is selected.

70. The method of claim 67, wherein the plurality of functions comprises a predetermined menu of options.

71. The method of claim 67, wherein the plurality of functions comprises a plurality of configuration options of an application related to the graphical element.

72. The method of claim 71, wherein at least one of the plurality of configuration options comprises a default configuration of the application.

73. The method of claim 71, further comprising:

implementing a hierarchy of functions; and

providing access to the hierarchy of functions via a corresponding hierarchy of graphical elements.

74. The method of claim 73, further comprising:

recognizing at least one actuation of the graphical element to select a first hierarchical level function;

prompting the creation of a second graphical element;

recognizing at least one actuation of the second graphical element to select a second hierarchical level function;

providing an audio output related to the second hierarchical level function; and

associating the second hierarchical level function with the second graphical element.

75. A method of interacting with a handheld device, said method comprising:

recognizing selection of a first graphical icon on a writable surface, said selection performed using a writing instrument of said handheld device;

in response to said selection, audibly rendering a listing of first options associated with said first graphical icon wherein said first options are operable to be invoked by said handheld device; and

in response to a selection of one of said first options, invoking said one of said first options.

76. A method as described in claim 75 wherein said first options comprise at least one application to be invoked.

77. A method as described in claim 75 wherein said one of said first options is an application program resident on said handheld device.

78. A method as described in claim 75 wherein said audibly rendering said listing of said first options comprises audibly rendering, one at a time, each of said first options in a round-robin fashion, in response to selections of said first graphical icon by said writing instrument.

79. A method as described in claim 78 further comprising identifying a selection of said one of said first options in response to said writing instrument selecting a portion of said first graphical icon after said one of said first options is audibly rendered.

80. A method as described in claim 79 wherein said portion of said first graphical icon is a symbol of a check mark.

81. A method as described in claim 79 wherein said selecting said portion comprises recognizing a gesture made by a user with said handheld device.

82. A method as described in claim 75 wherein said first graphical icon is user written on said surface and further comprising automatically identifying said first graphical icon and wherein said automatically identifying said first graphical icon is performed using a processor of said handheld device.

83. A method as described in claim 75 wherein said first graphical icon is pre-printed on said surface.

84. A method as described in claim 75 wherein said first graphical icon is a menu item and wherein said first options are submenu items within a hierarchy of options operable to be invoked by said handheld device.

85. A method as described in claim 75 wherein said first options comprise an option having an associated second graphical icon and further comprising:

recognizing selection of said second graphical icon on said writable surface, said selection performed using said writing instrument of said handheld device;

in response to said selection, audibly rendering a listing of second options associated with said second graphical icon wherein said second options are operable to be invoked by said handheld device; and

in response to a selection of one of said second options, invoking said one of said second options.

86. A method as described in claim 85 wherein said second options comprise at least one application to be invoked.

87. A method as described in claim 85 wherein said one of said second options is an application program resident on said handheld device.

88. A method as described in claim 85 wherein said audibly rendering said listing of said second options comprises audibly rendering, one at a time, each of said second options in a round-robin fashion, in response to selections of said second graphical icon by said writing instrument.

89. A method as described in claim 88 further comprising identifying selection of said one of said second options by responding to said writing instrument selecting a portion of said second graphical icon after said one of said second options is audibly rendered.

90. A method as described in claim 85 wherein said second graphical icon is user written on said surface and further comprising automatically identifying said second graphical icon and wherein said automatically identifying said second graphical icon is performed using a processor of said handheld device.

91. A method as described in claim 75 wherein said one of said first options comprises a text recognition function wherein said handheld device is configured to recognize the end of a written word by recognizing the user tapping the last character of the word.

92. A method as described in claim 75 wherein said one of said first options comprises a text recognition function wherein said handheld device is configured to recognize the end of a written word by recognizing the user drawing a box or circle around the word.

93. A method as described in claim 75 wherein said one of said first options comprises a dictionary function wherein said handheld device is configured to recognize a user written word and audibly render a definition related to said user written word.

94. A method as described in claim 75 wherein said one of said first options comprises a calculator function wherein said handheld device is configured to recognize a plurality of user written graphic elements, and wherein the plurality of graphic elements comprise a plurality of numbers and mathematical operators, and wherein said handheld device is configured to recognize the selection of a first number, a first mathematical operator, a second number, and a second mathematical operator, wherein the first number, the first mathematical operator, and the second mathematical operator together form a math problem, and audibly render at least one audio output that comprises the answer to the math problem.

95. A method as described in claim 75 wherein said one of said first options comprises a translator function wherein said handheld device is configured to recognize a plurality of user written graphic elements, and wherein a first graphic element comprises a name of a language and a second graphic element comprises a word that is in a language that is different than the language, and wherein said handheld device is configured to recognize the selection of the word and to recognize the selection of the name of the language and audibly render the word in the language.

96. A method as described in claim 75 wherein said one of said first options comprises a word scramble function wherein said handheld device is configured to recognize a plurality of user written graphic elements comprising words of a sentence, and wherein said handheld device is configured to recognize the sequential selection of the words and to audibly render the sentence upon a successful sequential selection of the words of the sentence.

97. A method as described in claim 75 wherein said one of said first options comprises an alarm clock function wherein said handheld device is configured to recognize a user written alarm time and audibly render an alarm related to said user written alarm time.

98. A method as described in claim 85 wherein said one of said first options comprises a phone list function, and wherein said audibly rendered listing of said second options comprises accessing a phone number, adding a phone number, or deleting a phone number, and in response to a selection of one of said second options, invoking said one of said second options of said phone list function.

99. A method as described in claim 75 wherein said handheld device comprises a processor in communication with a remote computer system external to the handheld device.

100. A method as described in claim 96 wherein said remote computer system is a server and said processor uses wireless communication to interact with said server.