Providing a user interface having interactive elements on a writable surface

• US 20060067576A1
• Filed: 01/12/2005
• Published: 03/30/2006
• Est. Priority Date: 03/17/2004
• Status: Abandoned Application

• Current Assignee: LeapFrog Enterprises, Inc. (VTech Holdings Ltd.)
• Original Assignee: LeapFrog Enterprises, Inc. (VTech Holdings Ltd.)
• Inventors: Marggraff, James, Edgecomb, Tracy L., Chisholm, Alexander
• Application Number:
  US11/034,489
• Publication Number:
  US 20060067576A1
• Time in Patent Office: Days
• US Class Current: 382/186
• CPC Class Codes: G06F 3/0321 by optically sensing the absolute position with respect to a regularly patterned surface forming a passive digitiser, e.g. pen optically detecting position indicative tags printed on a paper sheet constructional details of pen-shaped pointing devices G06F3/03545, G06F3/03542, G06F3/037
• CPC Class Codes: G06F 3/03545 Pens or stylus
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06V 10/19 by sensing codes defining pattern positions
• CPC Class Codes: G06V 30/1423 the instrument generating sequences of position coordinates corresponding to handwriting preprocessing or recognising digital ink G06V30/32
• CPC Class Codes: G06V 30/36 Matching; Classification

Abstract

An interactive device that allows a user to create and interact with selectable items written on a surface. The present invention provides a user with an interface that replaces an electronic display with any surface, such as a piece of paper. The user may create user-written selectable items on the surface that execute associated functions. The user-written selectable items are persistent on the surface, allowing a user to execute functions associated with different selectable items throughout operation of the interactive device. By allowing a user to interact directly with a piece of paper, the present invention provides a unique user interface with an interactive device that provides the user with increased flexibility with regard to creation of user-written selectable items and execution of functions associated with these items.

Claims

1. An interactive device comprising:

a bus;

a processor coupled to said bus;

a memory unit coupled to said bus;

an audio output device coupled to said bus;

a writing element for allowing a user to create a user-written selectable item on a surface; and

an optical detector coupled to said bus for detecting information on said surface;

wherein software implemented on said processor is responsive to a user selection of said user-written selectable item and operable to execute a function associated with said user-written selectable item in response to said selection.

2. The interactive device of claim 1 wherein said processor is operable to automatically identify said user-written selectable item in response to said selection using symbol recognition.

3. The interactive device of claim 1 wherein said processor is operable to automatically record a surface location of said user-written selectable item on said surface.

4. The interactive device of claim 3 wherein said processor is operable to automatically identify said user-written selectable item in response to said selection based on said surface location of said user-written selectable item.

5. The interactive device of claim 1 wherein said processor is operable to automatically identify said user-written selectable item in response to said selection based on a detected surface location of said user-written selectable item.

6. The interactive device of claim 1 wherein said audio output device is operable to audibly render a list of potential user-written selectable items in response to an interaction with one user-written selectable item on said surface.

7. The interactive device of claim 6 wherein said audio output device is operable to audibly render instructions describing how to draw a said potential user-written selectable item.

8. The interactive device of claim 1 wherein said user-written selectable item comprises a symbol representation of an application program executable by said processor.

9. The interactive device of claim 1 wherein said user-written selectable item comprises a menu item of an application program executable by said processor.

10. The interactive device of claim 1 wherein said user-written selectable item comprises a menu item of a hierarchy of menu items in a hierarchical relationship of menu items associated with an operating system of said interactive device.

11. The interactive device of claim 1 wherein said user-written selectable item comprises a navigation item.

12. The interactive device of claim 1 wherein said user-written selectable item comprises a data.

13. The interactive device of claim 1 wherein said user-written selectable item comprises an application option selector.

14. The interactive device of claim 1 wherein said surface comprises encoded paper that has surface location information encoded in a pattern on said encoded paper.

15. The interactive device of claim 1 wherein said bus, said processor, said memory unit, said audio output device, said optical detector, and said writing element are comprised within a single pen-shaped housing.

16. The interactive device of claim 1 wherein said software is operable to direct said audio output device to audibly render information in response to said selection of said user-written selectable item.

17. The interactive device of claim 1 wherein said optical detector is for detecting said user-written selectable item on said surface.

18. The interactive device of claim 1 wherein said software is responsive to a user selection of a pre-printed selectable item on said surface and operable to execute a function associated with said pre-printed selectable item in response to a user selecting said pre-printed selectable item.

19. The interactive device of claim 18 wherein said pre-printed selectable item and said user-written selectable item are coordinated in state such that options of said pre-printed selectable item and said user-written selectable item span across said pre-printed selectable item and said user-written selectable item.

20. The interactive device of claim 18 wherein said software is operable to automatically identify said pre-printed selectable item in response to a user selecting said pre-printed selectable item using symbol recognition.

21. The interactive device of claim 18 wherein said processor is operable to automatically identify said pre-printed selectable item in response to a user selecting said pre-printed selectable item based on a surface location of said pre-printed selectable item.

22. The interactive device of claim 1 wherein said optical detector is operable to provide said processor with information to recognize said user-written selectable item based on a location on said surface and is also operable to provide said processor with information such that said processor can recognize a selection of said user-written selectable item based on said location.

23. The interactive device of claim 1 wherein said processor is operable to identify an application program based on selection of a particular bounded region of said surface, wherein different bounded regions are associated with different application programs.

24. An operating system of a computer system, wherein said operating system is operable to detect a user-written selectable item on a writable surface, associate said user-written selectable item with a function, and, responsive to a user interaction with said user-written selectable item, automatically executing said function.

25. The operating system of claim 24 wherein said user interaction is a user contacting said user-written selectable item with a writing instrument.

26. The operating system of claim 24 further operable to automatically detect said user-written selectable item using symbol recognition.

27. The operating system of claim 24 further operable to automatically record a surface location of said user-written selectable item on said surface.

28. The operating system of claim 24 further operable to automatically detect said user-written selectable item in response to said user interaction based on a surface location of said user-written selectable item.

29. The operating system of claim 28 wherein said surface is a writable surface and said user interaction is a user contacting said user-written selectable item with a writing instrument.

30. The operating system of claim 24 further operable to direct an audio output device to audibly render information associated with said user-written selectable item in response to said user interaction.

31. The operating system of claim 24 wherein said user-written selectable item comprises a symbol representation of an application program executable by said processor.

32. The operating system of claim 24 wherein said user-written selectable item comprises a menu item of an application program executable by said processor.

33. The operating system of claim 24 wherein said user-written selectable item comprises a menu item of a hierarchy of menu items in a hierarchical relationship of menu items associated with said operating system.

34. The operating system of claim 24 wherein said user-written selectable item comprises a navigation item.

35. The operating system of claim 24 wherein said user-written selectable item comprises a data.

36. The operating system of claim 24 wherein said user-written selectable item comprises an application option selector.

37. The operating system of claim 24 wherein said surface comprises encoded paper that has surface location information encoded in a pattern on said encoded paper.

38. The operating system of claim 24 wherein said computer system is a pen-based device comprising an optical detector, an audio output device, a writing element, a processor, and a memory unit.

39. A method for facilitating interaction with user-written selectable items on a writable surface, said method comprising:

detecting at least one user-written selectable item associated with a function, wherein said user-written selectable item resides on said writable surface;

detecting interaction with said user-written selectable item, wherein said interaction comprises a writing element contacting said user-written selectable item; and

in response to said interaction, a pen device comprising a processor and said writing element executing said function.

40. The method as recited in claim 39 wherein said user-written selectable item comprises a symbol representation of an application program executable by said processor.

41. The method as recited in claim 39 wherein said user-written selectable item comprises a menu item of an application program executable by said processor.

42. The method as recited in claim 39 wherein said user-written selectable item comprises a menu item of a hierarchy of menu items in a hierarchical relationship of menu items associated with an operating system of said interactive device.

43. The method as recited in claim 39 wherein said user-written selectable item comprises a navigation item.

44. The method as recited in claim 39 wherein said user-written selectable item comprises a data.

45. The method as recited in claim 39 wherein said user-written selectable item comprises an application option selector.

46. The method as recited in claim 39 wherein said function is an application.

47. The method as recited in claim 39 wherein said function is an audible rendering of a menu comprising a listing of navigation items.

48. The method as recited in claim 39 further comprising audibly rendering an identification of at least one said user-written selectable item.

49. An interactive device comprising:

a bus;

a processor coupled to said bus;

a memory unit coupled to said bus;

an audio output device coupled to said bus;

a writing element for allowing a user to create a user-written selectable item on a writable surface; and

an optical detector coupled to said bus for detecting information on said writable surface;

wherein software implemented on said processor is operable to direct said audio output device to audibly render a listing identifying potential user-written selectable items, wherein said processor is operable to detect that a user has written a plurality of user-written selectable items, and wherein said processor responsive to a user selection of one or more user-written selectable items of said plurality of user-written selectable items and is operable to execute a different function associated with each of the user-written selectable items in response to selection thereof.

50. The interactive device of claim 49 wherein user selection comprises contact by said writing element with said user-written selectable item.

51. The interactive device of claim 49 wherein said processor is operable to automatically identify said plurality of user-written selectable items in response to said user selection using symbol recognition.

52. The interactive device of claim 49 wherein said processor is operable to automatically record a surface location of said plurality of user-written selectable items on said surface.

53. The interactive device of claim 49 wherein said processor is operable to automatically identify one or more said user-written selectable items in response to said user selection based on a detected surface location of said user-written selectable items.

54. The interactive device of claim 49 wherein said plurality of user-written selectable items comprises a symbol representation of an application program executable by said processor.

55. The interactive device of claim 49 wherein said plurality of user-written selectable items comprises a menu item of an application program executable by said processor.

56. The interactive device of claim 49 wherein said user-written selectable item comprises a menu item of a hierarchy of menu items in a hierarchical relationship of menu items associated with an operating system of said interactive device.

57. The interactive device of claim 49 wherein said plurality of user-written selectable items comprises a navigation item.

58. The interactive device of claim 49 wherein said plurality of user-written selectable items comprises data.

59. The interactive device of claim 49 wherein said plurality of user-written selectable items comprises an application option selector.

60. The interactive device of claim 49 wherein said surface comprises encoded paper that has surface location information encoded in a pattern on said encoded paper.

61. The interactive device of claim 49 wherein said software is responsive to a user selection of a pre-printed selectable item on said surface and operable to execute a function associated with said pre-printed selectable item in response to a user selecting said pre-printed selectable item.