Interactive apparatus with recording and playback capability usable with encoded writing medium

• US 7,453,447 B2
• Filed: 01/12/2005
• Issued: 11/18/2008
• Est. Priority Date: 03/17/2004
• Status: Expired due to Fees

• Current Assignee: LeapFrog Enterprises, Inc. (VTech Holdings Ltd.)
• Original Assignee: LeapFrog Enterprises, Inc. (VTech Holdings Ltd.)
• Inventors: Marggraff, James, Chisholm, Alexander
• Primary Examiner(s): Shankar; Vijay
• Application Number:
  US11/034,495
• Publication Number:
  US 20060080608A1
• Time in Patent Office: 1,406 Days
• Field of Search: 345173-179, 178 1801- 1807, 178 1901- 1906
• US Class Current: 345/179
• CPC Class Codes: G06F 3/0321 by optically sensing the absolute position with respect to a regularly patterned surface forming a passive digitiser, e.g. pen optically detecting position indicative tags printed on a paper sheet constructional details of pen-shaped pointing devices G06F3/03545, G06F3/03542, G06F3/037
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G11B 27/034 on discs G11B27/036, G11B27/038 take precedence

Abstract

An interactive apparatus is disclosed. The interactive apparatus has an audio recording mode and an audio playback mode. Further, the interactive apparatus includes a writing tool for manually writing on an encoded writing medium, an audio input device for recording audio, an audio output device for playing the recorded audio. Moreover, the interactive apparatus has an optical unit for determining position on the encoded writing medium of the writing tool and for recognizing information on the encoded writing medium, wherein recognition of a recording-enabling graphical element on the encoded writing medium activates the audio recording mode and enables time-synchronization of creation of writing on the encoded writing medium with the writing tool and recording of the audio, and further wherein recognition of a playback-enabling graphical element on the encoded writing medium activates the audio playback mode and enables multiple techniques to control playback of the recorded audio.

Claims

1. An interactive apparatus having an audio recording mode and an audio playback mode, comprising:

a writing tool for manually writing non-electronic information on an encoded writing medium comprising a plurality of visual non-electronic markings representing encoded position information;

an audio input device for recording audio;

an audio output device for playing said recorded audio; and

an optical unit for determining position on said encoded writing medium of said writing tool based on said encoded position information and for recognizing said encoded position information and non-electronic information on said encoded writing medium, wherein recognition of a non-electronic recording-enabling graphical element on said encoded writing medium by said optical unit activates said audio input device in said audio recording mode and enables time-synchronization of creation of writing on said encoded writing medium with said writing tool and recording of said audio, and further wherein recognition of a non-electronic playback-enabling graphical element on said encoded writing medium by said optical unit activates said audio output device in said audio playback mode and enables multiple techniques to control playback of said recorded audio.

2. The interactive apparatus as recited in claim 1 further comprising a memory unit for storing recorded audio and information associated with said writing.

3. The interactive apparatus as recited in claim 1 further comprising a communication device for transmitting recorded audio and information associated with said writing to a remote storage location.

4. The interactive apparatus as recited in claim 3 wherein said communication device comprises a wireless communication device.

5. The interactive apparatus as recited in claim 1 wherein said encoded writing medium comprises paper encoded with a grid of dots.

6. The interactive apparatus as recited in claim 1 wherein said writing tool is one of a pen and a pencil.

7. The interactive apparatus as recited in claim 1 wherein said non-electronic recording-enabling graphical element and said non-electronic playback-enabling graphical element are drawn on said encoded writing medium with said writing tool.

8. The interactive apparatus as recited in claim 1 wherein recognition of non-electronic control-enabling graphical elements on said encoded writing medium by said optical unit enables controlling said audio recording mode and said audio playback mode;

and wherein said multiple techniques include using said non-electronic control-enabling graphical elements to control playback of said recorded audio and using said writing and said time-synchronization to indicate where to start playback of said recorded audio based on location of said writing tool on said writing on said encoded writing medium.

9. The interactive apparatus as recited in claim 1 wherein said optical unit comprises a processor, an optical emitter, and an optical detector.

10. The interactive apparatus as recited in claim 1 wherein said optical unit further captures and stores optical images of writing on a surface of said encoded writing medium during recording of said audio in said audio recording mode.

11. A system comprising:

a network comprising a data storage unit; and

an interactive apparatus having an audio recording mode and an audio playback mode, comprising;

a writing tool for manually writing non-electronic information on an encoded writing medium comprising a plurality of visual non-electronic markings representing encoded position information;

an audio input device for recording audio;

an audio output device for playing said recorded audio;

an optical unit for determining position on said encoded writing medium of said writing tool based on said encoded position information and for recognizing said encoded position information and non-electronic information on said encoded writing medium, wherein recognition of a non-electronic recording-enabling graphical element on said encoded writing medium by said optical unit activates said audio input device in said audio recording mode and enables time-synchronization of creation of writing on said encoded writing medium with said writing tool and recording of said audio, and further wherein recognition of a non-electronic playback-enabling graphical element on said encoded writing medium by said optical unit activates said audio output device in said audio playback mode and enables multiple techniques to control playback of said recorded audio; and

a communication device for communicating with said network, wherein said communication device transmits at least a portion of said recorded audio and at least a portion of information associated with said writing to said data storage unit for storage.

12. The system as recited in claim 11 wherein said interactive apparatus further comprises a memory unit for storing at least a portion of said recorded audio and at least a portion of information associated with said writing.

13. The system as recited in claim 11 further comprising a computer system comprising a hard drive, wherein said communication device communicates with said computer system, and wherein said communication device transmits at least a portion of said recorded audio and at least a portion of information associated with said writing to said hard drive for storage.

14. The system as recited in claim 11 wherein said communication device comprises a wireless communication device.

15. The system as recited in claim 11 wherein said encoded writing medium comprises paper encoded with a grid of dots.

16. The system as recited in claim 11 wherein said writing tool is one of a pen and a pencil.

17. The system as recited in claim 11 wherein said non-electronic recording-enabling graphical element and said non-electronic playback-enabling graphical element are drawn on said encoded writing medium with said writing tool.

18. The system as recited in claim 11 wherein recognition of non-electronic control-enabling graphical elements on said encoded writing medium by said optical unit enables controlling said audio recording mode and said audio playback mode;

and wherein said multiple techniques include using said non-electronic control-enabling graphical elements to control playback of said recorded audio and using said writing and said time-synchronization to indicate where to start playback of said recorded audio based on location of said writing tool on said writing on said encoded writing medium.

19. The system as recited in claim 11 wherein said optical unit comprises a processor, an optical emitter, and an optical detector.

20. The system as recited in claim 11 wherein said optical unit further captures and stores optical images of writing on a surface of said encoded writing medium during recording of said audio in said audio recording mode.

21. A method of visually integrating audio and an encoded writing medium using an interactive apparatus, said method comprising:

activating an audio recording mode of said interactive apparatus in response to recognition of a non-electronic recording-enabling graphical element on said encoded writing medium comprising a plurality of visual non-electronic markings representing encoded position information;

wherein said interactive apparatus comprises a writing tool, an audio input device, and an audio output device;

while recording audio using said interactive apparatus, time-synchronizing creation of writing representing non-electronic information on said encoded writing medium with said writing tool and recording of said audio using said interactive apparatus; and

deactivating said audio recording mode of said interactive apparatus in response to recognition of at least one of a plurality of non-electronic control-enabling graphical elements on said encoded writing medium.

22. The method as recited in claim 21 further comprising:

activating an audio playback mode of said interactive apparatus in response to recognition of a non-electronic playback-enabling graphical element on said encoded writing medium;

in said audio playback mode, retrieving recorded audio associated with said writing representing said non-electronic information and playing said recorded audio using said interactive apparatus; and

deactivating said audio playback mode of said interactive apparatus in response to recognition of at least one of said plurality of non-electronic control-enabling graphical elements on said encoded writing medium.

23. The method as recited in claim 22 further comprising:

using said non-electronic control-enabling graphical elements to control playback of said recorded audio; and

using said writing and said time-synchronization to indicate where to start playback of said recorded audio based on location of said writing tool on said writing on said encoded writing medium.

24. The method as recited in claim 21 further comprising:

storing recorded audio and information associated with said writing in a memory unit of said interactive apparatus.

25. The method as recited in claim 21 further comprising:

wirelessly transmitting recorded audio and information associated with said writing to a remote storage location.

26. The method as recited in claim 21 wherein said encoded writing medium comprises paper encoded with a grid of dots.

27. The method as recited in claim 21 wherein said writing tool is one of a pen and a pencil.

28. The method as recited in claim 27 wherein said non-electronic recording-enabling graphical element is drawn with said writing tool.

29. The method as recited in claim 21 further comprising:

capturing and storing optical images of said writing on surface of said encoded writing medium during recording of said audio in said audio recording mode.

30. A method of processing audio recording comprising:

in response to recognition of a first non-electronic graphical element on a writing surface useable for writing in a non-electronic format thereon, performing a recording of an audio signal, wherein said recognition is made with a writing instrument comprising;

a processor;

memory; and

an audio recording device;

in response to recognition of a second non-electronic graphical element on said writing surface, ending said recording, wherein said recognition is made with said writing instrument; and

storing said recording in digital form and storing time-synchronization between said recording of said audio signal and any writing representing non-electronic information created with said writing instrument during said recording.

31. A method as described in claim 30 wherein said storing is performed on a computer system remote to said writing instrument.

32. A method as described in claim 30 wherein said writing instrument further comprises a sensor for sensing a surface location of said writing instrument on said writing surface and further comprising storing in memory real-time surface location information of said writing instrument at discrete periods of time during said recording.

33. A method as described in claim 30 further comprising invoking a playback of said recording in response to recognition of a third non-electronic graphical element by said writing instrument and wherein said invoking said playback comprises:

accessing said recording for playback; and

rendering audible said recording.

34. A method as described in claim 33 wherein said accessing is performed wirelessly involving a storage system remote to said writing instrument.

35. A method as described in claim 33 further comprising adjusting said playback of said recording in response to recognition of a non-electronic playback control graphical element on said writing surface.

36. A method as described in claim 33 further comprising adjusting said playback of said recording in response to location of said writing instrument on any writing made during said recording;

and wherein said playback is moved to a segment of said recording that was recorded near in time to when said writing was made.

37. A method as described in claim 30 wherein said writing instrument further comprises a sensor for capturing optical images of said writing on said writing surface during recording of said audio signal.