Input device with audio capabilities

• US 7,627,703 B2
• Filed: 06/29/2005
• Issued: 12/01/2009
• Est. Priority Date: 06/29/2005
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Oliver, Thomas C.
• Primary Examiner(s): Tsai; Henry W. H.
• Assistant Examiner(s): Roche; John B
• Application Number:
  US11/168,469
• Publication Number:
  US 20070005849A1
• Time in Patent Office: 1,616 Days
• Field of Search: 710/72
• US Class Current: 710/72
• CPC Class Codes: G06F 3/03545 Pens or stylus
• CPC Class Codes: G06F 40/169 Annotation, e.g. comment data or footnotes
• CPC Class Codes: G06F 40/171 by use of digital ink
• CPC Class Codes: G06F 40/174 Form filling; Merging
• CPC Class Codes: G06F 40/18 of spreadsheets form-filling G06F40/174
• CPC Class Codes: G06F 40/186 Templates
• CPC Class Codes: G06F 40/58 Use of machine translation, e.g. for multi-lingual retrieval, for server-side translation for client devices or for real-time translation
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence

Abstract

An input device is provided for capturing handwriting strokes while in a handwriting mode and for providing audio-related functionality. The input device may be formed in the shape of a pen, and may or may not include an ink cartridge to facilitate movement of the input device in a familiar manner. In one configuration, an input device provides verbal messages, such as warnings to a user. In another configuration, an input device receives audible commands and performs related actions. In a further configuration, an input device recognizes specific movements, such as a word traced therewith, and provides audible information based on the movements. Methods for using input devices and computer-readable instructions for performing the methods are also provided.

Claims

1. An electronic pen, comprising:

a semantic input interface on the electronic pen;

a sensory output interface on the electronic pen;

a microphone on the electronic pen;

a processor on the electronic pen; and

a memory on the electronic pen, wherein the memory includes computer-executable instructions stored on the memory, wherein the computer-executable instructions are configured for;

detecting one of;

audio information external to the electronic pen, wherein the audio information is detected by the microphone of the electronic pen device and recorded to the memory of the electronic pen, or

a semantic ink object, wherein the semantic ink object is detected by the semantic input interface on the electronic pen;

when audio information is detected by the microphone of the electronic pen device, associating the stored audio information with an electronic document presently being modified by the electronic pen, storing the audio information with the electronic document, and displaying an icon in the presently modified electronic document that indicates the audio information; and

when the semantic ink object is detected by the semantic input interface on the electronic pen, matching the semantic ink object to preexisting output interface information and outputting the preexisting output interface information on the sensory output interface of the electronic pen.

2. The electronic pen of claim 1, wherein the semantic input interface includes a least one member of a group comprising:

a virtual ink sensor, a non-virtual ink sensor, a pressure sensor, an image sensor, and a inertial sensor.

3. The electronic pen of claim 1, wherein the sensory output interface includes at least one member of a group comprising:

a display and a speaker.

4. The electronic pen of claim 1, wherein the semantic ink object includes at least one member of a group comprising:

a letter, a number, a word, a phase, a shape and a symbol.

5. The electronic pen of claim 1, wherein semantic ink object includes at least one member of a group comprising:

a preexisting virtual ink object, a preexisting non-virtual ink object, a virtual ink object being generated and a non-virtual ink object being generated.

6. The electronic pen of claim 1, wherein matching the semantic ink object to preexisting output interface information includes translating the preexisting output information from a first language to a second language and outputting the translated output interface information to at least one member of a group comprising:

a display and a speaker.

7. The electronic pen of claim 1, wherein detecting the semantic ink object includes at least one member of a group comprising:

detecting the generation of a semantic ink object by the electronic pen and detecting the tracing of a preexisting semantic ink object by the electronic pen.

19. The electronic pen of claim 1, wherein displaying the icon in the presently modified electronic document that indicates the audio information includes displaying the icon adjacent to a portion of the document presently being modified.

8. A computer-readable storage medium on an electronic pen, wherein the computer-readable storage medium has computer-executable instructions for causing the output of pen interface information in response to a detection of a semantic ink object, the instructions comprising:

detecting, on the electronic pen, one of;

audio information external to the electronic pen, wherein the audio information is detected by a microphone of the electronic pen and recorded to a memory of the electronic pen, or

a semantic ink object, wherein the semantic ink object is detected by a semantic input interface on the electronic pen;

when audio information is detected by the microphone of the electronic pen, associating the stored audio information with an electronic document presently being modified by the electronic pen, storing the audio information with the electronic document, and displaying an icon in the presently modified electronic document adjacent to the portion of the document presently being modified, wherein the icon indicates a presence of the audio information; and

when the semantic ink object is detected by the semantic input interface on the electronic pen, matching the semantic ink object to preexisting output interface information and outputting the preexisting output interface information on a sensory output interface of the electronic pen.

9. The computer-readable storage medium of claim 8, wherein the semantic input interface includes a least one member of a group comprising:

a virtual ink sensor on the pen, a non-virtual ink sensor on the pen, a pressure sensor on the pen, an image sensor on the pen, and a inertial sensor on the pen.

10. The computer-readable storage medium of claim 8, wherein the sensory output interface includes at least one member of a group comprising:

a display on the pen and a speaker on the pen.

11. The computer-readable storage medium of claim 8, wherein the semantic ink object includes at least one member of a group comprising:

a letter, a number, a word, a phrase, a shape and a symbol.

12. The computer-readable storage medium of claim 8, wherein semantic ink object includes at least one member of a group comprising:

a preexisting virtual ink object, a preexisting non-virtual ink object, a virtual ink object being generated and a non-virtual ink object being generated.

13. The computer-readable storage medium of claim 8, wherein matching the semantic ink object to preexisting output interface information includes translating the preexisting output information from a first language to a second language and outputting the translated output interface information to at least one member of a group comprising:

a display on the pen and a speaker on the pen.

14. The computer-readable storage medium of claim 8, wherein detecting the semantic ink object includes at least one member of a group comprising:

detecting the generation of a semantic ink object by the electronic pen and detecting the tracing of a preexisting semantic ink object by the electronic pen.

15. A computer-implemented method for causing an output of pen interface information in response to a detection of a semantic ink object, the method comprising:

detecting, on an electronic pen, one of;

audio information external to the electronic pen, wherein the audio information is detected by a microphone of the electronic pen device and recorded to a memory of the electronic pen, or

a semantic ink object, wherein the semantic ink object is detected by a semantic input interface on the electronic pen;

when audio information is detected by the microphone of the electronic pen device, associating the stored audio information with an electronic document presently being modified by the electronic pen, storing the audio information with the electronic document, and displaying an icon in the presently modified electronic document that indicates the audio information; and

when the semantic ink object is detected by the semantic input interface on the electronic pen, matching the semantic ink object to preexisting output interface information and outputting the preexisting output interface information on a sensory output interface of the electronic pen.

16. The computer-implemented method of claim 15, wherein the semantic ink object includes at least one member of a group comprising:

a preexisting virtual ink object, a preexisting non-virtual ink object, a virtual ink object being generated and a non-virtual ink object being generated.

17. The computer-implemented method of claim 15, wherein matching the semantic ink object to preexisting output interface information includes translating the preexisting output information from a first language to a second language and outputting the translated output interface information to at least one member of a group comprising:

a display and a speaker.

18. The computer-implemented method of claim 15, wherein detecting the semantic ink object includes at least one member of a group comprising:

detecting the generation of a semantic ink object by the electronic pen and detecting the tracing of a preexisting semantic ink object by the electronic pen.

20. The computer-implemented method of claim 15, wherein displaying the icon in the presently modified electronic document that indicates the audio information includes displaying the icon adjacent to a portion of the document presently being modified.