Multi-Modal Learning System

• US 20090253107A1
• Filed: 03/31/2009
• Published: 10/08/2009
• Est. Priority Date: 04/03/2008
• Status: Active Grant

• Current Assignee: Livescribe Incorporated (Anoto Group AB)
• Original Assignee: Livescribe Incorporated (Anoto Group AB)
• Inventors: Marggraff, Jim
• Granted Patent: US 8,944,824 B2
• Time in Patent Office: Days
• US Class Current: 434/162
• CPC Class Codes: G06F 3/0317 in co-operation with a patterned surface, e.g. absolute position or relative movement detection for an optical mouse or pen positioned with respect to a coded surface
• CPC Class Codes: G06F 3/0321 by optically sensing the absolute position with respect to a regularly patterned surface forming a passive digitiser, e.g. pen optically detecting position indicative tags printed on a paper sheet constructional details of pen-shaped pointing devices G06F3/03545, G06F3/03542, G06F3/037
• CPC Class Codes: G06F 3/03545 Pens or stylus
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06V 30/1423 the instrument generating sequences of position coordinates corresponding to handwriting preprocessing or recognising digital ink G06V30/32
• CPC Class Codes: G06V 30/387 using human interaction, e.g. selection of the best displayed recognition candidate
• CPC Class Codes: G09B 11/00 Teaching hand-writing, shorthand, drawing, or painting
• CPC Class Codes: G09B 19/06 Foreign languages with audible presentation of material to be studied G09B5/04

Abstract

An instruction is presented to a user for making a target gesture. The target gesture may be a portion of an exemplary or model symbol. The instruction may be presented in various ways, such as being printed on a writing surface or being played in audio format through a speaker of a smart pen device. A writing gesture made on a writing surface by the user is digitally captured using a smart pen device. The captured writing gesture is compared to the target gesture and feedback is determined based on this comparison. This feedback may indicate the correctness of the user'"'"'s writing gesture. The feedback may be presented to the user through various means, including through the smart pen device. The comparison may also be used to determine a next instruction for the user to follow.

Claims

1. A method for instruction, the method comprising:

presenting an instruction to a user for making a target gesture;

digitally capturing a gesture made on a dot-enabled surface using a smart pen device;

determining feedback based on a comparison of the gesture with the target gesture; and

presenting the feedback to the user.

2. The method of claim 1, further comprising:

receiving an instructional module, the instructional module comprising a plurality of instructions and a plurality of target gestures, each instruction corresponding to a target gesture, the instructions having a defined sequence.

3. The method of claim 1, further comprising:

determining a next instruction to present to the user based on the comparison of the gesture with the target gesture.

4. The method of claim 1, wherein the feedback is presented on a display of the smart pen device.

5. The method of claim 1, wherein the feedback is presented through a speaker of the smart pen device.

6. The method of claim 1, wherein the feedback is presented haptically through the smart pen device.

7. The method of claim 1, wherein the feedback is presented substantially in real time as the gesture is captured.

8. The method of claim 1, wherein the gesture is digitally captured using an imaging system in a smart pen.

9. The method of claim 1, wherein the dot-enabled surface comprises an encoded coordinate system that is readable by the smart pen.

10. The method of claim 1, further comprising:

digitally capturing a target gesture made on a dot-enabled surface by the second user using a smart pen device; and

digitally capturing an instruction corresponding to the target gesture from the second user through a speaker of the smart pen device.

11. A computer program product for instruction using a smart pen, the computer program product comprising a computer-readable storage medium containing computer program code for:

digitally capturing a gesture made on a dot-enabled surface using a smart pen device;

determining feedback based on a comparison of the gesture with a target gesture; and

presenting the feedback to the user.

12. The computer program product of claim 11, wherein written instructions for making the target gesture are provided to the user.

13. The computer program product of claim 11, wherein the computer program code is further configured for:

presenting an instruction to a user for making the target gesture.

14. The computer program product of claim 13, wherein the computer program code is further configured for:

determining a next instruction to present to the user based on the comparison of the gesture with the target gesture.

15. The computer program product of claim 11, wherein the computer program code is further configured for:

receiving an instructional module, the instructional module comprising a plurality of instructions and a plurality of target gestures, each instruction corresponding to a target gesture, the instructions having a defined sequence.

16. The computer program product of claim 11, wherein the computer program code is configured for presenting the feedback on a display of the smart pen device.

17. The computer program product of claim 11, wherein the computer program code is configured for presenting the feedback in substantially in real time as the gesture is captured.

18. The computer program product of claim 11, wherein the gesture is digitally captured using an imaging system in a smart pen.

19. The computer program product of claim 11, wherein the dot-enabled surface comprises an encoded coordinate system that is readable by the smart pen.

20. The computer program product of claim 11, wherein the computer program code is further configured for:

digitally capturing a target gesture made on a dot-enabled surface by the second user using a smart pen device; and

digitally capturing an instruction corresponding to the target gesture from the second user through a speaker of the smart pen device.