Multi-Modal Controller

• US 20090251441A1
• Filed: 03/31/2009
• Published: 10/08/2009
• Est. Priority Date: 04/03/2008
• Status: Abandoned Application

• Current Assignee: Livescribe Incorporated (Anoto Group AB)
• Original Assignee: Livescribe Incorporated (Anoto Group AB)
• Inventors: Marggraff, Jam, Edgecomb, Tracy L., Pesic, Alexander Sasha
• Application Number:
  US12/415,780
• Publication Number:
  US 20090251441A1
• Time in Patent Office: Days
• US Class Current: 345/179
• CPC Class Codes: G06F 3/0321 by optically sensing the absolute position with respect to a regularly patterned surface forming a passive digitiser, e.g. pen optically detecting position indicative tags printed on a paper sheet constructional details of pen-shaped pointing devices G06F3/03545, G06F3/03542, G06F3/037
• CPC Class Codes: G06F 3/03545 Pens or stylus
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text

Abstract

Control inputs are provided to an application executing on a mobile computing device by moving the mobile computing device in certain recognizable patterns. The control inputs may execute various functions in the application such as starting or stopping audio playback or navigating through a menu. A writing gesture made by a user on a writing surface using a smart pen device is digitally captured. This gesture may be, for example, a tap or a stroke of the smart pen device on the writing surface. A control on the writing surface is identified, where the control at least partially corresponds to a location of the writing gesture on the writing surface. A control input is determined based on the identified control and the writing gesture. Responsive to the control input, a command is executed in an application running on the smart pen device or an attached computing system.

Claims

1. A method for receiving inputs through controls, the method comprising:

digitally capturing a writing gesture made on a writing surface using a smart pen device;

identifying a control on the writing surface, the control at least partially corresponding to a location of the writing gesture on the writing surface;

identifying an application associated with the control based on stored control information describing the identified control;

determining a control input based on the identified control and the writing gesture; and

responsive to the control input, switching to the identified application and executing a command in the identified application running on the smart pen device or an attached computing system.

2. The method of claim 1, wherein the application associated with the control is the application that was active when the control was first used.

3. The method of claim 1, further comprising:

identifying content on the writing surface associated with the control based on the stored control information;

wherein the executed command performs an operation on the identified content.

4. The method of claim 1, wherein executing the command further comprises:

presenting a result of the command to the user using an output device of the smart pen device.

5. The method of claim 4, wherein the output device comprises a display of the smart pen device.

6. The method of claim 1, wherein executing the command further comprises:

presenting a result of the command to the user using haptic feedback through the smart pen device.

7. The method of claim 1, wherein the command comprises navigating to a menu item in a menu of the application.

8. The method of claim 1, wherein the application comprises a playback application, and wherein the command comprises starting or stopping playback.

9. A method initializing a user-created control, the method comprising:

digitally capturing a writing gesture made on a writing surface using a smart pen device;

recognizing that the writing gesture comprises a control, the recognizing based on a pattern of the writing gesture;

determining a type of the control based on the pattern of the writing gesture;

determining a location of the control based on the location of the gesture on the writing surface;

determining an application associated with the control, where the application associated with the control is a currently running application; and

storing the location of the control, the type of the control, and the application associated with the control in a memory of the smart pen device.

10. The method of claim 10, wherein recognizing that the writing gesture comprises a control further comprises:

identifying a signaling gesture as a part of the writing gesture.

11. A system for providing instruction, the system comprising:

a smart pen device comprising;

a processor;

a storage medium;

a gesture capture system configured to capture a writing gesture made on a writing surface; and

instructions contained on the storage medium and capable of being executed by the processor, the instructions for identifying a control on the writing surface, the control at least partially including the location of the writing gesture on the writing surface, for identifying an application associated with the control based on stored control information describing the identified control, for determining a control input based on the identified control and the writing gesture, and for, responsive to the control input, switching to the identified application and executing a command in the identified application running on the smart pen device.

12. The system of claim 11, wherein the application associated with the control is the application that was active when the control was first used.

13. The system of claim 11, wherein the instructions are further configured for identifying content on the writing surface associated with the control based on the stored control information and wherein the executed command performs an operation on the identified content.

14. The system of claim 11, wherein executing the command further comprises:

presenting a result of the command to the user using an output device of the smart pen device.

15. The system of claim 14, wherein the output device comprises a display of the smart pen device.

16. The system of claim 11, wherein executing the command further comprises:

presenting a result of the command to the user using haptic feedback through the smart pen device.

17. The system of claim 11, wherein the command comprises navigating to a menu item in a menu of the application.

18. The system of claim 11, wherein the application comprises a playback application, and wherein the command comprises starting or stopping playback.