ALWAYS-AVAILABLE INPUT THROUGH FINGER INSTRUMENTATION

• US 20140098067A1
• Filed: 10/02/2013
• Published: 04/10/2014
• Est. Priority Date: 10/02/2012
• Status: Active Grant

• Current Assignee: Autodesk, Inc.
• Original Assignee: Autodesk, Inc.
• Inventors: GROSSMAN, Tovi, FITZMAURICE, George, YANG, Xing-Dong, WIGDOR, Daniel
• Granted Patent: US 9,921,687 B2
• Time in Patent Office: Days
• US Class Current: 345/175
• CPC Class Codes: G06F 2203/0331 Finger worn pointing device
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0317 in co-operation with a patterned surface, e.g. absolute position or relative movement detection for an optical mouse or pen positioned with respect to a coded surface
• CPC Class Codes: G06F 3/0321 by optically sensing the absolute position with respect to a regularly patterned surface forming a passive digitiser, e.g. pen optically detecting position indicative tags printed on a paper sheet constructional details of pen-shaped pointing devices G06F3/03545, G06F3/03542, G06F3/037
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386

Abstract

A finger device initiates actions on a computer system when placed in contact with a surface. The finger device includes instrumentation that captures images and gestures. When in contact with a surface, the finger device captures images of the surface and gestures made on the surface. The finger device also transmits the images and gesture data to the computer system. An application on the computer system matches the images received from the finger device to a representation of the surface, identifies an action associated with the surface representation and gesture, and executes the action. Instrumenting the finger instead of the surface, allows a user to configure virtually any surface to accept touch input.

Claims

1. A method for initiating an action in response to a user touching a surface, the method comprising:

receiving an image from a device instrumenting a finger of the user, wherein the finger is in contact with a surface;

identifying, from the image, the surface contacted by the finger of the user;

matching the identified surface to an action executed by a computing device; and

executing the action.

2. The method of claim 1, wherein identifying the surface comprises:

identifying one or more features from the image; and

identifying a matching of a surface representation in a library based upon the identified features.

3. The method of claim 2, wherein identifying the surface representation in a library comprises passing the identified features to a classifier, wherein the classifier is configured to predict the surface representation in the library based upon the identified features.

4. The method of claim 2, wherein retrieving the image comprises:

determining when the device is in contact with the surface; and

capturing the image in response to the device contacting the surface.

5. The method of claim 2, wherein the identified features include features of a texture shown in the image.

6. The method of claim 1, wherein matching the identified surface to the action further comprises:

determining a gesture made by the user while touching the identified surface; and

identifying the action based on the identified surface and the gesture.

7. The method of claim 6, wherein determining the gesture comprises:

capturing one or more coordinates identifying an initial position for a coordinate system and then subsequent movement relative to that initial position;

identifying the gesture from the one or more coordinates.

8. A computer-readable storage medium storing instructions that, when executed by a processor, cause the processor to perform an operation for initiating an action in response to a user touching a surface, the method comprising:

receiving an image from a device instrumenting a finger of the user, wherein the finger is in contact with a surface;

identifying, from the image, the surface contacted by the finger of the user;

matching the identified surface to an action executed by a computing device; and

executing the action.

9. The computer readable storage medium of claim 8, wherein identifying the surface comprises:

identifying one or more features from the image; and

identifying a matching of a surface representation in a library based upon the identified features.

10. The computer readable storage medium of claim 9, wherein identifying the surface representation in a library comprises passing the identified features to a classifier, wherein the classifier is configured to predict the surface representation in the library based upon the identified features.

11. The computer readable storage medium of claim 9, wherein retrieving the image comprises:

determining when the device is in contact with the surface; and

capturing the image in response to the device contacting the surface.

12. The computer readable storage medium of claim 9, wherein the identified features include features of a texture shown in the image.

13. The computer readable storage medium of claim 8, wherein matching the identified surface to the action further comprises:

determining a gesture made by the user while touching the identified surface; and

identifying the action based on the identified surface and the gesture.

14. The computer readable storage medium of claim 13, wherein determining the gesture comprises:

capturing one or more coordinates identifying an initial position for a coordinate system and then subsequent movement relative to that initial position; and

identifying the gesture from the one or more coordinates.

15. A computer system, comprising:

a memory; and

a processor storing one or more programs configured to perform an operation for initiating an action in response to a user touching a surface, the method comprising;

receiving an image from a device instrumenting a finger of the user, wherein the finger is in contact with a surface,

identifying, from the image, the surface contacted by the finger of the user,

matching the identified surface to an action executed by a computing device, and

executing the action.

16. The system of claim 15, wherein identifying the surface comprises:

identifying one or more features from the image; and

identifying a matching of a surface representation in a library based upon the identified features.

17. The system of claim 16, wherein identifying the surface representation in a library comprises passing the identified features to a classifier, wherein the classifier is configured to predict the surface representation in the library based upon the identified features.

18. The system of claim 16, wherein retrieving the image comprises:

determining when the device is in contact with the surface; and

capturing the image in response to the device contacting the surface.

19. The system of claim 16, wherein the identified features include features of a texture shown in the image.

20. The system of claim 15, wherein matching the identified surface to the action further comprises:

determining a gesture made by the user while touching the identified surface; and

identifying the action based on the identified surface and the gesture.

21. The system of claim 20, wherein determining the gesture comprises:

capturing one or more coordinates identifying an initial position for a coordinate system and then subsequent movement relative to that initial position; and

identifying the gesture from the one or more coordinates

22. A device worn on a finger, comprising:

a camera configured to capture images of a surface; and

a microcontroller configured to detect when the finger of a user wearing the device touches a surface and, in response, (i) capture an image of the surface and (ii) transmit the image to a computing system.

23. The device of claim 22, wherein the microcontroller is further configured to measure a relative movement of the finger device while in contact with the surface.