Techniques for recognizing multi-shape, multi-touch gestures including finger and non-finger touches input to a touch panel interface

• US 8,643,615 B2
• Filed: 06/27/2011
• Issued: 02/04/2014
• Est. Priority Date: 12/18/2009
• Status: Active Grant

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: Adamson, Peter S., Yarvis, Mark D., Graumann, David L., Sharma, Sangita
• Primary Examiner(s): GIESY, ADAM
• Application Number:
  US13/169,546
• Publication Number:
  US 20110254797A1
• Time in Patent Office: 953 Days
• Field of Search: None
• US Class Current: 345/173
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text

Abstract

Briefly, a method and apparatus for recognizing multi-shape, multi-touch gestures including finger and non-finger touches input to a touch panel interface is disclosed. The method may include receiving user input with a touch panel interface, recognizing a multi-shape, multi-touch gesture including finger and non-finger touches in the user input, and performing an action associated with the multi-touch gesture including finger and non-finger touches.

Claims

1. A method, comprising:

receiving user input with a touch panel interface;

recognizing a multi-shape, multi-touch gesture including finger and non-finger touches in the user input; and

performing an action associated with the multi-touch gesture including finger and non-finger touches, the action comprising adding expressive features to text and removing resting interactions of the finger and non-finger touches on the touch panel interface, wherein the adding expressive features to the text comprise one or more of the following;

tilting an angle of the text based, at least in part, on an angle of the finger or non-finger touch;

bolding the text based, at least in part, on an amount of time the finger or non-finger touch is in contact with the touch panel interface;

bolding the text based, at least in part, on an amount of force applied by the finger or non-finger touch to the touch panel interface;

increasing or decreasing a size of the text based, at least in part, on an amount of force the finger or non-finger touch to the touch panel interface;

or

increasing or decreasing one or more distances between words in the text based, at least in part, on the finger touches comprising two or more fingers moving away from each other or the two or more fingers moving towards each other.

2. The method as recited in claim 1, wherein recognizing the multi-shape, multi-touch gesture including finger and non-finger touches comprises:

identifying a region of touches;

tracking changes in the region over a period of time;

extracting shape and location features of the changes in the region;

creating a time series of the extracted features;

normalizing the features; and

matching the normalized features to known patterns.

3. The method as recited in claim 1, wherein performing the action associated with the multi-touch gesture comprises executing a complex action which otherwise would require one of multiple user steps, a short cut action, and a macro function to execute.

4. The method of claim 1, wherein the user input is made on dynamically-allocated region of the touch panel.

5. The method of claim 1, wherein the touch panel interface is a screen associated with a handheld device, cell phone or consumer electronic device and the multi-shape, multi-touch gesture including finger and non-finger touches is input anywhere on the screen to activate a desired response.

6. The method of claim 1, further comprising:

determining the multi-shape, multi-touch gesture including finger and non-finger touches; and

implementing a command assigned to the multi-shape, multi-touch gesture including finger and non-finger touches.

7. The method of claim 1, further comprising:

determining the multi-shape, multi-touch gesture including finger and non-finger touches is not recognized; and

prompting the user to associate the multi-shape, multi-touch gesture including finger and non-finger touches with a computing event based, at least in part, on determining the multi-shape, multi-touch gesture including finger and non-finger touches is not recognized.

8. An apparatus, comprising:

a touch subsystem including a touch panel for receiving user input in the form of a multi-shape, multi-touch gesture including finger and non-finger touches; and

a processing unit to recognize the multi-shape, multi-touch gesture including finger and non-finger touches in the user input and perform an action associated with the multi-shape, multi-touch gesture including finger and non-finger touches, the action comprising adding expressive features to text and filtering to remove resting interactions of the finger and non-finger touches, wherein the adding expressive features to the text comprise one or more of the following;

tilting an angle of the text based, at least in part, on an angle of the finger or non-finger touch;

bolding the text based, at least in part, on an amount of time the finger or non-finger touch is in contact with the touch panel interface;

bolding the text based, at least in part, on an amount of force applied by the finger or non-finger touch to the touch panel interface;

increasing or decreasing a size of the text based, at least in part, on an amount of force the finger or non-finger touch to the touch panel interface;

or

increasing or decreasing one or more distances between words in the text based, at least in part, on the finger touches comprising two or more fingers moving away from each other or the two or more fingers moving towards each other.

9. The apparatus as recited in claim 8, wherein recognizing the multi-shape, multi-touch gesture including finger and non-finger touches comprises:

identifying a region of touches;

tracking changes in the region over a period of time;

extracting shape and location features of the changes in the region;

creating a time series of the extracted features;

normalizing the features; and

matching the normalized features to known patterns.

10. The apparatus as recited in claim 8, wherein executing the action comprises executing a complex action which otherwise would require one of multiple user steps, a short cut action, and a macro function to execute.

11. The apparatus of claim 8, wherein the user input is made on a dynamically-allocated region of the touch panel.

12. The apparatus of claim 8, wherein the touch panel interface is a screen associated with a handheld device, cell phone or consumer electronic device and the multi-shape, multi-touch gesture including finger and non-finger touches is input anywhere on the screen to activate a desired response.

13. The apparatus of claim 8, further comprising:

determining the multi-shape, multi-touch gesture including finger and non-finger touches; and

implementing a command assigned to the multi-shape, multi-touch gesture including finger and non-finger touches.

14. The apparatus of claim 8, further comprising:

determining the multi-shape, multi-touch gesture including finger and non-finger touches is not recognized; and

prompting the user to associate the multi-shape, multi-touch gesture including finger and non-finger touches with a computing event based, at least in part, on determining the multi-shape, multi-touch gesture including finger and non-finger touches is not recognized.

15. A non-volatile computer readable medium encoded with computer executable instructions, which when accessed, cause a machine to perform operations comprising:

receiving user input with a touch panel interface;

recognizing a multi-shape, multi-touch gesture including finger and non-finger touches in the user input, the recognizing of the multi-shape, multi-touch gesture including finger and non-finger touches comprises;

identifying a region for the finger and non-finger touches based, at least in part, on a location of an initiation gesture on the touch panel interface;

tracking set of finger and non-finger touches in the region over a predetermined period of time when the initiation gesture is detected, the region comprising area on the touch panel interface that encompasses the set of finger and non-finger touches that are grouped together based on their proximity to the region or to each other;

extracting shape and location features of the set of finger and non-finger touches in the region at an end of the period of time;

creating a time series of the extracted features;

normalizing the time series of the extracted features; and

matching the normalized time series of the extracted features to known patterns; and

performing an action associated with at least one of the known patterns.

16. The non-volatile computer readable medium encoded with computer executable instructions of claim 15, wherein the action is one of executing a system action, adding expressive features to text, and filtering to remove resting interactions.

17. The non-volatile computer readable medium encoded with computer executable instructions of claim 15, wherein executing a system action comprises executing a complex action which otherwise would require one of multiple user steps, a short cut action, and a macro function to execute.

18. The non-volatile computer readable medium encoded with computer executable instructions of claim 15, wherein the user input is made on a dynamically-allocated region of the touch panel.

19. The non-volatile computer readable medium encoded with computer executable instructions of claim 15, wherein the touch panel interface is a screen associated with a handheld device, cell phone or consumer electronic device and the multi-shape, multi-touch gesture including finger and non-finger touches is input anywhere on the screen to activate a desired response.

20. The non-volatile computer readable medium encoded with computer executable instructions of claim 15, further comprising:

determining the multi-shape, multi-touch gesture including finger and non-finger touches; and

implementing a command assigned to the multi-shape, multi-touch gesture including finger and non-finger touches.

21. The non-volatile computer readable medium encoded with computer executable instructions of claim 15, further comprising:

determining the multi-shape, multi-touch gesture including finger and non-finger touches is not recognized; and

prompting the user to associate the multi-shape, multi-touch gesture including finger and non-finger touches with a computing event based, at least in part, on determining the multi-shape, multi-touch gesture including finger and non-finger touches is not recognized.

22. The non-volatile computer readable medium encoded with computer executable instructions of claim 15, wherein the normalizing of the time series of the extracted features accounts for variations of speed, location, or rotation in the set of finger and non-finger gestures.

23. A handheld device, comprising:

a screen;

a touch subsystem including a touch panel incorporated into said screen for receiving user input in the form of a multi-shape, multi-touch gesture including finger and non-finger touches; and

a processing unit to recognize the multi-shape, multi-touch gesture including finger and non-finger touches and perform an action associated with the a multi-shape, multi-touch gesture including finger and non-finger touches, the action comprising adding expressive features to text, wherein recognizing the multi-shape, multi-touch gesture including finger and non-finger touches comprises;

identifying a region of touches;

tracking changes in the region over a period of time;

extracting shape and location features of the changes in the region;

creating a time series of the extracted features;

normalizing the features; and

matching the normalized features to known patterns.

24. The handheld device of claim 23, wherein executing a system action comprises executing a complex action which otherwise would require one of multiple user steps, a short cut action, and a macro function to execute.

25. The handheld device of claim 23, wherein the user input is made on a dynamically-allocated region of the touch panel.

26. The handheld device of claim 23, further comprising:

determining the multi-shape, multi-touch gesture including finger and non-finger touches; and

implementing a command assigned to the multi-shape, multi-touch gesture including finger and non-finger touches.

27. The handheld device of claim 23, further comprising:

determining the multi-shape, multi-touch gesture including finger and non-finger touches is not recognized; and

prompting the user to associate the multi-shape, multi-touch gesture including finger and non-finger touches with a computing event based, at least in part, on determining the multi-shape, multi-touch gesture including finger and non-finger touches is not recognized.