Gesture recognition method and touch system incorporating the same

• US 7,880,720 B2
• Filed: 12/23/2009
• Issued: 02/01/2011
• Est. Priority Date: 09/16/2003
• Status: Active Grant

• Current Assignee: PixArt Imaging Incorporated
• Original Assignee: Smart Technologies ULC (Hon Hai Precision Industry Co., Ltd.)
• Inventors: Hill, Douglas B., Morrison, Gerald D.
• Primary Examiner(s): OSORIO, RICARDO
• Application Number:
  US12/646,081
• Publication Number:
  US 20100164891A1
• Time in Patent Office: 405 Days
• Field of Search: 345/156, 345/157, 345/163, 345/173, 345/177, 178 1801- 1809, 178 1901- 1905, 341/22, 341/33, 341/34, 715/812, 715/773, 715863-866
• US Class Current: 345/156
• CPC Class Codes: G06F 2203/04806 Zoom, i.e. interaction techniques or interactors for controlling the zooming operation
• CPC Class Codes: G06F 2203/04808 Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
• CPC Class Codes: G06F 3/0421 by interrupting or reflecting a light beam, e.g. optical touch-screen
• CPC Class Codes: G06F 3/0485 Scrolling or panning
• CPC Class Codes: G06F 3/0486 Drag-and-drop
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text

Abstract

A gesture recognition method includes detecting multiple pointers in close proximity to a touch surface to determine if the multiple pointers are being used to perform a known gesture. When the multiple pointers are being used to perform a known gesture, executing a command associated with the gesture. A touch system incorporating the gesture recognition method is also provided.

Claims

1. An event generation method comprising the steps of:

capturing images using image devices looking generally across a touch surface;

processing the captured images to detect multiple pointer contacts on said touch surface and examining the multiple pointer contacts to determine if the spacing between the multiple pointer contacts is beyond a threshold distance;

if the spacing between the multiple pointer contacts is beyond the threshold distance, designating each pointer contact as a separate input and if the spacing between the multiple pointer contacts is within the threshold distance, further processing the captured images to detect relative movement of the multiple pointers on the touch surface; and

generating an event when the detected relative pointer movement represents a gesture.

2. The method of claim 1 wherein during said further processing, the images are processed to detect movement of multiple pointers towards one another.

3. The method of claim 1 wherein during said further processing, the images are processed to detect movement of multiple pointers away from one another.

4. The method of claim 1 wherein during said further processing, the images are processed to detect movement of one pointer in a generally arcuate path about another pointer.

5. The method of claim 1 wherein said event represents a graphical object manipulation command.

6. The method of claim 5 wherein said graphical object manipulation command is one of a graphical object resize command, a graphical object rotate and a graphical object translate command.

7. The method of claim 6 further comprising executing said graphical object manipulation command.

8. The method of claim 7 wherein during said further processing, the images are processed to detect movement of multiple pointers towards one another.

9. The method of claim 8 wherein during said further processing, the images are processed to detect movement of multiple pointers away from one another.

10. The method of claim 8 wherein during said further processing, the images are processed to detect movement of one pointer in a generally arcuate path about another pointer.

11. An event generation method comprising the steps of:

detecting successive pointers brought into contact with and oriented generally normal to a touch surface followed by relative movement of said pointers along said touch surface;

comparing the locations of the pointer contacts on the touch surface to determine if the pointer contacts are beyond a threshold distance from one another;

if the pointer contacts are beyond the threshold distance from one another, separately processing each pointer contact and if the pointer contacts are within the threshold distance, determining if the relative pointer movement along the touch surface represents a gesture; and

generating an event when the relative pointer movement represents a gesture.

12. The method of claim 11 wherein during said determining, the relative pointer movement is examined to determine if said pointers are moving towards one another.

13. The method of claim 11 wherein during said determining, the relative pointer movement is examined to determine if said pointers are moving away from one another.

14. The method of claim 11 wherein during said determining, the relative pointer movement is examined to determine if one pointer is moving in a generally arcuate path about another pointer.

15. The method of claim 11 wherein during said determining, the relative pointer movement is examined to determine if said pointers are moving in a generally similar direction.

16. The method of claim 11 wherein said event represents a graphical object manipulation command.

17. The method of claim 16 wherein said graphical object manipulation command is one of a graphical object resize command, a graphical object rotate and a graphical object translate command.

18. The method of claim 17 further comprising executing said graphical object manipulation command.

19. The method of claim 18 wherein during said determining, the relative pointer movement is examined to determine if said pointers are moving towards one another.

20. The method of claim 18 wherein during said determining, the relative pointer movement is examined to determine if said pointers are moving away from one another.

21. The method of claim 18 wherein during said determining, the relative pointer movement is examined to determine if one pointer is moving in a generally arcuate path about another pointer.

22. The method of claim 18 wherein during said determining, the relative pointer movement is examined to determine if said pointers are moving in a generally similar direction.

23. An event generation method comprising the steps of:

capturing images of a touch surface from different vantages using image devices that are proximate to and aimed to look generally across the touch surface;

processing the captured images to detect successive pointers brought into contact with said touch surface followed by relative movement of the pointers on said touch surface;

comparing the locations of the pointer contacts on the touch surface to determine if the pointer contacts are beyond a threshold distance from one another;

if the pointer contacts are beyond the threshold distance from one another, designating each pointer contact as a separate input and if the pointer contacts are within the threshold distance, determining if the relative pointer movement on the touch surface represents a gesture; and

generating an event when the relative pointer movement represents a gesture.

24. The method of claim 23 wherein during said determining, the relative pointer movement is examined to determine if the multiple pointers are moving towards one another.

25. The method of claim 23 wherein during said determining, the relative pointer movement is examined to determine if the multiple pointers are moving away from one another.

26. The method of claim 23 wherein during said determining, the relative pointer movement is examined to determine if one pointer is moving in a generally arcuate path about another pointer.

27. The method of claim 23 wherein said event represents a graphical object manipulation command.

28. The method of claim 27 wherein said graphical object manipulation command is one of a graphical object resize command, a graphical object rotate and a graphical object translate command.

29. The method of claim 28 further comprising executing said graphical object manipulation command.

30. The method of claim 29 wherein during said determining, the relative pointer movement is examined to determine if the multiple pointers are moving towards one another.

31. The method of claim 29 wherein during said determining, the relative pointer movement is examined to determine if the multiple pointers are moving away from one another.

32. The method of claim 31 wherein during said determining, the relative pointer movement is examined to determine if one pointer is moving in a generally arcuate path about another pointer.

33. An event generation method comprising the steps of:

detecting multiple pointers brought successively into contact with and oriented substantially normal to a touch surface;

examining the relative contact positions of the multiple pointers on the touch surface to determine if the relative contact positions are within a threshold spacing;

when the relative contact positions are within the threshold spacing, further examining the relative pointer movement to determine if the relative pointer movement represents a gesture; and

generating an event when the relative pointer movement represents a gesture.

34. The method of claim 33 wherein during said further examining, the relative pointer movement is examined to determine if said pointers are moving towards one another.

35. The method of claim 33 wherein during said further examining, the relative pointer movement is examined to determine if said pointers are moving away from one another.

36. The method of claim 33 wherein during said further examining, the relative pointer movement is examined to determine if one pointer is moving in a generally arcuate path about another pointer.

37. The method of claim 33 wherein during said further examining, the relative pointer movement is examined to determine if said pointers are moving in a generally similar direction.

38. The method of claim 33 wherein said event represents a graphical object manipulation command.

39. The method of claim 38 wherein said graphical object manipulation command is one of a graphical object resize command, a graphical object rotate and a graphical object translate command.

40. The method of claim 39 further comprising executing said graphical object manipulation command.

41. The method of claim 40 wherein during said further examining, the relative pointer movement is examined to determine if said pointers are moving towards one another.

42. The method of claim 40 wherein during said further examining, the relative pointer movement is examined to determine if said pointers are moving away from one another.

43. The method of claim 40 wherein during said further examining, the relative pointer movement is examined to determine if one pointer is moving in a generally arcuate path about another pointer.

44. The method of claim 40 wherein during said further examining, the relative pointer movement is examined to determine if said pointers are moving in a generally similar direction.

45. An event generation method comprising the steps of:

capturing images from different vantages using image devices that are proximate to and aimed to look generally across a touch surface;

processing the captured images to detect multiple pointers brought into successive contact with the touch surface followed by relative movement of the pointers on the touch surface;

comparing the locations of the pointer contacts on the touch surface to determine if the pointer contacts are beyond a threshold distance from one another;

if the pointer contacts are beyond the threshold distance from one another, processing each pointer contact separately and if the pointer contacts are within the threshold distance, determining if the relative pointer movement on the touch surface represents a gesture; and

generating an event when the relative pointer movement represents a gesture.

46. The method of claim 45 wherein during said determining, the relative pointer movement is examined to determine if the multiple pointers are moving towards one another.

47. The method of claim 45 wherein during said determining, the relative pointer movement is examined to determine if the multiple pointers are moving away from one another.

48. The method of claim 45 wherein during said determining, the relative pointer movement is examined to determine if the one pointer is moving in a generally arcuate path about another pointer.

49. The method of claim 45 wherein during said determining, the relative pointer movement is examined to determine if the multiple pointers are moving in a generally similar direction.

50. The method of claim 45 wherein said event represents a graphical object manipulation command.

51. The method of claim 50 wherein said graphical object manipulation command is one of a graphical object resize command, a graphical object rotate and a graphical object translate command.

52. The method of claim 51 further comprising executing said graphical object manipulation command.

53. The method of claim 52 wherein during said determining, the relative pointer movement is examined to determine if the multiple pointers are moving towards one another.

54. The method of claim 52 wherein during said determining, the relative pointer movement is examined to determine if the multiple pointers are moving away from one another.

55. The method of claim 52 wherein during said determining, the relative pointer movement is examined to determine if the one pointer is moving in a generally arcuate path about another pointer.

56. The method of claim 52 wherein during said determining, the relative pointer movement is examined to determine if the multiple pointers are moving in a generally similar direction.