Gesture recognition method and touch system incorporating the same

• US 7,411,575 B2
• Filed: 09/16/2003
• Issued: 08/12/2008
• Est. Priority Date: 09/16/2003
• Status: Active Grant

• Current Assignee: PixArt Imaging Incorporated
• Original Assignee: Smart Technologies ULC (Hon Hai Precision Industry Co., Ltd.)
• Inventors: Hill, Douglas B., Morrison, Gerald D.
• Primary Examiner(s): OSORIO, RICARDO
• Application Number:
  US10/662,813
• Publication Number:
  US 20050057524A1
• Time in Patent Office: 1,792 Days
• Field of Search: 345/156, 345/157, 345/163, 345173-177, 178 1801- 1809, 178 1901- 1905, 341/22, 341/33, 341/34, 382/181, 715/812, 715/773, 715863-866
• US Class Current: 345/156
• CPC Class Codes: G06F 2203/04806 Zoom, i.e. interaction techniques or interactors for controlling the zooming operation
• CPC Class Codes: G06F 2203/04808 Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
• CPC Class Codes: G06F 3/0421 by interrupting or reflecting a light beam, e.g. optical touch-screen
• CPC Class Codes: G06F 3/0485 Scrolling or panning
• CPC Class Codes: G06F 3/0486 Drag-and-drop
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text

Abstract

A gesture recognition method includes detecting multiple pointers in close proximity to a touch surface to determine if the multiple pointers are being used to perform a known gesture. When the multiple pointers are being used to perform a known gesture, executing a command associated with the gesture. A touch system incorporating the gesture recognition method is also provided.

Claims

1. A gesture recognition method comprising the steps of:

displaying an image on a touch surface;

capturing images looking generally across said touch surface;

processing the captured images to detect pointer contacts on said touch surface and examining said pointer contacts to recognize different successive pointer contacts representing a gesture based on the relative positions of said pointer contacts; and

when different successive pointer contacts representing a gesture occur, updating the displayed image in accordance with said gesture.

2. The method of claim 1 wherein said successive pointer contacts representing a gesture include successive finger contacts on said touch surface.

3. The method of claim 1 wherein said successive pointer contacts representing a gesture include a finger contact on said touch surface and an object contact on said touch surface.

4. The method of claim 1 wherein said successive pointer contacts representing a gesture include successive object contacts on said touch surface.

5. The method of claim 1 wherein said successive pointer contacts representing a gesture include one of successive finger contacts on said touch surface, a finger contact on said touch surface and an object contact on said touch surface, and multiple object contacts on said touch surface.

6. The method of claim 1 wherein said gesture represents a right-click event.

7. The method of claim 6 wherein said right-click event is represented by a first pointer contact on a displayed application, and a subsequent second pointer contact adjacent said first pointer contact.

8. The method of claim 6 wherein said right-click event is represented by a first pointer contact on a displayed application, and a subsequent second pointer contact that occurs within a threshold distance of said first pointer contact and while said first pointer contact is maintained.

9. A gesture recognition method comprising the steps of:

capturing images of a pointer input region;

processing the images to detect different successive pointers within said input region to determine if said different successive pointers are being used to perform a known gesture based on movement and type of said pointers within said input region and the relative positions of said pointers; and

when said different successive pointers are being used to perform a known gesture, executing a command associated with said gesture.

10. The method of claim 9 wherein during said processing, pointer contacts with or close pointer hovers over a touch surface are detected to determine if a known gesture is being performed.

11. The method of claim 10 wherein said multiple pointers include multiple fingers, at least one finger and at least one object, and multiple objects in close proximity to said touch surface.

12. The method of claim 10 wherein during said detecting the multiple pointers are examined to determine if one of a number of known gestures is being performed, each known gesture being associated with a different command.

13. The method of claim 12 wherein the movement of the multiple pointers relative to the touch surface determines the gesture being performed.

14. The method of claim 12 wherein the pointer type determines the gesture being performed.

15. The method of claim 12 wherein the movement of the multiple pointers relative to the touch surface and the pointer type determines the gesture being performed.

16. An input detection method in an interactive system capable of detecting movement of multiple pointers generally simultaneously within an input region, said method comprising the steps of:

capturing images looking generally across said input region;

analyzing said images to detect different successive pointers within said input region;

when different successive pointers are detected, examining data associated with said different successive pointers to determine if the data represents an input gesture, said data representing at least movement and type of said pointers within said input region and the relative positions of said pointers; and

when the data represents an input gesture, executing a command corresponding to the recognized input gesture.

17. The method of claim 16 wherein said gesture represents a right-click event.

18. The method of claim 16 wherein said gesture represents a scroll event.

19. A touch system comprising:

a touch surface;

at least one imaging device having a field of view looking generally across said touch surface; and

processing structure communicating with said at least one imaging device and analyzing images acquired by said at least one imaging device to determine the location on said touch surface where pointer contacts are made, when said touch surface is contacted by different pointers in succession, said processing structure examining the positions of said successive pointer contacts to determine if said successive pointer contacts represent a gesture and when said successive pointer contacts represent a gesture, said processing structure executing a command associated with said gesture.

20. A touch system according to claim 19 wherein a first pointer contact followed by a subsequent second pointer contact adjacent said first pointer contact represents said gesture.

21. A touch system according to claim 19 wherein each gesture is represented by specified successive pointer actions.

22. A touch system according to claim 21 wherein each gesture is further represented by pointer type.

23. An interactive input system comprising:

at least one imaging device having an input region within its field of view into which pointers are moved to generate user input; and

processing structure communicating with said at least one imaging device and analyzing each image acquired by said at least one imaging device to determine the action of said pointers in said input region, said processing structure determining when different successive pointer actions within said input region represent a gesture, when said different successive pointer actions represent a gesture, said processing structure executing a command corresponding to said gesture.

24. An interactive input system according to claim 23 wherein each gesture is further represented by pointer type.

25. In an interactive touch system, a method of simulating a fight-click mouse event comprising the steps of:

capturing images of a touch surface;

processing the images to detect a first pointer contact on the touch surface that represents a left-click mouse event;

processing the images to detect when a subsequent second pointer contact on said touch surface occurs within a threshold distance of said first pointer contact; and

generating a right-click mouse event in response to said detected second pointer contact, said second pointer contact being different that said first pointer contact.

26. The method of claim 25 wherein said second pointer contact must also occur during said first pointer contact in order for said right-click mouse event to be generated.

27. The method of claim 26 wherein said second pointer contact must also occur to the right of said first pointer contact in order for said right-click mouse event to be generated.

28. A gesture recognition method comprising:

detecting movement and type of different successive pointers relative to a touch surface to determine if the different successive pointers are being used to perform a known gesture; and

when the different successive pointers are being used to perform a known gesture, executing a command associated with said gesture.

29. The method of claim 28 wherein during said detecting, pointer contacts with or close pointer hovers over a touch surface are detected to determine if a known gesture is being performed.

30. The method of claim 29 wherein said pointers include one of multiple fingers, at least one finger and at least one object, and multiple objects in contact with or in close proximity to said touch surface.

31. The method of claim 29 wherein during said detecting the pointers are examined to determine if one of a number of known gestures is being performed, each known gesture being associated with a different command.

32. A gesture recognition method comprising the steps of:

capturing images of a pointer input region;

processing the images to detect movement and type of different successive pointers within said input region to determine if said different successive pointers are being used to perform a known gesture; and

when said different successive pointers are being used to perform a known gesture, executing a command associated with said gesture.

33. The method of claim 32 wherein during said processing the multiple pointers are examined to determine if one of a number of known gestures is being performed, each known gesture being associated with a different command.

34. A gesture recognition method comprising the steps of:

displaying an image on a touch surface;

capturing images looking generally across said touch surface;

processing the captured images to detect movement and type of different successive pointers used to contact said touch surface thereby to recognize an input gesture; and

when an input gesture is recognized, updating the displayed image in accordance with said recognized gesture.

35. The method of claim 34 wherein said different pointers comprise different fingers.

36. The method of claim 34 wherein said different pointers comprise a finger and an object.

37. The method of claim 34 wherein said input gesture represents a right-click event.

38. The method of claim 37 wherein said right-click event is represented by a first pointer contact on a displayed application, and a subsequent second pointer contact adjacent said first pointer contact.

39. The method of claim 37 wherein said right-click event is represented by a first pointer contact on a displayed application, and a subsequent second pointer contact that occurs within a threshold distance of said first pointer contact and while said first pointer contact is maintained.

40. The method of claim 34 wherein said input gesture represents a scroll event.