Bounding box gesture recognition on a touch detecting interactive display

• US 8,139,043 B2
• Filed: 11/09/2009
• Issued: 03/20/2012
• Est. Priority Date: 08/06/2004
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: TOUCHTABLE, INC.
• Inventors: Hillis, W. Daniel
• Primary Examiner(s): Nguyen, Jimmy H
• Application Number:
  US12/615,224
• Publication Number:
  US 20100117979A1
• Time in Patent Office: 862 Days
• Field of Search: 345173-178, 178 1801- 1811, 715/863, 382/181, 382/182, 382/184, 382/187, 382/189
• US Class Current: 345/173
• CPC Class Codes: G06F 2203/04806 Zoom, i.e. interaction techniques or interactors for controlling the zooming operation
• CPC Class Codes: G06F 2203/04808 Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text

Abstract

The invention provides a method and apparatus for identifying gestures performed by a user to control an interactive display. The gestures are identified based on a bounding box enclosing the points at which a user contacts a touch sensor corresponding with the display surface. The invention thus permits the use of inexpensive and highly reliable grid-based touch sensors that provide a bounding box to describe contact information. In identifying the gestures, the position, motion, shape, and deformation of the bounding box may all be considered. In particular, the center, width, height, aspect ratio, length of the diagonal, and orientation of the diagonal of the bounding box may be determined. A stretch factor, defined as the maximum of the ratio of the height of the bounding box to the width of the bounding box and the ratio of the width of the bounding box to the height of the bounding box, may also be computed. Finally, gestures may be identified based on the changes in time of these characteristics and quantities.

Claims

1. In an interactive display, a method for identifying a specific user gesture comprising the steps of:

displaying imagery on an imaging surface;

detecting a plurality of contact locations at which at least one user contacts a touch sensor to control said display;

responsive to a determination that a deformation of a bounding box enclosing at least two of said plurality of contacts locations exceeds a predetermined deformation threshold and responsive to a determination that a motion of said bounding box is less than a predetermined motion threshold, identifying said specific user gesture as a rotate gesture;

associating said specific user gesture with a display command; and

executing said display command to alter the display of said imagery.

2. The method of claim 1, wherein a display command affects any of:

a translation of said imagery;

a change in magnification of said imagery; and

a change in orientation of said imagery.

3. The method of claim 1, further comprising the step of:

responsive to a determination that a motion of said bounding box exceeds said predetermined motion threshold and that deformation of said bounding box is less than said predetermined deformation threshold, identifying said specific user gesture as a pan command.

4. The method of claim 1, further comprising the step of:

responsive to a determination that a stretch factor of said bounding box exceeds a predetermined stretch threshold and responsive to a determination that said stretch factor is either increasing or decreasing, identifying said specific user gesture as a zoom gesture.

5. In an interactive display, a method for identifying a specific user gesture comprising the steps of:

displaying imagery on an imaging surface;

detecting a plurality of contact locations at which at least one user contacts a touch sensor to control said display;

responsive to a determination that a stretch factor of a bounding box surrounding at least two of said contact locations exceeds a predetermined stretch threshold and responsive to a determination that said stretch factor is either increasing or decreasing, identifying said specific user gesture as a zoom gesture;

associating said specific user gesture with a display command; and

executing said display command to alter the display of said imagery.

6. The method of claim 5, wherein a display command affects any of:

a translation of said imagery;

a change in magnification of said imagery; and

a change in orientation of said imagery.

7. The method of claim 5, further comprising the step of:

responsive to a determination that a motion of said bounding box exceeds said predetermined motion threshold and that deformation of said bounding box is less than said predetermined deformation threshold, identifying said specific user gesture as a pan command.

8. In an interactive display, a method for identifying a specific user gesture comprising the steps of:

displaying imagery on an imaging surface;

detecting a plurality of contact locations at which at least one user contacts a touch sensor to control said display;

responsive to a determination that a motion of a bounding box enclosing at least two of said plurality of contact locations exceeds said predetermined motion threshold and that deformation of said bounding box is less than said predetermined deformation threshold, identifying said specific user gesture as a pan gesture;

responsive to a determination that a stretch factor of a bounding box exceeds a predetermined stretch threshold and responsive to a determination that said stretch factor is either increasing or decreasing, identifying said specific user gesture as a zoom gesture;

associating said specific user gesture with a display command; and

executing said display command to alter the display of said imagery.

9. The method of claim 8, wherein a display command affects any of:

a translation of said imagery;

a change in magnification of said imagery; and

a change in orientation of said imagery.

10. The method of claim 8, further comprising the step of:

responsive to a determination that a deformation of a bounding box enclosing at least two of said plurality of contacts locations exceeds a predetermined deformation threshold and responsive to a determination that a motion of said bounding box is less than a predetermined motion threshold, identifying said specific user gesture as a rotate gesture.

11. In an interactive display, a method for identifying a specific user gesture comprising the steps of:

displaying imagery on an imaging surface;

detecting a plurality of contact locations at which at least one user contacts a touch sensor to control said display;

responsive to a determination that a deformation of a bounding box enclosing at least two of said plurality of contacts locations exceeds a predetermined deformation threshold and responsive to a determination that a motion of said bounding box is less than a predetermined motion threshold, identifying said specific user gesture as a rotate gesture;

responsive to a determination that a motion of said bounding box exceeds said predetermined motion threshold and that deformation of said bounding box is less than said predetermined deformation threshold, identifying said specific user gesture as a pan gesture;

responsive to a determination that a stretch factor of said bounding box exceeds a predetermined stretch threshold and responsive to a determination that said stretch factor is either increasing or decreasing, identifying said specific user gesture as a zoom gesture;

associating said specific user gesture with a display command; and

executing said display command to alter the display of said imagery.

12. An interactive display comprising:

means for displaying imagery on an imaging surface;

a touch sensor corresponding to said imaging surface;

means for detecting a plurality of contact locations at which at least one user contacts said touch sensor to control said display;

means for identifying a zoom gesture, comprising;

means for determining if a stretch factor of a bounding box surrounding at least two of said contact locations is greater than a predetermined stretch threshold; and

means for determining if said stretch factor is either increasing or decreasing;

means for associating said zoom gesture with a corresponding display command; and

means for executing said display command to alter the display of said imagery.

13. The display of claim 12, further comprising means for identifying a pan gesture, comprising:

means for determining if a motion of said bounding box is greater than a predetermined motion threshold;

means for determining if a deformation of said bounding box is less than a predetermined deformation threshold; and

means for associating said pan gesture with a corresponding display command.

14. The display of claim 12, further comprising means for identifying a rotate gesture, comprising:

means for determining if a deformation of said bounding box is greater than a predetermined deformation threshold;

means for determining if said motion of said bounding box is less than a predetermined threshold; and

means for associating said rotate gesture with a corresponding display command.

15. An interactive display comprising:

means for displaying imagery on an imaging surface;

a touch sensor corresponding to said imaging surface;

means for detecting a plurality of contact locations at which at least one user contacts said touch sensor to control said display;

means for identifying a rotate gesture, comprising;

means for determining if a deformation of a bounding box surrounding at least two of said contact locations is greater than a predetermined deformation threshold; and

means for determining if said motion of said bounding box is less than a predetermined threshold;

means for associating said rotate gesture with a corresponding display command; and

means for executing said display command to alter the display of said imagery.

16. The display of claim 15, further comprising means for identifying a pan gesture, comprising:

means for determining if a motion of said bounding box is greater than a predetermined motion threshold;

means for determining if a deformation of said bounding box is less than a predetermined deformation threshold; and

means for associating said pan gesture with a corresponding display command.

17. The display of claim 15, further comprising means for identifying a zoom gesture comprising:

means for determining if a stretch factor of a bounding box surrounding at least two of said contact locations is greater than a predetermined stretch threshold;

means for determining if said stretch factor is either increasing or decreasing; and

means for associating said zoom gesture with a corresponding display command.

18. An interactive display comprising:

means for displaying imagery on an imaging surface;

a touch sensor corresponding to said imaging surface;

means for detecting a plurality of contact locations at which at least one user contacts said touch sensor to control said display;

means for identifying a pan gesture, comprising;

means for determining if a motion of a bounding box surrounding at least two of said contact locations is greater than a predetermined motion threshold; and

means for determining if a deformation of said bounding box is less than a predetermined deformation threshold;

means for associating said pan gesture with a corresponding display command; and

means for identifying a rotate gesture, comprising;

means for determining if a deformation of said bounding box is greater than a predetermined deformation threshold;

means for determining if said motion of said bounding box is less than a predetermined threshold;

means for associating said rotate gesture with a corresponding display command; and

means for executing said display commands to alter the display of said imagery.

19. The display of claim 18, further comprising means for identifying a zoom gesture comprising:

means for determining if a stretch factor of a bounding box surrounding at least two of said contact locations is greater than a predetermined stretch threshold;

means for determining if said stretch factor is either increasing or decreasing; and

means for associating said zoom gesture with a corresponding display command.

20. In an interactive display, a method for identifying a specific user gesture comprising the steps of:

displaying imagery on an imaging surface;

detecting a plurality of contact locations at which at least one user contacts a touch sensor to control said display;

responsive to a determination that a motion of a bounding box enclosing at least two of said plurality of contact locations exceeds said predetermined motion threshold and that deformation of said bounding box is less than said predetermined deformation threshold, identifying said specific user gesture as a pan gesture;

responsive to a determination that a deformation of a bounding box enclosing at least two of said plurality of contacts locations exceeds a predetermined deformation threshold and responsive to a determination that a motion of said bounding box is less than a predetermined motion threshold, identifying said specific user gesture as a rotate gesture;

associating said specific user gesture with a display command; and

executing said display command to alter the display of said imagery.

21. An interactive display comprising:

means for displaying imagery on an imaging surface;

a touch sensor corresponding to said imaging surface;

means for detecting a plurality of contact locations at which at least one user contacts said touch sensor to control said display;

means for identifying a pan gesture, comprising;

means for determining if a motion of a bounding box surrounding at least two of said contact locations is greater than a predetermined motion threshold; and

means for determining if a deformation of said bounding box is less than a predetermined deformation threshold;

means for associating said pan gesture with a corresponding display command;

means for identifying a zoom gesture comprising;

means for determining if a stretch factor of a bounding box surrounding at least two of said contact locations is greater than a predetermined stretch threshold;

means for determining if said stretch factor is either increasing or decreasing;

means for associating said zoom gesture with a corresponding display command; and

means for executing said display commands to alter the display of said imagery.