Bounding box gesture recognition on a touch detecting interactive display

• US 20060288313A1
• Filed: 05/20/2005
• Published: 12/21/2006
• Est. Priority Date: 08/06/2004
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: TOUCHTABLE, INC.
• Inventors: Hillis, W. Daniel
• Granted Patent: US 7,719,523 B2
• Time in Patent Office: Days
• US Class Current: 715/863
• CPC Class Codes: G06F 2203/04806 Zoom, i.e. interaction techniques or interactors for controlling the zooming operation
• CPC Class Codes: G06F 2203/04808 Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text

Abstract

The invention provides a method and apparatus for identifying gestures performed by a user to control an interactive display. The gestures are identified based on a bounding box enclosing the points at which a user contacts a touch sensor corresponding with the display surface. The invention thus permits the use of inexpensive and highly reliable grid-based touch sensors that provide a bounding box to describe contact information. In identifying the gestures, the position, motion, shape, and deformation of the bounding box may all be considered. In particular, the center, width, height, aspect ratio, length of the diagonal, and orientation of the diagonal of the bounding box may be determined. A stretch factor, defined as the maximum of the ratio of the height of the bounding box to the width of the bounding box and the ratio of the width of the bounding box to the height of the bounding box, may also be computed. Finally, gestures may be identified based on the changes in time of these characteristics and quantities.

Claims

1. A method of operating an interactive display comprising the steps of:

displaying imagery on an imaging surface;

providing a touch sensor corresponding to said imaging surface;

detecting at least two contact locations at which at least one user contacts said touch sensor to control said display;

determining a bounding box enclosing said at least two contact locations;

identifying a specific user gesture based on said bounding box;

associating said user gesture with a corresponding display command; and

executing said display command to alter the display of said imagery.

2. The method of claim 1, said identifying step comprising the step of considering any of:

a position of said bounding box;

a motion of said bounding box;

a shape of said bounding box; and

a deformation of said bounding box.

3. The method of claim 1, said identifying step comprising the step of determining any of:

a center of said bounding box;

a width of said bounding box;

a height of said bounding box;

an aspect ratio of said bounding box;

a length of a diagonal of said bounding box; and

an orientation of a diagonal of said bounding box.

4. The method of claim 1, said identifying step comprising the step of determining a time rate of change of any of:

a center of said bounding box;

a width of said bounding box;

a height of said bounding box;

an aspect ratio of said bounding box;

a length of a diagonal of said bounding box;

and an orientation of a diagonal of said bounding box.

5. The method of claim 1, said identifying step comprising the step of determining a stretch factor comprising a maximum of a ratio of said bounding box width to said bounding box height and a ratio of said bounding box height to said bounding box width.

6. The method of claim 1, said identifying step comprising the step of determining a time rate of change of a stretch factor comprising a maximum of a ratio of said bounding box width to said bounding box height and a ratio of said bounding box height to said bounding box width.

7. The method of claim 1, wherein said gesture comprises any of:

a pan gesture;

a zoom gesture; and

a rotate gesture.

8. The method of claim 1, wherein said display command effects any of:

a translation of said imagery;

a change in magnification of said imagery; and

a change in orientation of said imagery.

9. The method of claim 2, said identifying step comprising the step of identifying a pan gesture by executing the steps of:

determining if said motion is greater than a predetermined motion threshold; and

determining if said deformation is less than a predetermined deformation threshold.

10. The method of claim 5, said identifying step comprising the step of identifying a zoom gesture by executing the steps of:

determining if said stretch factor is greater than a predetermined stretch threshold; and

determining if said stretch factor is either increasing or decreasing.

11. The method of claim 2, said identifying step further comprising the step of identifying a rotate gesture by executing the steps of:

determining if said deformation is greater than a predetermined deformation threshold; and

determining if said motion is less than a predetermined threshold.

12. The method of claim 8, wherein a direction of said change in the orientation of said imagery is determined by any of the steps of:

applying a convention to indicate a particular diagonally opposed orientation of a pair of points used to specify said bounding box;

determining a relative intensity for said at least one contact location; and

measuring a torque applied to said touch sensor by said user.

13. An interactive display comprising:

means for displaying imagery on an imaging surface;

a touch sensor corresponding to said imaging surface;

means for detecting at least two contact locations at which at least one user contacts said touch sensor to control said display;

means for determining a bounding box enclosing said at least two contact locations;

means for identifying a specific user gesture based on said bounding box;

means for associating said user gesture with a corresponding display command; and

means for executing said display command to alter the display of said imagery.

14. The display of claim 13, said means for identifying comprising means for identifying any of:

a position of said bounding box;

a motion of said bounding box;

a shape of said bounding box; and

a deformation of said bounding box.

15. The display of claim 13, said means for identifying comprising means for determining any of:

a center of said bounding box;

a width of said bounding box;

a height of said bounding box;

an aspect ratio of said bounding box;

a length of a diagonal of said bounding box; and

an orientation of a diagonal of said bounding box.

16. The display of claim 13, said means for identifying comprising means for determining a time rate of change of any of:

a center of said bounding box;

a width of said bounding box;

a height of said bounding box;

an aspect ratio of said bounding box;

a length of a diagonal of said bounding box; and

an orientation of a diagonal of said bounding box.

17. The display of claim 13, said means for identifying comprising means for determining a stretch factor comprising a maximum of a ratio of said bounding box width to said bounding box height and a ratio of said bounding box height to said bounding box width.

18. The display of claim 13, said means for identifying comprising means for determining a time rate of change of a stretch factor comprising a maximum of a ratio of said bounding box width to said bounding box height and a ratio of said bounding box height to said bounding box width.

19. The display of claim 13, wherein said gesture comprises any of:

a pan gesture;

a zoom gesture;

and a rotate gesture.

20. The display of claim 13, wherein said display command effects any of:

a translation of said imagery;

a change in magnification of said imagery; and

a change in orientation of said imagery.

21. The display of claim 14, said means for identifying comprising means for identifying a pan gesture, comprising:

means for determining if said motion is greater than a predetermined motion threshold; and

means for determining if said deformation is less than a predetermined deformation threshold.

22. The display of claim 17, said means for identifying comprising means for identifying a zoom gesture, comprising:

means for determining if said stretch factor is greater than a predetermined stretch threshold; and

means for determining if said stretch factor is either increasing or decreasing.

23. The display of claim 14, said identifying step means for comprising means for identifying a rotate gesture, comprising:

means for determining if said deformation is greater than a predetermined deformation threshold; and

means for determining if said motion is less than a predetermined threshold.

24. The display of claim 20, wherein a direction of said change in the orientation of said imagery is determined by any of the steps of:

means for applying a convention to indicate a particular diagonally opposed orientation of a pair of points used to specify said bounding box;

means for determining a relative intensity for said at least one contact location; and

means for measuring a torque applied to said touch sensor by said user.