Bounding box gesture recognition on a touch detecting interactive display

• US 10,073,610 B2
• Filed: 03/12/2014
• Issued: 09/11/2018
• Est. Priority Date: 08/06/2004
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Hillis, W. Daniel
• Primary Examiner(s): Nguyen, Jimmy H
• Application Number:
  US14/207,201
• Publication Number:
  US 20140195991A1
• Time in Patent Office: 1,644 Days
• Field of Search: 345173-178, 178 1801- 1811, 382181, 382182, 382184, 382187, 382189, 715863
• CPC Class Codes: G06F 2203/04806 Zoom, i.e. interaction techniques or interactors for controlling the zooming operation
• CPC Class Codes: G06F 2203/04808 Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text

Abstract

The invention provides a method and apparatus for identifying gestures performed by a user to control an interactive display. The gestures are identified based on a bounding box enclosing the points at which a user contacts a touch sensor corresponding with the display surface. The invention thus permits the use of inexpensive and highly reliable grid-based touch sensors that provide a bounding box to describe contact information. In identifying the gestures, the position, motion, shape, and deformation of the bounding box may all be considered. In particular, the center, width, height, aspect ratio, length of the diagonal, and orientation of the diagonal of the bounding box may be determined. A stretch factor, defined as the maximum of the ratio of the height of the bounding box to the width of the bounding box and the ratio of the width of the bounding box to the height of the bounding box, may also be computed. Finally, gestures may be identified based on the changes in time of these characteristics and quantities.

Claims

1. In an interactive display, a method for identifying a specific user gesture, the method comprising:

displaying imagery on an imaging surface of the interactive display;

detecting a plurality of contact locations at which at least one user contacts a touch sensor to control the interactive display;

responsive to a determination that a deformation of a bounding box enclosing at least two of said plurality of contact locations exceeds a predetermined deformation threshold and responsive to a determination that a motion of said bounding box is less than a predetermined motion threshold, identifying said specific user gesture as a rotate gesture;

determining a direction of rotation of the bounding box based on characteristics of the at least two of said plurality of contact locations;

associating said specific user gesture with a display command; and

executing said display command to alter the imagery.

2. The method of claim 1, wherein the determining the direction of rotation further comprises designating said bounding box with one of an upper left corner and a lower right corner or a lower left corner and an upper right corner.

3. The method of claim 1, further comprising arbitrarily choosing the direction of rotation responsive to a determination the direction of rotation is ambiguous.

4. The method of claim 1, wherein the characteristics of the at least two of said plurality of contact locations includes a measurement of a torque applied to the touch sensor.

5. The method of claim 1, wherein the characteristics of the at least two of said plurality of contact locations includes a relative intensity of the at least two of said plurality of contact locations.

6. The method of claim 5, wherein the characteristics of the at least two of said plurality of contact locations includes a strength of contact for each of the at least two of said plurality of contact locations.

7. The method of claim 5, further comprising determining at least two true points of contact based on the relative intensity of the at least two of said plurality of contact locations.

8. An interactive display comprising:

means for displaying imagery on an imaging surface of the interactive display;

a touch sensor corresponding to said imaging surface;

means for detecting a plurality of contact locations at which at least one user contacts said touch sensor to control said interactive display;

means for identifying a rotate gesture that determines if a deformation of a bounding box surrounding at least two of said plurality of contact locations is greater than a predetermined deformation threshold and determines if a motion of said bounding box is less than a predetermined motion threshold;

means for determining a direction of rotation of the bounding box based on characteristics of the at least two of said plurality of contact locations;

means for associating said rotate gesture with a corresponding display command; and

means for executing said corresponding display command to alter the imagery.

9. The interactive display of claim 8, wherein the means for determining the direction of rotation designates said bounding box with one of an upper left corner and a lower right corner or a lower left corner and an upper right corner.

10. The interactive display of claim 8, further comprising means for arbitrarily choosing the direction of rotation responsive to a determination the direction of rotation is ambiguous.

11. The interactive display of claim 8, wherein the characteristics of the at least two of said plurality of contact locations includes a measurement of a torque applied to the touch sensor.

12. The interactive display of claim 8, wherein the characteristics of the at least two of said plurality of contact locations includes a relative intensity of the at least two of said plurality of contact locations.

13. The interactive display of claim 12, wherein the characteristics of the at least two of said plurality of contact locations includes a strength of contact for each of the at least two of said plurality of contact locations.

14. The interactive display of claim 12, further comprising determining at least two true points of contact based on the relative intensity of the at least two of said plurality of contact locations.

15. A non-transitory machine-readable medium having instructions which, when executed by a machine, cause the machine to perform operations comprising:

displaying imagery on an imaging surface of an interactive display;

detecting a plurality of contact locations at which at least one user contacts a touch sensor to control the interactive display;

responsive to a determination that a deformation of a bounding box enclosing at least two of said plurality of contact locations exceeds a predetermined deformation threshold and responsive to a determination that a motion of said bounding box is less than a predetermined motion threshold, identifying a specific user gesture as a rotate gesture;

determining a direction of rotation of the bounding box based on characteristics of the at least two of said plurality of contact locations;

associating said specific user gesture with a display command; and

executing said display command to alter the imagery.

16. The non-transitory machine-readable medium of claim 15, wherein the determining the direction of rotation further comprises designating said bounding box with one of an upper left corner and a lower right corner or a lower left corner and an upper right corner.

17. The non-transitory machine-readable medium of claim 15, further comprising arbitrarily choosing the direction of rotation responsive to a determination the direction of rotation is ambiguous.

18. The non-transitory machine-readable medium of claim 15, wherein the characteristics of the at least two of said plurality of contact locations includes a measurement of a torque applied to the touch sensor.

19. The non-transitory machine-readable medium of claim 15, wherein the characteristics of the at least two of said plurality of contact locations includes a relative intensity of the at least two of said plurality of contact locations.

20. The non-transitory machine-readable medium of claim 19, further comprising determining at least two true points of contact based on the relative intensity of the at least two of said plurality of contact locations.