Touch detecting interactive display

• US 8,072,439 B2
• Filed: 09/24/2010
• Issued: 12/06/2011
• Est. Priority Date: 08/06/2004
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: TOUCHTABLE, INC.
• Inventors: Hillis, W. Daniel, Ferren, Bran
• Primary Examiner(s): Nguyen, Kimnhung
• Application Number:
  US12/890,453
• Publication Number:
  US 20110022991A1
• Time in Patent Office: 438 Days
• Field of Search: 345156-158, 345173-179, 178 1801- 1804
• US Class Current: 345/173
• CPC Class Codes: G06F 2203/04808 Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/041 Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text

Abstract

The invention provides an interactive display that is controlled by user gestures identified on a touch detecting display surface. In the preferred embodiment of the invention, imagery is projected onto a horizontal projection surface from a projector located above the projection surface. The locations where a user contacts the projection surface are detected using a set of infrared emitters and receivers arrayed around the perimeter of the projection surface. For each contact location, a computer software application stores a history of contact position information and, from the position history, determines a velocity for each contact location. Based upon the position history and the velocity information, gestures are identified. The identified gestures are associated with display commands that are executed to update the displayed imagery accordingly. Thus, the invention enables users to control the display through direct physical interaction with the imagery.

Claims

1. A computer implemented method performed in a system including a processor coupled to digital data storage and a display having a touch-sensitive display surface, the display surface including a material layer with at least one optical property that changes responsive to pressure applied to the material layer, the method comprising the tasks of:

in the digital data storage, storing a record defining a collection of multiple user gestures, each gesture executable by touching the display, and further storing for each user gesture an assignment of one or more of multiple prescribed operations of modifying subject matter presented by the display;

the processor performing visual analysis of the material layer;

for each of one or more touches experienced by the display surface, the processor using input including at least the visual analysis to track a position history of the touch;

the processor identifying from the collection of user gestures at least one user gesture executed by the one or more touches, the identifying being performed based at least in part upon the position history of the one or more of the touches experienced by the display surface; and

the processor identifying the one or more prescribed operations assigned to the executed user gesture, and causing the display to modify the subject matter presented by the display according to the identified one or more operations.

10. The invention of claim 1 or 2 or 3 or 4 or 5 or 6, wherein the subject matter presented by the display includes geographic information systems imagery.

11. The invention of claim 1 or 2 or 3 or 4 or 5 or 6, wherein the display surface is a projection surface, and the subject matter is produced by a projector.

12. The invention of claim 1 or 2 or 3 or 4, wherein the multiple prescribed operations include any of:

a panning movement;

a zoom; and

a rotation.

13. The invention of claim 1 or 2 or 3 or 4 or 5 or 6, the tasks further comprising detecting touches experienced by the display surface by performing any of:

operating at least one infrared emitter and receiver pair;

operating a capacitive touch pad;

operating an ultrasound system;

operating a resistive touch pad.

2. At least one non-transitory computer-readable storage medium containing a program of machine-readable instructions executable by a digital data processing machine to perform tasks for operating an interactive display system including a processor coupled to digital data storage and a display having a touch-sensitive display surface, the display surface including a material layer with at least one optical property that changes responsive to pressure applied to the material layer, where the digital data storage contains a record defining a collection of multiple user gestures, each user gesture executable by touching the display, and where the digital data storage further contains for each user gesture an assignment of one or more of multiple prescribed operations of modifying subject matter presented by the display, where the tasks comprise:

the processor performing visual analysis of the material layer;

for each of one or more touches experienced by the display surface, the processor using input including at least the visual analysis to track a position history of the touch;

the processor identifying from the collection of user gestures at least one user gesture executed by the one or more touches, the identifying being performed based at least in part upon the position history of the one or more of the touches experienced by the display surface; and

the processor identifying the one or more prescribed operations assigned to the executed user gesture, and causing the display to modify the subject matter presented by the display according to the identified one or more operations.

3. A computer implemented method performed in a system including a processor coupled to digital data storage and a display having a touch-sensitive display surface, the method comprising the tasks of:

in the digital data storage, storing a record defining a collection of multiple user gestures, each user gesture executable by touching the display, and further storing for each user gesture an assignment of one or more of multiple prescribed operations of modifying subject matter presented by the display;

for each of one or more touches experienced by the display surface, the processor determining a force exerted by the touch upon the display surface;

based on one or more prescribed properties of the one or more touches experienced by the display surface, the processor identifying from the collection of user gestures at least one user gesture executed by the one or more touches;

the processor identifying the one or more prescribed operations assigned to the executed user gesture, and causing the display to modify the subject matter presented by the display according to the identified one or more operations; and

where the tasks are further performed according to any or both of;

(1) the identification of the executed user gesture is performed based on properties including the determined force of the one or more touches;

(2) as to the manner in which the subject matter presented by the display is modified according to the identified one or more operations, said manner is further responsive to the determined force of the one or more touches.

7. The invention of claim 3 or 4, where:

the processor establishes a velocity at which the subject matter presented by the display is modified in response to the sensed force.

8. The invention of claim 3 or 4, where:

subject matter presented by the display includes a combined representation of multiple layers of imagery; and

one of the prescribed operations comprises modifying transparency of one or more but less than all of the imagery layers responsive to the determined force.

9. The invention of claim 3 or 4, where the force is determined using measurements obtained by load cells mechanically coupled to the display surface.

4. At least one non-transitory computer-readable storage medium containing a program of machine-readable instructions executable by a digital data processing machine to perform tasks for operating an interactive display system including a processor coupled to digital data storage and a display having a touch-sensitive display surface, where the digital data storage contains a record defining a collection of one or more user gestures, each user gesture executable by touching the display, and where the digital data storage further contains for each user gesture an assignment of one or more prescribed operations of modifying subject matter presented by the display, where the tasks comprise:

for each of one or more touches experienced by the display surface, the processor determining a force exerted by the touch upon the display surface;

based on one or more prescribed properties of the one or more touches experienced by the display surface, the processor identifying from the collection of user gestures at least one user gesture executed by the one or more touches;

the processor identifying the one or more prescribed operations assigned to the executed user gesture, and causing the display to modify the subject matter presented by the display according to the identified one or more operations; and

where the tasks are further performed according to any or both of;

(1) the identification of the executed user gesture is performed based on properties including the determined force of the one or more touches;

(2) as to the manner in which the subject matter presented by the display is modified according to the identified one or more operations, said manner is further responsive to the determined force of the one or more touches.

5. A computer implemented method performed in a system including a processor coupled to digital data storage and a display having a touch-sensitive display surface, the method comprising the tasks of:

in the digital data storage, storing a record defining a collection of one or more user gestures, each user gesture executable by touching the display, and further storing for each user gesture an assignment of one or more prescribed operations of modifying subject matter presented by the display;

the processor causing the display to present subject matter including a representation of at least one control interface or menu positioned proximate to and oriented towards a first location on a periphery of the display surface;

for each touch experienced by the display surface, the processor determining a position of the touch;

based on data including the determined position or a history of the determined position of one or more of the touches experienced by the display surface, the processor identifying from the collection of user gestures at least one user gesture executed by the one or more touches;

the processor identifying the one or more prescribed operations assigned to the executed user gesture, and causing the display to begin modifying the subject matter presented by the display according to the identified one or more operations;

where the at least one executed user gesture includes a first gesture, and the identified one or more prescribed operations includes a first operation assigned to the first gesture, and where the first operation causes the display to effect a movement and reorientation of the control interface or menu proximate to and oriented towards a second location on the periphery of the display surface, the second location being different than the first location.

14. The invention of claim 5 or 6, where the first gesture comprises occurrence of a touch upon the display surface within a prescribed proximity to the second location.

15. The invention of claim 5 or 6, where the periphery of the display surface includes multiple edges, and the first location comprises a first one of the edges, and the second location comprises a second one of the edges different than the first edge, and where the first gesture comprises occurrence of a touch within a prescribed distance of the second edge.

6. At least one non-transitory computer-readable storage medium containing a program of machine-readable instructions executable by a digital data processing machine to perform tasks for operating an interactive display system including a processor coupled to digital data storage and a display having a touch-sensitive display surface, where the digital data storage contains a record defining a collection of multiple user gestures, each user gesture executable by touching the display, and where the digital data storage further contains for each user gesture an assignment of one or more of multiple prescribed operations of modifying subject matter presented by the display, where the tasks comprise:

the processor causing the display to present subject matter including a representation of at least one control interface or menu positioned proximate to and oriented towards a first location on a periphery of the display surface;

for each touch experienced by the display surface, the processor determining a position of the touch;

based on data including the determined position or a history of the determined position of one or more of the touches experienced by the display surface, the processor identifying from the collection of user gestures at least one user gesture executed by the one or more touches;

the processor identifying the one or more prescribed operations assigned to the executed user gesture, and causing the display to begin modifying the subject matter presented by the display according to the identified one or more operations;

where the at least one executed user gesture includes a first gesture, and the identified one or more prescribed operations includes a first operation assigned to the first gesture, and where the first operation causes the display to effect a movement and reorientation of the control interface or menu proximate to and oriented towards a second location on the periphery of the display surface, the second location being different than the first location.