Touch prediction for visual displays

• US 10,203,796 B2
• Filed: 06/24/2016
• Issued: 02/12/2019
• Est. Priority Date: 06/04/2014
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Hewitt, Trudy L., McKinney, Debra J., Wetli, Christina L.
• Primary Examiner(s): Lu, William
• Application Number:
  US15/191,680
• Publication Number:
  US 20160306494A1
• Time in Patent Office: 963 Days
• CPC Class Codes: G06F 16/248 Presentation of query results
• CPC Class Codes: G06F 2203/04104 Multi-touch detection in digitiser, i.e. details about the simultaneous detection of a plurality of touching locations, e.g. multiple fingers or pen and finger
• CPC Class Codes: G06F 2203/04808 Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
• CPC Class Codes: G06F 3/0416 Control or interface arrangements specially adapted for digitisers
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06N 5/04 Inference or reasoning models
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks
• CPC Class Codes: G16H 40/63 for local operation
• CPC Class Codes: G16Z 99/00 Subject matter not provided for in other main groups of this subclass

Abstract

Touch events can be predicted relative to a visual display by maintaining a database of aggregated touch event history data relative to the visual display and from a plurality of touch screen devices. The database can be queried according to a set of input parameters defining an environment for use of the visual display. The results from the querying of the database can be analyzed to predict a set of touch events within the environment and based upon inferences obtained from the results. A representation of the set of touch events can be displayed along with the visual display.

Claims

1. A method for tracking touch events relating to a visual display on a first electronic device, the method comprising:

identifying a set of environmental parameters, wherein the environmental parameters provide information regarding an environment in which the visual display is used on the first electronic device by a user;

accessing a plurality of touch event data sets for the visual display, the plurality of touch event data sets corresponding to the environmental parameters and associated with the visual display as used on at least a second electronic device;

identifying touch patterns from the plurality of touch event data sets;

monitoring subsequent touch events by the user of the visual display;

detecting a match between the touch patterns and the subsequent touch events of the user;

predicting, in response to detecting the match, a possible touch event for the user, wherein the predicting the possible touch event indicates that the user is likely to perform the possible touch event and wherein the user has not previously performed the possible touch event; and

modifying, in response to the possible touch event, the visual display on the first electronic device for the user, wherein the modifying automatically updates the visual display as if the user had performed the possible touch event on the visual display.

2. The method of claim 1, wherein the touch event data sets specify sequences of touch events, touch coordinates relative to the visual display and different types of touch events.

3. The method of claim 1, wherein modifying the visual display includes moving an item into a field of view of the visual display.

4. The method of claim 1, wherein modifying the visual display includes displaying touch event data of other users.

5. The method of claim 1, wherein the touch patterns comprise at least one touch event performed on a visual display that lacks an interactive component.

6. A system comprising:

a processor; and

a memory in communication with the processor, the memory containing program instructions that, when executed by the processor, are configured to cause the processor to perform a method for tracking touch events relating to a visual display on a first electronic device, the method comprising;

identifying a set of environmental parameters, wherein the environmental parameters provide information regarding an environment in which the visual display is used on the first electronic device by a user;

accessing a plurality of touch event data sets for the visual display, the plurality of touch event data sets corresponding to the environmental parameters and associated with the visual display as used on at least a second electronic device;

identifying touch patterns from the plurality of touch event data sets;

monitoring subsequent touch events by the user of the visual display;

detecting a match between the touch patterns and the subsequent touch events of the user;

predicting, in response to detecting the match, a possible touch event for the user, wherein the predicting the possible touch event indicates that the user is likely to perform the possible touch event and wherein the user has not previously performed the possible touch event; and

modifying, in response to the possible touch event, the visual display on the first electronic device for the user, wherein the modifying automatically updates the visual display as if the user had performed the possible touch event on the visual display.

7. The system of claim 6, wherein the touch event data sets specify sequences of touch events, touch coordinates relative to the visual display and different types of touch events.

8. The system of claim 6, wherein modifying the visual display includes moving an item into a field of view of the visual display.

9. The system of claim 6, wherein modifying the visual display includes displaying touch event data of other users.

10. The system of claim 6, wherein the touch patterns comprise at least one touch event performed on a visual display that lacks an interactive component.

11. A computer program product, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a computer to cause the computer to track touch events relating to a visual display on a first electronic device, the tracking comprising:

identifying a set of environmental parameters, wherein the environmental parameters provide information regarding an environment in which the visual display is used on the first electronic device by a user;

accessing a plurality of touch event data sets for the visual display, the plurality of touch event data sets corresponding to the environmental parameters and associated with the visual display as used on at least a second electronic device;

identifying touch patterns from the plurality of touch event data sets;

monitoring subsequent touch events by the user of the visual display;

detecting a match between the touch patterns and the subsequent touch events of the user;

predicting, in response to detecting the match, a possible touch event for the user, wherein the predicting the possible touch event indicates that the user is likely to perform the possible touch event, and wherein the user has not previously performed the possible touch event; and

modifying, in response to the possible touch event, the visual display on the first electronic device for the user, wherein the modifying automatically updates the visual display as if the user had performed the possible touch event on the visual display.

12. The computer program product of claim 11, wherein the touch event data sets specify sequences of touch events, touch coordinates relative to the visual display and different types of touch events.

13. The computer program product of claim 11, wherein modifying the visual display includes moving an item into a field of view of the visual display.

14. The computer program product of claim 11, wherein modifying the visual display includes displaying touch event data of other users.

15. The computer program product of claim 11, wherein the touch patterns comprise at least one touch event performed on a visual display that lacks an interactive component.