Dynamic modification of displayed elements of obstructed region

• US 10,509,563 B2
• Filed: 01/15/2018
• Issued: 12/17/2019
• Est. Priority Date: 01/15/2018
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Budd, Colin McGrath
• Primary Examiner(s): Kohlman, Christopher J
• Application Number:
  US15/871,216
• Publication Number:
  US 20190220187A1
• Time in Patent Office: 701 Days
• Field of Search: None
• CPC Class Codes: G06F 2203/0381 Multimodal input, i.e. interface arrangements enabling the user to issue commands by simultaneous use of input devices of different nature, e.g. voice plus gesture on digitizer
• CPC Class Codes: G06F 2203/04104 Multi-touch detection in digitiser, i.e. details about the simultaneous detection of a plurality of touching locations, e.g. multiple fingers or pen and finger
• CPC Class Codes: G06F 3/0418 for error correction or compensation, e.g. based on parallax, calibration or alignment
• CPC Class Codes: G06F 3/04186 Touch location disambiguation
• CPC Class Codes: G06F 3/044 by capacitive means
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06F 3/0486 Drag-and-drop
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus

Abstract

A method is disclosed, as well as a related input device and computer program product, for use with an input device having an input region at least partly overlapping with a display region of a display device. The method comprises determining an input occurring in the input region, and determining an obstructed region of the display region corresponding to the input. The obstructed region overlaps a displayed element in the display region. The method further comprises determining a relevance of the displayed element to a user interaction with the input device. The user interaction is determined using at least the input. The method further comprises modifying, using the determined relevance, the displayed element to be non-overlapping with the obstructed region.

Claims

1. A computer-implemented method for use with an input device having an input region at least partly overlapping with a display region of a display device, the method comprising:

determining an input occurring in the input region;

determining an obstructed region of the display region corresponding to the input, wherein the obstructed region overlaps a displayed element in the display region;

determining, using at least the input, a type of a user interaction with the input device, wherein the type is selected from a plurality of predetermined types describing user interactions with content displayed by the display device;

determining a relevance of content of the displayed element to the type of the user interaction; and

modifying, using the determined relevance, the displayed element to be non-overlapping with the obstructed region.

2. The method of claim 1, further comprising:

accessing interaction correlation information associated with one or more users; and

predicting the user interaction using the interaction correlation information.

3. The method of claim 2, further comprising:

categorizing the input as one of an active input and an inactive input,

wherein predicting the user interaction comprises;

relating, using the interaction correlation information, the categorized input with one or more previous user interactions.

4. The method of claim 3, further comprising:

generating a model that relates the one or more previous user interactions with one or more previous inputs, wherein the one or more previous inputs comprise one or more of previous active inputs and previous inactive inputs,

wherein relating the categorized input with one or more previous user interactions comprises;

applying the categorized input to the model to predict the user interaction.

5. The method of claim 1, wherein the input device comprises one or more computer processors, and

wherein determining the relevance of the content of the displayed element is based on one or both of (i) a current application executed by the one or more computer processors, and (ii) a current mode of the input device.

6. The method of claim 5, wherein selecting the type from the plurality of predetermined types comprises:

based on one or both of (i) the current application and (ii) the current mode of the input device, disregarding one or more types of the plurality of predetermined types.

7. The method of claim 1, wherein modifying the displayed element comprises one or more of:

modifying a text wrapping when the displayed element comprises a textual element;

modifying a positioning when the displayed element comprises a graphical element;

modifying a sizing of the graphical element; and

generating an overlay element comprising at least a portion of the displayed element.

8. The method of claim 1, further comprising:

determining that the obstructed region no longer obstructs the displayed element; and

modifying the displayed element to be overlapping with the obstructed region.

9. The method of claim 1, wherein determining the obstructed region of the display region comprises:

determining, using a visual sensor of the input device, a position of a user relative to the display device.

10. An input device comprising:

one or more computer processors communicatively coupled with a display device and with one or more sensor electrodes, wherein the one or more sensor electrodes defines an input region at least partly overlapping with a display region of the display device, wherein the one or more computer processors are configured to;

determine an input occurring in the input region;

determine an obstructed region of the display region corresponding to the input, wherein the obstructed region overlaps a displayed element in the display region;

determine, using at least the input, a type of a user interaction with the input device, wherein the type is selected from a plurality of predetermined types describing user interactions with content displayed by the display device;

determine a relevance of content of the displayed element to the type of the user interaction; and

modify, using the determined relevance, the displayed element to be non-overlapping with the obstructed region.

11. The input device of claim 10, wherein the one or more computer processors are further configured to:

access interaction correlation information associated with one or more users; and

predict the user interaction using the interaction correlation information.

12. The input device of claim 10, wherein determining the relevance of the content of the displayed element is based on one or both of (i) a current application executed by the one or more computer processors, and (ii) a current mode of the input device.

13. The input device of claim 10, wherein modifying the displayed element comprises one or more of:

modifying a text wrapping when the displayed element comprises a textual element;

modifying a positioning when the displayed element comprises a graphical element;

modifying a sizing of the graphical element; and

generating an overlay element comprising at least a portion of the displayed element.

14. The input device of claim 10, wherein the one or more computer processors are further configured to:

determine that the obstructed region no longer obstructs the displayed element; and

modify the displayed element to be overlapping with the obstructed region.

15. The input device of claim 10, wherein determining the obstructed region of the display region comprises:

determine, using a visual sensor of the input device, a position of a user relative to the display device.

16. A computer program product comprising:

a computer-readable storage medium having computer-readable program code embodied therewith, the computer-readable program code executable by one or more computer processors to;

determine an input occurring in an input region of an input device, wherein the input region is defined using one or more sensor electrodes and is at least partly overlapping with a display region of a display device;

determine an obstructed region of the display region corresponding to the input, wherein the obstructed region overlaps a displayed element in the display region;

determine, using at least the input, a type of a user interaction with the input device, wherein the type is selected from a plurality of predetermined types describing user interactions with content displayed by the display device;

determine a relevance of content of the displayed element to the type of the user interaction; and

modify, using the determined relevance, the displayed element to be non-overlapping with the obstructed region.

17. The computer program product of claim 16, wherein the computer-readable program code is further executable to:

access interaction correlation information associated with one or more users; and

predict the user interaction using the interaction correlation information.

18. The computer program product of claim 16, wherein determining the relevance of the content of the displayed element is based on one or both of (i) a current application executed by the one or more computer processors, and (ii) a current mode of the input device.

19. The computer program product of claim 16, wherein modifying the displayed element comprises one or more of:

modifying a text wrapping when the displayed element comprises a textual element;

modifying a positioning when the displayed element comprises a graphical element;

modifying a sizing of the graphical element; and

generating an overlay element comprising at least a portion of the displayed element.

20. The computer program product of claim 16, wherein the computer-readable program code is further executable to:

determine that the obstructed region no longer obstructs the displayed element; and

modify the displayed element to be overlapping with the obstructed region.