Hover-based interaction with rendered content

• US 10,444,961 B2
• Filed: 02/22/2017
• Issued: 10/15/2019
• Est. Priority Date: 08/12/2014
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Niranjani, Himanshu, Lichtenstein, Craig, Mittal, Shailendra, Jensen, Devin
• Primary Examiner(s): Bhargava, Anil K
• Application Number:
  US15/439,808
• Publication Number:
  US 20170160914A1
• Time in Patent Office: 965 Days
• CPC Class Codes: G06F 2203/04101 2.5D-digitiser, i.e. digitiser detecting the X/Y position of the input means, finger or stylus, also when it does not touch, but is proximate to the digitiser's interaction surface and also measures the distance of the input means within a short range in the Z direction, possibly with a separate measurement setup
• CPC Class Codes: G06F 2203/04805 Virtual magnifying lens, i.e. window or frame movable on top of displayed information to enlarge it for better reading or selection
• CPC Class Codes: G06F 2203/04806 Zoom, i.e. interaction techniques or interactors for controlling the zooming operation
• CPC Class Codes: G06F 3/0416 Control or interface arrangements specially adapted for digitisers
• CPC Class Codes: G06F 3/04186 Touch location disambiguation
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06F 3/04847 Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures

Abstract

Disclosed are techniques and systems for enabling “hover-based” interaction with content that is rendered on a display of a viewing device. A process may include rendering content on a display, detecting an object hovering in front of a front surface of the display, and in response to detecting the object, determining a location on the front surface of the display corresponding to a position of the object. The determined location on the front surface of the display may then be used to determine a portion of the content that is rendered at the location or within a threshold distance from the location, and a magnified window of the portion of the content may then be displayed in a region of the display. The portion of the content within the magnified window may be actionable by responding to user input when the user input is provided within the magnified window.

Claims

1. A method comprising:

rendering content on a display;

detecting an object in front of, but not in contact with, a front surface of the display;

determining, at least partly in response to detecting the object, a location on the front surface of the display that is spaced a shortest distance from the object relative to distances from the object to other locations on the front surface of the display;

determining a portion of the content that is rendered at the location or within a threshold distance from the location;

displaying a magnified window in a region of the display, the magnified window including a magnified view of the portion of the content over the region of the display;

detecting that the object has moved within the magnified window and in front of, but not in contact with, the front surface of the display;

selecting content in response to detecting the object hovering over the selected content within the magnified window;

detecting that the object has made two successive contacts with the screen within a predetermined period of time; and

performing a display-related function in response to detecting that the object has moved within the magnified window, the display-related function comprising at least one of changing an appearance of the selected content within the magnified view of the portion of the content within the magnified window, or displaying additional content within the magnified window that relates to the selected content, wherein performing the display related function further comprises zooming the display to the portion of the content within the magnified window in response to the detected two successive contacts.

2. The method of claim 1, wherein changing the appearance of the selected content within the magnified view of the portion of the content within the magnified window comprises at least one of underlining or highlighting at least some of the selected content within the magnified window.

3. The method of claim 1, wherein displaying the additional content within the magnified window comprises displaying at least one of a pop-up with the additional information, or a drop-down menu with the additional information.

4. The method of claim 1, wherein detecting the object in front of, but not in contact with, the front surface of the display comprises determining an input gesture from the object.

5. The method of claim 4, wherein the input gesture is determined by:

detecting that the object is at a first position that is within a threshold distance from the front surface measured in a direction perpendicular to the front surface; and

determining that the object is within a predetermined area of the first position for a predetermined period of time, the predetermined area being parallel to the front surface of the display.

6. The method of claim 1, wherein the front surface of the display comprises a top portion, a bottom portion, a left side, and a right side, a positive vertical direction pointing toward the top portion, and wherein the region includes a bottom boundary that is at a predetermined distance from the location in the positive vertical direction.

7. The method of claim 1, further comprising:

determining that the object has moved farther away from the front surface of the display, in a direction perpendicular to the front surface; and

in response to determining that the object has moved farther away from the front surface, decreasing a magnification level of the portion of the content within the magnified window.

8. The method of claim 1, further comprising causing the magnified window to disappear from the display in response to at least one of:

(i) determining that the object moves outside of a control area of the display, (ii) determining that the object moves across the front surface of the display above a predetermined speed, (iii) determining that the object contacts the front surface of the display outside of the region, or (iv) determining that the object has, moved away from the front surface of the display in a direction perpendicular to the front surface of the display beyond a threshold distance from the front surface of the display measured along the direction.

9. The method of claim 1, wherein rendering content on the display comprises rendering the content in a first window, and wherein displaying the magnified window comprises rendering a second window different from the first window.

10. The method of claim 9, wherein rendering the second window comprises overlaying the second window on, top of the first window.

11. A system comprising:

a display configured to display content;

one or more sensors configured to detect an object in front of, but not in contact with, a front surface of the display;

one or more processors; and

memory storing computer-executable instructions that, when executed by the one or more processors, cause the one or more processors to;

determine, at least partly in response to detecting the object in front of, but not in contact with, the front surface of the display, a location on the front surface of the display that is spaced a shortest distance from the object relative to distances from the object to other locations on the front surface of the display;

determine a portion of the content that is rendered at the location or within a threshold distance from the location;

cause a presentation of a magnified window in a region of the display, the magnified window including a magnified view of the portion of the content within the magnified window;

detect that the object has moved within the magnified window and in front of, but not in contact with, the front surface of the display;

select content in response to detecting the object hovering over the selected content within the magnified window;

detecting that the object has made two successive contacts with the screen within a predetermined period of time; and

perform a display-related function in response to detecting that the object has moved within the magnified window, the display-related function comprising at least one of changing an appearance of the selected content within the magnified view of the portion of the content within the magnified window, or displaying additional content within the magnified window that relates to the selected content, wherein performing the display related function further comprises zooming the display to the portion of the content within the magnified window in response to the detected two successive contacts.

12. The system of claim 11, wherein changing the appearance of the selected content within the magnified view of the portion of the content within the magnified window comprises at least one of underlining or highlighting at least some of the selected content within the magnified window.

13. The system of claim 11, wherein displaying the additional content within the magnified window comprises displaying at least one of a pop-up with the additional information, or a drop-down menu with the additional information.

14. The system of claim 11, wherein detecting the object in front of, but not in contact with, the front surface of the display comprises determining an input gesture from the object.

15. The system of claim 14, wherein the computer-executable instructions, when executed by the one or more processors, further cause the one or more processors to determine the input gesture by:

detecting that the object is at a first position that is within a threshold distance from the front surface measured in a direction perpendicular to the front surface; and

determining that the object is within a predetermined area of the first position for a predetermined period of time, the predetermined area being parallel to the front surface of the display.

16. The system of claim 11, wherein the front surface of the display comprises a top portion, a bottom portion, a left side, and a right side, a positive vertical direction pointing toward the top portion, and wherein the region includes a bottom boundary that is at a predetermined distance from the location in the positive vertical direction.

17. The system of claim 11, wherein the computer-executable instructions, when executed by the one or more processors, further cause the one or more processors to:

determine that the object has moved farther away from the front surface of the display in a direction perpendicular to the front surface; and

in response to determining that the object has moved farther away from the front surface, decrease a magnification level of the portion of the content within the magnified window.

18. One or more computer-readable storage media comprising memory storing a plurality of programming instructions that are executable by one or more processors of a computing device to cause the computing device to perform acts comprising:

rendering content on a display of the computing device;

detecting an object in front of, but not in contact with, a front surface of the display;

determining, at least partly in response to detecting the object, a location on the front surface of the display that is spaced a shortest distance from the object relative to distances from the object to other locations on the front surface of the display;

determining a portion of the content that is rendered at the location or within a threshold distance from the location;

displaying a magnified window in a region of the display, the magnified window including a magnified view of the portion of the content within the magnified window;

detecting that the object has moved within the magnified window and in front of, but not in contact with, the front surface of the display;

selecting content in response to detecting the object hovering over the selected content within the magnified window;

detecting that the object has made two successive contacts with the screen within a predetermined period of time; and

performing a display-related function in response to detecting that the object has moved within the magnified window, the display-related function comprising at least one of changing an appearance of the selected content within the magnified view of the portion of the content rendered within the magnified window, or displaying additional content within the magnified window that relates to the selected content,

wherein performing the display related function further comprises zooming the display to the portion of the content within the magnified window in response to the detected two successive contacts.

19. The one or more computer-readable storage media of claim 18, wherein changing the appearance of the magnified view of the portion of the content rendered within the magnified window comprises at least one of underlining or highlighting at least some of the selected content within the magnified window.

20. The one or more computer-readable storage media of claim 18, wherein displaying the additional content within the magnified window comprises displaying at least one of a pop-up with the additional information, or a drop-down menu with the additional information.

21. The one or more computer-readable storage media of claim 18, the acts further comprising:

determining that the object has moved farther away from the front surface of the display in a direction perpendicular to the front surface; and

in response to determining that the object has moved farther away from the front surface, decreasing a magnification level of the portion of the content within the magnified window.

22. The one or more computer-readable storage media of claim 18, the acts further, comprising causing the magnified window to disappear from the display in response to at least one of:

(i) determining that the object moves outside of a control area of the display, (ii) determining that the object moves across the front surface of the display above a predetermined speed, (iii) determining that the object contacts the front surface of the display outside of the region, or (iv) determining that the object has moved away from the front surface of the display in a direction perpendicular to the front surface of the display beyond a threshold distance from the front surface of the display measured along the direction.