Unified presentation of contextually connected information to improve user efficiency and interaction performance

• US 10,691,292 B2
• Filed: 02/23/2015
• Issued: 06/23/2020
• Est. Priority Date: 02/24/2014
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Lu, Qi, Connell, Derrick Leslie, Shakib, Darren, Gates, III, William H.
• Primary Examiner(s): Weber, Joy M
• Application Number:
  US14/629,214
• Publication Number:
  US 20150242091A1
• Time in Patent Office: 1,947 Days
• CPC Class Codes: G06F 16/435 Filtering based on additional data, e.g. user or group profiles
• CPC Class Codes: G06F 16/487 using geographical or spatial information, e.g. location
• CPC Class Codes: G06F 16/489 using time information
• CPC Class Codes: G06F 3/04817 using icons graphical or visual programming using iconic symbols G06F8/34
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/0485 Scrolling or panning
• CPC Class Codes: G06F 9/451 Execution arrangements for user interfaces
• CPC Class Codes: G06Q 10/10 Office automation; Time management
• CPC Class Codes: G06Q 30/0241 Advertisements

Abstract

A unified experience environment supports mechanisms that collect and utilize contextual metadata to associate information in accordance with its relevance to a user'"'"'s current context. An ambient data collector obtains contextual and activity information coincident with a user'"'"'s creation, editing or consumption of data and associates it with such data as contextual metadata. A context generator that utilizes contextual metadata to identify relationships between data and enable the proactive presentation of data relevant to a user'"'"'s current context. Proactive presentation includes a context panel that is alternatively displayable and hideable in an application-independent manner and a unified activity feed that comprises correlated data groupings identified by correlation engines, including a universal, cross-application correlation engine and individual, application-specific correlation engines that exchange information through data correlation interfaces. The context panel and unified activity feed enable users to more efficiently access data and increase their interaction performance with a computing device.

Claims

1. A method for presenting contextually connected information, the method comprising:

accessing, by a first application executing on a computing device, a first data that has a first contextual metadata corresponding to it, the first contextual metadata comprising ambient data indicative of activity that occurred coincidentally with creation or editing of the first data;

transmitting, by the first application, a request to a data correlation interface of a second application also executing on the computing device, the request providing the first contextual metadata;

searching, by the second application, in response to receiving the request via the data correlation interface of the second application, other contextual metadata to identify a relationship between the first contextual metadata and a second, different contextual metadata, the identifying the relationship being based at least in part on existing, established data correlations;

obtaining, by the second application, a second data to which the second contextual metadata corresponds;

physically generating, by the computing device on a display device communicationally coupled to the computing device, a graphical user interface comprising;

a primary content area comprising a display of at least a portion of the first data; and

a context panel visually adjacent to the primary content area, the context panel comprising a first user interface element that is indicative of the second data, the first user interface element being user-selectable to access the second data;

detecting a user action comprising a drag action from the primary content area to the context panel;

generating, in response to the detecting, an explicit association between the first data and the second data, the explicit association now being one of the established data correlations; and

updating the graphical user interface to comprise an updated context panel now comprising a second user interface element that is indicative of the first data;

wherein the first application is a dedicated application directed to the consumption of the first data and the second application is a dedicated application directed to the consumption of the second data, thereby resulting in a fractured data consumption paradigm.

2. The method of claim 1, further comprising:

receiving, by a unified activity feed component executing on the computing device, a first correlated data grouping from the second application, the first correlated data grouping comprising the second data and the first data as correlated to the second data;

wherein the context panel comprises user interface elements indicative of data correlated to the first data, the data correlated to the first data being selected by the unified activity feed.

3. The method of claim 2, further comprising:

receiving, by the unified activity feed component, multiple separate correlated data groupings from multiple separate applications; and

ranking, by the unified activity feed component, data that was identified by at least one of the second correlated data grouping or the multiple separate correlated data groupings as being correlated to the first data;

wherein the context panel comprises user interface elements indicative of highest ranked, according to the ranking, data correlated to the first data.

4. One or more computer-readable storage media comprising computer-executable instructions for presenting contextually connected information, the computer-executable instructions, when executed by a computing device, causing the computing device to perform steps comprising:

accessing, by a first application, a first data that has a first contextual metadata corresponding to it, the first contextual metadata comprising ambient data indicative of activity that occurred coincidentally with creation or editing of the first data;

transmitting, by the first application, a request to a data correlation interface of a second application, the request providing the first contextual metadata;

searching, by the second application, in response to receiving the request via the data correlation interface of the second application, other contextual metadata to identify a relationship between the first contextual metadata and a second, different contextual metadata, the identifying the relationship being based at least in part on existing, established data correlations;

obtaining, by the second application, a second data to which the second contextual metadata corresponds;

physically generating, on a display device communicationally coupled to the computing device, a graphical user interface comprising;

a primary content area comprising a display of at least a portion of the first data; and

a context panel visually adjacent to the primary content area, the context panel comprising a first user interface element that is indicative of the second data, the first user interface element being user-selectable to access the second data;

detecting a user action comprising a drag action from the primary content area to the context panel;

generating, in response to the detecting, an explicit association between the first data and the second data, the explicit association now being one of the established data correlations; and

updating the graphical user interface to comprise an updated context panel now comprising a second user interface element that is indicative of the first data;

wherein the first application is a dedicated application directed to the consumption of the first data and the second application is a dedicated application directed to the consumption of the second data, thereby resulting in a fractured data consumption paradigm.

5. A system for presenting contextually connected information, the system comprising:

one or more processing units;

a display device communicationally coupled to the one or more processing units; and

one or more computer-readable storage media communicationally coupled to the one or more processing units, the one or more computer-readable storage media comprising computer-executable instructions, which, when executed by the one or more processing units, cause the system to perform steps comprising;

accessing, by a first application executing on the system, a first data that has a first contextual metadata corresponding to it, the first contextual metadata comprising ambient data indicative of activity that occurred coincidentally with creation or editing of the first data;

transmitting, by the first application, a request to a data correlation interface of a second application, the request providing the first contextual metadata;

searching, by the second application, in response to receiving the request via the data correlation interface of the second application, other contextual metadata to identify a relationship between the first contextual metadata and a second, different contextual metadata, the identifying the relationship being based at least in part on existing, established data correlations;

obtaining, by the second application, a second data to which the second contextual metadata corresponds;

physically generating, on the display device, a graphical user interface comprising;

a primary content area comprising a display of at least a portion of the first data; and

a context panel visually adjacent to the primary content area, the context panel comprising a first user interface element that is indicative of the second data, the first user interface element being user-selectable to access the second data;

detecting a user action comprising a drag action from the primary content area to the context panel;

generating, in response to the detecting, an explicit association between the first data and the second data, the explicit association now being one of the established data correlations; and

updating the graphical user interface to comprise an updated context panel now comprising a second user interface element that is indicative of the first data;

wherein the first application is a dedicated application directed to the consumption of the first data and the second application is a dedicated application directed to the consumption of the second data, thereby resulting in a fractured data consumption paradigm.

6. The system of claim 5, wherein the graphical user interface further comprises a third user interface element, the third user interface element being displayed within the graphical user interface externally to, but visually connected, to the context panel;

and wherein further the computer-readable media comprise further computer-executable instructions, which, when executed by the one or more processing units, cause the system to perform further steps comprising;

detecting user action directed to the third user interface element; and

, in response to the detecting, variously displaying or hiding the context panel within the graphical user interface.

7. The system of claim 5, wherein the first user interface element comprises a tile comprising a textual identification of the second data and a graphical icon representative of the second data.

8. The system of claim 5, wherein the second data is a document and the first user interface element is a graphical icon representative of the document;

and wherein further the computer-readable media comprise further computer-executable instructions, which, when executed by the one or more processing units, cause the system to perform further steps comprising;

detecting a user action directed to the graphical icon;

accessing, in response to the detected user action, the second data; and

updating the graphical user interface to comprise an updated primary content area now comprising a display of at least a portion of the document.

9. The system of claim 5, wherein the second data references an action performable by the system;

and wherein further the computer-readable media comprise further computer-executable instructions, which, when executed by the one or more processing units, cause the system to perform further steps comprising;

detecting a user action directed to the first user interface element; and

initiating, in response to the detected user action, the action performable by the system that is referenced by the second data.

10. The system of claim 5, wherein the context panel is automatically displayed within the graphical user interface based on a current time or a current location of the user.

11. The system of claim 5, wherein the computer-readable media comprise further computer-executable instructions, which, when executed by the one or more processing units, cause the system to perform further steps comprising:

physically generating, on the display device, a second graphical user interface prior to the accessing, the second graphical user interface comprising;

a unified activity feed comprising multiple distinct visual elements visually separated from each other by space, each distinct visual element being surrounded by the space, the multiple distinct visual elements comprising;

a first distinct visual element;

a second distinct visual element visually positioned horizontally adjacent to the first distinct visual element; and

a third distinct visual element visually positioned vertically adjacent to the first distinct visual element;

wherein each distinct visual element represents a distinct grouping of correlated data, each distinct visual element comprising multiple tile interface elements, the multiple tile interface elements comprising;

a first tile interface element;

a second tile interface element visually positioned horizontally adjacent to the first tile interface element; and

a third tile interface element visually positioned vertically adjacent to the first tile interface element;

wherein at least some of the tile interface elements comprise action icons.

12. The system of claim 11, wherein the computer-readable media comprise further computer-executable instructions, which, when executed by the one or more processing units, cause the system to perform further steps comprising:

detecting user action directed to scrolling the unified activity feed;

updating the second graphical user interface, in response to the detecting the user action directed to scrolling the unified activity feed, such that all of the multiple distinct visual elements together move in response to the scrolling of the unified activity feed;

detecting user action directed to scrolling within a first distinct visual element; and

updating the second graphical user interface, in response to the detecting the user action directed to scrolling within the first distinct visual element, such that the multiple tile interface elements of the first distinct visual element together move within the first distinct visual element, exclusive of movement of others of the multiple distinct visual elements in the unified activity feed and exclusive of movement of the multiple tile user interface elements in the others of the multiple distinct visual elements.

13. The system of claim 12, wherein the scrolling of the unified activity feed is in a same direction as the scrolling within the first distinct visual element.

14. The system of claim 11, wherein the computer-readable media comprise further computer-executable instructions, which, when executed by the one or more processing units, cause the system to perform further steps comprising:

determining a current context of the user;

visually arranging, in response to the ranking, the multiple distinct visual elements within the unified activity feed; and

repeating the determining the current context of the user, the ranking and the visually arranging as the current context of the user changes.

15. The system of claim 5, wherein the computer-readable media comprise further computer-executable instructions, which, when executed by the one or more processing units, cause the system to perform further steps comprising:

receiving, by a unified activity feed component, a first correlated data grouping from the second application, the first correlated data grouping comprising the second data and the first data as correlated to the second data;

wherein the context panel comprises user interface elements indicative of data correlated to the first data, the data correlated to the first data being selected by the unified activity feed.

16. The system of claim 15, wherein the computer-readable media comprise further computer-executable instructions, which, when executed by the one or more processing units, cause the system to perform further steps comprising:

receiving, by the unified activity feed component, a second correlated data grouping from the first application, the second correlated data grouping differing from the first correlated data grouping.

17. The system of claim 15, wherein the computer-readable media comprise further computer-executable instructions, which, when executed by the one or more processing units, cause the system to perform further steps comprising:

receiving, by the unified activity feed component, multiple separate correlated data groupings from multiple separate applications; and

ranking data that was identified by at least one of the second correlated data grouping or the multiple separate correlated data groupings as being correlated to the first data;

wherein the context panel comprises user interface elements indicative of highest ranked, according to the ranking, data correlated to the first data.

18. The system of claim 5, wherein

a second user drag action directed to the first user interface element and proceeding to drag to the primary content area causes at least a portion of the second data to be inserted into the first data.

19. The system of claim 5, wherein the graphical user interface further comprises a second user interface element, the second user interface element being displayed within the graphical user interface externally to, but visually connected, to the context panel, the second user interface element being user actionable to trigger displaying or hiding the context panel within the graphical user interface.