METHOD, APPARATUS, AND SYSTEM FOR MODELING INTERACTIONS OF A GROUP OF USERS WITH A COMPUTING SYSTEM

• US 20130311925A1
• Filed: 09/28/2012
• Published: 11/21/2013
• Est. Priority Date: 05/17/2012
• Status: Active Grant

• Current Assignee: SRI International, Inc.
• Original Assignee: Rukman Senanayake
• Inventors: Denker, Grit, Senanayake, Rukman
• Granted Patent: US 9,158,370 B2
• Time in Patent Office: Days
• US Class Current: 715/771
• CPC Class Codes: G06F 16/2425 Iterative querying; Query formulation based on the results of a preceding query
• CPC Class Codes: G06F 16/332 Query formulation
• CPC Class Codes: G06F 16/3329 Natural language query formulation or dialogue systems
• CPC Class Codes: G06F 16/3349 Reuse of stored results of previous queries
• CPC Class Codes: G06F 16/90328 using search space presentation or visualization, e.g. category or range presentation and selection
• CPC Class Codes: G06F 3/01 Input arrangements or combined input and output arrangements for interaction between user and computer G06F3/16 takes precedence
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence

Abstract

A method, apparatus, and system for modeling interactions of a group of users with a computing system includes monitoring passive and active interaction data of multiple users and discovering common interests among the users based on the interaction data. Collective contextual user models may be dynamically formed based on the common interests. Such models have many useful applications that can benefit the group or the individual members of the group.

Claims

1. A common interest discovery module to discover common interests between or among users interacting with a computing system, comprising executable instructions embodied in one or more machine-readable media, configured to:

access a plurality of contextual user models, each of the contextual user models comprising user-specific interaction data, the user-specific interaction data comprising;

gaze-tracking data comprising one or more real-time sensor inputs indicative of user gaze in relation to on-screen locations of a display of the computing device; and

semantic descriptions of user interface elements displayed at the on-screen locations corresponding to the gaze-tracking data, the semantic descriptions comprising information about the user interface elements;

identify a collective interaction context that is common across the plurality of contextual user models; and

form a collective contextual user model based on the collective interaction context, the collective contextual user model comprising user-specific interaction data from the plurality of contextual user models, the user-specific interaction data relating to the common interaction context.

2. The common interest discovery module of claim 1, configured to apply one or more cognitive parameters to the user-specific interaction data and store the user-specific interaction data in the collective contextual user model according to the cognitive parameters.

3. The common interest discovery module of claim 1, wherein the collective interaction context relates to a common interest derived from the user-specific interaction data.

4. The common interest discovery module of claim 1, wherein the collective interaction context relates to content viewed by users associated with the contextual user models.

5. The common interest discovery module of claim 1, wherein the collective interaction context relates to prior interactions with the computing system by users associated with the contextual user models.

6. The common interest discovery module of claim 1, wherein the collective interaction context relates to current interactions with the computing system by users associated with the contextual user models.

7. The common interest discovery module of claim 1, wherein the collective contextual user model comprises a short term memory, a long term memory, and pre-cognitive content.

8. The common interest discovery module of claim 1, configured to generate a notification relating to user-specific interaction data based on the collective interaction context.

9. The common interest discovery module of claim 1, configured to aggregate user-specific interaction data from the plurality of contextual user models and present an inference based on the aggregated user-specific interaction data.

10. The common interest discovery module of claim 1, wherein the user-specific interaction data comprises data relating to user interactions with a plurality of software applications.

11. The common interest discovery module of claim 1, configured to present content associated with user-specific interaction data of a contextual user model to one or more users associated with the collective contextual user model.

12. The common interest discovery module of claim 11, configured to receive user feedback in response to the presentation of content and update the collective contextual user model based on the user feedback.

13. The common interest discovery module of claim 1, configured to initiate a software application event based on the collective contextual user model.

14. The common interest discovery module of claim 13, configured to receive user feedback in response to the initiated software application event and update the collective contextual user model based on the user feedback.

15. The common interest discovery module of claim 1, configured to prepare a shared reading record indicating portions of content viewed by users associated with the collective contextual model.

16. The common interest discovery module of claim 15, configured to indicate in the shared reading record durations of visual attention associated with the portions of content viewed by the users associated with the collective contextual model, wherein the durations of visual attention are derived from the gaze-tracking data.

17. The common interest discovery module of claim 16, configured to apply one or more cognitive parameters to the durations of visual attention to associate levels of cognitive processing with the durations of visual attention.

18. The common interest discovery module of claim 17, configured to identify portions of content having higher or lower levels of cognitive processing.

19. A method for determining a collective interaction context of users interacting with a computing system, comprising:

maintaining a plurality of contextual user models, each of the contextual user models comprising user-specific interaction data, the user-specific interaction data comprising;

passive interaction data comprising one or more real-time sensor inputs indicative of a passive user interaction in relation to on-screen locations of a display of the computing device, the passive interaction data relating to a user interaction that does not initiate an application event; and

semantic descriptions of user interface elements displayed at the on-screen locations corresponding to the passive interaction data, the semantic descriptions comprising information about the user interface elements;

identifying a plurality of common interests across the plurality of contextual user models based on the user-specific interaction data; and

dynamically associating the plurality of contextual user models with one another based on the common interests.

20. The method of claim 19, comprising forming a plurality of collective contextual user models based on the associations of contextual user models, wherein each collective contextual user model comprises user-specific interaction data relating to the common interest.

21. The method of claim 20, comprising changing the plurality of collective contextual user models as the common interests change.

22. The method of claim 19, wherein the user-specific interaction data comprises active interaction data relating to a user interaction that results in an application event.

23. The method of claim 19, wherein the real-time sensor inputs comprise one or more of gaze-tracking inputs, proximity inputs, force inputs and pressure inputs.

24. A computing system comprising:

a display;

a sensor subsystem to obtain the passive interaction data;

one or more processors; and

one or more machine-readable media having stored therein a plurality of instructions that when executed by the processor cause the computing system to perform the method of claim 19.

25. A common interest discovery module to discover common interests between or among users interacting with a computing system, comprising executable instructions embodied in one or more machine-readable media, configured to:

access a plurality of contextual user models, each of the contextual user models comprising user-specific interaction data, the user-specific interaction data comprising;

passive interaction data comprising one or more real-time sensor inputs indicative of a passive user interaction in relation to on-screen locations of a display of the computing device, the passive interaction data relating to a user interaction that does not initiate an application event;

active interaction data, the active interaction data relating to a user interaction that results in an application event; and

semantic descriptions of user interface elements displayed at the on-screen locations corresponding to the passive interaction data and the active interaction data, the semantic descriptions comprising information about the user interface elements;

identify a collective interaction context that is common across the plurality of contextual user models; and

form a collective contextual user model based on the collective interaction context, the collective contextual user model comprising user-specific interaction data from the plurality of contextual user models, the user-specific interaction data relating to the common interaction context.