Systems and methods for estimating and integrating measures of human cognitive load into the behavior of computational applications and services

• US 6,999,955 B1
• Filed: 06/28/2002
• Issued: 02/14/2006
• Est. Priority Date: 04/20/1999
• Status: Expired due to Fees

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Horvitz, Eric J.
• Primary Examiner(s): STARKS, WILBERT L
• Application Number:
  US10/186,135
• Time in Patent Office: 1,327 Days
• Field of Search: 702/57, 706/10, 706/45, 114/382, 700/47
• US Class Current: 706/45
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 20/10 using kernel methods, e.g. support vector machines [SVM]
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks

Abstract

The present invention relates to a system and methodology for extending and making more appropriate the interactive decisions made by automated services by taking into consideration estimates of the time required by users for the cognitive processing of problems, visualizations, and content based on several factors, including the competency and familiarity of the user with the output and nature of the sequences of output, the complexity of the output and overall tasks being considered, and the context of the user. The methods allow automated applications to control the rate at which the applications interact with users. Portions of automated services are controlled in view of limited human processor capabilities in design/operation of such services, and/or visualizations/output from the services (e.g., amount of dwell time provided/considered before a next automated sequence is displayed/invoked). Models can be formulated and utilized in conjunction with automated services such as automatically determining and pausing a period of time before a next sequence in a service is enabled or disabled. The models can also be employed to determine multiple dwells for comprehension, as a function of complexity of graphical/conceptual relationships, at different phases of an animation.

Claims

1. A computer-implemented automated system for controlling applications, comprising:

at least one model that is adapted to human cognitive processing capability; and

a dwell component that receives results from the model to control a rate at which applications are performed.

2. The system of claim 1, further comprising an output to drive at least one of a graphical display, a text display, an audio output, and a mechanical output.

3. The system of claim 1, the applications include at least one of a service, a display, an e-mail, a calendar, desktop service(s), a desktop control, a web service, a browser, and animated display sequences that are executable by a computer.

4. The system of claim 1, the dwell component provides dwell controls that represent at least one of codes, flags, and signals related to timing data.

5. The system of claim 4, the dwell controls at least one of pause, delay, start, end, and at the applications.

6. The system of claim 4, further comprising a cognitive processor that processes user data to determine current user state information.

7. The system of claim 6, the cognitive processor executes the at least one model to process the user state information and provides data to the dwell component, the at least one model determines information relating to an ability to process information by a user.

8. The system of claim 6, the cognitive processor processes feedback from the application indicating at least one of a complexity, a number of objects, and object associations between objects, that are at least one of currently and about to be processed by the user.

9. The system of claim 6, the user data is derived from at least one of user inputs, a keyboard, a mouse, a microphone, a camera, an infrared detector, previous computer activities that are monitored over time, past applications employed, number of times the past applications have been accessed, and speed of interaction with a respective application.

10. The system of claim 9, the user inputs are employed to provide at least one of manual indications, selections, and adjustments to enable fine-tuning of the rate at which the applications are performed.

11. The system of claim 4, the dwell controls include at least one of formatting timer values, and controlling a signal in accordance with determined time values that are at least one of larger and smaller depending on cognitive capabilities of the user.

12. The system of claim 11, the dwell controls are dynamically according to at least one of determined cognitive capabilities of the user and feedback from the application.

13. The system of claim 1, the at least one model is adapted in part from the following expression:

C_T=ƒ

(C_P1, C_P2 . . . C_PN)

wherein C_T relates to a comprehension time, ƒ

is a function, C_P relates to cognitive parameters or variables, and N is an integer.

14. The system of claim 13, the at least one model is employed with at least one of a probability model, a Bayesian inference models, a Support Vector Machine (SVM), a Naive Bayes model, Bayes Net model, a decision tree model, a similarity-based model, and a vector-based model to determine cognitive capabilities of the user.

15. The system of claim 1, the at least one model is adapted in part from the following expression:

C_T=ƒ

(C_C, C_E, C_B, C_O)

wherein C_C relates to a nature and complexity of items at a user'"'"'s focus of attention, C_E relates to an at least one of an environmental and a work context, C_B relates to background knowledge of the user (e.g., can shorten dwell times in an animated visualization as the user may be familiar with the data structure), and C_O relates to an overall competency of the user with regards to the item at the user'"'"'s focus of attention.

16. A computer-implemented method to facilitate automated cognitive processing and control, comprising:

determining at least one parameter associated with a cognitive function of a user; and

controlling a tempo of an application based at least in part on the at least one parameter.

17. The method of claim 16, further comprising determining one or more user states, the states including at least one of whether the user is busy, having a conversation, how many applications are currently active, what of type applications are currently active, the user'"'"'s goals and attentional resources.

18. The method of claim 16, providing feedback from the application indicating a complexity involved with a current application being accessed by the user.

19. The method of claim 16, further comprising determining a dwell based upon the at least one parameter.

20. The method of claim 19, further comprising automatically determining if a user'"'"'s cognitive capabilities have changed over time and dynamically adjusting the dwell.

21. A computer-implemented method to facilitate dynamic visual processing, comprising:

segmenting portions of an application into a set of sub-problems via a decomposition of a conceptual structure into a sequence of structures; and

determining transformations among the structures, the portions of the application and the transformations relate to cognitive determinations for presentation to a user.

22. The method of claim 21, further comprising automatically providing information to the sub-problems as feedback to a cognitive processor.

23. The method of claim 21, further comprising searching through a variety of abstractions to select a set of the sub-problems at a level of abstraction to determine suitably sized chunks for cognitive processing.

24. The method of claim 23, further comprising determining at least one of natural clusters and break points in an overall complexity of a structure of the sub-problems and a complexity of the transformations among sub-problems that are captured by animated transformations.

25. The method of claim 21, further comprising determining a user'"'"'s familiarity with a visualization, a presentation and a measure of complexity to derive a suitable temporal sequence.

26. The method of claim 21, further comprising controlling at least one of a dwell and a rate of an animation.

27. The method of claim 26, further comprising applying the at least one dwell and the rate hierarchically to a set of sub-animations.

28. The method of claim 26, further comprising at least one of determining the dwell as a function of a complexity of a structure and determining the rate of animation as a function of the complexity of a transformations.

29. The method of claim 21, further comprising determining a measure of complexity, the measure includes functions of at least one of a number of concepts, a number of objects, objects, and a number of relationships among the objects.

30. The method of claim 21, further comprising controlling an animation via at least one of:

controlling an initial dwell;

controlling a pace of transformation i, i being an integer; and

controlling a rate of transformations of at least one of a relationship and a view 1 . . . n, n being an integer.

31. The method of claim 30, further comprising at least one of:

providing a review dwell;

providing a review presentation;

controlling a pace of preparation; and

preparing a new transformation including at least one a change in orientation of a structure and a viewpoint change.

32. The method of claim 30, further comprising at least one of:

collapsing at least one of visualization phases and sub-problems into a single animation that is controlled by a rate; and

expanding the phases and sub-problems into more detailed phases and sub-problems.

33. The method of claim 21, further comprising at least one of;

enabling a user to control a speed of a visualization, the control including at least one of freezing, slowing down, and speeding up a pace of animations; and

controlling a duration of dwells in-stream with an ongoing visualization sequence.

34. The method of claim 33, further comprising at least one of:

receiving an input gestures to control at least one of a pause and a repeat of a transformation;

transforming a viewpoint; and

at least one of marking and scaling a size of a selected component of a transformation.

35. The method of claim 34, further comprising highlighting at least one of text and an object to facilitate visualization of at least one of a flow of objects, components, relationships from a first sub-problem to another sub-problem, and from a previous sub-problem to a final rendering.

36. The method of claim 21, further comprising analyzing feedback data to determine how a suitable viewing rate is influenced by complexity.

37. The method of claim 21, further comprising determining that a complexity of a visualization is influenced positively by a number of objects and relationships among objects.

38. The method of claim 21, further comprising determining that a complexity of a visualization is influenced negatively by a function of a familiarity of a user with objects and relations represented in a visualization.

39. The method of claim 21, further comprising identifying an effective complexity based on at least one of short-term and long-term memory considerations.

40. The method of claim 21, further comprising determining as a complexity of structure and a transformation associated with an animation increases, controlling a pace of the animation in a manner wherein more complex transformations are associated with a slower pace and less complex transformations are associated with a faster pace.

41. The method of claim 21, further comprising determining that a duration of dwell on a substructure can be diminished with growing familiarity within the substructure and in accordance with the complexity of the substructure.

42. The method of claim 21, further comprising computing a transformation complexity as a function of at least one of initial relationships in a structure being transformed, new objects being introduced, new relationships being introduced, the relationships being deleted (negative correlation), and a magnitude of change in orientation of an object.

43. The method of claim 21, further comprising enhancing an effectiveness and satisfaction associated with a visualization by optimizing at least one of a duration of dwell on a substructure, a pace of a transformations, increasing the pace, and performing frank chunking of subsequences of visualizations having familiarity.