Empathetic human-machine interfaces

• US 20040250218A1
• Filed: 06/06/2003
• Published: 12/09/2004
• Est. Priority Date: 06/06/2003
• Status: Abandoned Application

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Eltezadi, Cameron A., Wecker, David B.
• Application Number:
  US10/456,365
• Publication Number:
  US 20040250218A1
• Time in Patent Office: Days
• US Class Current: 715/812
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
• CPC Class Codes: G06F 3/04812 Interaction techniques based on cursor appearance or behaviour, e.g. being affected by the presence of displayed objects

Abstract

An empathetic interface individualizes the interaction of a user and a computer. The empathetic interface educes user intentions to move or perform actions with a pointing device without the user having to objectively communicate in an explicit manner, such as manually moving the mouse from one location to another location or performing a user interface action. The user initiates a movement or an action and the empathetic interface completes the movement or the action.

Claims

1. A computer system for educing intentions of a user, comprising:

an on-screen cursor for performing user interface actions in a user interface, the on-screen cursor being controlled by a pointing device; and

an empathetic interface for educing a user'"'"'s intention to move the on-screen cursor from a first location to a target location in the user interface and moving the on-screen cursor to the target location from the first location when the user initiates actuation of the pointing device to cause the on-screen cursor to begin to move toward the target location.

2. The computer system of claim 1, wherein the on-screen cursor includes an on-screen icon.

3. The computer system of claim 2, wherein the on-screen cursor is selected from a group consisting of a blinking underline, a blinking rectangle, a blinking vertical bar, and an arrowhead leaning slightly leftward or rightward.

4. The computer system of claim 1, wherein the pointing device is selected from a group consisting of a mouse, a graphics tablet, a stylus, a light pen, a joystick, a puck, and a trackball.

5. The computer system of claim 1, wherein the empathetic interface receives information pertaining to the actuation of the pointing device and produces the target location to which the user intends to move the on-screen cursor.

6. A computer system for building an empathetic interface to educe a user'"'"'s intentions to navigate an on-screen cursor in a user interface, the computer system comprising:

a model builder adapted for receiving events generated by a pointing device when the user navigates the on-screen cursor toward a user interface target to perform an action, the model builder being further adapted for receiving model parameters; and

a model for empathetically educing a user'"'"'s intention to navigate the on-screen cursor toward the user interface target to perform the action, the model being built by the model builder in accordance with the received events and the model parameters.

7. The computer system of claim 6, wherein the pointing device includes a mouse.

8. The computer system of claim 7, further comprising a mouse information gatherer for gathering mouse, keyboard, and timing information to produce stream files with which to build the empathetic interface.

9. The computer system of claim 8, wherein the mouse information gatherer includes a mouse data hook for collecting events generated by the actuations of the mouse.

10. The computer system of claim 9, wherein the mouse information gatherer includes a keyboard data hook for collecting events generated by keyboard key presses.

11. The computer system of claim 10, wherein the mouse information gatherer includes a timer for generating time-out events to indicate inactivity after the pointer is moved but no action is performed.

12. The computer system of claim 11, wherein the mouse information gatherer includes a movement detector that extracts the Cartesian coordinates x, y, and the absolute time from an event received from the mouse data hook.

13. The computer system of claim 12, wherein the mouse information gatherer includes a button detector that extracts from the event a button on the mouse that was pressed and the state of the depressed button.

14. The computer system of claim 13, wherein the mouse information gatherer includes a decimator for reducing the number of data samples comprising the events.

15. The computer system of claim 14, wherein the mouse information gatherer includes a collector for moving stream files to a storage location so as to prevent taxing a computing system on which the mouse information gatherer is executed.

16. The computer system of claim 6, further comprising an aggregator for presenting a stream file to a developer for selecting the contents of the stream file for building the model.

17. The computer system of claim 16, further comprising a database for storing the contents of the selected stream file.

18. The computer system of claim 6, further comprising the empathetic interface that incorporates the model so as to educe the user'"'"'s intentions.

19. The computer system of claim 18, further comprising a trainer to retrain the model when the eduction of the model is not sufficiently accurate.

20. A computer-readable medium having a data structure stored thereon for use by a computing system to educe user intentions, the data structure comprising:

a header field that is indicative of a screen resolution and a screen origin; and

a data field that is indicative of a start event, a move event, and a termination event of a pointing device being actuated by a user in navigating an on-screen cursor in a user interface.

21. The data structure of claim 20, wherein nesting within the header field is a screen resolution field that indicates a width and a height of a screen.

22. The data structure of claim 20, wherein nesting within the header field is a screen origin field that indicates Cartesian coordinates of the origin of a screen to facilitate normalization.

23. The data structure of claim 20, wherein nesting within the data field is a start event field, which indicates the beginning of a new pointing device action, and wherein nesting within the data field are fields for storing Cartesian coordinates x, y, which define the location of the pointing device, and an absolute time.

24. The data structure of claim 20, wherein nesting within the data field is a window event field, which includes a window class field and an application type field for distinguishing one event from another event.

25. The data structure of claim 20, wherein nesting within the data field is a move event field, which includes fields for storing Cartesian coordinates x, y, and a relative time, for indicating a move event.

26. The data structure of claim 20, wherein nesting within the data field is a termination event field, which includes fields for storing Cartesian coordinates x, y, and a relative time, for indicating a termination event.

27. The data structure of claim 20, wherein nesting within the data field is another window event field, which includes fields for storing a window class and an application type in which the on-screen cursor ends up.

28. The data structure of claim 20, wherein the header allows normalization of negative coordinates so as enable referencing every location of any screen to Cartesian coordinates 0, 0.

29. The data structure of claim 20, wherein the header allows normalization of pointing device information gathered from different users with different screen resolutions.

30. A computer system for educing intentions of a user, comprising:

an operating system that controls usage of resources in the computer system; and

an empathetic interface coupled to the operating system for educing a user'"'"'s intention to move an on-screen cursor from a first location to a target location in a user interface to perform an action and moving the on-screen cursor to the target location from the first location to perform the action when the user initiates actuation of a pointing device to cause the on-screen cursor to begin to move toward the target location.

31. The computer system of claim 30, wherein the empathetic interface includes a personal gatherer for gathering mouse, keyboard, and timer information.

32. The computer system of claim 30, wherein the empathetic interface includes a personal model that can educe the user'"'"'s intention to move the on-screen cursor to the target location to perform the action.

33. The computer system of claim 30, further comprising a trainer for retraining the empathetic interface when its eduction is not sufficiently accurate.

34. The system of claim 33, wherein the trainer includes a trainer model, which can educe a user'"'"'s intention, and a trainer gatherer, which gathers both actual actions and predicted actions for comparison.

35. A method implemented in a computer system for educing a user'"'"'s intention for navigating a pointer in a user interface, the method comprising:

inputting a set of Cartesian coordinates, which are indicative of a first location of the pointer when the user initiates the actuation of a pointing device toward a target location, the act of inputting including inputting a velocity of the actuation into a model; and

empathetically educing the target location including an action to be taken by the pointer when the pointer has been moved by the method to the target location.

36. The method of claim 35, wherein the act of inputting includes inputting data selected from a group consisting of changes in the Cartesian coordinates, changes in the vertical acceleration of the actuation of the pointing device, changes in the horizontal acceleration of the actuation of the pointing device, a size of the screen, and an acceleration of the actuation of the pointing device.

37. The method of claim 35, wherein the act of educing includes educing an action that causes a button state of a pointing device to be up or down.

38. The method of claim 35, wherein the act of educing includes educing an action that causes the pointer to move to a menu item in a menu that is likely to be used by the user.

39. The method of claim 35, wherein the act of educing includes educing an action that causes scrolling of a document at a speed desired by the user.

40. A computer-readable medium having computer-executable instructions for educing a user'"'"'s intention for navigating a pointer in a user interface, the method comprising:

inputting a set of Cartesian coordinates, which are indicative of a first location of the pointer when the user initiates the actuation of a pointing device toward a target location, the act of inputting including inputting a velocity of the actuation into a model; and

empathetically educing the target location including an action to be taken by the pointer when the pointer has been moved by the method to the target location.

41. The computer-readable medium of claim 40, wherein the act of inputting includes inputting data selected from a group consisting of changes in the Cartesian coordinates, changes in the vertical acceleration of the actuation of the pointing device, changes in the horizontal acceleration of the actuation of the pointing device, a size of the screen, and an acceleration of the actuation of the pointing device.

42. The computer-readable medium of claim 40, wherein the act of educing includes educing an action that causes a button state of a pointing device to be up or down.

43. The computer-readable medium of claim 40, wherein the act of educing includes educing an action that causes the pointer to move to a menu item in a menu that is likely to be used by the user.

44. The computer-readable medium of claim 40, wherein the act of educing includes educing an action that causes the scrolling of a document at a speed desired by the user.

45. A method implementable in a computer system for building an empathetic interface that educes user intentions for navigating a pointer, the method comprising:

accumulating data relating to a pointing device;

building a model for educing user intentions using the accumulated data;

educing the user'"'"'s intention to navigate the pointer from a first location of the pointer to perform an action at a target location when the user initiates the actuation of a pointing device toward the target location; and

retraining the model if the model is not sufficiently accurate.

46. A computer-readable medium having computer-executable instructions for building an empathetic interface that educes user intentions for navigating a pointer, the method comprising:

accumulating data relating to a pointing device;

building a model for educing user intentions using the accumulated data;

educing the user'"'"'s intention to navigate the pointer from a first location of the pointer to perform an action at a target location when the user initiates the actuation of a pointing device toward the target location; and

retraining the model if the model is not sufficiently accurate.