Method and program product for resolving ambiguities through fading marks in a user interface

• US 20050251746A1
• Filed: 05/04/2004
• Published: 11/10/2005
• Est. Priority Date: 05/04/2004
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: International Business Machines Corporation
• Inventors: Faisman, Alexander, Basson, Sara H., Kanevsky, Dimitri
• Granted Patent: US 7,676,754 B2
• Time in Patent Office: Days
• US Class Current: 715/708
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 3/16 Sound input; Sound output speech processing G10L

Abstract

Resolving an ambiguous input. When an ambiguous user input is received, the user input is displayed, for example, in a way that either the ambiguity is resolved or the ambiguity is indicated to the user by unique and distinctive graphical indications. Then, after waiting either for a time to receive a corrective input from the user or for a predetermined time, the display of the user input is faded.

Claims

1. A method of resolving an ambiguous multimodal input to a computer system comprising

a). receiving an ambiguous user input wherein said ambiguity is in at least one of:

i) user intention,

ii) system interpretation of user input, and

iii) user perceptual satisfaction;

b). displaying the user input in a way that either the ambiguity is resolved or the ambiguity is indicated to the user by graphical means;

c). waiting a time to receive a corrective input from the user; and

d). fading the display of the user input.

2. The method of claim 1 wherein the multimodal input is chosen from the group consisting of touch, gestures, head mounted mouse movements, eye movements, voice, and voice intonations.

3. A computer system for resolving multimodal input ambiguity by

a) receiving a user input;

b) displaying the user input;

c) pausing processing the user input to receive a corrective user input; and

d) thereafter fading the feature from said graphical display.

4. The computer system of claim 3 wherein the multimodal input is chosen from the group consisting of touch, gestures, head mounted mouse, eye movements, voice, and voice intonations.

5. The system of claim 3 wherein the ambiguity is chosen from the group consisting of one or more of:

a). ambiguity in a user intention,

b). ambiguity in interpretation of a user intention,

c). ambiguity in user interpretation of computer actions;

d). ambiguity in user perception of computer actions;

or

e). user perceptual dissatisfaction in interactions between the user and the computer system.

6. The system of claim 3 wherein displaying a user input comprises either resolving ambiguity or indicating unresolved ambiguity

7. The system of claim 6 adapted to indicate ambiguity to the user by graphical output.

8. The system of claim 7 wherein the graphical output comprises fading mark.

9. The system of claim 3 wherein the input is a tactile input chosen from the group consisting of a touch screen sensor, a mouse pointer, or a head mounted mouse pointer.

10. The system of claim 9 wherein when the user applies a tactile input, the system displays a cursor and a circle associated with the cursor and collapsing on the cursor position.

11. The system of claim 10 wherein when the user positions the cursor around an area of interest and clicks a tactile drag button while dragging the tactile input, the system displays a rectangle where user dragging of the rectangle indicates the area that has been grabbed and dragged.

12. The system of claim 11 wherein when the user releases the tactile input drag button, the system updates the display filling in additional detail.

13. The system of claim 12 wherein the system fades the rectangle.

14. The system of claim 3 wherein the duration of pausing processing of the user input to receive a corrective user input is a function of different multimodal input events.

15. The system of claim 3 wherein the system generates a corrective input in the absence of a user reply.

16. The system of claim 3 wherein the system resolves user intention ambiguities in multimodal actions based upon in time delays between user input multimodal events.

17. The system of claim 3 adapted for geographic use wherein the sequence of and the time intervals between subsequent user input events touches and marks shows different location sequences.

18. A method of resolving an ambiguous multimodal input to a computer system comprising:

a) receiving a user input;

b) displaying the user input;

c) pausing processing the user input to receive a corrective user input; and

d) thereafter fading the feature from said graphical display.

19. The method of claim 18 wherein the multimodal input is chosen from the group consisting of touch, gestures, a head mounted mouse pointer, a mouse pointer, eye movements, voice, and voice intonations.

20. The method of claim 18 wherein the ambiguity is chosen from the group consisting of one or more of:

a). ambiguity in a user intention,

b). ambiguity in interpretation of a user intention,

c). ambiguity in user interpretation of computer actions;

d). ambiguity in user perception of computer actions;

or

e). user perceptual dissatisfaction in interactions between the user and the computer system.

21. The method of claim 18 comprising displaying a user input and either resolving ambiguity or indicating unresolved ambiguity

22. The method of claim 18 comprising graphically indicating ambiguity to the user.

23. The method of claim 22 comprising indicating ambiguity to the user by fading a mark.

24. The method of claim 18 wherein the input is chosen from the group consisting of a touch screen sensor, a gesture, a head mounted mouse pointer, and a mouse pointer.

25. The method of claim 24 comprising applying a tactile input, and the system displaying a cursor and a circle associated with the cursor, and collapsing the circle on the cursor position.

26. The method of claim 25 comprising positioning the cursor around an area of interest and clicking a tactile drag button while dragging the tactile input, displaying a rectangle where user dragging of the rectangle indicates the area that has been grabbed and dragged.

27. The method of claim 26 wherein when the user releases the tactile input drag button, the system updates the display filling in additional detail.

28. The method of claim 27 wherein the system fades the rectangle.

29. The method of claim 18 wherein the duration of pausing processing the user input to receive a corrective user input is a function of different multimodal input events.

30. The method of claim 18 comprising the system generating a corrective input in the absence of a user reply.

31. The method of claim 18 comprising the system resolving user intention ambiguities in multimodal actions based upon in time delays between user input multimodal events.

32. The method of claim 18 adapted for geographic use wherein the sequence of and the time intervals between subsequent user input events touches and marks shows different location sequences.

33. A program product comprising computer readable code on a medium, said computer readable code configuring and directing a computer to perform a method for resolving an ambiguous multimodal input to a computer system comprising:

a) receiving a user input;

b) displaying the user input;

c) pausing processing the user input to receive a corrective user input; and

d) thereafter fading the feature from said graphical display.

34. The program product of claim 33 wherein the multimodal input is chosen from the group consisting of touch, gestures, a mouse pointer, a head mounted mouse pointer, eye movements, voice, and voice intonations.

35. The program product of claim 33 wherein the ambiguity is chosen from the group consisting of one or more of:

a). ambiguity in a user intention,

b). ambiguity in interpretation of a user intention,

c). ambiguity in user interpretation of computer actions;

d). ambiguity in user perception of computer actions;

or

e). user perceptual dissatisfaction in interactions between the user and the computer system.

36. The program product of claim 33 adapted for displaying a user input and either resolving ambiguity or indicating unresolved ambiguity

37. The program product of claim 36 adapted for graphically indicating ambiguity to the user.

38. The program product of claim 33 adapted for indicating ambiguity to the user by fading a mark.

39. The program product of claim 33 wherein the input means is chosen from the group consisting of a touch screen sensor, a mouse pointer, and a head mounted mouse pointer.

40. The program product of claim 39 adapted for receiving a user tactile input, displaying a cursor and a circle associated with the cursor in response thereto, and collapsing the circle on the cursor position.

41. The program product of claim 40 adapted for the user positioning the cursor around an area of interest and clicking a tactile drag button while dragging the tactile input, and thereafter displaying a rectangle where user dragging of the rectangle indicates the area that has been grabbed and dragged.

42. The program product of claim 41 adapted for updating the display when the user releases the tactile input drag button.

43. The program product of claim 42 adapted for fading the rectangle.

44. The program product of claim 33 adapted for pausing processing the user input to receive a corrective user input as a function of different multimodal input events.

45. The program product of claim 33 adapted for generating a corrective input in the absence of a user reply.

46. The program product of claim 33 adapted for system resolving user intention ambiguities in multimodal actions based upon in time delays between user input multimodal events.

47. The program product of claim 33 adapted for geographic use wherein the sequence of and the time intervals between subsequent user input events touches and marks indicates and displays location sequences.