Technique for implementing an on-demand tool glass for use in a desktop user interface

• US 6,232,957 B1
• Filed: 11/25/1998
• Issued: 05/15/2001
• Est. Priority Date: 09/14/1998
• Status: Expired due to Term

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Hinckley, Kenneth Paul
• Primary Examiner(s): Hjerpe, Richard
• Assistant Examiner(s): Laneau, Ronald
• Application Number:
  US09/200,321
• Time in Patent Office: 902 Days
• Field of Search: 345/156, 345/157, 345/163, 345/326
• US Class Current: 345/156
• CPC Class Codes: G06F 2203/0382 Plural input, i.e. interface arrangements in which a plurality of input device of the same type are in communication with a PC
• CPC Class Codes: G06F 2203/04804 Transparency, e.g. transparent or translucent windows
• CPC Class Codes: G06F 2203/04805 Virtual magnifying lens, i.e. window or frame movable on top of displayed information to enlarge it for better reading or selection
• CPC Class Codes: G06F 3/03543 Mice or pucks G06F3/03541 takes precedence
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance

Abstract

A technique, specifically apparatus and accompanying methods, for implementing an on-demand “Tool Glass” based desktop user interface. The interface uses at least one input device capable of detecting touch. A sensed touch transition reflective of a user then making or breaking contact with the device, such by touching the device with a finger of a non-preferred hand or lifting his(her) finger from the device, causes a Tool Glass sheet to be displayed or dismissed. To prevent user distraction, these detected transitions preferably initiate corresponding predefined animation sequences that occur over preset time intervals in which the Tool Glass sheet either begins to fade into view as soon as user contact begins and then begins to fade out from view as soon as user contact ends. Such touch sensing can readily be used to provide “on-demand” display and dismissal of substantially any display widget, e.g., a toolbar, based on sensed contact between each hand of a user and a corresponding input device, such as between a preferred hand and a touch sensitive mouse. Through use of this interface, display clutter can be reduced and displayed application screen area increased at appropriate times during program execution consistent with user action and without imposing any significant cognitive burden on the user to do so; thereby, advantageously improving a “user experience”.

Claims

1. Apparatus for use with a computer system for implementing a Tool Glass based user interface, the system having a processor and a memory, connected to the processor, for storing computer executable instructions therein, and a display connected to the system, the apparatus comprising:

first and second input devices for connection to the system, wherein the first and second devices are manipulated by non-preferred and preferred hands, respectively, of a user, and each of the devices directly senses both whether the user is touching said each device and movement of said each device as it is being manipulated by a corresponding one of the hands, the first and second devices having first and second touch sensors for generating first and second output signals responsive to whether the user is touching the first and second devices with the non-preferred and preferred hands, respectively;

a graphical display, rendered on the display and produced by the system in response to the executable instructions, having a Tool Glass, containing at least one icon, a display area and a cursor, wherein the Tool Glass, as a first display widget, is situated over the display area but below the cursor, the Tool Glass and cursor being separately positionable in response to movement of the first and second devices, respectively; and

wherein the system displays or dismisses the Tool Glass in response to the first output signal produced by the first touch sensor and reflective of whether the non-preferred hand of the user then establishes or breaks physical contact with the first device by touching or releasing the first device, respectively.

2. The apparatus in claim 1 wherein the first device is a touchpad, touch sensitive mouse, touch sensitive trackball or other touch sensitive input pointing device.

3. The apparatus in claim 2 wherein the processor, in response to the executable instructions, controls fading of the Tool Glass in response to the first output signal such that upon initiation of the physical contact the Tool Glass fades into view from a substantially invisible state having a first predefined opacity to a semi-transparent state having a second predefined opacity and upon cessation of the physical contact the Tool Glass fades out of view from the semi-transparent state back to the substantially invisible state.

4. The apparatus in claim 3 wherein the first and second predefined opacities are approximately 0α

and at least approximately 0.7α

, respectively, where 0α and

1α

represent full transparency and full opacity, respectively.

5. The apparatus in claim 3 wherein the processor, in response to the executable instructions, fades the Tool Glass into and out of view over predefined first and second time intervals, respectively, and controls opacity of the Tool Glass while fading during each of the first and second intervals in response to predefined first and second functions, respectively.

6. The apparatus in claim 5 wherein the processor, in response to the executable instructions, initiates fading of the Tool Glass into view and out of view in response to corresponding transitions in the first output signal indicative of the non-preferred hand of the user establishing and breaking physical contact with the first device.

7. The apparatus in claim 6 wherein either of the predefined first and second functions is a predefined non-linear function of opacity with respect to time.

8. The apparatus in claim 7 wherein the non-linear function provides slow-in/slow-out fading.

9. The apparatus in claim 6 wherein either of the first and second predefined functions is a linear function.

10. The apparatus in claim 6 wherein the first and second intervals are, in duration, approximately 0.5 seconds or less, and at least approximately 1 second, respectively.

11. The apparatus in claim 6 wherein the first and second predefined opacities are approximately 0α

and at least approximately 0.7α

, respectively, where 0α and

1α

represent full transparency and full opacity, respectively.

12. The apparatus in claim 6 further comprising graphics and output circuitry, connected to the processor and the display, capable of implementing alpha-transparent graphic operations wherein the processor, in response to the executable instructions, issues instructions to the circuitry so as to change an alpha-transparency value of the Tool Glass rendered on the display for successive display frames during the first and second intervals and so implement a controlled animation sequence which, during the successive frames, fades the Tool Glass into or out of view, respectively.

13. The apparatus in claim 12 wherein the processor, in response to the executable instructions, varies coloration or luminance of the Tool Glass between successive ones of the frames during the animation sequence.

14. The apparatus in claim 12 wherein either of the predefined first and second functions is a predefined non-linear function of opacity with respect to time.

15. The apparatus in claim 14 wherein the non-linear function provides slow-in/slow-out fading.

16. The apparatus in claim 13 wherein either of the first and second predefined functions is a linear function.

17. The apparatus in claim 13 wherein the first and second intervals are, in duration, approximately 0.5 seconds or less, and at least approximately 1 second, respectively.

18. The apparatus in claim 13 wherein the first and second predefined opacities are approximately 0α

and at least approximately 0.7α

, respectively, where 0α and

1α

represent full transparency and full opacity, respectively.

19. The apparatus in claim 3 wherein the second device is a touchpad, touch sensitive mouse, touch sensitive trackball or other touch sensitive input pointing device and the processor, in response to the executable instructions, controls fading on the display of a second display widget, other than the Tool Glass, in response to the second output signal such that, in response to the preferred hand touching the second input device, the second display widget fades into view from a substantially invisible state having a third predefined opacity to an opaque state having a fourth predefined opacity and, in response to the preferred hand releasing the second input device, the second display widget fades out of view from the substantially opaque state back to the substantially invisible state.

20. The apparatus in claim 19 wherein if both the Tool Glass and the second display widget are simultaneously being rendered on the display and the physical contact ceases between the preferred hand and the second input device, the processor, in response to the executable instructions and which specific widgets are then being displayed, an operation then occurring or a relative display prioritization among widgets that could be simultaneously displayed, does not fade the second display widget from view.

21. The apparatus in claim 19 wherein the processor, in response to the executable instructions, fades the second display widget into and out of view over predefined third and fourth time intervals, respectively, and controls opacity of the second display widget while fading during each of the third and fourth intervals in response to predefined third and fourth functions.

22. The apparatus in claim 21 wherein the processor, in response to the executable instructions, initiates fading of the second display widget into view and out of view in response to corresponding transitions in the second output signal indicative of the preferred hand of the user establishing and breaking physical contact with the second device.

23. The apparatus in claim 22 wherein either of the predefined third and fourth functions is a predefined non-linear function of opacity with respect to time.

24. The apparatus in claim 23 wherein the non-linear function provides slow-in/slow-out fading.

25. The apparatus in claim 22 wherein either of the third and fourth predefined functions is a linear function.

26. The apparatus in claim 22 wherein the third and fourth intervals are, in duration, approximately 0.5 seconds or less, and at least approximately 1 second, respectively.

27. The apparatus in claim 22 wherein the third and fourth predefined opacities are approximately full transparency and full opacity, respectively.

28. The apparatus in claim 22 further comprising graphics and output circuitry, connected to the processor and the display, capable of implementing alpha-transparent graphic operations wherein the processor, in response to the executable instructions, issues instructions to the circuitry so as to change an alpha-transparency value of the second display widget rendered on the display for successive display frames occurring during the third and fourth intervals and so implement a controlled animation sequence which, during the successive frames, fades the second display widget into or out of view, respectively.

29. The apparatus in claim 28 wherein the processor, in response to the executable instructions, varies coloration or luminance of the second display widget between successive ones of the frames during the animation sequence.

30. The apparatus in claim 28 wherein either of the predefined third and fourth functions is a predefined non-linear function of opacity with respect to time.

31. The apparatus in claim 30 wherein the non-linear function provides slow-in/slow-out fading.

32. The apparatus in claim 29 wherein either of the third and fourth predefined functions is a linear function.

33. The apparatus in claim 29 wherein the third and fourth intervals are, in duration, approximately 0.5 seconds or less, and at least approximately 1 second, respectively.

34. The apparatus in claim 29 wherein the third and fourth predefined opacities are approximately full transparency and full opacity, respectively.

35. The apparatus in claim 19 wherein each of the first and second touch sensors produces a one-bit output signal.

36. The apparatus in claim 19 wherein the second input device is a touch sensitive mouse comprising:

a palm support; and

a palm sensor, as the second touch sensor and situated directly on the palm support, for determining whether a palm of the preferred hand is abutting against the palm support so as to generate the second output signal.

37. The apparatus in claim 36 wherein the processor, in response to the executable instructions, fades the second display widget into and out of view over predefined third and fourth time intervals, respectively, and controls opacity of the second display widget while fading during each of the third and fourth intervals in response to predefined third and fourth functions.

38. The apparatus in claim 37 wherein the processor, in response to the executable instructions, initiates fading of the second display widget into view and out of view in response to corresponding transitions in the second output signal indicative of the preferred hand of the user establishing and breaking physical contact with the second device.

39. The apparatus in claim 38 wherein either of the predefined third and fourth functions is a predefined non-linear function of opacity with respect to time.

40. The apparatus in claim 39 wherein the non-linear function provides slow-in/slow-out fading.

41. The apparatus in claim 38 wherein either of the third and fourth predefined functions is a linear function.

42. The apparatus in claim 38 wherein the third and fourth intervals are, in duration, approximately 0.5 seconds or less, and at least approximately 1 second, respectively.

43. The apparatus in claim 38 wherein the third and fourth predefined opacities are approximately full transparency and full opacity, respectively.

44. The apparatus in claim 38 further comprising graphics and output circuitry, connected to the processor and the display, capable of implementing alpha-transparent graphic operations wherein the processor, in response to the executable instructions, issues instructions to the circuitry so as to change an alpha-transparency value of the second display widget rendered on the display for successive display frames occurring during the third and fourth intervals and so implement a controlled animation sequence which, during the successive frames, fades the second display widget into or out of view, respectively.

45. The apparatus in claim 44 wherein the processor, in response to the executable instructions, varies coloration or luminance of the second display widget between successive ones of the frames during the animation sequence.

46. The apparatus in claim 44 wherein either of the predefined third and fourth functions is a predefined non-linear function of opacity with respect to time.

47. The apparatus in claim 46 wherein the non-linear function provides slow-in/slow-out fading.

48. The apparatus in claim 45 wherein either of the third and fourth predefined functions is a linear function.

49. The apparatus in claim 45 wherein the third and fourth intervals are, in duration, approximately 0.5 seconds or less, and at least approximately 1 second, respectively.

50. The apparatus in claim 45 wherein the third and fourth predefined opacities are approximately full transparency and full opacity, respectively.

51. A method for implementing a Tool Glass based user interface for use with a computer system, the system having a processor and a memory, connected to the processor, for storing computer executable instructions therein, and a display connected to the system, the method comprising the steps of:

manipulating first and second input devices connected to the system by non-preferred and preferred hands, respectively, of a user, wherein each of the devices directly senses both whether the user is touching said each device and movement of said each device as it is being manipulated by a corresponding one of the hands, the first and second devices having first and second touch sensors for generating first and second output signals responsive to whether the user is touching the first and second devices with the non-preferred and preferred hands, respectively;

rendering a graphical display on the display connected to the system, the graphical display being produced by the system in response to the executable instructions and having a Tool Glass, containing at least one icon, a display area and a cursor, wherein the Tool Glass, as a first display widget, is situated over the display area but below the cursor, the Tool Glass and cursor being separately positionable in response to movement of the first and second devices, respectively; and

displaying or dismissing the Tool Glass in response to the first output signal produced by the first touch sensor and reflective of whether the non-preferred hand of the user then establishes or breaks physical contact with the first device by touching or releasing the first device, respectively.

52. The method in claim 51, wherein the first device is a touch sensitive input pointing device, further comprising the step of controlling fading of the Tool Glass in response to the first output signal such that upon initiation of the physical contact the Tool Glass fades into view from a substantially invisible state having a first predefined opacity to a semi-transparent state having a second predefined opacity and upon cessation of the physical contact the Tool Glass fades out of view from the semi-transparent state back to the substantially invisible state.

53. The method in claim 52 wherein the first and second predefined opacities are approximately 0α

and at least approximately 0.7α

, respectively, where 0α and

1α

represent full transparency and full opacity, respectively.

54. The method in claim 52 further comprising the steps of:

fading the Tool Glass into and out of view over predefined first and second time intervals, respectively; and

controlling opacity of the Tool Glass while fading during each of the first and second intervals in response to predefined first and second functions, respectively.

55. The method in claim 54 further comprising the step of initiating fading of the Tool Glass into view and out of view in response to corresponding transitions in the first output signal indicative of the non-preferred hand of the user establishing and breaking physical contact with the first device.

56. The method in claim 55 wherein either of the predefined first and second functions is a predefined non-linear function of opacity with respect to time.

57. The method in claim 56 wherein the non-linear function provides slow-in/slow-out fading.

58. The method in claim 55 wherein either of the first and second predefined functions is a linear function.

59. The method in claim 55 wherein the first and second intervals are, in duration, approximately 0.5 seconds or less, and at least approximately 1 second, respectively.

60. The method in claim 55 wherein the first and second predefined opacities are approximately 0α

and at least approximately 0.7α

, respectively, where 0α and

1α

represent full transparency and full opacity, respectively.

61. The method in claim 55 further comprising the step of changing an alpha-transparency value of the Tool Glass rendered on the display for successive display frames occurring during the first and second intervals and so implement a controlled animation sequence which, during the successive frames, fades the Tool Glass into or out of view, respectively.

62. The method in claim 61 further comprising the step of varying coloration or luminance of the Tool Glass between successive ones of the frames during the animation sequence.

63. The method in claim 61 wherein either of the predefined first and second functions is a predefined non-linear function of opacity with respect to time.

64. The method in claim 63 wherein the non-linear function provides slow-in/slow-out fading.

65. The method in claim 62 wherein either of the first and second predefined functions is a linear function.

66. The method in claim 62 wherein the first and second intervals are, in duration, approximately 0.5 seconds or less, and at least approximately 1 second, respectively.

67. The method in claim 62 wherein the first and second predefined opacities are approximately 0α

and at least approximately 0.7α

, respectively, where 0α and

1α

represent full transparency and full opacity, respectively.

68. The method in claim 52, wherein the second device is a touch sensitive input pointing device, further comprising the step of controlling fading on the display of a second display widget, other than the Tool Glass, in response to the second output signal such that, in response to the preferred hand touching the second input device, the second display widget fades into view from a substantially invisible state having a third predefined opacity to an opaque state having a fourth predefined opacity and, in response to the preferred hand releasing the second input device, the second display widget fades out of view from the substantially opaque state back to the substantially invisible state.

69. The method in claim 68 further comprising the step, if both the Tool Glass and the second display widget are simultaneously being rendered on the display and the physical contact ceases between the preferred hand and the second input device, of not fading the second display widget from view in response to which specific widgets are then being displayed, an operation then occurring or a relative display prioritization among widgets that could be simultaneously displayed.

70. The method in claim 68 further comprising the steps of:

fading the second display widget into and out of view over predefined third and fourth time intervals, respectively; and

controlling opacity of the second display widget while fading during each of the first and second intervals in response to predefined third and fourth functions.

71. The method in claim 70 further comprising the step of initiating fading of the second display widget into view and out of view in response to corresponding transitions in the second output signal indicative of the preferred hand of the user establishing and breaking physical contact with the second device.

72. The method in claim 71 wherein either of the predefined third and fourth functions is a predefined non-linear function of opacity with respect to time.

73. The method in claim 72 wherein the non-linear function provides slow-in/slow-out fading.

74. The method in claim 71 wherein either of the third and fourth predefined functions is a linear function.

75. The method in claim 71 wherein the third and fourth intervals are, in duration, approximately 0.5 seconds or less, and at least approximately 1 second, respectively.

76. The method in claim 71 wherein the third and fourth predefined opacities are approximately full transparency and full opacity, respectively.

77. The method in claim 71 further comprising the step of changing an alpha-transparency value of the second display widget rendered on the display for successive display frames occurring during the third and fourth intervals and so implement a controlled animation sequence which, during the successive frames, fades the second display widget into or out of view, respectively.

78. The method in claim 77 further comprising the step of varying coloration or luminance of the second display widget between successive ones of the frames during the animation sequence.

79. The method in claim 77 wherein either of the predefined third and fourth functions is a predefined non-linear function of opacity with respect to time.

80. The method in claim 79 wherein the non-linear function provides slow-in/slow-out fading.

81. The method in claim 78 wherein either of the third and fourth predefined functions is a linear function.

82. The method in claim 78 wherein the third and fourth intervals are, in duration, approximately 0.5 seconds or less, and at least approximately 1 second, respectively.

83. The method in claim 78 wherein the third and fourth predefined opacities are approximately full transparency and full opacity, respectively.