Thematic response to a computer user's context, such as by a wearable personal computer

• US 7,137,069 B2
• Filed: 07/16/2001
• Issued: 11/14/2006
• Est. Priority Date: 12/18/1998
• Status: Expired due to Fees

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Tangis Corporation
• Inventors: Robarts, James O., Newell, Dan, Abbott, Kenneth H., Freedman, Joshua M., Apacible, Johnson
• Primary Examiner(s): Hailu, Tadesse
• Application Number:
  US09/907,008
• Publication Number:
  US 20010043231A1
• Time in Patent Office: 1,947 Days
• Field of Search: 345733-748, 700 28- 31, 700/91, 700/93, 706/10, 706/11, 706 45- 61, 709201-207, 709223-226, 715733-748
• US Class Current: 715/744
• CPC Class Codes: G06F 1/163 Wearable computers, e.g. on a belt
• CPC Class Codes: G06F 16/40 of multimedia data, e.g. slideshows comprising image and additional audio data retrieval of still image data G06F16/50; retrieval of audio data G06F16/60; retrieval of video data G06F16/70
• CPC Class Codes: G06F 21/00 Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
• CPC Class Codes: G06F 2221/2149 Restricted operating environment
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06N 20/00 Machine learning

Abstract

Techniques are disclosed for creating, modifying, categorizing, modeling, distributing, purchasing, selling, and otherwise using themes and theme-related information. Themes can represent various types of contextual aspects or situations, and can model high-level concepts of activities or states not reflected in individual contextual attributes that each model a single aspect of the state of a user, their computing device, the surrounding physical environment, and/or the current cyber-environment. Such themes specify inter-relationships among a set of contextual attributes, and can have associated theme-related information such as theme-specific attributes, theme layouts used to present information and functionality, CSes that provide theme attribute values, and CCs that process theme information. Disclosed techniques can identify one or more themes that currently match the modeled context, select one of the matching themes as a current theme, and provide an appropriate response (e.g., by presenting appropriate information and/or providing appropriate functionality) based on the current theme.

Claims

1. A method in a computing device for receiving themes representing contextual situations and customizing the received themes for use in providing appropriate responses to the represented contextual situations, the method comprising:

receiving at the computing device multiple uncustomized themes supplied from one or more external sources that further supply the multiple uncustomized themes to multiple other distinct computing devices whose users are distinct from a user of the computing device, each of the received themes representing a contextual situation and associated with a type of response that is appropriate to the represented contextual situation;

customizing at least one of the received themes to reflect the computing device and/or the user of the computing device;

receiving an indication of a current contextual situation;

determining that one of the customized themes represents the current contextual situation; and

providing the type of response that is associated with the one theme.

2. The method of claim 1 wherein the received multiple themes include a first theme from a first external source, wherein the first theme is customized by the computing device to reflect the computing device and/or a user of the computing device, and including, under control of each of multiple other computing devices:

receiving from one of the external sources the first theme; and

customizing the first theme to reflect the other computing device and/or a user of the other computing device, the customized first theme for the other computing device being distinct from the customized first theme for the computing device.

3. The method of claim 1 wherein the representing of a contextual situation by a theme is with multiple context attributes related to that contextual situation and with criteria for determining values of those context attributes that match that contextual situation, wherein the current contextual situation includes multiple context attribute values, and wherein the determining that the one theme represents the current contextual situation is performed automatically based on the included values of the current contextual situation containing values for the context attributes of the one theme that the criteria of the one theme determines to be matching values.

4. The method of claim 1 wherein the customizing of the received themes is based on explicit indications received from the user of the computing device.

5. The method of claim 1 wherein the customizing of the received themes is performed automatically without intervention by the user.

6. The method of claim 1 wherein the customizing of the received themes is based on likely preferences of the user of the computing device that are determined without intervention by the user.

7. The method of claim 6 including, after the determining of the likely preferences and before the customizing of the received theme, receiving explicit user approval of the customizing.

8. The method of claim 6 including monitoring actions of the user in order to learn the likely user preferences.

9. The method of claim 1 wherein the customizing of the received themes is to specialize the received themes to the user of the computing device.

10. The method of claim 1 wherein the customizing of the received themes is to specialize the received themes to the computing device.

11. The method of claim 1 wherein the customizing of the received themes is to modify the types of response that are associated with the received themes.

12. The method of claim 1 wherein the customizing of the received themes is to modify the contextual situations represented by the received themes.

13. The method of claim 1 wherein the receiving of one of the themes includes receiving at least one context attribute that is specified by that one theme, a context server module able to provide values for one or more context attributes specified by that one theme and/or a context client module capable of producing responses appropriate to the contextual situation represented by that one theme, and wherein the customizing of that received theme includes customizing one of the received context attributes, the received context server module, or the received context client module.

14. The method of claim 1 wherein the receiving of one of the themes includes receiving an associated theme layout that specifies information appropriate to the contextual situation represented by that theme and that specifies a manner of presenting the specified information, and wherein the customizing of that received theme includes customizing the associated theme layout.

15. The method of claim 1 including determining that the provided type of response associated with the one theme is inappropriate for the current contextual situation, and automatically customizing the one theme based on the determining.

16. The method of claim 15 wherein the determining is based on a received user indication.

17. The method of claim 1 including receiving an indication of a new contextual situation, determining that no available theme represents the new contextual situation, and automatically customizing one of the available themes based on the determining.

18. The method of claim 1 wherein the provided type of response includes presenting information to the user of the computing device that is appropriate to the current contextual situation.

19. The method of claim 1 wherein the customizing of one of the received themes is in such a manner that the associated type of response for the customized theme includes presenting information specific to the user of the computing device.

20. The method of claim 1 wherein the contextual situations represented by the received themes are activities of the user of the computing device.

21. The method of claim 1 wherein one of the represented contextual situations relates to a situation of the user of the computing device.

22. The method of claim 1 wherein the one represented contextual situation reflects a modeled mental state of the user.

23. The method of claim 1 wherein one of the represented contextual situations relates to a situation of the computing device.

24. The method of claim 1 wherein at least one of the represented contextual situations relates to a physical environment.

25. The method of claim 1 wherein at least one of the represented contextual situations relates to a cyber-environment of the user of the computing device.

26. The method of claim 1 wherein at least one of the received themes includes permission information that specifies how that theme can be customized, and wherein the customizing of that theme is performed in accordance with that permission information.

27. The method of claim 26 wherein the permission information specifies that the theme cannot be modified, and wherein the customizing of that theme in accordance with that permission information includes not customizing the theme.

28. The method of claim 1 wherein the computing device is a wearable computer.

29. The method of claim 1 wherein the computing device is a cellular phone.

30. The method of claim 1 wherein the computing device is a mobile phone.

31. The method of claim 1 wherein the computing device is a handheld device mounted in a holster on a user of the handheld device.

32. The method of claim 1 wherein the computing device is a thin client device.

33. A computer-readable storage medium whose contents cause a computing device to receive themes representing contextual situations and customize the received themes for use in providing appropriate responses to the represented contextual situations, by performing a method comprising:

receiving multiple themes each representing a contextual situation and associated with a type of response that is appropriate to the represented contextual situation, each of the received themes distributed by a remote source and available to be distributed to one or more other distinct computing devices;

customizing at least one of the received themes;

determining that one of the received themes represents a current contextual situation; and

providing the type of response that is associated with the one theme.

34. The computer-readable medium of claim 33 wherein the computer-readable medium is a memory of a computer system.

35. The computer-readable medium of claim 33 wherein the representing of a contextual situation by a theme is with multiple context attributes related to that contextual situation and with criteria for determining values of those context attributes that match that contextual situation, wherein the current contextual situation includes multiple context attribute values, and wherein the determining that the one theme represents the current contextual situation is performed automatically based on the included values of the current contextual situation containing values for the context attributes of the one theme that the criteria of the one theme determines to be matching values.

36. The computer-readable medium of claim 33 wherein the customizing of the received themes is performed automatically without intervention by a user of the computing device.

37. The computer-readable medium of claim 33 wherein the customizing of the received themes is to specialize the received themes to a user of the computing device.

38. The computer-readable medium of claim 33 wherein the customizing of the received themes is to specialize the received themes to the computing device.

39. The computer-readable medium of claim 33 wherein the customizing of the received themes is to modify the types of response that are associated with the received themes.

40. The computer-readable medium of claim 33 wherein the customizing of the received themes is to modify the contextual situations represented by the received themes.

41. The computer-readable medium of claim 33 wherein the receiving of one of the themes includes receiving at least one context attribute that is specified by that one theme, a context server module able to provide values for one or more context attributes specified by that one theme and/or a context client module capable of producing responses appropriate to the contextual situation represented by that one theme, and wherein the customizing of that received theme includes customizing at least one of the received context attributes, the received context server module, and the received context client module.

42. The computer-readable medium of claim 33 wherein the receiving of one of the themes includes receiving an associated theme layout that specifies information appropriate to the contextual situation represented by that theme and that specifies a manner of presenting the specified information, and wherein the customizing of that received theme includes customizing the associated theme layout.

43. The computer-readable medium of claim 33 wherein the method further comprises determining that the provided type of response associated with the one theme is inappropriate for the current contextual situation, and automatically customizing the one theme based on the determining.

44. The computer-readable medium of claim 33 wherein the method further comprises receiving an indication of a new contextual situation, determining that no available theme represents the new contextual situation, and automatically customizing one of the available themes based on the determining.

45. The computer-readable medium of claim 33 wherein the provided type of response includes presenting information to a user of the computing device that is appropriate to the current contextual situation.

46. The computer-readable medium of claim 33 wherein the contextual situations represented-by the received themes are activities-of a user of the computing device.

47. The computer-readable medium of claim 33 wherein the computing device is a wearable computer.

48. The computer-readable medium of claim 33 wherein the computing device is a cellular phone.

49. The computer-readable medium of claim 33 wherein the computing device is a mobile phone.

50. The computer-readable medium of claim 33 wherein the computing device is a handheld device mounted in a holster on a user of the handheld device.

51. The computer-readable medium of claim 33 wherein the computing device is a thin client device.

52. A computing device for receiving themes representing contextual situations and customizing the received themes for use in providing appropriate responses to the represented contextual situations, comprising:

one or more processors;

an input component configured to receive at least one theme from an external source that distributes the at least one theme to multiple computing devices having distinct users, each of the received themes representing a contextual situation and associated with a type of response that is appropriate to the represented contextual situation;

a customizing component that when executed by at least one or more processors is configured to customize at least one of the received themes in a manner specific to the computing device; and

a response provider component that when executed by at least one or more processors is configured to determine that one of the received themes represents a current contextual situation and providing the type of response that is associated with the one theme.

53. The computing device of claim 52 wherein the customizing component and the response provider component include software instructions stored in a memory.

54. The computing device of claim 52 wherein the computing device is a wearable computer.

55. The computing device of claim 52 wherein the representing of a contextual situation by a theme is with multiple context attributes related to that contextual situation and with criteria for determining values of those context attributes that match that contextual situation, wherein the current contextual situation includes multiple context attribute values, and wherein the determining that the one theme represents the current contextual situation is performed automatically based on the included values of the current contextual situation containing values for the context attributes of the one theme that the criteria of the one theme determines to be matching values.

56. The computing device of claim 53 wherein the customizing of the received themes is performed automatically without intervention by a user of the computing device.

57. The computing device of claim 53 wherein the customizing of the received themes is to specialize the received themes to a user of the computing device.

58. The computing device of claim 53 wherein the customizing of the received themes is to specialize the received themes to the computing device.

59. The computing device of claim 53 wherein the provided type of response includes presenting information to a user of the computing device.

60. The computing device of claim 53 wherein the computing device is a cellular phone.

61. The computing device of claim 53 wherein the computing device is a mobile phone.

62. The computing device of claim 53 wherein the computing device is a handheld device.

63. The computing device of claim 53 wherein the computing device is a thin client device.

64. A computer system for receiving themes representing contextual situations and customizing the received themes for use in providing appropriate responses to the represented contextual situations, comprising:

a memory;

means for receiving at least one theme from an external source that distributes the at least one theme to multiple computing devices having distinct users, each of the received themes representing a contextual situation and associated with a type of response that is appropriate to the represented contextual situation;

means for customizing at least one of the received themes in a manner specific to the computing device; and

means for determining that one of the received themes represents a current contextual situation and providing the type of response that is associated with the one theme.

65. The computer system of claim 64 wherein the provided type of response includes presenting information to a user of the computer system.

66. A computer-implemented method for defining a default theme and customizing that theme for use by a portable computing device in presenting information that is appropriate to a contextual situation represented by the theme, the method comprising:

defining a default uncustomized theme representing a contextual situation by,

receiving a specification of the represented contextual situation;

receiving one or more indications of information that is appropriate to the represented contextual situation; and

defining a theme based on the received indications that can be used by any of multiple computing devices to present the appropriate information when a current contextual situation matches the represented contextual situation; and

customizing the defined default theme based on the portable computing device and based on a user of the portable computing device so that the portable computing device can use the customized theme to present the appropriate information to the user in a manner specific to the portable computing device and the user.

67. The method of claim 66 including distributing the customized theme to the portable computing device.

68. The method of claim 66 including customizing the defined default theme for each of multiple other computing devices having distinct users such that each of the customized themes are distinct.

69. The method of claim 66 wherein the received specification of the represented contextual situation includes one or more indications of multiple context attributes that are related to the represented contextual situation and of criteria for determining values of the indicated context attributes that match the represented contextual situation, and wherein the defined theme is determined to match a contextual situation when context information for that contextual situation includes values for the indicated context attributes that are determined by the indicated criteria to match the represented contextual situation.

70. The method of claim 66 wherein the customizing of the defined theme is based on explicit indications received from a user of the portable computing device.

71. The method of claim 66 wherein the customizing of the defined theme is based on likely preferences of a user of the portable computing device that are determined without intervention by the user.

72. The method of claim 71 including, after the determining of the likely preferences and before the customizing of the received theme, receiving explicit user approval of the customizing.

73. The method of claim 66 wherein the customizing of the defined theme includes specializing the defined theme to the user of the portable computing device.

74. The method of claim 66 wherein the customizing of the defined theme includes specializing the defined theme to the portable computing device.

75. The method of claim 66 wherein the customizing of the defined theme is to modify the appropriate information for the defined theme.

76. The method of claim 66 wherein the customizing of the defined theme is to modify the contextual situation represented by the defined theme.

77. The method of claim 66 including:

receiving an indication of a current contextual situation;

determining that a copy of the defined theme that has not been customized matches the current contextual situation; and

in response, presenting the indicated appropriate information for the defined theme.

78. The method of claim 66 wherein the defining of the theme includes associating a priority with the defined theme, and wherein the customizing of the theme includes modifying the priority.

79. The method of claim 66 wherein the defining of the theme includes associating permission information with the defined theme that specifies types of access to the defined theme that are allowed, and wherein the customizing of the theme includes modifying the permission information.

80. The method of claim 66 wherein the defining of the theme includes associating security information with the defined theme, and wherein the customizing of the theme includes modifying the security information.

81. The method of claim 66 wherein the defining of the theme includes specifying a relationship of the defined theme to other themes, and wherein the customizing of the theme includes modifying the specified relationship.

82. The method of claim 81 wherein the specified relationship is a category of themes that is associated with the defined theme.

83. The method of claim 66 wherein the defining of the theme includes associating a theme layout with the defined theme that includes one or more indications of the indicated appropriate information for the defined theme, and wherein the customizing of the theme includes modifying the theme layout.

84. The method of claim 66 wherein the defining of the theme includes generating a new context server module capable of generating context information for the represented contextual situation or of generating a new context client module capable of producing responses appropriate to the represented contextual situation, and wherein the customizing of the theme includes modifying the new context server module or new context client module.

85. The method of claim 66 wherein the method is performed by the portable computing device.

86. The method of claim 85 wherein the customizing of the theme is performed automatically without intervention by the user of the portable computing device.

87. The method of claim 66 wherein the method is performed by a computing system, and wherein the customizing of the theme is performed automatically without intervention by a user of the computing system.

88. The method of claim 66 wherein the portable computing device is a wearable computer.

89. The method of claim 66 wherein the portable computing device is a cellular phone.

90. The method of claim 66 wherein the portable computing device is a mobile phone.

91. The method of claim 66 wherein the portable computing device is a handheld device mounted in a holster on a user of the handheld device.

92. The method of claim 66 wherein the portable computing device is a thin client device.

93. A computer-readable storage medium containing instructions that when executed cause a computing device to define a default theme and customize that theme for use in providing a response that is appropriate to a contextual situation represented by the theme, by performing a method comprising:

defining a default theme representing a contextual situation by,

receiving a specification of the represented contextual situation;

receiving one or more indications of a type of response that is appropriate to the represented contextual situation; and

defining a theme based on the received indications that can be used by any of multiple computing devices to provide the appropriate type of response when a current contextual situation matches the represented contextual situation; and

customizing the defined default theme in a specified manner so that the customized theme can be used to provide an appropriate type of response in a manner distinct from that of the default theme.

94. The computer-readable medium of claim 93 wherein the customizing of the default theme is performed in a manner specific to an indicated computing device and/or to a user of the indicated computing device.

95. The computer-readable medium of claim 94 wherein the method further comprises distributing the customized theme to the indicated computing device.

96. The computer-readable medium of claim 93 wherein the received specification of the represented contextual situation includes one or more indications of multiple context attributes that are related to the represented contextual situation and of criteria for determining values of the indicated context attributes that match the represented contextual situation, and wherein the defined theme is determined to match a contextual situation when context information for that contextual situation includes values for the indicated context attributes that are determined by the indicated criteria to match the represented contextual situation.

97. The computer-readable medium of claim 93 wherein the customizing of the theme is performed automatically without intervention by any users.

98. The computer-readable medium of claim 93 wherein the method further comprises using the customized theme to provide the appropriate type of response when the represented contextual situation occurs.

99. The computer-readable medium of claim 98 wherein the providing of the appropriate type of response includes presenting information to a user of the computing device.

100. The computer-readable medium of claim 93 wherein the method further comprises using the customized theme to provide the appropriate type of response to a wearable computer.

101. The computer-readable medium of claim 93 wherein the method further comprises using the customized theme to provide the appropriate type of response to a cellular phone.

102. The computer-readable medium of claim 93 wherein the method further comprises using the customized theme to provide the appropriate type of response to a mobile phone.

103. The computer-readable medium of claim 93 wherein the method further comprises using the customized theme to provide the appropriate type of response to a handheld device.

104. The computer-readable medium of claim 93 wherein the method further comprises using the customized theme to provide the appropriate type of response to a thin client device.

105. A computing device for defining a default theme and customizing that theme for use by a specified computing device in presenting information that is appropriate to a contextual situation represented by the theme, comprising:

one or more processors;

a theme definition component that when executed by at least one or more processors is configured to define a default theme representing a contextual situation by receiving a specification of the represented contextual situation, receiving one or more indications of information that is appropriate to the represented contextual situation, and defining a theme based on the received indications that can be used by any of multiple computing devices to present the appropriate information when a current contextual situation matches the represented contextual situation; and

a theme customization component that when executed by at least one or more processors is configured to customize the defined default theme to reflect a specified computing device so that the specified computing device can use the customized theme to present the appropriate information in a manner specific to the specified computing device.

106. The computing device of claim 105 wherein the customizing of the default theme to reflect the specified computing device is performed in a manner based on a user of the specified computing device.

107. The computing device of claim 105 further comprising a component capable of distributing the customized theme to the specified computing device.

108. The computing device of claim 105 wherein the received specification of the represented contextual situation includes one or more indications of multiple context attributes that are related to the represented contextual situation and of criteria for determining values of the indicated context attributes that match the represented contextual situation, and wherein the defined theme is determined to match a contextual situation when context information for that contextual situation includes values for the indicated context attributes that are determined by the indicated criteria to match the represented contextual situation.

109. The computing device of claim 105 wherein the customizing of the theme is performed automatically without intervention by any users.

110. The computing device of claim 105 wherein the computing device is the specified computing device, and further comprising a component capable of using the customized theme to provide the appropriate type of response when the represented contextual situation occurs.

111. The computing device of claim 105 wherein the specified computing device is a wearable computer.

112. The computing device of claim 105 wherein the specified computing device is a cellular phone.

113. The computing device of claim 105 wherein the specified computing device is a mobile phone.

114. The computing device of claim 105 wherein the specified computing device is a handheld device.

115. The computing device of claim 105 wherein the specified computing device is a thin client device.