System and method for multimodal short-cuts to digital services

• US 9,374,451 B2
• Filed: 02/04/2002
• Issued: 06/21/2016
• Est. Priority Date: 02/04/2002
• Status: Active Grant

• Current Assignee: Nokia Technologies Oy (Nokia Corporation)
• Original Assignee: Nokia Technologies Oy (Nokia Corporation)
• Inventors: Salmenkaita, Jukka-Pekka, Sorvari, Antti
• Primary Examiner(s): Augustine, Nicholas
• Application Number:
  US10/061,277
• Publication Number:
  US 20040176958A1
• Time in Patent Office: 5,251 Days
• Field of Search: 455/563, 455/550.1, 455/550, 715/810, 715/845, 715/863
• US Class Current: 1/1
• CPC Class Codes: H04L 67/306 User profiles
• CPC Class Codes: H04M 1/6041 Portable telephones adapted for handsfree use
• CPC Class Codes: H04M 1/72403 with means for local support of applications that increase the functionality
• CPC Class Codes: H04M 1/72445 for supporting Internet browser applications
• CPC Class Codes: H04M 1/72454 according to context-related or environment-related conditions
• CPC Class Codes: H04M 1/72457 according to geographic location
• CPC Class Codes: H04M 2201/40 using speech recognition speech recognition per se G10L15/00
• CPC Class Codes: H04M 2203/251 where a voice mode or a visual mode can be used interchangeably
• CPC Class Codes: H04M 2203/253 where a visual mode is used instead of a voice mode
• CPC Class Codes: H04M 2250/12 including a sensor for measuring a physical value, e.g. temperature or motion telephones including GPS signal receivers H04M2250/10
• CPC Class Codes: H04M 2250/74 with voice recognition means voice activated dialling H04M1/271; voice recognition algorithms G10L15/00
• CPC Class Codes: H04M 3/4938 comprising a voice browser which renders and interprets, e.g. VoiceXML
• CPC Class Codes: H04W 8/183 Processing at user equipment or user record carrier

Abstract

A method and system for facilitating user access to services through a wireless device of a user, involves recommending to a user a subset of services from a plurality of services available to the user in which each recommended service of the subset has at least one voice short-cut associated therewith, and selecting a service to be accessed through the user'"'"'s wireless device from the subset of services according to a voice command by the user corresponding to the voice short-cut of the service.

Claims

1. A method comprising:

storing on a user'"'"'s wireless device information identifying a plurality of services;

determining a current context based on current environment of the wireless device sensed by one or more sensors and at least one of a selected profile of the wireless device or user profile from an online server or web site, wherein the current context is represented as a metadata vector, and wherein the wireless device includes at least one privacy setting;

recommending to the user through the wireless device a subset of services, the subset dynamically selected from the plurality of services in the wireless device in response to the current environment sensed by the one or more sensors and the profile of the wireless device or user, each recommended service of the subset having at least one voice short-cut associated therewith, each voice short-cut in the form of a voice tag based, at least in part, on metadata associated with the recommended service or a service provider of the recommended service; and

selecting a service to be accessed through the wireless device from the subset of services according to a voice command by the user corresponding to the at least one voice short-cut of the service,

wherein the recommending comprises determining a dynamic subset of services of interest to the user from the identified plurality of services based on the determined current context.

2. The method according to claim 1, wherein the plurality of services comprises services previously accessed by the user.

3. The method according to claim 1, further comprising displaying visual information corresponding to the subset of recommended services to the user.

4. The method according to claim 3, wherein the visual information includes identification of a voice short-cut for each service of the subset of services.

5. The method according to claim 4, wherein the visual information includes an icon to notify the user of the availability of voice access of the recommended services.

6. The method according to claim 1, further comprising outputting audio information corresponding to the subset of recommended services to the user.

7. The method according to claim 6, wherein the audio information includes identification of a voice short-cut for each service of the subset of recommended services.

8. The method according to claim 1, wherein the recommending determines the subset of recommended services based also on the user-related filter criteria.

9. The method according to claim 1, wherein the current context is determined based also on one of a temporal variable or a position variable.

10. The method according to claim 1, further comprising determining the current context based also on a current sensed environment of the wireless device.

11. The method according to claim 1, wherein the determining of the current context includes parsing a unique operational code, and the recommending determines the subset of recommended services based also on a service usage history or preference of the user.

12. The method according to claim 1, wherein the recommending determines the subset of recommended services based also on the current environment of the wireless device sensed by one or more sensors.

13. The method according to claim 12, wherein the sensor is a positioning sensor, a compass, a touch sensor, an audio sensor, a light sensor, or a temperature sensor.

14. The method according to claim 1, further comprising receiving voice short-cuts for a service in an XML format.

15. The method according to claim 1, further comprising assigning temporary voice short-cuts to the subset of recommended services.

16. The method according to claim 1, further comprising generating a voice short-cut for a service from the plurality of services based on metadata associated with the service.

17. The method according to claim 1, further comprising enabling the user to predefine a voice short-cut for a service from the plurality of services.

18. The method according to claim 1, further comprising enabling the user to add or modify voice short-cuts for any one of the plurality of services.

19. The method according to claim 1, wherein a voice short-cut for a service of the plurality of services is predefined by the service provider.

20. The method according to claim 1, wherein the voice short-cut for a service is received from a remote location, across an ad hoc wireless network including the wireless device.

21. The method according to claim 1, wherein the voice short-cut for a service is received from a voice tag registry, across a wireless network.

22. The method according to claim 1, wherein the selecting comprises performing voice recognition processing to identify the service with a voice short-cut matching the voice command.

23. The method according to claim 22, wherein voice recognition processing is performed by comparing the voice command only to voice short-cuts of the recommended services.

24. The method according to claim 22, wherein the voice recognition processing is performed to identify the service with a voice short-cut matching the voice command according to a priority of the recommended services.

25. The method according to claim 1, wherein the recommending of a subset of services is performed by the wireless device.

26. The method according to claim 1, wherein the selecting of a service is performed by the wireless device.

27. The method according to claim 1, wherein the selecting of a service is performed at a remote server and information identifying the selected service from the subset of recommended services is provided by the remote server to the wireless device.

28. The method according to claim 1, further comprising filtering down the subset of recommended services for selection according to past recommendations to the user.

29. The method according to claim 1, further comprising filtering down the subset of recommended services for selection according to user-defined filter categories.

30. The method according to claim 1, further comprising accessing the selected service using the wireless device across a wireless network.

31. The method according to claim 30, further comprising enabling multi modal input, including at least voice input, to browse the accessed service.

32. The method according to claim 30, further comprising enabling multi-modal input selection, including at least by voice, of a service from the subset of recommended services.

33. The method according to claim 1, wherein the voice command matches a plurality of the recommended services, further comprising outputting an indication of the matched services for selection by the user.

34. The method according to claim 33, wherein the indication of the matched services comprises voice short-cuts corresponding thereto.

35. The method according to claim 1, further comprising storing one or more databases of information corresponding to services and associated voice short-cuts.

36. The method according to claim 1, further comprising interacting with at least one of a user input device and a user output device across a wireless short-range link.

37. The method according to claim 1, wherein the voice short-cuts of the recommended services change hierarchically according to a level of service.

107. The method according to claim 1, wherein at least one voice short-cut is defined by an entity other than the user, the method further comprising:

storing a voice short-cut received from the service provider on the user device.

110. The method according to claim 1, wherein the recommending recommends a dynamic subset of services.

111. The method according to claim 1, further comprising:

providing the user with a selection of different categories of characteristic from which a dynamic subset of services is able to be determined; and

receiving user selection of a particular category of characteristic, wherein the determination of the dynamic subset of services is based also on information corresponding to the category of characteristic.

112. The method according to claim 1, wherein the current context and the subset of recommended services are determined locally on the wireless device.

113. The method according to claim 1, wherein the determination to store on a user'"'"'s wireless device stores a service history log which includes at least the information identifying a plurality of services previously used by a user through the wireless device, and the determined subset of services is selected from the identified plurality of services on the service history log, the identified plurality of services being services accessible through the Internet.

114. The method according to claim 1, further comprising generating a voice short-cut for a service from the plurality of services based on address attributes of the service.

115. The method according to claim 1, further comprising receiving a request for voice tags for the subset of recommended services prior to a processing of the voice command by the user.

38. An apparatus comprising:

at least one processor; and

at least one memory including computer program code,

the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus to perform at least the following,

determine a current context based on current environment sensed by one or more sensors and at least one of a selected profile of the apparatus or user profile from an online server or web site, wherein the current context is represented as a metadata vector, and wherein the apparatus includes at least one privacy setting,

determine to recommend to a user a subset of services from a plurality of services, the subset dynamically selected in response to the current environment sensed by the one or more sensors and the profile of the apparatus or user, each recommended service of the subset having at least one voice short-cut associated therewith, each voice short-cut in the form of a voice tag based, at least in part, on metadata associated with the recommended service or a service provider of the recommended service, and

select a service to be accessed through the apparatus from the subset of services according to a user'"'"'s voice command corresponding to the voice short-cut of the service,

wherein the recommending comprises determining a dynamic subset of services of interest to the user from a plurality of services previously used by the user through the user'"'"'s device based on the determined current context.

39. The apparatus according to claim 38, wherein the plurality of services comprises services previously accessed by the user.

40. The apparatus according to claim 38, wherein visual information corresponding to the subset of recommended services is determined to be visually displayed via an output device to the user.

41. The apparatus according to claim 40, wherein the visual information includes identification of a voice short-cut for each service of the subset of services.

42. The apparatus according to claim 41, wherein the visual information includes an icon to notify the user of the availability of voice access of the recommended services.

43. The apparatus according to claim 38, wherein audio information corresponding to the subset of recommended services is determined to be audibly outputted via an output device to the user.

44. The apparatus according to claim 43, wherein the audio information includes identification of a voice short-cut for each service of the subset of recommended services.

45. The apparatus according to claim 38, wherein the subset of recommended services is determined based also on the user-related filter criteria.

46. The apparatus according to claim 38, wherein the current context is determined based also on at least one of a temporal variable or a position variable.

47. The apparatus according to claim 38, wherein the processor further determines the current context based also on a current sensed environment.

48. The apparatus according to claim 38, wherein the subset of recommended services is determined based also on a service usage history or preference of the user.

49. The apparatus according to claim 38, comprising one or more sensors, wherein the apparatus is further caused to determine to sense an environment of the apparatus, wherein the subset of recommended services is determined based also on a current environment of the apparatus sensed by the one or more sensors.

50. The apparatus according to claim 49, wherein the sensor is a positioning sensor, a compass, a touch sensor, an audio sensor, a light sensor, or a temperature sensor.

51. The apparatus according to claim 38, wherein the voice short-cuts for a service is received in an XML format via a wireless interface.

52. The apparatus according to claim 38, wherein the apparatus is further caused to determine to assign temporary voice short-cuts to the subset of recommended services.

53. The apparatus according to claim 38, wherein the apparatus is further caused to determine to generate a voice short-cut for a service from the plurality of services based on metadata associated with the service.

54. The apparatus according to claim 38, wherein the apparatus is further caused to determine to enable the user to predefine a voice short-cut for a service from the plurality of services.

55. The apparatus according to claim 38, wherein the apparatus is further caused to determine to enable the user to add or modify voice short-cuts for any one of the plurality of services.

56. The apparatus according to claim 38, wherein a voice short-cut for a service of the plurality of services is predefined by the service provider.

57. The apparatus according to claim 38, wherein the voice short-cut for a service is received from the remote location, across an ad hoc wireless network.

58. The apparatus according to claim 38, wherein the voice short-cut for a service is received from a voice tag registry, across a wireless network.

59. The apparatus according to claim 38, wherein the apparatus is further caused to determine to perform voice recognition processing to identify the service with a voice short-cut matching the voice command.

60. The apparatus according to claim 59, wherein the apparatus is further caused to determine to perform voice recognition processing by comparing the voice command only to the voice short-cuts of the recommended services.

61. The apparatus according to claim 59, wherein the voice recognition processing is performed to identify the service with a voice short-cut matching the voice command according to a priority of the recommended services.

62. The apparatus according to claim 38, wherein the apparatus is further caused to determine to filter down the subset of recommended services for selection according to past recommendations to the user.

63. The apparatus according to claim 38, wherein the apparatus is further caused to determine to filter down the subset of recommended services for selection according to user-defined filter categories.

64. The apparatus according to claim 38, wherein the selected service is accessed across a wireless network, via a wireless interface.

65. The apparatus according to claim 64, further comprising a plurality of input devices, including a voice input device, wherein the apparatus is further caused to determine to enable multi-modal input to browse the accessed service.

66. The apparatus according to claim 38, further comprising a plurality of input devices, including a voice input device, wherein the apparatus is further caused to determine to enable multi-modal input selection, via the input devices, of a service to be accessed from the subset of recommended services.

67. The apparatus according to claim 38, wherein the voice command matches a plurality of the recommended services, the apparatus is further caused to determine to output an indication of the matched services for selection by the user.

68. The apparatus according to claim 67, wherein the indication of the matched services comprises voice short-cuts corresponding thereto.

69. The apparatus according to claim 38, wherein the apparatus is further caused to determine to store one or more databases of information corresponding to services and associated voice short-cuts.

70. The apparatus according to claim 38, wherein at least one of the voice input devices and an output device interact with the at least one processor or other components of the wireless device across a wireless short-range link.

71. The apparatus according to claim 38, wherein the voice short-cuts of the recommended services change hierarchically according to a level of service.

72. A system comprising:

an apparatus, the apparatus comprising at least one processor and at least one memory including computer program code,

the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus to perform at least the following,

determine to store information identifying a plurality of services, determine to enable communication between the apparatus and a server,

determine to cooperate with the server to determine a current context based on current environment of the apparatus sensed by one or more sensors and at least one of a selected profile of the apparatus or user profile from an online server or web site, wherein the current context is represented as a metadata vector, and wherein the wireless device includes at least one privacy setting, to recommend a subset of services from the plurality of identified services, the subset dynamically selected in the apparatus in response to the current environment sensed by the one or more sensors and the profile of the apparatus or user, each recommended service of the subset having at least one voice short-cut associated therewith, each voice short-cut in the form of a voice tag based, at least in part, on metadata associated with the recommended service or a service provider of the recommended service, and to select a service to be accessed through the apparatus from the subset of services according to the user'"'"'s voice command corresponding to the voice short-cut of the service,

wherein the recommending of a subset of services comprises determining a dynamic subset of services of interest to the user from a plurality of services available to the user through the apparatus based on the determined current context.

73. The system according to claim 72, wherein the apparatus is further caused to determine to display visual information corresponding to the subset of recommended services.

74. The system according to claim 73, wherein the visual information includes identification of a voice short-cut for each service of the subset of recommended services.

75. The system according to claim 74, wherein the visual information includes an icon to notify the user of the availability of voice access of the recommended services.

76. The system according to claim 72, wherein the apparatus is further caused to determine to output audio information corresponding to the subset of recommended services.

77. The system according to claim 76, wherein the audio information includes identification of a voice short-cut for each service of the subset of recommended services.

78. The system according to claim 72, wherein the subset of recommended services is determined based also on the user-related filter criteria.

79. The system according to claim 72, wherein the current context is determined based also on at least one of a temporal variable, a position variable and a selected profile of the device.

80. The system according to claim 72, wherein the apparatus is further caused to determine the current context based also on a current sensed environment of the wireless device.

81. The system according to claim 72, wherein the subset of recommended services is determined based also on a service usage history or preference of the user.

82. The system according to claim 72, wherein the wireless device includes one or more sensors, wherein the apparatus is further caused to determine to sense a surrounding environment, the subset of recommended services being determined based also on a current environment of the wireless device sensed by the one or more sensors.

83. The system according to claim 82, wherein the sensor 1s a positioning sensor, a compass, a touch sensor, an audio sensor, a light sensor, or a temperature sensor.

84. The system according to claim 72, wherein the voice short-cut for a service is received in an XML format via a wireless interface.

85. The system according to claim 72, wherein the apparatus is further caused to determine to assign temporary voice short-cuts to the subset of recommended services.

86. The system according to claim 72, wherein the apparatus is further caused to determine to generate a voice short-cut for a service from the plurality of services based on metadata associated with the service.

87. The system according to claim 72, wherein the apparatus is further caused to determine to enable the user to predefine a voice short-cut for a service from the plurality of services.

88. The system according to claim 72, wherein the apparatus is further caused to determine to enable the user to add or modify voice short-cuts for any one of the plurality of services.

89. The system according to claim 72, wherein a voice short-cut for a service of the plurality of services is predefined by the service provider.

90. The system according to claim 72, wherein the apparatus is further caused to determine to receive the voice short-cut for a service from a remote location, across an ad hoc wireless network.

91. The system according to claim 72, wherein the apparatus is further caused to determine to receive the voice short-cut for a service from a voice tag registry, via a wireless network infrastructure.

92. The system according to claim 72, wherein the apparatus is further caused to determine to perform voice recognition processing to identify the service with a voice short-cut matching the voice command.

93. The system according to claim 92, wherein apparatus is further caused to determine to perform voice recognition processing by comparing the voice command only to the voice short-cuts of the recommended services.

94. The system according to claim 92, wherein the voice recognition processing is performed to identify the service with a voice short-cut matching the voice command according to a priority of the recommended services.

95. The system according to claim 72, wherein the apparatus is further caused to determine to filter down the subset of recommended services for selection according to past recommendations to the user.

96. The system according to claim 72, wherein the apparatus is further caused to determine to filter down the subset of recommended services for selection according to user-defined filter categories.

97. The system according to claim 72, wherein the apparatus is further caused to determine to access the selected service across a wireless network infrastructure.

98. The system according to claim 97, wherein the apparatus further comprises a plurality of input devices, including a voice input device, and wherein the apparatus is further caused to enable multi-modal input by the user to browse the accessed service.

99. The system according to claim 72, wherein the apparatus further comprises a plurality of input devices, including a voice input device, and wherein the apparatus is further caused to enable multi-modal input selection, via the input devices, of a service to be accessed from the subset of recommended services.

100. The system according to claim 72, wherein the voice command matches a plurality of the recommended services, and wherein the apparatus is further caused to determine to output an indication of the matched services for selection by the user.

101. The system according to claim 100, wherein the indication of the matched services comprises voice short-cuts corresponding thereto.

102. The system according to claim 72, wherein the apparatus is further caused to determine to store one or more databases of information corresponding to services and associated voice short-cuts.

103. The system according to claim 72, wherein the wireless device includes at least one of the voice input device and the output device which are communicatively coupled thereto across a wireless short-range link.

104. The system according to claim 72, wherein the voice short-cuts of the recommended services change hierarchically according to a level of service.

105. A system comprising:

at least one processor;

at least one memory including computer program code for storing on an apparatus operated by a user information identifying a plurality of services;

the at least one memory and the computer program code configured to, with the at least one processor, cause the following,

determining a current environment of the apparatus;

sensing a current context based on at least one of a selected profile of the apparatus or user profile from an online server or web site, wherein the current context is represented as a metadata vector, and wherein the apparatus includes at least one privacy setting;

recommending to the user a subset of services from the identified plurality of services, the subset dynamically selected in the apparatus in response to the sensed current environment and the profile of the apparatus or user, each recommended service of the subset having at least one voice short-cut associated therewith, each voice short-cut in the form of a voice tag based, at least in part, on metadata associated with the recommended service or a service provider of the recommended service;

receiving a voice command from the user and comparing the user'"'"'s voice command to the voice short-cuts of the recommended service; and

accessing a service, from the subset of recommended services, with a voice short-cut matching the voice command,

wherein the recommending comprises determining a dynamic subset of services of interest to the user from a plurality of services available to the user through the apparatus based on the current context.

106. A computer-readable storage medium carrying one or more sequences of one or more instructions which, when executed by one or more processors, cause an apparatus to at least perform the following steps:

storing on a wireless device of a user information identifying a plurality of services determining a current context based on current environment of the wireless device sensed by one or more sensors and at least one of a selected profile of the wireless device or user profile from an online server or web site, wherein the current context is represented as a metadata vector, and wherein the wireless device includes at least one privacy setting;

recommending to a user a subset of services from the identified plurality of services, the subset dynamically selected in the wireless device in response to the current environment sensed by the one or more sensors and the profile of the wireless device or user, each recommended service of the subset having at least one voice short-cut associated therewith, each voice short-cut in the form of a voice tag based, at least in part, on metadata associated with the recommended service or a service provider of the recommended service; and

selecting a service to be accessed through the wireless device from the subset of services according to a voice command by the user corresponding to the voice short-cut of the service,

wherein the recommending comprises determining a dynamic subset of services of interest to the user from a plurality of services available to the user through the wireless device based on the determined current context.

108. A method comprising:

storing on a wireless device of a user information identifying a plurality of services;

determining a current context based on current environment of the wireless device sensed by one or more sensors and at least one of a selected profile of the wireless device or user profile from an online server or web site, wherein the current context 1s represented as a metadata vector, and wherein the wireless device includes at least one privacy setting;

recommending a subset of selectable services of interest to the user from the identified plurality of services, the subset dynamically selected in the wireless device in response to the current environment sensed by the one or more sensors and the profile of the wireless device or user, each selectable service of the subset having at least one voice short-cut associated therewith and being transparent to the user, each voice short-cut in the form of a voice tag based, at least in part, on metadata associated with the recommended service or a service provider of the recommended service; and

selecting a service to be accessed through the wireless device from the subset of services according to a voice command by the user corresponding to the voice short-cut of the service,

wherein the recommending comprises determining a dynamic subset of services of interest to the user from a plurality of services available to the user through the wireless device based on the determined current context.

109. An apparatus comprising:

at least one processor; and

at least one memory including computer program code,

the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus to perform at least the following,

determining a current context based on current environment of the apparatus sensed by one or more sensors and at least one of a selected profile of the apparatus or user profile from an online server or web site, wherein the current context is represented as a metadata vector, and wherein the wireless device includes at least one privacy setting;

recommending a subset of selectable services of interest to the user from the identified plurality of services, the subset dynamically selected in the apparatus in response to the current environment sensed by the one or more sensors and the profile of the apparatus or user, each selectable service of the subset having at least one voice short-cut associated therewith and being transparent to the user, each voice short-cut in the form of a voice tag based, at least in part, on metadata associated with the recommended service or a service provider of the recommended service; and

selecting a service to be accessed through the apparatus from the subset of services according to a voice command by the user corresponding to the voice short-cut of the service,

wherein the recommending operation comprises determining a dynamic subset of services of interest to the user from a plurality of services previously used by the user through the apparatus based on the determined current context.