VIRTUAL ASSISTANT SYSTEM TO ENABLE ACTIONABLE MESSAGING

• US 20170264715A1
• Filed: 05/24/2017
• Published: 09/14/2017
• Est. Priority Date: 12/11/2014
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Wand Labs, Inc. (Microsoft Corporation)
• Inventors: Sharma, Vishal, Amjadi, Elhum
• Granted Patent: US 10,585,685 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 3/04817 using icons graphical or visual programming using iconic symbols G06F8/34
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 3/165 Management of the audio stream, e.g. setting of volume, audio stream path
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06F 40/30 Semantic analysis
• CPC Class Codes: G06F 9/453 Help systems
• CPC Class Codes: H04L 51/00 User-to-user messaging in packet-switching networks, transmitted according to store-and-forward or real-time protocols, e.g. e-mail
• CPC Class Codes: H04L 51/02 using automatic reactions or user delegation, e.g. automatic replies or chatbot-generated messages
• CPC Class Codes: H04L 51/046 Interoperability with other network applications or services
• CPC Class Codes: H04L 51/18 Commands or executable codes
• CPC Class Codes: H04L 63/0884 by delegation of authentication, e.g. a proxy authenticates an entity to be authenticated on behalf of this entity vis-à-vis an authentication entity
• CPC Class Codes: H04L 67/01 Protocols
• CPC Class Codes: H04L 67/1001 for accessing one among a plurality of replicated servers
• CPC Class Codes: H04L 67/12 specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
• CPC Class Codes: H04L 67/131 Protocols for games, networked simulations or virtual reality
• CPC Class Codes: H04M 1/72409 by interfacing with external accessories hands-free H04M1/60
• CPC Class Codes: H04M 1/724094 Interfacing with a device worn on the user's body to provide access to telephonic functionalities, e.g. accepting a call, reading or composing a message
• CPC Class Codes: H04M 1/724097 Worn on the head
• CPC Class Codes: H04M 1/72412 using two-way short-range wireless interfaces
• CPC Class Codes: H04M 1/72415 for remote control of appliances
• CPC Class Codes: H04W 12/06 : Authentication
• CPC Class Codes: H04W 12/43 : using shared identity modules, e.g. SIM sharing
• CPC Class Codes: H04W 4/00 : Services specially adapted for wireless communication networks; Facilities therefor
• CPC Class Codes: H04W 88/02 : Terminal devices

Abstract

A virtual assistant system includes a mobile device to receive an input command corresponding to a function to be performed at one or more external services, to translate the input command into a semantic atom representing the command, and to transmit the semantic atom, and an assistant server configured to receive the transmitted semantic atom, the assistant server including a plurality of plugins, each plugin corresponding to a respective one of the external services and configured to generate a proprietary language command corresponding to the received semantic atom for each external service in which the function is to be performed.

Claims

1. A mobile device for use in presenting a virtual environment to a user, the mobile device comprising:

a sensor configured to receive an input command from the user, the input command corresponding to a function to be performed at one or more external services;

an assistant interface configured to translate the input command into a semantic atom representing the input command; and

a communicator configured to transmit the semantic atom to at least one other user of the virtual environment.

2. The mobile device of claim 1, the virtual environment being presented as part of a virtual reality (VR) session amongst a plurality of participants.

3. The mobile device of claim 1, the virtual environment being presented as part of an augmented reality (AR) session amongst a plurality of participants.

4. The mobile device of claim 3, further configured to receive a semantic atom representing output data from the AR session, and to present the output data to the user of the mobile device.

5. The mobile device of claim 1, wherein the sensor comprises a stationary sensor.

6. The mobile device of claim 1, wherein the sensor comprises a wearable sensor worn by the user.

7. The mobile device of claim 3, wherein at least one of the plurality of participants includes at least one of a plugin, another mobile device, an assistant server, and another user.

8. The mobile device of claim 3, wherein a participant in the AR session transmits one or more semantic atoms to one or more other participants in the AR session.

9. The mobile device of claim 7, wherein a first plugin participating in the AR session transmits a semantic atom representing at least one of settings, user preferences, user status, user credentials, and command inputs to a second plugin participating in the AR session.

10. The mobile device of claim 9, wherein the first plugin transmits the semantic atom representing at least one of settings, user preferences, user status, user credentials, and command inputs to the second plugin automatically when the user inputs a command to the second plugin.

11. The mobile device of claim 1, further configured to analyze context information of the input command before the input command is converted to a proprietary language command to accurately determine the intended function to be performed by the input command.

12. The mobile device of claim 11, wherein the analyzed context information includes at least one of which plugin is receiving the input command, a status of a participant in an AR session, a time of day when the command is input, co-presence of specific individuals, involuntary actions by a user, and action sequences taken by at least one user.

13. The mobile device of claim 12, wherein the action sequences taken by the at least one user includes a plurality of input commands made simultaneously or sequentially.

14. The mobile device of claim 12, wherein the action sequences taken by the at least one user includes an input command that is invalidated and followed by another input command.

15. The mobile device of claim 12, wherein the action sequences taken by the at least one user includes an input command that is made by a first user and another input command that is made by a second user.

16. The mobile device of claim 12, further configured to update a stored library of recognizable input commands according to the analyzed context information.

17. The mobile device of claim 12, further configured to update a stored library of recognizable input commands corresponding to a first AR external service according to analyzed context information corresponding to a second AR external service.

18. The mobile device of claim 13, wherein when the mobile device recognizes a repeated cluster of input commands, the mobile device creates a new consolidated input command to represent performing the cluster of input commands simultaneously and stores the consolidated input command in a library of recognizable input commands.

22. The mobile device of claim 3, further configured to store credentials including data to authorize one or more participants of the AR session to issue commands for the one or more AR external services.

23. The mobile device of claim 22, wherein:

at least one participant of the AR session controls an assistant server to extend authority to input commands to the one or more AR external services to at least one other participant of the AR session.

24. The mobile device of claim 23, wherein the at least one participant of the AR session sets a level of authority the at least one other participant of the AR session has to input commands to the one or more AR external services.

25. The mobile device of claim 22, further configured to automatically authorize one or more participants of the AR session to control the one or more AR external services through input commands according to the stored credentials, the authorization being based on context information.

26. The mobile device of claim 22, wherein a plurality of the participants of the AR session controls the assistant server to extend authority to input commands to the corresponding AR external service to each other.

27. The mobile device of claim 22, wherein:

the stored credentials include at least one payment credential; and

at least one participant of the AR sessions controls the assistant server to extend authority to use the at least one payment credential to at least one other participant of the AR session.

28. A method for presenting a virtual environment to a user, the method comprising:

receiving an input command from the user through a sensor, the input command corresponding to a function to be performed at one or more external services;

translating the input command into a semantic atom representing the input command; and

transmitting the semantic atom to at least one other user of the virtual environment.

29. The method of claim 28, the virtual environment being presented as part of a virtual reality (VR) session amongst a plurality of participants.

30. The mobile device of claim 28, the virtual environment being presented as part of an augmented reality (AR) session amongst a plurality of participants.