Voice interface for virtual area interaction

• US 10,649,724 B2
• Filed: 01/15/2016
• Issued: 05/12/2020
• Est. Priority Date: 01/15/2009
• Status: Active Grant

• Current Assignee: Sococo, Inc.
• Original Assignee: Sococo, Inc.
• Inventors: Van Wie, David
• Primary Examiner(s): Bashore, William L
• Assistant Examiner(s): Parcher, Daniel W
• Application Number:
  US14/997,301
• Publication Number:
  US 20160139878A1
• Time in Patent Office: 1,579 Days
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06Q 50/01 Social networking
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: G10L 2015/225 Feedback of the input speech
• CPC Class Codes: H04L 12/1827 Network arrangements for conference optimisation or adaptation
• CPC Class Codes: H04L 51/04 Real-time or near real-time messaging, e.g. instant messaging [IM]
• CPC Class Codes: H04L 51/043 using or handling presence information
• CPC Class Codes: H04L 51/046 Interoperability with other network applications or services
• CPC Class Codes: H04L 51/18 Commands or executable codes

Abstract

Examples of interface systems and methods for voice-based interaction in one or more virtual areas that define respective persistent virtual communication contexts are described. These examples enable communicants to use voice commands to, for example, search for communication opportunities in the different virtual communication contexts, enter specific ones of the virtual communication contexts, and bring other communicants into specific ones of the virtual communication contexts. In this way, these examples allow communicants to exploit the communication opportunities that are available in virtual areas, even when hands-based or visual methods of interfacing with the virtual areas are not available.

Claims

1. A computer-implemented method in a network communications environment implemented by at least one server network node and supporting realtime communications between a user and one or more other communicants who are operating respective client network nodes and are co-present in any of multiple virtual locations of a virtual area that define different respective communication contexts in the network communications environment, the method comprising:

processing voice input from the user;

calling one or more services associated with the network communications environment to perform operations comprising

detecting in the processed voice input a navigation command in a fixed set of navigation commands each of which is associated with a respective syntax, wherein the fixed set of navigation commands comprises a look command and a find command, wherein in response to the user vocalizing the look command with no modifier causes one of the services to send to the user a report of who else is in the user'"'"'s current virtual location of presence in the virtual area and in response to the user vocalizing the look command with a specified virtual location in the virtual area causes one of the services to send to the user a report of who is in the specified virtual location in the virtual area, wherein the user vocalizing the find command with a specified contact operand causes one of the services to send to the user a report of the current virtual location of the specified contact in the virtual area, and the user vocalizing the find command with a specified name of a meeting in the virtual area causes one of the services to send to the user a report of the virtual location of the meeting with the specified name in the virtual area, wherein the detecting comprises probabilistically scoring a set of navigation commands based on application of semantic rules that measure consistency of the navigation commands with the user'"'"'s current state or activity in the virtual area and, based on the scoring, selecting one of the navigation commands as the detected navigation command,

ascertaining one or more operands based on the processed voice input and the syntax associated with the detected navigation command, wherein the ascertaining comprises selecting the one or more operands from one or more of names associated with respective ones of the virtual locations and names associated with respective ones of the communicants based on relationship data indicating one or more of whether the user is a member of any of the respective ones of the virtual locations and whether any of the respective ones of the communicants is a contact of the user, and

producing navigation results comprising navigating an attentional focus or state of the user from a current virtual location to another virtual location based on application of the detected navigation command to the one or more ascertained operands.

2. The method of claim 1, wherein the one or more of names associated with respective ones of the virtual locations and names associated with respective ones of the communicants are configured dynamically by the at least one server network node based on data currently associated with the user.

3. The method of claim 1, wherein the one or more of names associated with respective ones of the virtual locations and names associated with respective ones of the communicants comprise names that are related to the user and names that are unrelated to the user, and the selecting comprises preferentially selecting ones of the names that currently are associated with the user over other ones of the names.

4. The method of claim 1, wherein the detecting and the ascertaining comprises:

forming grammatically correct word combinations of commands selected from the fixed set of navigation commands and operands selected from the one or more names associated with respective ones of the virtual locations and the names associated with respective ones of the communicants based on the relationship data according to the respective navigation command syntaxes; and

preferentially selecting a respective one of the formed grammatically correct word combinations based on information about the user.

5. The method of claim 4, wherein the information about the user comprises names of communicants who are contacts of the user.

6. The method of claim 1, wherein the fixed set of navigation commands comprises a look command;

and

wherein when the look command is detected in the processed voice input;

an unsuccessful ascertainment of a virtual location operand for the look command in the processed voice input results in selecting a current virtual location of presence of the user in the network communications environment as the ascertained operand, and returning in the navigation results a list of communicants who are present in the user'"'"'s current virtual location of presence; and

a successful ascertainment of a virtual location operand for the look command in the processed voice input results in returning in the navigation results a list of communicants who are present in the respective one of the virtual locations in the network communications environment corresponding to the ascertained virtual location operand without the user being present in the respective virtual location.

7. The method of claim 6, wherein each returning comprises returning in the navigation results one or more status attributes of each communicant in the respective list of communicants.

8. The method of claim 1, wherein, when the find command is detected in the processed voice input, a successful ascertainment of a communicant name operand for the find command in the processed voice input results in returning in the navigation results an identification of a respective one of the virtual locations in which a communicant corresponding to the communicant name operand is present.

9. The method of claim 8, wherein the returning comprises returning in the navigation results one or more status attributes of each communicant who is present in the identified virtual location.

10. The method of claim 1, wherein, when the find command is detected in the processed voice input, a successful ascertainment of a meeting name operand for the find command in the processed voice input results in returning in the navigation results an identification of a respective one of the virtual locations in the network communications environment corresponding to the ascertained meeting name operand.

11. The method of claim 10, wherein the returning comprises returning in the navigation results one or more status attributes of each communicant who is present in the identified virtual location.

12. The method of claim 1, wherein the fixed set of commands comprises a go command and, when the go command is detected in the processed voice input, a successful ascertainment of a communicant name operand for the go command in the processed voice input results in moving the presence of the user to a respective one of the virtual locations in which a communicant corresponding to the communicant name operand is present.

13. The method of claim 1, wherein the fixed set of commands comprises a go command and, when the go command is detected in the processed voice input, a successful ascertainment of a virtual location name operand for the go command in the processed voice input results in moving the presence of the user to a respective one of the virtual locations corresponding to the virtual location name.

14. The method of claim 1, further comprising generating audio output comprising synthesized speech reporting the navigation results.

15. The method of claim 1, wherein application of the detected navigation command to the one or more operands produces navigation results comprising initiating a meeting between the user and one or more other communicants in a designated one of the virtual locations.

16. The method of claim 15, wherein application of the detected navigation command to the one or more operands produces navigation results comprising inviting the one or more other communicants to the designated virtual location.

17. The method of claim 1, wherein application of the detected navigation command to the one or more operands generates a recording of speech.

18. The method of claim 17, wherein application of the detected navigation command to the one or more operands produces a text based representation of the recorded speech.

19. The method of claim 1, wherein application of the detected navigation command to the one or more operands produces navigation results comprising reporting communicant activity in a designated meeting in a virtual location to the user.

20. Apparatus in a network communications environment implemented by at least one server network node and supporting realtime communications between a user and one or more other communicants who are operating respective client network nodes and are co-present in any of multiple virtual locations of a virtual area that define different respective communication contexts in the network communications environment, the apparatus comprising a non-transitory memory storing processor-readable instructions, and a processor coupled to the memory, operable to execute the instructions, and based at least in part on the execution of the instructions operable to perform operations comprising:

processing voice input from the user;

calling one or more services associated with the network communications environment to perform operations comprising

detecting in the processed voice input a navigation command in a fixed set of navigation commands each of which is associated with a respective syntax, wherein the fixed set of navigation commands comprises a look command and a find command, wherein in response to the user vocalizing the look command with no modifier causes one of the services to send to the user a report of who else is in the user'"'"'s current virtual location of presence in the virtual area and in response to the user vocalizing the look command with a specified virtual location in the virtual area causes one of the services to send to the user a report of who is in the specified virtual location in the virtual area, wherein the user vocalizing the find command with a specified contact operand causes one of the services to send to the user a report of the current virtual location of the specified contact in the virtual area, and the user vocalizing the find command with a specified name of a meeting in the virtual area causes one of the services to send to the user a report of the virtual location of the meeting with the specified name in the virtual area,

ascertaining one or more operands based on the syntax associated with the detected navigation command, the processed voice input, and one or more of names of respective ones of the virtual locations that are associated with the user and names of communicants who are contacts of the user, and

returning navigation results based on application of the detected navigation command to the one or more ascertained operands,

wherein the detecting and the ascertaining comprises forming grammatically correct word combinations of commands selected from the fixed set of navigation commands and operands selected from one or more of the names associated with respective ones of the virtual locations and the names associated with respective ones of the communicants based on relationship data according to the respective navigation command syntaxes, and preferentially selecting a respective one of the formed grammatically correct word combinations based on information about the user; and

generating audio output comprising synthesized speech reporting the navigation results.