Voice command processing for conferencing

• US 10,424,297 B1
• Filed: 02/02/2017
• Issued: 09/24/2019
• Est. Priority Date: 02/02/2017
• Status: Active Grant

• Current Assignee: Mitel Networks Incorporated (Mitel Networks Corporation)
• Original Assignee: Mitel Networks Incorporated (Mitel Networks Corporation)
• Inventors: Carino, Eric N.
• Primary Examiner(s): Leland, III, Edwin S
• Application Number:
  US15/423,028
• Time in Patent Office: 964 Days
• Field of Search: 704235
• CPC Class Codes: G06F 16/90332 Natural language query formulation or dialogue systems
• CPC Class Codes: G10L 15/1822 Parsing for meaning understanding
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 17/00 Speaker identification or verification techniques
• CPC Class Codes: G10L 17/04 Training, enrolment or model building
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: G10L 25/78 Detection of presence or absence of voice signals switching of direction of transmission by voice frequency in two-way loud-speaking telephone systems H04M9/10
• CPC Class Codes: H04L 65/403 Arrangements for multi-party communication, e.g. for conferences data switching systems for conference H04L12/18; arrangements for connecting several subscribers to a common circuit, i.e. affording conference facilities H04M3/56; television conferencing systems H04N7/15
• CPC Class Codes: H04N 7/152 Multipoint control units therefor

Abstract

An example method includes receiving at a conference bridge media from a plurality of participants during a conference session and mixing the media received from the plurality of participants to provide mixed media. At least one utterance of the mixed media is parsed using natural language processing to determine a command and at least one subject or object associated with the command. The method also includes invoking a selected conference function during the conference session based on the determined command and each identified subject or object.

Claims

1. A method, comprising:

receiving at a conference bridge media from a plurality of participants during a conference session;

mixing the media received from the plurality of participants to provide mixed media;

parsing a plurality of utterances of the mixed media using natural language processing, wherein each utterance of the plurality of utterances is associated with a respective participant of the plurality of participants;

converting each utterance of the plurality of utterances into text; and

analyzing a grammatical relationship and relative context of the text for a given utterance associated with a given participant of the plurality of participants to determine a command and the text for another utterance associated with an other participant of the plurality of participants that is prior to or after the given utterance to determine at least one subject or object associated with the command; and

invoking a selected conference function during the conference session based on the determined command and each identified subject or object.

2. The method of claim 1, wherein the parsing further comprises:

detecting voice audio in the mixed media;

extracting the plurality of utterances from the mixed media in response to the detecting the voice audio, each extracted utterance including the detected voice audio and metadata associated with a corresponding participant specifying a start time and an end time for each extracted utterance; and

wherein the analyzing is further based on the metadata for each extracted utterance to determine the command and the at least one subject or object according to a command schema defined for the conference session.

3. The method of claim 1, wherein the at least one object or subject includes one or more of the plurality of participants different from the given participant associated with the given utterance.

4. The method of claim 1, wherein the conference bridge includes a plurality of conference functions stored in memory, the method further comprising identifying the selected conference function from the plurality of conference functions based on the determined command.

5. The method of claim 4, further comprising:

determining a role of each participant within the conference session; and

assigning different subsets of the plurality of conference functions and rules associated with controlling invocation each of the plurality of conference functions according to the role of the participant who made the utterance.

6. The method of claim 4, wherein the invoking the selected conference function during the conference session is performed automatically by the conference bridge in response to the analyzing.

7. The method of claim 4 further comprising:

in response to the analyzing and prior to invoking the selected conference function, sending a request to the other participant of the plurality of participants for the selected conference function, wherein the selected conference function is invoked in response to a positive user input from the other participant to confirm the request, or the selected conference function is canceled in response to a negative user input from the other participant.

8. The method of claim 7, wherein the other participant is the at least one subject or object associated with the command.

9. The method of claim 7, wherein the request is sent to the other participant via a graphical user interface associated with the other participant or a private media stream to the other participant.

10. The method of claim 1, further comprising:

monitoring each stream of the received media from the plurality of participants; and

identifying each participant from the plurality of participants who is speaking the given utterance and the other utterance; and

determining the at least one subject or object further based on an identity of the each participant.

11. The method of claim 10, further comprising:

registering an identity of at least some of the plurality of participants before or during the conference session; and

associating each registered identity with a respective stream of the received media, the at least one subject or object being determined based on the association of a registered identity for the other participant.

12. The method of claim 1, in response to determining that the command is ambiguous, the method further comprises:

sending a request to the other participant with at least one option for performing the command; and

executing the command in response to receiving a user input from the other participant confirming the command.

13. The method of claim 1, further comprising:

analyzing metadata associated with each utterance to determine an identity of the other participant from the plurality of participants, the metadata comprising information identifying the other participant; and

invoking the selected conference function for the other participant further based on the identity of the other participant.

14. The method of claim 1, further comprising confirming an identity of the other participant prior to invoking the selected conference function for the other participant, wherein the selected conference function for the other participant is invoked in response to the confirming of the identity of the other participant.

15. The method of claim 1, wherein the selected conference function is further invoked based on one of a predetermined contract and a subscription.

16. The method of claim 1, wherein in response to invoking the selected conference function, at least one participant of the plurality of participants is disconnected from the conference session.

17. The method of claim 1, wherein the selected conference function is invoked for the other participant, wherein the selected conference function is a screen-sharing function.

18. A conference system, comprising:

memory to store machine readable instructions and data;

one or more processors to access the memory and execute the instructions, the instructions comprising;

a natural language processing (NLP) engine to;

parse a plurality of utterances of mixed media using natural language processing, wherein each utterance of the plurality of utterances is associated with a respective participant of the plurality of participants, wherein the mixed media comprises media received from a plurality of participants during a conference session;

convert each utterance of the plurality of utterances into text, and

analyze a grammatical relationship and relative context of the text for a given utterance associated with a given participant of the plurality of participants to determine a command and the text for another utterance associated with an other participant of the plurality of participants that is prior to or after the given utterance to determine at least one subject or object associated with the command; and

a conference manager to invoke a selected conference function for execution for the other participant of the plurality of participants during the conference session based on the determined command and each identified subject or object.

19. The system of claim 18, wherein the conference manager further comprises a command processor to receive the determined command and each associated subject or object and to invoke the selected conference function for the other participant during the conference session.

20. The system of claim 19, wherein the command processor further comprises:

an NLP command interface to receive the determined command from the NLP engine; and

a participant interface to provide an interactive interface to enable one or more of the plurality of participants different from the given participant to input instructions to invoke the selected conference function.

21. The system of claim 19, wherein the conference manager further comprises conference controls to control conference connections and flow of media associated with the conference session, and wherein the conference controls modify the conference connections and/or the flow of media in response to the selected conference function being invoked for the other participant during the conference session.

22. The system of claim 20, wherein the NLP engine further comprises:

an NLP analyzer to analyze the corresponding text and search a conference database to ascertain if each utterance includes the determined command and the at least one subject or object for the selected conference function, the NLP engine providing a command input to the NLP command interface to invoke the selected conference function for the other participant.

23. The system of claim 22, wherein the NLP engine further analyzes conference metadata associated with each utterance and provides the command input, corresponding to the determined command and the associated at least one subject or object, based on the metadata.