Voice-based input using natural language processing for interfacing with one or more devices

• US 9,805,717 B2
• Filed: 03/25/2015
• Issued: 10/31/2017
• Est. Priority Date: 03/06/2015
• Status: Active Grant

• Current Assignee: Dell Products LP (Dell Technologies Inc.)
• Original Assignee: Dell Products LP (Dell Technologies Inc.)
• Inventors: Panemangalore, Sachinrao, Sawal, Vinay, Dharmadhikari, Vivek, Patel, Kuntal, Cato, Gavin Richard
• Primary Examiner(s): Godbold, Douglas
• Assistant Examiner(s): Villena, Mark
• Application Number:
  US14/668,850
• Publication Number:
  US 20160260430A1
• Time in Patent Office: 951 Days
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06F 40/58 Use of machine translation, e.g. for multi-lingual retrieval, for server-side translation for client devices or for real-time translation
• CPC Class Codes: G10L 15/1822 Parsing for meaning understanding
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command

Abstract

Aspects of the present invention provide a more universal, easy, natural, and vendor-agnostic interface to configure, manage, and/or monitor devices in networks. In embodiments, a user-friendly natural language interface voice interface may be used to “live chat” with one or more devices. In embodiments, a natural language input from a user intended for a target device is received and converted into one or more properly formed commands that are target-specific for the target device and may be executed by the target device. In embodiments, results from the execution of the one or more commands may be appropriately formatted for presentation to the user.

Claims

1. A computer-implemented method for converting a natural language input into a command input for a target device comprising:

converting an audio input into text comprising a natural language input, at least a portion of the natural language input being intended for a target device from among a plurality of potential target devices that requires commands to be properly formed for the target device to be executed;

performing natural language processing (NLP) input cleaning on the natural language input; and

performing input transformation on at least a portion of the cleaned natural language input to convert the cleaned natural language input into one or more properly formed commands that are target-specific for the target device by performing the steps comprising;

performing a lookup in a lemmatization database to convert at least some of the cleaned natural language input to a target-specific input that comprises target-specific vocabulary that corresponds to the target device; and

using the target-specific input to perform a lookup in a command template database to obtain a target-specific command template match corresponding to the target-specific input that comprises target-specific vocabulary, the target-specific command template being used to create at least one of the one or more properly formed commands.

2. The computer-implemented method of claim 1 wherein the step of performing natural language processing (NLP) input cleaning on the natural language input comprises performing one or more of the steps comprising:

responsive to the natural language input having plurals or reflexive pronouns, stemming the natural language input to remove any plurals and reflexive pronouns; and

responsive to the natural language input comprising compound statements, complex statements, or both, splitting the natural language input into two or more simple natural language inputs.

3. The computer-implemented method of claim 1 further comprising:

responsive to the target-specific command template match having one or more empty values, filling the one or more empty values from data obtained from the natural language input to obtain a properly formed target-specific command.

4. The computer-implemented method of claim 1 further comprising:

receiving the natural language input via a messaging application.

5. The computer-implemented method of claim 4 further comprising the step of:

presenting to a user via the messaging application one or more avatars corresponding to one or more target devices, which can be communicated with via the messaging application.

6. The computer-implemented method of claim 1 further comprising the step of:

submitting the one or more properly formed target-specific commands to the target device for execution.

7. The computer-implemented method of claim 6 further comprising:

receiving a set of results from the target device after execution of the one or more properly formed target-specific commands; and

formatting at least some of the results from the set of results for presentation to a user.

8. A natural language processing (NLP) system for converting a natural language input into a target-specific command input for a target device, the system comprising:

an audio sensor for receiving audio input from a user;

a voice-to-text converter, communicatively coupled to the audio sensor, that converts the audio input from the user into text comprising a natural language input;

an end-point interface system comprising a natural language processing (NLP) interaction end-point server that receives the natural language input from the user, the natural language input being intended for a target device from among a plurality of potential target devices;

a transformation system that cleans the natural language input and transforms at least a portion of the cleaned natural language input into one or more properly formed commands that are target-specific for the target device, the transformation system comprising;

an NLP input cleaning and transformation logic module, which is communicatively coupled to the end-point interface system to receive the natural language input and is communicatively coupled to a lemmatization database, that performs a lookup in the lemmatization database to convert at least some of the natural language input into a target-specific input corresponding to the target device; and

a target-specific NLP processing engine module, which is communicatively coupled to the NLP input cleaning and transformation logic module and to a command template database, that performs a lookup in the command template database using the target-specific input to obtain a target-specific command template match for creating at least one of the one or more properly formed commands; and

a target-device interface module that communicates the one or more properly formed commands to the target device for execution by the target device.

9. The natural language processing (NLP) system of claim 8 wherein the target-specific NLP processing engine module is further configured to:

responsive to the target-specific command template match having one or more empty values, fill the one or more empty values from data obtained from the natural language input to obtain at least one of the one or more properly formed commands that are target-specific for the target device.

10. The natural language processing (NLP) system of claim 9 wherein the lemmatization database comprises:

a collection of structured records having a list of commands and each command is associated with list of terms related to different devices, different vendors, or both that are considered synonyms.

11. The natural language processing (NLP) system of claim 9 wherein the command template database comprises:

target-specific command templates, each target-specific command template being in a structured format and having an associated set of one or more tags related to usage of the target-specific command template; and

one or more indexes of the target-specific command templates.

12. The natural language processing (NLP) system of claim 9 further comprising:

a training system that is communicatively coupled to the one or more sets of target-related documentation and that uses the target-related documentation to develop the lemmatization database and the command template database.

13. The natural language processing (NLP) system of claim 8 further comprising:

an NLP response engine that is communicatively coupled to the target device interface module and that receives results data related to at least one of the one or more properly formed commands.

14. The natural language processing (NLP) system of claim 8 further comprising:

a media engine that receives results data related to at least one of the one or more properly formed commands and formats at least some of the results data for presentation to the user into one or more of a plurality of formats.

15. A non-transitory computer-readable medium or media comprising one or more sequences of instructions which, when executed by at least one processor, causes steps to be performed comprising:

converting an audio input into text comprising a natural language input, at least a portion of the natural language input being intended for a target device from among a plurality of potential target devices that requires commands to be properly formed for the target device to be executed;

performing natural language processing (NLP) input cleaning on the natural language input; and

performing input transformation on at least a portion of the cleaned natural language input to convert the cleaned natural language input into one or more properly formed commands that are target-specific for the target device by performing the steps comprising;

performing a lookup in a lemmatization database to convert at least some of the cleaned natural language input to a target-specific input that comprises target-specific vocabulary that corresponds to the target device; and

using the target-specific input to perform a lookup in a command template database to obtain a target-specific command template match corresponding to the target-specific input that comprises target-specific vocabulary, the target-specific command template being used to create at least one of the one or more properly formed commands.

16. The non-transitory computer-readable medium or media of claim 15 wherein the step of performing natural language processing (NLP) input cleaning on the natural language input comprises performing one or more of the steps comprising:

responsive to the natural language input having plurals or reflexive pronouns, stemming the natural language input to remove any plurals and reflexive pronouns; and

responsive to the natural language input comprising compound statements, complex statements, or both, splitting the natural language input into two or more simple natural language inputs.

17. The non-transitory computer-readable medium or media of claim 15 further comprising:

responsive to the target-specific command template match having one or more empty values, filling the one or more empty values from data obtained from the natural language input to obtain a properly formed target-specific command.

18. The non-transitory computer-readable medium or media of claim 15 further comprising one or more sequences of instructions which, when executed by at least one processor, causes steps to be performed for generating one or more databases for use in converting a natural language input into one or more properly formed command inputs for a target device, the steps comprising:

generating a lemmatization database that comprise a collection of structured records having a list of commands and each command is associated with list of terms related to different devices, different vendors, or both that are considered synonyms; and

generating a command template database comprises;

target-specific command templates, each target-specific command template being in a structured format and having an associated set of one or more tags related to usage of the target-specific command template; and

one or more indexes of the target-specific command templates.

19. The non-transitory computer-readable medium or media of claim 18 wherein the step of generating a lemmatization database comprises the step of:

extracting command name and corresponding descriptions for a domain from a support corpus of documentation that include documentation across different devices, vendors, or both;

generating a set of word vectors from at least some of the support corpus documentation; and

clustering the set of word vectors into clusters in which a cluster represents word vectors that are deemed to be synonyms.

20. The non-transitory computer-readable medium or media of claim 18 wherein the step of generating a command template database comprises the step of:

extracting target-specific command templates from a set of documentation;

generating a target-specific command templates corpus in a structured format and label each target-specific command template with an associated set of one or more tags related to usage of the target-specific command template; and

indexing the target-specific command templates corpus.