Natural language processing utilizing grammar templates

• US 9,928,836 B2
• Filed: 07/13/2016
• Issued: 03/27/2018
• Est. Priority Date: 09/18/2012
• Status: Active Grant

• Current Assignee: Adobe Inc.
• Original Assignee: Adobe Systems Incorporated (Adobe Inc.)
• Inventors: Wilensky, Gregg D., Chang, Walter W., Dontcheva, Lubomira A., Laput, Gierad P., Agarwala, Aseem O.
• Primary Examiner(s): Godbold, Douglas
• Application Number:
  US15/209,676
• Publication Number:
  US 20160321242A1
• Time in Patent Office: 622 Days
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06F 40/186 Templates
• CPC Class Codes: G06F 40/205 Parsing
• CPC Class Codes: G06F 40/253 Grammatical analysis; Style critique
• CPC Class Codes: G06F 40/279 Recognition of textual entities
• CPC Class Codes: G06F 40/284 Lexical analysis, e.g. tokenisation or collocates
• CPC Class Codes: G06F 40/30 Semantic analysis
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue

Abstract

Natural language input processing utilizing grammar templates are described. In one or more implementations, a natural language input indicating an operation to be performed is parsed into at least one part-of-speech, a grammar template corresponding to the part-of-speech is located, an arbitrary term in the part-of-speech is detected based on the located grammar template, a term related to the arbitrary term and describing a modification for the operation is determined based on the sentence expression of the grammar template, and the indicated operation is performed with the described modification.

Claims

1. A method comprising:

receiving, by a computing device, a natural language input indicative of an operation to be performed;

parsing, by the computing device, the natural language input into at least one part-of-speech;

locating, by the computing device, at least one grammar template of a plurality of grammar templates that corresponds to the at least one part-of-speech, each of the plurality of grammar templates defining a sentence expression;

detecting, by the computing device, an arbitrary term in the at least one part-of-speech based on the located grammar template;

identifying, by the computing device, a known operational term in the at least one part-of-speech, the known operational term corresponding to the operation to be performed;

determining, by the computing device, a term that is related to the arbitrary term based on matching the sentence expression of the at least one grammar template against the known operational term, the term describing a modification for the operation to be performed and selected from a set of base terms corresponding to the known operational term; and

initiating performance, by the computing device, of the operation as including the modification described by the term.

2. A method as described in claim 1, wherein the determining of the term includes producing a measure of closeness based on a term distance between the arbitrary term and a respective base terms from the set of base terms.

3. A method as described in claim 1, wherein the determining of the term includes comparing a part-of-speech tag for the arbitrary term to respective base terms from the set of base terms.

4. A method as described in claim 1, wherein the operation to be performed is tagged with a specific class of sentence patterns.

5. A method as described in claim 1, wherein the set of base terms includes a noun term, a verb term, or an adjective term encapsulating the modification for the operation to be performed.

6. The method as described in claim 5, wherein the noun term, verb term, and adjective term are based on definitions contained in a lexical database.

7. A method as described in claim 1, wherein the operation to be performed is a combination of at least two filter functions with specified filter parameters, the specified filter parameters including a modification described by the term.

8. A method as described in claim 1, wherein the operation to be performed is one or more image editing operations.

9. A method as described in claim 1, wherein the operation to be performed is one or more filter functions with specified filter parameters, the specified filter parameters including a modification described by the term.

10. A method as described in claim 1, wherein the known operational term is identified by using lexical ontologies and semantic distances to map the at least one part-of-speech to the operation to be performed.

11. A method as described in claim 1, further comprising:

receiving, by the computing device, a gesture input; and

determining, by the computing device, the term that is related to the arbitrary term based on the sentence expression of the at least one grammar template and the gesture input, the term describing a modification for the operation to be performed.

12. A system comprising:

a speech-to-text engine implemented at least partially in hardware of a computing device to receive a natural language input indicative of an operation to be performed and to convert the natural language input into text; and

a natural language processing module implemented at least partially in hardware of the computing device to;

parse the text into at least one part-of-speech;

locate at least one grammar template of a plurality of grammar templates that correspond to the at least one part-of-speech by matching the part-of-speech against a sentence expression defined by each of the plurality of grammar templates, the at least one grammar template selected based on a scoring mechanism;

detect an arbitrary term in the at least one part-of-speech based on the located grammar template;

identify a known operational term in the at least one part-of-speech, the known operational term corresponding to the operation to be performed;

determine a term that is related to the arbitrary term based on matching the sentence expression of the at least one grammar template against the known operational term, the term describing a modification for the operation to be performed and selected from a set of base terms corresponding to the known operational term; and

initiate performance of the operation as including the modification described by the term.

13. The system as described in claim 12, wherein the determination of the term includes producing a measure of closeness using a term distance formula between the arbitrary term and a respective base terms from the set of base terms.

14. The system as described in claim 12, wherein the operation to be performed is one or more image editing operations.

15. The system as described in claim 12, wherein the operation to be performed is one or more filter functions with specified filter parameters, the specified filter parameters including a modification described by the term.

16. The system as described in claim 12, wherein the known operational term is identified by using lexical ontologies and semantic distances to map the at least one part-of-speech to the operation to be performed.

17. One or more computer-readable storage media comprising instructions stored thereon that, responsive to execution by a computing device, causes the computing device to perform operations comprising:

parsing, by the computing device, a natural language input into at least one part-of-speech;

identifying, by the computing device, a known operational term in the at least one part-of-speech, the known operational term corresponding to an operation to be performed;

locating, by the computing device, at least one grammar template of a plurality of grammar templates that corresponds to the at least one part-of-speech, each of the plurality of grammar templates defining a sentence expression;

detecting, by the computing device, an arbitrary term in the at least one part-of-speech based on the located grammar template;

determining, by the computing device, a term that is related to the arbitrary term based on matching the sentence expression of the at least one grammar template against the known operational term, the term describing a modification for the operation to be performed; and

initiating performance, by the computing device, of the operation as including the modification described by the term.

18. One or more computer-readable storage media as described in claim 17, wherein determining the term includes producing a measure of closeness using a term distance formula between the arbitrary term and a respective base terms from a set of base terms corresponding to the known operational term.

19. One or more computer-readable storage media as described in claim 17, wherein the operation to be performed is one or more image editing operations.

20. One or more computer-readable storage media as described in claim 17, wherein the operation to be performed is one or more filter functions with specified filter parameters, the specified filter parameters including a modification described by the term.