System and method for delivering targeted advertisements and/or providing natural language processing based on advertisements

• US 9,406,078 B2
• Filed: 08/26/2015
• Issued: 08/02/2016
• Est. Priority Date: 02/06/2007
• Status: Active Grant

• Current Assignee: VB Assets, LLC
• Original Assignee: VoiceBox Technologies Corporation (Microsoft Corporation)
• Inventors: Kennewick, Mike, Freeman, Tom
• Primary Examiner(s): ALBERTALLI, BRIAN LOUIS
• Application Number:
  US14/836,606
• Publication Number:
  US 20150364133A1
• Time in Patent Office: 342 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 40/263 Language identification
• CPC Class Codes: G06Q 30/0241 Advertisements
• CPC Class Codes: G06Q 30/0242 Determining effectiveness of advertisements
• CPC Class Codes: G06Q 30/0251 Targeted advertisements
• CPC Class Codes: G06Q 30/0269 based on user profile or attribute
• CPC Class Codes: G10L 15/18 using natural language modelling
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence

Abstract

The system and method described herein may use various natural language models to deliver targeted advertisements and/or provide natural language processing based on advertisements. In one implementation, an advertisement associated with a product or service may be provided for presentation to a user. A natural language utterance of the user may be received. The natural language utterance may be interpreted based on the advertisement and, responsive to the existence of a pronoun in the natural language utterance, a determination of whether the pronoun refers to one or more of the product or service or a provider of the product or service may be effectuated.

Claims

1. A method for processing natural language utterances that include requests and selecting and presenting advertisements based thereon, the method being implemented by one or more physical processors programmed with computer program instructions, which when executed cause the one or more physical processors to perform the method, the computer program instructions comprising at least a conversational language processor configured to interpret a natural language utterance, which relates to a request, based on words or phrases recognized from the natural language utterance, the method comprising:

in response to receiving the natural language utterance, providing the natural language utterance as an input to a speech recognition engine;

in response to receiving the words or phrases, recognized from the natural language utterance, as an output of the speech recognition engine, providing the words or phrases as an input to the conversational language processor;

interpreting the natural language utterance, at the conversational language processor, based on the recognized words or phrases;

determining the request based on the interpretation of the natural language utterance;

determining a context for the natural language utterance based at least on the recognized words or phrases;

selecting an advertisement based at least on the determined context;

providing the selected advertisement via an output device coupled to the conversational language processor;

obtaining a response to the request;

providing the response to the request;

in response to receiving a second natural language utterance, providing the second natural language utterance as an input to the speech recognition engine;

in response to receiving a second set of words or phrases, recognized from the second natural language utterance, as a second output of the speech recognition engine, providing the second set of words or phrases as a second input to the conversational language processor;

interpreting the second natural language utterance at the conversational language processor based on the recognized second set of words or phrases and the determined context; and

determining a second request, related to the advertisement, based on the interpretation of the second natural language utterance; and

obtaining and providing a second response to the second request.

2. The method of claim 1, wherein the conversational language processor comprises one or more domain agents, where a domain agent is configured to assist in:

i) interpreting requests related to its domain; and

ii) determining a response to the requests related to its domain.

3. The method of claim 2, wherein the domain agents comprise an electronic commerce agent and selecting an advertisement comprises selecting an advertisement that corresponds to an electronic commerce opportunity.

4. The method of claim 1, wherein the conversational language processor is configured to communicate with one or more domain-specific applications, and in response to a request relating to a domain specific application, the conversational language processor invoking the domain specific application to assist in processing the request.

5. The method of claim 4, wherein the domain-specific applications include an electronic commerce application and selecting an advertisement comprises selecting an advertisement that corresponds to an electronic commerce opportunity.

6. The method of claim 1, the method further comprising:

using an environmental model to determine environmental information, wherein determining a context for the natural language utterance is based further on the environmental information.

7. The method of claim 6, wherein the environmental information comprises a user location, a user activity, or a user action.

8. The method of claim 6, wherein determining the request is based further on the context.

9. The method of claim 1, wherein presenting the advertisement comprises:

audibly presenting a natural language conversational advertisement.

10. The method of claim 1, wherein obtaining the response comprises:

(i) servicing the request and determining the response, or (ii) providing the request to a remote device and receiving the response from the remote device.

11. The method of claim 1, wherein the response and the advertisement are provided together or separately.

12. The method of claim 1, wherein obtaining and providing the second response comprises facilitating a purchase or obtaining additional information related to the advertisement.

13. A method for processing natural language utterances that include requests and selecting and presenting advertisements based thereon, the method being implemented by one or more physical processors programmed with computer program instructions, which when executed cause the one or more physical processors to perform the method, the computer program instructions comprising at least a conversational language processor configured to interpret a natural language utterance, which relates to a request, based on words or phrases recognized from the natural language utterance, the method comprising:

in response to receiving the natural language utterance, providing the natural language utterance as an input to a speech recognition engine;

in response to receiving the words or phrases, recognized from the natural language utterance, as an output of the speech recognition engine, providing the words or phrases as an input to the conversational language processor;

interpreting the natural language utterance, at the conversational language processor, based on the recognized words or phrases;

determining the request based on the interpretation of the natural language utterance;

determining a context for the natural language utterance based at least on the recognized words or phrases;

selecting an advertisement based at least on the determined context;

providing the selected advertisement via an output device coupled to the conversational language processor;

obtaining a response to the request;

providing the response to the request;

receiving an indication that a user interaction with the advertisement has occurred;

determining whether the interpretation of the natural language utterance was correct based on the indication; and

interpreting a subsequent natural language utterance based on the determination of whether the interpretation of the natural language utterance was correct.

14. A method for processing natural language utterances that include requests and selecting and presenting advertisements based thereon, the method being implemented by one or more physical processors programmed with computer program instructions, which when executed cause the one or more physical processors to perform the method, the computer program instructions comprising at least a conversational language processor configured to interpret a natural language utterance, which relates to a request, based on words or phrases recognized from the natural language utterance, the method comprising:

in response to receiving the natural language utterance, providing the natural language utterance as an input to a speech recognition engine;

in response to receiving the words or phrases, recognized from the natural language utterance, as an output of the speech recognition engine, providing the words or phrases as an input to the conversational language processor;

interpreting the natural language utterance, at the conversational language processor, based on the recognized words or phrases;

determining the request based on the interpretation of the natural language utterance;

determining a context for the natural language utterance based at least on the recognized words or phrases;

selecting an advertisement based at least on the determined context;

providing the selected advertisement via an output device coupled to the conversational language processor;

obtaining a response to the request;

providing the response to the request;

receiving an indication that a user interaction with the advertisement has occurred;

determining that a subsequent natural language utterance relates to an advertisement context based on the indication;

in response to receiving a second natural language utterance after the user interaction, providing the second natural language utterance as an input to the speech recognition engine;

in response to receiving a second set of words or phrases, recognized from the second natural language utterance, as a second output of the speech recognition engine, providing the second set of words or phrases as a second input to the conversational language processor;

interpreting the second natural language utterance at the conversational language processor based on the recognized second set of words or phrases and the advertisement context; and

determining a second request, related to the advertisement, based on the interpretation of the second natural language utterance; and

obtaining and providing a second response to the second request.

15. A method for processing natural language utterances that include requests and selecting and presenting advertisements based thereon, the method being implemented by one or more physical processors programmed with computer program instructions, which when executed cause the one or more physical processors to perform the method, the computer program instructions comprising at least a conversational language processor configured to interpret a natural language utterance, which relates to a request, based on words or phrases recognized from the natural language utterance, the method comprising:

in response to receiving the natural language utterance, providing the natural language utterance as an input to a speech recognition engine;

in response to receiving the words or phrases, recognized from the natural language utterance, as an output of the speech recognition engine, providing the words or phrases as an input to the conversational language processor;

interpreting the natural language utterance, at the conversational language processor, based on the recognized words or phrases;

determining the request based on the interpretation of the natural language utterance;

determining a context for the natural language utterance based at least on the recognized words or phrases;

selecting an advertisement based at least on the determined context;

providing the selected advertisement via an output device coupled to the conversational language processor;

obtaining a response to the request;

providing the response to the request;

in response to a determination that the natural language utterance includes incomplete or unrecognized words or phrases such that insufficient information is available to determine the request, selecting a plurality of advertisements, each advertisement relating to its own context;

presenting the plurality of advertisements to the output device; and

receiving an indication that a user interaction has occurred with at least one of the plurality of advertisements, wherein determining the request is based further on a corresponding context of an advertisement, among the plurality of advertisements, with which the user interaction occurred.

16. A system for processing natural language utterances that include requests and selecting and presenting advertisements based thereon, the system comprising:

one or more physical processors programmed with computer program instructions, the computer program instructions comprising at least a conversational language processor configured to interpret a natural language utterance, which relates to a request, based on words or phrases recognized from the natural language utterance, the computer program instructions which when executed cause the one or more physical processors to;

in response to receipt of the natural language utterance, provide the natural language utterance as an input to a speech recognition engine;

in response to receipt of the words or phrases, recognized from the natural language utterance, as an output of the speech recognition engine, provide the words or phrases as an input to the conversational language processor;

interpret the natural language utterance, at the conversational language processor, based on the recognized words or phrases;

determine the request based on the interpretation of the natural language utterance;

determine a context for the natural language utterance based at least on the recognized words or phrases;

select an advertisement based at least on the determined context;

provide the selected advertisement via an output device coupled to the conversational language processor;

obtain a response to the request;

provide the response to the request;

in response to receipt of a second natural language utterance, provide the second natural language utterance as an input to the speech recognition engine;

in response to receipt of a second set of words or phrases, recognized from the second natural language utterance, as a second output of the speech recognition engine, provide the second set of words or phrases as a second input to the conversational language processor;

interpret the second natural language utterance at the conversational language processor based on the recognized second set of words or phrases and the determined context; and

determine a second request, related to the advertisement, based on the interpretation of the second natural language utterance; and

obtaining and providing a second response to the second request.

17. The system of claim 16, wherein the conversational language processor comprises one or more domain agents, where a domain agent is configured to assist:

i) interpret requests related to its domain; and

ii) determine a response to the requests related to its domain.

18. The system of claim 17, wherein the domain agents comprise an electronic commerce agent and wherein to select an advertisement, the one or more physical processors are further programmed to:

select an advertisement that corresponds to an electronic commerce opportunity.

19. The system of claim 16, wherein the one or more physical processors are further programmed to communicate with one or more domain-specific applications, and in response to a request relating to a domain specific application, invoke the domain specific application to assist process the request.

20. The system of claim 19, wherein the domain-specific applications include an electronic commerce application and wherein to select an advertisement, the one or more physical processors are programmed to select an advertisement that corresponds to an electronic commerce opportunity.

21. The system of claim 16, wherein the one or more physical processors are further programmed to:

use an environmental model to determine environmental information, wherein the context for the natural language utterance is determined based further on the environmental information.

22. The system of claim 21, wherein the environmental information comprises a user location, a user activity, or a user action.

23. The system of claim 21, wherein the determined request is based further on the context.

24. The system of claim 16, wherein to present the advertisement, the one or more physical processors are programmed to:

audibly present a natural language conversational advertisement.

25. The system of claim 16, wherein to obtain the response, the one or more physical processors are further programmed to:

(i) service the request and determine the response, or (ii) provide the request to a remote device and receive the response from the remote device.

26. The system of claim 16, wherein the response and the advertisement are provided together or separately.

27. The system of claim 16, wherein to obtain and provide the second response, the one or more physical processors are programmed to facilitate a purchase or obtain additional information related to the advertisement.

28. A system for processing natural language utterances that include requests and selecting and presenting advertisements based thereon, the system comprising:

one or more physical processors programmed with computer program instructions, the computer program instructions comprising at least a conversational language processor configured to interpret a natural language utterance, which relates to a request, based on words or phrases recognized from the natural language utterance, the computer program instructions which when executed cause the one or more physical processors to;

in response to receipt of the natural language utterance, provide the natural language utterance as an input to a speech recognition engine;

in response to receipt of the words or phrases, recognized from the natural language utterance, as an output of the speech recognition engine, provide the words or phrases as an input to the conversational language processor;

interpret the natural language utterance, at the conversational language processor, based on the recognized words or phrases;

determine the request based on the interpretation of the natural language utterance;

determine a context for the natural language utterance based at least on the recognized words or phrases;

select an advertisement based at least on the determined context;

provide the selected advertisement via an output device coupled to the conversational language processor;

obtain a response to the request;

provide the response to the request;

receive an indication that a user interaction with the advertisement has occurred;

determine whether the interpretation of the natural language utterance was correct based on the indication; and

interpret a subsequent natural language utterance based on the determination of whether the interpretation of the natural language utterance was correct.

29. A system for processing natural language utterances that include requests and selecting and presenting advertisements based thereon, the system comprising:

one or more physical processors programmed with computer program instructions, the computer program instructions comprising at least a conversational language processor configured to interpret a natural language utterance, which relates to a request, based on words or phrases recognized from the natural language utterance, the computer program instructions which when executed cause the one or more physical processors to;

in response to receipt of the natural language utterance, provide the natural language utterance as an input to a speech recognition engine;

in response to receipt of the words or phrases, recognized from the natural language utterance, as an output of the speech recognition engine, provide the words or phrases as an input to the conversational language processor;

interpret the natural language utterance, at the conversational language processor, based on the recognized words or phrases;

determine the request based on the interpretation of the natural language utterance;

determine a context for the natural language utterance based at least on the recognized words or phrases;

select an advertisement based at least on the determined context;

provide the selected advertisement via an output device coupled to the conversational language processor;

obtain a response to the request;

provide the response to the request;

receive an indication that a user interaction with the advertisement has occurred;

determine that a subsequent natural language utterance relates to an advertisement context based on the indication;

in response to receipt of a second natural language utterance after the user interaction, provide the second natural language utterance as an input to the speech recognition engine;

in response to receipt of a second set of words or phrases, recognized from the second natural language utterance, as a second output of the speech recognition engine, provide the second set of words or phrases as a second input to the conversational language processor;

interpret the second natural language utterance at the conversational language processor based on the recognized second set of words or phrases and the advertisement context; and

determine a second request, related to the advertisement, based on the interpretation of the second natural language utterance; and

obtain and provide a second response to the second request.

30. A system for processing natural language utterances that include requests and selecting and presenting advertisements based thereon, the system comprising:

one or more physical processors programmed with computer program instructions, the computer program instructions comprising at least a conversational language processor configured to interpret a natural language utterance, which relates to a request, based on words or phrases recognized from the natural language utterance, the computer program instructions which when executed cause the one or more physical processors to;

in response to receipt of the natural language utterance, provide the natural language utterance as an input to a speech recognition engine;

in response to receipt of the words or phrases, recognized from the natural language utterance, as an output of the speech recognition engine, provide the words or phrases as an input to the conversational language processor;

interpret the natural language utterance, at the conversational language processor, based on the recognized words or phrases;

determine the request based on the interpretation of the natural language utterance;

determine a context for the natural language utterance based at least on the recognized words or phrases;

select an advertisement based at least on the determined context;

provide the selected advertisement via an output device coupled to the conversational language processor;

obtain a response to the request;

provide the response to the request;

in response to a determination that the natural language utterance includes incomplete or unrecognized words or phrases such that insufficient information is available to determine the request, select a plurality of advertisements, each advertisement relating to its own context;

present the plurality of advertisements to the output device;

receive an indication that a user interaction has occurred with at least one of the plurality of advertisements, wherein the determination of the request is based further on a corresponding context of an advertisement, among the plurality of advertisements, with which the user interaction occurred.