Feedback controller for data transmissions

• US 10,431,209 B2
• Filed: 12/30/2016
• Issued: 10/01/2019
• Est. Priority Date: 12/30/2016
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: Bhaya, Gaurav, Stets, Robert
• Primary Examiner(s): Neway, Samuel G
• Application Number:
  US15/395,694
• Publication Number:
  US 20180190271A1
• Time in Patent Office: 1,005 Days
• CPC Class Codes: G06F 16/3329 Natural language query formulation or dialogue systems
• CPC Class Codes: G06F 16/3344 using natural language analysis
• CPC Class Codes: G06F 16/90332 Natural language query formulation or dialogue systems
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06F 40/205 Parsing
• CPC Class Codes: G10L 15/1822 Parsing for meaning understanding
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 2015/088 Word spotting
• CPC Class Codes: G10L 25/69 for evaluating synthetic or decoded voice signals
• CPC Class Codes: G10L 25/87 Detection of discrete points within a voice signal
• CPC Class Codes: G10L 25/90 Pitch determination of speech signals
• CPC Class Codes: H04B 17/309 Measuring or estimating channel quality parameters
• CPC Class Codes: H04L 65/1069 Session establishment or de-establishment
• CPC Class Codes: H04L 65/762 at the source reformatting of additional data in video distribution servers H04N21/2355
• CPC Class Codes: H04L 65/80 Responding to QoS
• CPC Class Codes: H04L 67/01 Protocols
• CPC Class Codes: H04L 67/53 using third party service providers
• CPC Class Codes: H04L 67/535 Tracking the activity of the user network monitoring arrangements H04L43/00; recording of computer activity G06F11/34
• CPC Class Codes: H04M 3/2236 Quality of speech transmission monitoring
• CPC Class Codes: H04M 3/4931 Directory assistance systems

Abstract

A feedback control system for data transmissions in voice activated data packet based computer network environment is provided. A system can receive audio signals detected by a microphone of a device. The system can parse the audio signal to identify trigger keyword and request. The system can select a content item using the trigger keyword or request. The content item can be configured to establish a communication session between the device and a third party device. The system can monitor the communication session to measure a characteristic of the communication session. The system can generate a quality signal based on the measured characteristic.

Claims

1. A feedback control system for data transmissions over a computer network, comprising:

a natural language processor component executed by a data processing system to receive, via an interface of the data processing system, data packets comprising an input audio signal detected by a sensor of a client device;

the natural language processor component to parse the input audio signal to identify a request and a trigger keyword corresponding to the request;

a content selector component executed by the data processing system to receive the trigger keyword identified by the natural language processor component and to select, based on the trigger keyword, a content item via a real-time content selection process;

a feedback monitor component to;

receive data packets carrying auditory signals transmitted between the client device and a conversational application programming interface that established a communication session with the client device responsive to interaction with the content item;

measure a characteristic of the communication session based on the auditory signals; and

generate a quality signal based on the measured characteristic; and

the content selector component to adjust the real-time content selection process based on the quality signal.

2. The system of claim 1, comprising the conversational application programming interface executed by the data processing system.

3. The system of claim 1, comprising the conversational application programming interface executed by a third party provider device.

4. The system of claim 1, comprising the data processing system to:

intercept the data packets transmitted from the client device;

parse the data packets to identify a third party device; and

route the data packets to the third party device.

5. The system of claim 1, comprising the data processing system to:

parse the data packets to determine an absence of auditory signals; and

generate the quality signal indicating a low level of quality based on the absence of the auditory signals.

6. The system of claim 1, comprising:

the feedback monitor component to forward the data packets carrying the auditory signals to the natural language processor component to determine a first characteristic of the auditory signals at a first time interval, and a second characteristic of the auditory signals at a second time interval subsequent to the first time interval; and

measure the characteristic based on a comparison of the first characteristic and the second characteristic.

7. The system of claim 6, wherein the first characteristic and the second characteristic include at least one of amplitude, frequency, tempo, tone, and pitch.

8. The system of claim 1, comprising the data processing system to:

transmit a plurality of voice-based queries to the client device; and

measure the characteristic based on responses to the plurality of voice-based queries.

9. The system of claim 1, comprising the data processing system to:

generate a query based on the quality signal being less than a threshold;

receive a response to the query from the client device; and

generate a second quality signal based on the response.

10. The system of claim 1, comprising the data processing system to:

measure the characteristic based on a duration of the communication session.

11. The system of claim 1, comprising the data processing system to:

measure the characteristic based on network activity on multiple electronic surfaces; and

aggregate quality signals measured from the multiple electronic surfaces to generate a summed quality signal.

12. The system of claim 1, comprising the data processing system to:

prevent the content selector component from selection, in the real-time content selection process, of the content item responsive to the quality signal being less than a threshold.

13. The system of claim 1, comprising:

permit the content selector component to select, in the real-time content selection process, the content item responsive to the quality signal being greater than or equal to a threshold.

17. The system of claim 1, comprising:

the feedback monitor component to determine a reaction of a user associated with the input audio signal based on the characteristic of the communication session.

18. The system of claim 1, comprising:

the content selector component to adjust the real-time content selection process based on the quality signal, wherein the characteristic of the communication session is indicative of a sentiment of a user associated with the client device.

19. The system of claim 1, wherein the characteristic of the communication session includes at least one of amplitude, frequency, tempo, tone, and pitch.

14. A method of transmitting data over a computer network using a feedback control system, comprising:

receiving, by a natural language processor component executed by a data processing system, via an interface of the data processing system, data packets comprising an input audio signal detected by a sensor of a client device;

parsing, by the data processing system, the input audio signal to identify a request and a trigger keyword corresponding to the request;

receiving, by a content selector component executed by the data processing system, the trigger keyword identified by the natural language processor component;

selecting, by the content selector component, based on the trigger keyword, a content item via a real-time content selection process;

receiving, by a feedback monitor component, data packets carrying auditory signals transmitted between the client device and a conversational application programming interface that established a communication session with the client device responsive to interaction with the content item;

measuring, by the feedback monitor component, a quality of the communication session based on the auditory signals;

generating, by the feedback monitor component, a quality signal based on the measured characteristic; and

adjusting, by the content selector component, the real-time content selection process based on the quality signal.

15. The method of claim 14, comprising:

forwarding, by the feedback monitor component, the data packets carrying the auditory signals to the natural language processor component to determine a first characteristic of the auditory signals at a first time interval, and a second characteristic of the auditory signals at a second time interval subsequent to the first time interval; and

measuring, by the feedback monitor component, the quality based on a comparison of the first characteristic and the second characteristic.

16. The method of claim 15, wherein the first characteristic and the second characteristic include at least one of amplitude, frequency, tempo, tone, and pitch.

20. The method of claim 14, wherein the auditory signals indicate at least one of amplitude, frequency, tempo, tone, and pitch.