Natural language grammar enablement by speech characterization

• US 10,347,245 B2
• Filed: 01/20/2017
• Issued: 07/09/2019
• Est. Priority Date: 12/23/2016
• Status: Active Grant

• Current Assignee: Soundhound AI IP LLC
• Original Assignee: SoundHound Incorporated
• Inventors: Stahl, Karl
• Primary Examiner(s): Lerner, Martin
• Application Number:
  US15/411,567
• Publication Number:
  US 20180182385A1
• Time in Patent Office: 900 Days
• Field of Search: 704250, 704255, 704275, 704236, 704257
• CPC Class Codes: G10L 15/1807 using prosody or stress
• CPC Class Codes: G10L 15/1822 Parsing for meaning understanding
• CPC Class Codes: G10L 15/19 Grammatical context, e.g. disambiguation of the recognition hypotheses based on word sequence rules
• CPC Class Codes: G10L 17/02 Preprocessing operations, e.g. segment selection; Pattern representation or modelling, e.g. based on linear discriminant analysis [LDA] or principal components; Feature selection or extraction
• CPC Class Codes: G10L 2015/025 Phonemes, fenemes or fenones being the recognition units
• CPC Class Codes: G10L 25/63 for estimating an emotional state

Abstract

Either or both of voice speaker identification or utterance classification such as by age, gender, accent, mood, and prosody characterize speech utterances in a system that performs automatic speech recognition (ASR) and natural language processing (NLP). The characterization conditions NLP, either through application to interpretation hypotheses or to specific grammar rules. The characterization also conditions language models of ASR. Conditioning may comprise enablement and may comprise reweighting of hypotheses.

Claims

1. A non-transitory computer-readable medium comprising code effective to cause one or more processors to:

characterize a speech utterance to determine at least one characteristic;

recognize the speech utterance, without regard to the at least one characteristic, to produce at least one transcription hypothesis;

parse the at least one transcription hypothesis according to a set of grammar rules to produce a plurality of interpretation hypotheses, each having a corresponding likelihood score;

provide a plurality of grammar rule weights for the plurality of grammar rules corresponding to a plurality of speech characteristics;

select one or more grammar rule weights corresponding to the at least one characteristic from the plurality of grammar rule weights;

for each interpretation hypothesis of the plurality of interpretation hypotheses, adjust the likelihood score of the each interpretation hypothesis according to the selected one or more grammar rule weights corresponding to the at least one characteristic;

determine a set of authorized domains based on the at least one characteristic; and

filter the plurality of interpretation hypotheses according to the set of authorized domains; and

select a selected interpretation hypothesis from the plurality of interpretation hypotheses according to the likelihood scores thereof.

2. The non-transitory computer-readable medium of claim 1 wherein the at least one characteristic is mood.

3. The non-transitory computer-readable medium of claim 1 wherein the at least one characteristic is prosody.

4. The non-transitory computer-readable medium of claim 1 wherein the at least one characteristic is a rising intonation at the end of the speech utterance that indicates a yes or no question.