Method and apparatus for automatically determining speaker characteristics for speech-directed advertising or other enhancement of speech-controlled devices or services

• US 8,793,127 B2
• Filed: 10/31/2007
• Issued: 07/29/2014
• Est. Priority Date: 10/31/2002
• Status: Expired due to Term

• Current Assignee: Promptu Systems Corporation
• Original Assignee: Promptu Systems Corporation
• Inventors: Printz, Harry, Gulati, Vikas
• Primary Examiner(s): Lerner, Martin
• Application Number:
  US11/930,993
• Publication Number:
  US 20080103761A1
• Time in Patent Office: 2,463 Days
• Field of Search: 704/231, 704/243, 704/246, 704/250, 704/270, 704/275, 704/232, 704/257, 379/88.01, 379/88.04, 379/88.18, 705/14, 705/14.66
• US Class Current: 704/232
• CPC Class Codes: G06F 16/95 Retrieval from the web
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation
• CPC Class Codes: G06Q 30/02 Marketing; Price estimation or determination; Fundraising
• CPC Class Codes: G10L 15/02 Feature extraction for speech recognition; Selection of recognition unit
• CPC Class Codes: G10L 15/142 Hidden Markov Models [HMMs]
• CPC Class Codes: G10L 15/18 using natural language modelling
• CPC Class Codes: G10L 15/187 Phonemic context, e.g. pronunciation rules, phonotactical constraints or phoneme n-grams
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 17/26 Recognition of special voice characteristics, e.g. for use in lie detectors; Recognition of animal voices
• CPC Class Codes: G10L 2015/025 Phonemes, fenemes or fenones being the recognition units

Abstract

In addition to conveying primary information, human speech also conveys information concerning the speaker'"'"'s gender, age, socioeconomic status, accent, language spoken, emotional state, or other personal characteristics, which is referred to as secondary information. Disclosed herein are both the means of automatic discovery and use of such secondary information to direct other aspects of the behavior of a controlled system. One embodiment of the invention comprises an improved method to determine, with high reliability, the gender of an adult speaker. A further embodiment of the invention comprises the use of this information to display a gender-appropriate advertisement to the user of an information retrieval system that uses a cell phone as the input and output device. The invention is not limited to gender and such secondary information can include, for example, any of information concerning the speaker'"'"'s age, socioeconomic status, accent, language spoken, emotional state, or other personal characteristics.

Claims

1. An apparatus for automatically determining desired speaker characteristics, configured to operate in an application mode or a learning mode or in both an application mode and a learning mode, comprising:

a speech input device comprising a mobile device having an input and output, wherein said input comprises a means for receiving speech inputs in the form of one or more utterances and for receiving non-speech inputs, wherein said mobile device is configured for transmitting and receiving data over a network;

a primary information extraction module for receiving one or more utterances from said speech input device, said primary information extraction module comprising an automatic speech recognition module for collecting primary information in the form of;

transcribed text, symbolic representation of associated meaning, or a combination of transcribed text and symbolic representation of associated meaning;

a secondary information extraction module for receiving utterances from said speech input device, said secondary information extraction module comprising an automatic speech characteristics module that estimates or extracts secondary information comprising explicit or implicit speech indicators of interest, corresponding to desired speaker characteristics; and

a controlled system for using primary and secondary information extracted by said automatic speech recognition system module and said automatic speech characteristics module to produce a system action or response as a system output;

a learning module configured to record both secondary information and user behavior, said learning module configured to analyze said information to determine any of economically valuable relationships between speech and behavior, and economically valuable relationships between speech and speaker'"'"'s personal characteristics, wherein said economically valuable relationships indicate any of a high propensity to purchase and a high socioeconomic status;

said learning module analyzing either said secondary information and user behavior or said secondary information and speaker personal characteristics, for the discovery of relations between said secondary information and user behavior or said secondary information and speaker personal characteristics, by application of any of;

multivariate correlation analysis and linear regression;

mutual information statistics;

clustering and decision trees; and

perceptrons, neural networks, support vector machines, and linear classifiers;

wherein, during said application mode, said controlled system presents one or more advertisements to said user in addition to producing said system action, wherein the content of said advertisement is based on factors selected from among a group of factors consisting of;

primary information;

secondary information; and

a combination of primary information and secondary information;

wherein, during said learning mode, learning results from a series of queries or interactions, including at least some conducted by speech, followed by user behavior.

2. The apparatus of claim 1, said secondary information comprising at least one of:

gender, age, socioeconomic status, accent, language spoken, emotional state, propensity to execute a purchase, phone rate (speech rate), or power spectral density estimates.

3. The apparatus of claim 1, said primary information extraction module further comprising:

a means for natural language understanding of said primary information.

4. The apparatus of claim 1, further comprising:

a non-speech input that is provided to said controlled system.