Speech recognizer adapted to reject machine articulations

• US 9,653,068 B2
• Filed: 07/19/2016
• Issued: 05/16/2017
• Est. Priority Date: 06/23/2008
• Status: Active Grant

• Current Assignee: Knapp Investment Company Limited
• Original Assignee: John Nicholas and Kristin Gross Trust
• Inventors: Gross, John Nicholas
• Primary Examiner(s): COLUCCI, MICHAEL C
• Application Number:
  US15/214,049
• Publication Number:
  US 20160329046A1
• Time in Patent Office: 301 Days
• Field of Search: 704246, 704243, 704244, 7042564, 704273, 705 18, 705 1469, 379168, 455410, 709225, 726 2, 726 23, 726 7
• CPC Class Codes: G06F 21/32 using biometric data, e.g. fingerprints, iris scans or voiceprints
• CPC Class Codes: G10L 13/027 Concept to speech synthesisers; Generation of natural phrases from machine-based concepts generation of parameters for speech synthesis out of text G10L13/08
• CPC Class Codes: G10L 13/08 Text analysis or generation of parameters for speech synthesis out of text, e.g. grapheme to phoneme translation, prosody generation or stress or intonation determination
• CPC Class Codes: G10L 15/02 Feature extraction for speech recognition; Selection of recognition unit
• CPC Class Codes: G10L 15/063 Training
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 17/00 Speaker identification or verification techniques
• CPC Class Codes: G10L 17/02 Preprocessing operations, e.g. segment selection; Pattern representation or modelling, e.g. based on linear discriminant analysis [LDA] or principal components; Feature selection or extraction
• CPC Class Codes: G10L 17/04 Training, enrolment or model building
• CPC Class Codes: G10L 17/06 Decision making techniques; Pattern matching strategies
• CPC Class Codes: G10L 17/22 Interactive procedures; Man-machine interfaces
• CPC Class Codes: G10L 17/26 Recognition of special voice characteristics, e.g. for use in lie detectors; Recognition of animal voices
• CPC Class Codes: H04L 63/102 Entity profiles
• CPC Class Codes: H04L 63/123 received data contents, e.g. message integrity
• CPC Class Codes: H04M 2203/2027 Live party detection

Abstract

A speech recognizer on a portable computing device is trained and configured to reject machine articulations of utterances attempting access to resources on such device. The recognizer is trained with scoring feedback from human listeners identifying articulations as originating from a machine.

Claims

1. A method of controlling access to a portable computing device which includes a speech recognition system for interpreting user commands, the method comprising:

configuring the speech recognition system to process and recognize speech presented in speech utterances originating both from humans and machines;

selecting first text data to be articulated as a speech utterance by an entity attempting to access resources of the portable computing device;

training the speech recognition system with human feedback from one or more human listeners indicating a machine origin for test utterances articulating said first text data;

wherein the speech recognition system is biased through such human feedback and said test utterances to behave like a human ear and reject utterances originating from non-live sources;

receiving first speech utterance input from a first entity corresponding to said first text data with the portable computing device;

processing said first speech utterance input with the biased speech recognition system to convert such input into recognized speech to determine if said first speech utterance corresponds to said first text data;

determining an origin of said first speech utterance input, including whether said first speech utterance input corresponds to a human or a machine with the biased speech recognition system;

wherein said determining of said origin is based on measuring whether speech data characteristics of said first speech utterance input by said first entity differ from and exceed human reference speech data characteristics by a controllable threshold;

controlling whether said first entity is allowed to access resources and/or data on the portable computing device based on said origin of said first speech utterance input.

2. The method of claim 1 further including a step:

comparing said first speech utterance input to a database of machine voiceprints to uniquely identify a machine attempting access.

3. The method of claim 1 further including steps:

receiving and storing a first human user voiceprint for said first text data; and

comparing said first speech utterance input to said first human user voiceprint during said determining of origin.

4. The method of claim 1 wherein the biased speech recognition system is configured to not recognize articulations of selected utterances originating from machine sources.

5. The method of claim 1 further including a step:

selecting said first text data from a set of sentences based on a score assigned by a human to said test utterances.

6. The method of claim 1 further including a step:

training the speech recognition system with a seed set of machine utterances that are rejected at a high rate by said one or more human listeners.

7. The method of claim 1 further including a step:

training the speech recognition system to hear and recognize content in said first text data utterance as a human ear hears and recognizes such utterance.

8. The method of claim 1 wherein the content of said first text data is processed to determine if a correct answer was provided in response to a question posed by a challenge routine operating on the portable computing device.

9. The method of claim 1 wherein multiple speech utterances from said first entity are processed during an interactive dialog to determine an origin of said utterances.

10. The method of claim 1 wherein said first text data is selected randomly from a set of challenge sentences optimized for discriminating between humans and machines.

11. The method of claim 10 wherein said set of challenge sentences are optimized through a phoneme selection process to ensure a minimum threshold of acoustic differences between humans and machines.

12. The method of claim 10 wherein said set of challenge sentences are customized for different demographic groups.

13. The method of claim 1 further including steps:

determining a prosody score for said first speech utterance input; and

using said prosody score also to determine said origin of said first speech utterance input.

14. The method of claim 1 further including a step:

estimating a first machine entity that best correlates to said first entity; and

selecting said first text data based on an identity of said first entity so as to maximize a difference in acoustic characteristics.

15. A challenge system for identifying a source of data input to a portable computing system using prosodic elements of speech comprising:

one or more software routines implemented in a computer readable medium and configured as a speech recognition system that can process and recognize speech presented in speech utterances originating both from humans and machines;

wherein the speech recognition system is configured with human feedback from one or more human listeners indicating a machine origin for test utterances articulating first text data;

further wherein the speech recognition system is biased through such human feedback and said test utterances to behave like a human ear and reject utterances originating from non-live sources;

one or more software routines implemented in a computer readable medium and adapted to cause the challenge system to;

select said first text data to be articulated as a speech utterance by an entity attempting to access resources of the portable computing device;

receive first speech utterance input from a first entity corresponding to said first text data;

process said first speech utterance input with the biased speech recognition system to convert such input into recognized speech to determine if said first speech utterance corresponds to said first text data;

determine an origin of said first speech utterance input, including whether said first speech utterance input corresponds to a human or a machine using the biased speech recognition system;

wherein said determination of said origin is made based on a measurement of whether speech data characteristics of said first speech utterance input by said first entity differ from and exceed human reference speech data characteristics by a controllable threshold;

control whether said first entity is allowed to access resources and/or data on the portable computing device based on said origin of said first speech utterance input.