System and Method for Multi-User GPU-Accelerated Speech Recognition Engine for Client-Server Architectures

• US 20170236518A1
• Filed: 02/16/2017
• Published: 08/17/2017
• Est. Priority Date: 02/16/2016
• Status: Active Grant

• Current Assignee: Carnegie Mellon University
• Original Assignee: Carnegie Mellon University
• Inventors: Lane, Ian Richard, Kim, Jungsuk
• Granted Patent: US 10,453,445 B2
• Time in Patent Office: Days
• CPC Class Codes: G10L 15/14 using statistical models, e.g. Hidden Markov Models [HMMs] G10L15/18 takes precedence
• CPC Class Codes: G10L 15/28 Constructional details of speech recognition systems

Abstract

Disclosed herein is a GPU-accelerated speech recognition engine optimized for faster than real time speech recognition on a scalable server-client heterogeneous CPU-GPU architecture, which is specifically optimized to simultaneously decode multiple users in real-time. In order to efficiently support real-time speech recognition for multiple users, a “producer/consumer” design pattern is applied to decouple speech processes that run at different rates in order to handle multiple processes at the same time. Furthermore, the speech recognition process is divided into multiple consumers in order to maximize hardware utilization. As a result, the platform architecture is able to process more than 45 real-time audio streams with an average latency of less than 0.3 seconds using one-million-word vocabulary language models.

Claims

1. A method for multi-user speech recognition comprising:

providing a computing platform comprising a Central Processing Unit (CPU) and a Graphical Processing Unit (GPU);

initializing search hypotheses for one or more inputs;

capturing a sample of each input signal of the one or more inputs;

computing sample features for each input of the one or more inputs;

jointly computing observation likelihoods for each of the sample features using a consumer/producer design pattern;

updating likelihoods for one or more inputs based on observation likelihoods for the each of the sample features and likelihoods in previous time step to form a set of partial hypotheses;

computing likelihood corrections for each partial hypothesis of the set of partial hypotheses for one or more inputs;

updating likelihoods for the each partial hypothesis of the set of partial hypotheses for one or more inputs; and

performing back-track on the set of updated partial hypotheses and determining a most likely partial hypothesis of the set of updated partial hypotheses for one or more inputs.