System and method for validating natural language content using crowdsourced validation jobs
First Claim
1. A computer-implemented method, the method being implemented in a computer system having one or more physical processors programmed with computer program instructions that, when executed by the one or more physical processors, program the computer system to perform the method, the method comprising:
- obtaining, by the computer system, a transcription pair comprising natural language content and text, wherein the natural language content comprises audio content received from one or more audio input components, and wherein the text corresponds to a transcription of the natural language content;
creating, by the computer system, a first crowdsourced validation job to be performed at one or more first validation devices, the first crowdsourced validation job comprising first instructions for a crowd user to provide an indication of an accuracy of the transcription of the natural language content;
causing, by the computer system, the transcription pair and the first crowdsourced validation job to be provided to the one or more first validation devices;
receiving, by the computer system, at least a first response from at least a first validation device from among the one or more first validation devices, wherein the first response includes a first indication of an accuracy of the transcription of the natural language content; and
storing, by the computer system, the transcription pair in a validated transcription library based, at least in part, on the first response.
5 Assignments
0 Petitions
Accused Products
Abstract
Systems and methods of validating transcriptions of natural language content using crowdsourced validation jobs are provided herein. In various implementations, a transcription pair comprising natural language content and text corresponding to a transcription of the natural language content may be gathered. A group of validation devices may be selected for reviewing the transcription pair. A crowdsourced validation job may be created for the group of validation devices. The crowdsourced validation job may be provided to the group of validation devices. One or more votes representing whether or not the text accurately represents the natural language content may be received from the group of validation devices. Based on the one or more votes received, the transcription pair may be stored in a validated transcription library, which may be used to process end-user voice data.
-
Citations
28 Claims
-
1. A computer-implemented method, the method being implemented in a computer system having one or more physical processors programmed with computer program instructions that, when executed by the one or more physical processors, program the computer system to perform the method, the method comprising:
-
obtaining, by the computer system, a transcription pair comprising natural language content and text, wherein the natural language content comprises audio content received from one or more audio input components, and wherein the text corresponds to a transcription of the natural language content; creating, by the computer system, a first crowdsourced validation job to be performed at one or more first validation devices, the first crowdsourced validation job comprising first instructions for a crowd user to provide an indication of an accuracy of the transcription of the natural language content; causing, by the computer system, the transcription pair and the first crowdsourced validation job to be provided to the one or more first validation devices; receiving, by the computer system, at least a first response from at least a first validation device from among the one or more first validation devices, wherein the first response includes a first indication of an accuracy of the transcription of the natural language content; and storing, by the computer system, the transcription pair in a validated transcription library based, at least in part, on the first response. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14)
-
-
15. A system comprising:
one or more physical processors programmed with one or more computer program instructions which, when executed, program the one or more physical processors to; gather a transcription pair comprising natural language content and text, wherein the natural language content comprises audio content received from one or more audio input components, and wherein the text corresponds to a transcription of the natural language content; create a first crowdsourced validation job to be performed at one or more first validation devices, the first crowdsourced validation job comprising first instructions for a crowd user to provide an indication of an accuracy of the transcription of the natural language content; cause the transcription pair and the first crowdsourced validation job to be provided to the one or more first validation devices; receive at least a first response from at least a first validation device from among the one or more first validation devices, wherein the first response includes a first indication of an accuracy of the transcription of the natural language content; and store the transcription pair in a validated transcription library based, at least in part, on the first response. - View Dependent Claims (16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28)
Specification