Sound rate modification

• US 10,249,321 B2
• Filed: 11/20/2012
• Issued: 04/02/2019
• Est. Priority Date: 11/20/2012
• Status: Active Grant

• Current Assignee: Adobe Inc.
• Original Assignee: Adobe Inc.
• Inventors: King, Brian John, Mysore, Gautham J., Smaragdis, Paris
• Primary Examiner(s): Colucci, Michael C
• Application Number:
  US13/681,643
• Publication Number:
  US 20140142947A1
• Time in Patent Office: 2,324 Days
• Field of Search: 704270, 704 4, 704 9, 704235, 704260
• CPC Class Codes: G10L 21/043 by changing speed

Abstract

Sound rate modification techniques are described. In one or more implementations, an indication is received of an amount that a rate of output of sound data is to be modified. One or more sound rate rules are applied to the sound data that, along with the received indication, are usable to calculate different rates at which different portions of the sound data are to be modified, respectively. The sound data is then output such that the calculated rates are applied.

Claims

1. A method implemented by at least one computing device, the method comprising:

receiving, as a user input, by the at least one computing device, an indication of an amount of time in which sound data is to be output, the sound data including a waveform representation and a plurality of portions, the indicated amount of time being different from an unmodified amount of time for playback of the sound data;

identifying, by the at least one computing device, at least one active portion and at least one inactive portion of the plurality of portions of the sound data based on spectral characteristics of the sound data, the at least one active portion containing multiple different units of speech, the at least one inactive portion corresponding to pauses in speech;

modifying, by the at least one computing device, the sound data to be output in the indicated amount of time using a set of sound rate rules generated to capture sound rate characteristics of units of speech in a natural language model by;

calculating different relative rates at which the multiple different units of speech are to be output, respectively, based on the set of sound rate rules and the indicated amount of time,

applying a first calculated rate to a first unit of speech in the active portion to cause the first unit of speech to be output at the first calculated rate, and

applying a second different calculated rate to a second unit of speech in the active portion to cause the second unit of speech to be output at the second different calculated rate; and

outputting, by the at least one computing device, the sound data as modified by the first calculated rate and the second different calculated rate in the indicated amount of time.

2. A method as described in claim 1, further comprising receiving, by the at least one computing device, at least one sound rate rule of the set of sound rate rules specified manually by a user.

3. A method as described in claim 1, further comprising learning, by the at least one computing device, at least one sound rate rule of the set of sound rate rules automatically and without user intervention through processing of a corpus of sound data.

4. A method as described in claim 1, wherein the indication specifies that the sound data is to be output in a longer amount of time than the unmodified amount of time for playback of the sound data.

5. A method as described in claim 1, wherein the at least one active portion includes a plurality of active portions, and the set of sound rate rules is usable to calculate a rate for each of the plurality of active portions.

6. A method as described in claim 1, wherein at least one of the set of sound rate rules specifies a value for a corresponding unit of speech usable to calculate the rate.

7. A method as described in claim 6, wherein the value is a cost, weight, or threshold value.

8. A method as described in claim 6, wherein the unit of speech is a syllable, phrase, pause, word, sentence, transient sound, or phone.

9. A method as described in claim 6, wherein the set of sound rate rules specify a plurality of values for a single said corresponding unit of speech, at least one said value of which is specified for a context in which the single said corresponding unit of speech is encountered in the sound data.

10. A method as described in claim 1, wherein the set of sound rate rules are arranged in a hierarchy such that a first said rule that corresponds to a first active portion is to be applied before a second said rule that corresponds to a second active portion.

11. A system comprising:

at least one module implemented at least partially in hardware and configured to;

receive input specifying a time period over which sound data is to be output, the sound data including a plurality of portions;

identify at least one active portion and at least one inactive portion of the plurality of portions of the sound data based on spectral characteristics of the sound data, the at least one active portion containing multiple different units of speech, the at least one inactive portion corresponding to pauses in speech;

modify the sound data using a set of sound rate rules that reflect a natural language model rule to the sound data by;

calculating different relative rates at which the different units of speech are to be output, respectively, based on the set of sound rate rules;

applying a first calculated rate to a first unit of speech in the active portion to cause the first unit of speech to be output at the first calculated rate; and

applying a second different calculated rate to a second unit of speech in the active portion to cause the second unit of speech to be output at the second different calculated rate; and

output the sound data as modified by the first calculated rate and the second different calculated rate over the specified time period.

12. A system as described in claim 11, wherein the at least one module if configured to receive at least one sound rate rule of the set of sound rate rules specified manually by a user.

13. A system as described in claim 11, wherein the indication specifies that the rate of the output of the sound data is to be generally unchanged while the sound data is being output.

14. A system as described in claim 11, wherein the at least one active portion includes a plurality of active portions, and the set of sound rate rules is usable to calculate a rate for each of the plurality of active portions.

15. A system as described in claim 11, wherein the set of sound rate rules are arranged in a hierarchy such that a first said rule that corresponds to a first active portion is to be applied before a second said rule that corresponds to a second active portion.

16. At least one computer-readable storage medium having instructions stored thereon that, responsive to execution on a computing device, causes the computing device to perform operations comprising:

receiving input specifying a time period over which sound data is to be output, the sound data including a plurality of portions;

identifying at least one active portion and at least one inactive portion of the plurality of portions of the sound data based on spectral characteristics of the sound data, the at least one active portion containing multiple different units of speech, the at least one inactive portion corresponding to pauses in speech;

modifying the sound data using a set of sound rate rules that reflect a natural language model rule to the sound data by;

calculating different relative rates at which the different units of speech are to be output, respectively, based on the set of sound rate rules to enable the sound data to be output within the specified period of time;

applying a first calculated rate to a first unit of speech in the active portion to cause the first unit of speech to be output at the first calculated rate; and

applying a second different calculated rate to a second unit of speech in the active portion to cause the second unit of speech to be output at the second different calculated rate; and

outputting the sound data as modified by the first calculated rate and the second different calculated rate over the specified time period.

17. At least one computer-readable storage medium as described in claim 16, wherein the input specifying the time period is specified manually by a user.

18. At least one computer-readable storage medium as described in claim 16, wherein the input specifying the time period specifies that the rate of the output of the sound data is to be generally unchanged while the sound data is being output.

19. At least one computer-readable storage medium as described in claim 16, wherein the at least one active portion includes a plurality of active portions, and the set of sound rate rules is usable to calculate a rate for each of the plurality of active portions.

20. At least one computer-readable storage medium as described in claim 16, wherein the set of sound rate rules are arranged in a hierarchy such that a first said rule that corresponds to a first active portion is to be applied before a second said rule that corresponds to a second active portion.