Efficient beat-matched crossfading

• US 8,805,693 B2
• Filed: 08/18/2010
• Issued: 08/12/2014
• Est. Priority Date: 08/18/2010
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Lindahl, Aram, Powell, Richard Michael
• Primary Examiner(s): Godbold, Douglas
• Application Number:
  US12/858,900
• Publication Number:
  US 20120046954A1
• Time in Patent Office: 1,455 Days
• Field of Search: 704500-504
• US Class Current: 704/500
• CPC Class Codes: G10H 1/40 Rhythm
• CPC Class Codes: G10H 2210/076 for extraction of timing, tempo; Beat detection tempo display G10H2220/081; tempo control G10H2210/375
• CPC Class Codes: G10H 2210/125 Medley, i.e. linking parts of different musical pieces in one single piece, e.g. sound collage, DJ mix
• CPC Class Codes: G10H 2230/015 PDA [personal digital assistant] or palmtop computing devices used for musical purposes, e.g. portable music players, tablet computers, e-readers or smart phones in which mobile telephony functions need not be used touch-screen interfaces G10H2220/096
• CPC Class Codes: G10H 2240/075 Musical metadata derived from musical analysis or for use in electrophonic musical instruments additional information unrelated to its juxtaposed musical file data G10H2240/091; audio retrieval G06F16/60
• CPC Class Codes: G10H 2240/125 Library distribution, i.e. distributing musical pieces from a central or master library
• CPC Class Codes: G10H 2240/135 Library retrieval index, i.e. using an indexing scheme to efficiently retrieve a music piece
• CPC Class Codes: G10H 2240/155 Library update, i.e. making or modifying a musical database using musical parameters as indices data structures involving musical parameters G10H2240/005
• CPC Class Codes: G10H 2240/325 Synchronizing two or more audio tracks or files according to musical features or musical timings synchronised lyrics, e.g. for karaoke G10H2220/011
• CPC Class Codes: G10H 2250/031 Spectrum envelope processing
• CPC Class Codes: G10H 2250/035 Crossfade, i.e. time domain amplitude envelope control of the transition between musical sounds or melodies, obtained for musical purposes, e.g. for ADSR tone generation, articulations, medley, remix audio mixers H04H60/04
• CPC Class Codes: G10H 2250/261 Window, i.e. apodization function or tapering function amounting to the selection and appropriate weighting of a group of samples in a digital signal within some chosen time interval, outside of which it is zero valued
• CPC Class Codes: G10H 7/008 Means for controlling the transition from one tone waveform to another
• CPC Class Codes: G10L 19/025 Detection of transients or attacks for time/frequency resolution switching
• CPC Class Codes: G10L 2025/937 Signal energy in various frequency bands

Abstract

Methods and devices to enable efficient beat-matched, DJ-style crossfading are provided. For example, such a method may involve determining beat locations of a first audio stream and a second audio stream and crossfading the first audio stream and the second audio stream such that the beat locations of the first audio stream are substantially aligned with the beat locations of the second audio stream. The beat locations of the first audio stream or the second audio stream may be determined based at least in part on an analysis of frequency data unpacked from one or more compressed audio files.

Claims

1. An electronic device comprising:

nonvolatile storage configured to store a first compressed audio file;

data processing circuitry configured to unpack the first compressed audio file into frequency data and to estimate locations of beats in the first compressed audio file based at least in part on the frequency data before the frequency data is converted into an audio stream in the time domain; and

an audio decoder configured to decode a second compressed audio file into a time domain audio stream while the data processing circuitry unpacks the first compressed audio file into frequency data and estimates the locations of beats in the first compressed audio file.

2. The electronic device of claim 1, wherein the data processing circuitry is configured to detect the locations of the beats in the first compressed audio file based at least in part on a periodic pattern of spectral change over a series of frames of frequency data.

3. The electronic device of claim 1, wherein the data processing circuitry is configured to detect the locations of the beats in the first compressed audio file based at least in part on a periodic pattern of time window sizes of a series of frames of frequency data.

4. The electronic device of claim 1, comprising an audio decoder configured to decode the frequency data of the first compressed audio file unpacked by the data processing circuitry to obtain a time domain audio stream.

5. An article of manufacture comprising:

one or more tangible, machine-readable storage media having non-transitory instructions encoded thereon for execution by a processor, the instructions comprising;

instructions to receive a compressed audio file that encodes an audio stream;

instructions to partially decode the compressed audio file to obtain frames of time window sizes of frequency data representing frequencies present during windows of time of the audio file;

instructions to analyze a first series of the frames of frequency data to determine a first plurality of likely beat locations in the audio stream based at least in part on frequency changes over the first series of the frames of frequency data;

instructions to extrapolate beat locations elsewhere in the audio stream based at least in part on the first plurality of likely beat locations in the audio stream;

instructions to convert the frames of time window sizes of frequency data into time domain audio; and

instructions to verify the extrapolated beat locations by analyzing a second series of the frames of frequency data where a beat has been extrapolated and determining whether a likely beat location occurs at that location.

6. The article of manufacture of claim 5, wherein the instructions to analyze the first plurality of the frames of frequency data comprise instructions to identify frequency changes over the first series of the frames of frequency data in a frequency band.

7. The article of manufacture of claim 6, wherein the frequency band comprises a frequency associated with a percussion instrument.

8. The article of manufacture of claim 6, comprising instructions to determine the frequency band by identifying a likely-beat-containing set of frames via a time window analysis and determining what spectral components change in the likely-beat-containing set of frames.

9. The article of manufacture of claim 5, wherein the instructions to analyze the first series of the frames of frequency data comprise instructions to determine a likely beat location when a frequency band of the first series of the frames of frequency data reaches a peak magnitude.

10. A method comprising:

unpacking, using data processing circuitry, a compressed audio file into frames of frequency data representing frequencies present during time windows of the audio file before the frequency data is converted into an audio stream in the time domain;

analyzing, using the data processing circuitry, the frames of frequency data to select a plurality of frames of frequency data representing a frequency band present during short-term time windows of the audio file;

identifying, using the data processing circuitry, likely beat locations in the selected plurality of frames based at least in part on a peak magnitude of a frequency band of interest smaller than the entire frequency band present during the short-term time windows;

converting the frames of frequency data of the time windows of the audio file into time domain audio after identifying the likely beat locations; and

crossfading the audio file with another audio file using the likely beat locations.

11. The method of claim 10, wherein the likely beat locations are identified based at least in part on a periodic occurrence of the plurality of frames of frequency data representing the short-term time windows.

12. The method of claim 10, wherein the likely beat locations are identified based at least in part on a periodic occurrence of the plurality of frames of frequency data representing the short-term time windows punctuating frames of frequency data having relatively long-term time windows.

13. The method of claim 10, comprising identifying a specific frame of frequency data as a likely beat location by identifying a likely-beat-containing set of frames and selecting a centermost frame from among the likely-beat-containing set of frames.

14. A non-transitory machine readable medium providing instructions which when executed by data processing circuitry cause the data processing circuitry to perform a method of processing data, the method comprising:

unpacking, using data processing circuitry, a compressed audio file into frames of frequency data representing frequencies present during time windows of the audio file before the frequency data is converted into an audio stream in the time domain;

analyzing, using the data processing circuitry, the frames of frequency data to select a plurality of frames of frequency data representing a frequency band present during short-term time windows of the audio file;

identifying, using the data processing circuitry, likely beat locations in the selected plurality of frames based at least in part on a peak magnitude of a frequency band of interest smaller than the entire frequency band present during the short-term time windows;

converting the frames of frequency data of the time windows of the audio file into time domain audio after identifying the likely beat locations; and

crossfading the audio file with another audio file using the likely beat locations.

15. The medium of claim 14, wherein the likely beat locations are identified based at least in part on a periodic occurrence of the plurality of frames of frequency data representing the short-term time windows.

16. The medium of claim 14, wherein the likely beat locations are identified based at least in part on a periodic occurrence of the plurality of frames of frequency data representing the short-term time windows punctuating frames of frequency data having relatively long-term time windows.

17. The medium of claim 14, comprising identifying a specific frame of frequency data as a likely beat location by identifying a likely-beat-containing set of frames and selecting a centermost frame from among the likely-beat-containing set of frames.