System and method for generating an audio thumbnail of an audio track

• US 20040064209A1
• Filed: 09/30/2002
• Published: 04/01/2004
• Est. Priority Date: 09/30/2002
• Status: Active Grant

• Current Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Original Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Inventors: Zhang, Tong
• Granted Patent: US 7,386,357 B2
• Time in Patent Office: Days
• US Class Current: 700/94
• CPC Class Codes: G10H 2210/031 Musical analysis, i.e. isolation, extraction or identification of musical elements or musical parameters from a raw acoustic signal or from an encoded audio signal neural networks for electrophonic musical instruments or musical processing G10H2250/311
• CPC Class Codes: G10L 25/78 Detection of presence or absence of voice signals switching of direction of transmission by voice frequency in two-way loud-speaking telephone systems H04M9/10
• CPC Class Codes: G11B 27/036 Insert-editing
• CPC Class Codes: G11B 27/28 by using information signals recorded by the same method as the main recording G11B27/22 takes precedence

Abstract

A method and system for generating an audio thumbnail of an audio track in which a first content feature, such as singing, is detected as a characteristic of an audio track. A predetermined length of the detected portion of the audio track corresponding to the first content feature is extracted from the audio track. A highlight of the audio track, such as a portion of the audio track having a sudden increase in temporal energy within the audio track, is detected; and a portion of the audio track corresponding to the highlight is extracted from the audio track. The two extracted portions of the audio track are combined as a thumbnail of the audio track.

Claims

1. A method for generating an audio thumbnail of an audio track, comprising:

detecting a first content feature within an audio track;

extracting a first portion of the audio track corresponding to the detected first content feature;

detecting an occurrence of an increase in energy within the audio track;

extracting a second portion of the audio track corresponding to the detected increase in energy; and

combining the extracted first and second portions of the audio track into an audio thumbnail of the audio track.

2. The method according to claim 1, wherein the audio track comprises singing and instrumental music.

3. The method according to claim 1, wherein detecting a first content feature includes detecting the first occurrence of a human sound on the audio track.

4. The method according to claim 1, wherein the audio track is a song and wherein the first content feature is the start of a human voice within the song, comprising:

storing the audio thumbnail; and

listening to the stored audio thumbnail to preview the song.

5. The method according to claim 1, wherein detecting a first content feature includes application of at least one of voice detection methods of short-time average zero crossing rate, harmonic coefficient, spectral flux, filter analysis, and short-time energy function.

6. The method according to claim 1, wherein the first content feature within the audio track corresponds to a sudden increase in the energy level of the audio signal on the audio track.

7. The method according to claim 1, comprising:

selecting a first duration of time; and

extracting the first portion of the audio track having a duration corresponding to the selected first duration of time.

8. The method according to claim 1, wherein the increase in energy exceeds a predetermined threshold.

9. The method according to claim 1, wherein detecting an occurrence of an increase in energy comprises comparing an increase in the temporal energy between two adjacent portions of the audio track with a predetermined threshold.

10. The method according to claim 9, wherein detecting an occurrence of an increase in energy comprises:

computing a temporal energy envelope for the audio track;

mapping two adjacent windows on the audio track;

detecting locations on the audio track corresponding to human sound;

for each detected location of human sound on the audio track, comparing the computed temporal energy of the window corresponding to the detected location of the human sound with the computed temporal energy of the prior adjacent window;

determining the portions of the audio track whose temporal energy increase over the energy of the prior adjacent portion of the audio track exceeds the predetermined threshold; and

selecting as the second portion of the audio track the determined portion of the audio track having the greatest increase in temporal energy.

11. The method according to claim 10, wherein the locations corresponding to a human sound are detected by one or more voice detection methods including zero crossing rate, harmonic coefficient, spectral flux, filter analysis, and short term energy function.

12. The method according to claim 10, wherein the locations corresponding to human sound are detected by applying a zero crossing rate algorithm, and wherein if the application of the zero crossing rate algorithm produces inconclusive results, applying one or more of a harmonic coefficient algorithm, a spectral flux algorithm, an energy function algorithm, or a filter analysis algorithm.

13. The method according to claim 10, wherein the audio thumbnail of the audio track consists of the extracted first portion of the audio track.

14. The method according to claim 1, comprising:

selecting a second duration of time; and

extracting the second portion of the audio track having a duration corresponding to the selected second duration of time.

15. The method according to claim 1, wherein the extracted second portion is a refrain of a song.

16. The method according to claim 1, comprising:

detecting the first occurrence of singing on an audio track;

extracting a predetermined portion of the audio track corresponding to the start of the detected first occurrence of singing;

detecting a second occurrence of human sound on the audio track as characterized by a greatest increase in temporal energy within human sound portions on the audio track;

extracting a predetermined portion of the audio track corresponding to the start of the detected second occurrence of human sound; and

combining the extracted first occurrence of singing and second occurrence of human sound as an audio thumbnail of the song on the audio track.

17. A method for generating an audio thumbnail of an audio track, comprising:

detecting a first content feature within an audio track;

mapping a pointer to the detected first content feature within the audio track;

setting a first duration of time;

detecting an occurrence of an increase in energy within the audio track;

mapping a pointer to the detected occurrence of an increase in energy within the audio track;

setting a second duration of time; and

storing the pointer to the detected first content feature, the first duration of time, the pointer to the detected occurrence of an increase in energy, and the second duration of time as an audio thumbnail of the audio track.

18. The method according to claim 17, wherein the first and second durations of time are measured as a duration of music.

19. A method for detecting a highlight on an audio track, comprising:

determining a location of human sound on an audio track;

computing a first temporal energy envelope of a first segment of the audio track;

computing a second temporal energy envelope of a second segment of the audio track;

comparing the computed first and second temporal energy envelopes; and

if the second segment corresponds to a location of human sound on the audio track and if the computed temporal energy of the second segment exceeds the computed temporal energy of the first segment by a predetermined threshold, selecting a location on the audio track corresponding to the location of the second segment as a highlight on the audio track.

20. A computer-based system for generating a thumbnail of an audio track, comprising:

a recorder configured to record an audio track comprised of singing and instrumental music; and

a processor configured to;

detect a first characteristic within an audio track;

extract a first portion of the audio track corresponding to the detected first characteristic;

detect an occurrence of an increase in energy within the audio track;

extract a second portion of the audio track corresponding to the detected increase in energy; and

combine the extracted first and second portions of the audio track into an audio thumbnail of the audio track.

21. A computer readable medium encoded with software for generating an audio thumbnail of an audio track, by detecting a first characteristic within an audio track;

extracting a first portion of the audio track corresponding to the detected first characteristic;

detecting an occurrence of an increase in energy within the audio track;

extracting a second portion of the audio track corresponding to the detected increase in energy; and

combining the extracted first and second portions of the audio track into an audio thumbnail of the audio track.