Systems and methods for the automatic extraction of audio excerpts

• US 20030083871A1
• Filed: 11/01/2001
• Published: 05/01/2003
• Est. Priority Date: 11/01/2001
• Status: Active Grant

• Current Assignee: Fuji Xerox Company Limited (Xerox Holdings Corp.)
• Original Assignee: Fuji Xerox Company Limited (Xerox Holdings Corp.)
• Inventors: Cooper, Matthew L., Foote, Jonathan T., Wilcox, Lynn D.
• Granted Patent: US 7,260,439 B2
• Time in Patent Office: Days
• US Class Current: 704/233
• CPC Class Codes: G11B 27/28 by using information signals recorded by the same method as the main recording G11B27/22 takes precedence

Abstract

A method of extracting audio excerpts comprises: segmenting audio data into a plurality of audio data segments; setting a fitness criteria for the plurality of audio data segments; analyzing the plurality of audio data segments based on the fitness criteria; and selecting one of the plurality of audio data segments that satisfies the fitness criteria. In various exemplary embodiments, the method of extracting audio excerpts further comprises associating the selected one of the plurality of audio data segments with video data. In such embodiments, associating the selected one of the plurality of audio data segments with video data may comprise associating the selected one of the plurality of audio data segments with a keyframe.

Claims

1. A method of extracting an audio excerpt, comprising:

segmenting audio data into a plurality of audio data segments;

setting a fitness criteria for the plurality of audio data segments;

analyzing the plurality of audio data segments based on the fitness criteria; and

selecting one of the plurality of audio data segments that satisfies the fitness criteria.

2. The method of claim 1, wherein segmenting the audio data into a plurality of audio data segments comprises segmenting the audio data based on a self-similarity analysis.

3. The method of claim 2, wherein segmenting the audio data based on a self-similarity analysis comprises identifying novelty points in a source signal of the audio data.

4. The method of claim 1, wherein:

setting a fitness criteria for the plurality of audio data segments comprises setting a duration constraint for the audio data segments; and

analyzing the plurality of audio data segments based on the fitness criteria comprises comparing a duration of each audio data segment to the duration constraint.

5. The method of claim 4, wherein selecting one of the plurality of audio data segments comprises selecting a first of the plurality of audio data segments that satisfies the duration constraint.

6. The method of claim 4, wherein analyzing the plurality of audio data segments based on the fitness criteria further comprises computing a fitness score for each audio data segment based on the comparison to the duration constraint.

7. The method of claim 6, wherein selecting one of the plurality of audio data segments comprises selecting an audio data segment having a maximum for the fitness score.

8. The method of claim 1, wherein:

setting a fitness criteria for the plurality of audio data segments comprises selecting at least one of duration, pitch contour, acoustic similarity, video features, keyframe time and associated data as a constraint for the audio data segments; and

analyzing the plurality of audio data segments based on the fitness criteria comprises identifying at least one of duration, pitch contour, acoustic similarity, video features, keyframe time and associated data for each audio data segment and applying the constraint.

9. The method of claim 1, further comprising associating the selected one of the plurality of audio data segments with video data.

10. The method of claim 9, wherein associating the selected one of the plurality of audio data segments with video data comprises associating the selected one of the plurality of audio data segments with a keyframe.

11. A method of extracting an audio excerpt, comprising:

segmenting audio data into a plurality of audio data segments;

setting a duration constraint for the plurality of audio data segments;

comparing a duration of each audio data segment to the duration constraint; and

selecting one of the plurality of audio data segments that satisfies the duration constraint.

12. The method of claim 11, wherein segmenting the audio data into a plurality of audio data segments comprises segmenting the audio data based on a self-similarity analysis.

13. The method of claim 12, wherein segmenting the audio data based on a self-similarity analysis comprises identifying novelty points in a source signal of the audio data.

14. The method of claim 11, wherein selecting one of the plurality of audio data segments comprises selecting a first of the plurality of audio data segments that satisfies the duration constraint.

15. The method of claim 11, further comprising computing a fitness score for each audio data segment based on the comparison to the duration constraint.

16. The method of claim 15, wherein selecting one of the plurality of audio data segments comprises selecting an audio data segment having a maximum for the fitness score.

17. The method of claim 11, further comprising associating the selected one of the plurality of audio data segments with video data.

18. The method of claim 17, wherein associating the selected one of the plurality of audio data segments with video data comprises associating the selected one of the plurality of audio data segments with a keyframe.

19. A system of extracting an excerpt of audio data, comprising:

a controller;

a segmentation circuit for segmenting audio data into a plurality of audio data segments;

an analysis circuit for analyzing the plurality of audio data segments based on at least one fitness criteria; and

a selection circuit for selecting one of the plurality of audio data segments that satisfies the fitness criteria.

20. The system of claim 19, further comprising an association circuit for associating the selected one of the plurality of audio data segments with video data.

21. A system of extracting an audio excerpt, comprising:

a controller;

a segmentation circuit for segmenting audio data into a plurality of audio data segments;

an analysis circuit for comparing a duration of each audio data segment to a duration constraint; and

a selection circuit for selecting one of the plurality of audio data segments that satisfies the duration constraint.

22. The system of claim 21, further comprising an association circuit for associating the selected one of the plurality of audio data segments with video data.

23. A computer readable storage medium comprising:

computer readable program code embodied on the computer readable storage medium, the computer readable program code usable to program a computer to perform a method of extracting an audio excerpt, comprising

segmenting audio data into a plurality of audio data segments;

setting a fitness criteria for the plurality of audio data segments;

analyzing the plurality of audio data segments based on the fitness criteria; and

selecting one of the plurality of audio data segments that satisfies the fitness criteria.

24. A carrier wave encoded to transmit a control program usable for a method of extracting an audio excerpt to a device for executing the control program, the control program comprising:

instructions for segmenting audio data into a plurality of audio data segments;

instructions for setting a fitness criteria for the plurality of audio data segments;

instructions for analyzing the plurality of audio data segments based on the fitness criteria; and

instructions for selecting one of the plurality of audio data segments that satisfies the fitness criteria.

25. A computer readable storage medium comprising:

computer readable program code embodied on the computer readable storage medium, the computer readable program code usable to program a computer to perform a method of extracting an audio excerpt, comprising

segmenting audio data into a plurality of audio data segments;

setting a duration constraint for the plurality of audio data segments;

comparing a duration of each audio data segment to the duration constraint; and

selecting one of the plurality of audio data segments that satisfies the duration constraint.

26. A carrier wave encoded to transmit a control program usable for a method of extracting an audio excerpt to a device for executing the control program, the control program comprising:

instructions for segmenting audio data into a plurality of audio data segments;

instructions for setting a duration constraint for the plurality of audio data segments;

instructions for comparing a duration of each audio data segment to the duration constraint; and

instructions for selecting one of the plurality of audio data segments that satisfies the duration constraint.

27. A method of selecting an audio excerpt from a plurality of audio data segments, comprising:

selecting a start candidate boundary from a list of candidate boundaries;

selecting an adjacent boundary candidate from the list of candidate boundaries to define a candidate segment;

analyzing the candidate segment based on at least one fitness criteria;

adding the candidate segment to a list of candidate segments if the candidate segment meets the at least one fitness criteria;

repeating the above steps to evaluate every combination of start candidate boundary and adjacent boundary candidate from the list of candidate boundaries; and

selecting one of the candidate segments from the generated list of candidate segments.

28. The method of claim 27, wherein selecting one of the candidate segments from the generated list of candidate segments is based on the at least one fitness criteria.

29. The method of claim 27, wherein selecting one of the candidate segments from the generated list of candidate segments is based on at least a second fitness criteria different from the at least one fitness criteria.