Editing audio directly in frequency space

• US 7,640,069 B1
• Filed: 03/30/2006
• Issued: 12/29/2009
• Est. Priority Date: 03/30/2006
• Status: Active Grant

• Current Assignee: Adobe Inc.
• Original Assignee: Adobe Systems Incorporated (Adobe Inc.)
• Inventors: Johnston, David E.
• Primary Examiner(s): Kuntz; Curtis
• Assistant Examiner(s): Elbin; Jesse A
• Application Number:
  US11/395,044
• Time in Patent Office: 1,370 Days
• Field of Search: 381/98, 381/100, 715/716, 715/727
• US Class Current: 700/94
• CPC Class Codes: H04R 5/02 Spatial or constructional arrangements of loudspeakers

Abstract

Systems and methods for audio editing are provided. In one implementation, a computer-implemented method is provided. The method includes displaying audio data in a visual form. A user input is received demarcating an arbitrary region within the visual display of the audio data. A portion of the audio data is isolated corresponding to the region demarcated by the user. The isolated portion of the audio data is edited and the edited audio data is mixed into the audio data to create edited audio data.

Claims

1. A computer-implemented method, comprising:

receiving a user input demarcating an arbitrary region within a visual display of audio data, where the arbitrary region has a boundary with respect to frequency and time such that a bounded range of frequencies varies with respect to time;

isolating a portion of the audio data, the portion corresponding to the region demarcated by the user input;

editing the isolated portion of the audio data; and

mixing the edited portion of the audio data into the audio data to create edited audio data.

2. The method of claim 1, where isolating the portion of the audio data comprises:

dividing the region into blocks;

processing each block, including;

defining a window function for the block;

performing a Fourier transform on the audio data over the time of the block to extract frequency components;

applying the window to the Fourier transform results to remove extracted frequency components external to the block;

performing an inverse Fourier transform to provide a time domain isolated block audio data corresponding only to the audio data of the block;

combining the isolated block audio data for each block to generate the isolated audio data for the demarcated region.

3. The method of claim 2, where dividing the demarcated regions into blocks includes creating overlapping rectangular blocks having a predefined width in time and a height corresponding to the frequency range of the demarcated region at that time.

4. The method of claim 2, where windowing the block defines a function that is zero-valued outside the boundary of the window.

5. The method of claim 4, where the window boundary substantially matches the width of the block and the height of the demarcated region for block time.

6. The method of claim 5, where applying the window function to the Fourier transform results removes all frequency components of the audio data outside the defined window.

7. The method of claim 1, where displaying the audio data includes generating a frequency spectrogram display.

8. The method of claim 1, where editing the isolated audio data includes applying one or more effects to the isolated audio data.

9. The method of claim 1, where mixing the edited isolated audio data includes:

subtracting the isolated audio data of the demarcated region from the displayed audio data; and

adding the edited audio data into the displayed audio data.

10. The method of claim 1, where editing operations are performed directly within the display of the audio data.

11. A computer program product, encoded on a computer-readable storage medium, operable to cause data processing apparatus to perform operations comprising:

displaying audio data in a visual form;

receiving a user input demarcating an arbitrary region within the visual display of the audio data, where the arbitrary region has a boundary with respect to frequency and time such that a bounded range of frequencies varies with respect to time;

isolating a portion of the audio data, the portion corresponding to the region demarcated by the user;

editing the isolated portion of the audio data; and

mixing the edited portion of the audio data into the audio data to create edited audio data.

12. The computer program product of claim 11, where isolating the portion of the audio data comprises:

dividing the region into blocks;

processing each block, including;

defining a window function for the block;

performing a Fourier transform on the audio data over the time of the block to extract frequency components;

applying the window to the Fourier transform results to remove extracted frequency components external to the block;

performing an inverse Fourier transform to provide a time domain isolated block audio data corresponding only to the audio data of the block;

combining the isolated block audio data for each block to generate the isolated audio data for the demarcated region.

13. The computer program product of claim 12, where dividing the demarcated regions into blocks includes creating overlapping rectangular blocks having a predefined width in time and a height corresponding to the frequency range of the demarcated region at that time.

14. The computer program product of claim 12, where windowing the block defines a function that is zero-valued outside the boundary of the window.

15. The computer program product of claim 14, where the window boundary substantially matches the width of the block and the height of the demarcated region for block time.

16. The computer program product of claim 15, where applying the window function to the Fourier transform results removes all frequency components of the audio data outside the defined window.

17. The computer program product of claim 11, where displaying the audio data includes generating a frequency spectrogram display.

18. The computer program product of claim 11, where editing the isolated audio data includes applying one or more effects to the isolated audio data.

19. The computer program product of claim 11, where mixing the edited isolated audio data includes:

subtracting the isolated audio data of the demarcated region from the displayed audio data; and

adding the edited audio data into the displayed audio data.

20. The computer program product of claim 11, where editing operations are performed directly within the display of the audio data.

21. A computer-implemented method, comprising:

receiving a selection of an arbitrary region within a visual display of audio data, where the arbitrary region has a boundary with respect to frequency and time such that a bounded range of frequencies varies with respect to time;

isolating a portion of the audio data, the portion corresponding to the selected arbitrary region;

editing the isolated portion of the audio data; and

mixing the edited portion of the audio data into the audio data to create edited audio data.

22. An apparatus, comprising:

a display region for displaying audio data of an audio file;

one or more selection tools for selecting an arbitrary region within a particular displayed audio data, where the arbitrary region has a boundary with respect to frequency and time such that a bounded range of frequencies varies with respect to time; and

one or more editing tools for directly editing a portion of the audio data contained within the selected region of the displayed audio data.

23. A system comprising:

one or more processors configured to perform operations including;

receiving a user input demarcating an arbitrary region within a visual display of audio data, where the arbitrary region has a boundary with respect to frequency and time such that a bounded range of frequencies varies with respect to time;

isolating a portion of the audio data, the portion corresponding to the region demarcated by the user input;

editing the isolated portion of the audio data; and

mixing the edited portion of the audio data into the audio data to create edited audio data.

24. The system of claim 23, where isolating the portion of the audio data comprises:

dividing the region into blocks;

processing each block, including;

defining a window function for the block;

performing a Fourier transform on the audio data over the time of the block to extract frequency components;

applying the window to the Fourier transform results to remove extracted frequency components external to the block;

performing an inverse Fourier transform to provide a time domain isolated block audio data corresponding only to the audio data of the block;

combining the isolated block audio data for each block to generate the isolated audio data for the demarcated region.

25. The system of claim 24, where dividing the demarcated regions into blocks includes creating overlapping rectangular blocks having a predefined width in time and a height corresponding to the frequency range of the demarcated region at that time.

26. The system of claim 24, where windowing the block defines a function that is zero-valued outside the boundary of the window.

27. The system of claim 26, where the window boundary substantially matches the width of the block and the height of the demarcated region for block time.

28. The system of claim 27, where applying the window function to the Fourier transform results removes all frequency components of the audio data outside the defined window.

29. The system of claim 23, where displaying the audio data includes generating a frequency spectrogram display.

30. The system of claim 23, where editing the isolated audio data includes applying one or more effects to the isolated audio data.

31. The system of claim 23, where mixing the edited isolated audio data includes:

subtracting the isolated audio data of the demarcated region from the displayed audio data; and

adding the edited audio data into the displayed audio data.

32. The system of claim 23, where editing operations are performed directly within the display of the audio data.

33. A computer program product, encoded on a computer-readable storage medium, operable to cause data processing apparatus to perform operations comprising:

receiving a selection of an arbitrary region within a visual display of audio data, where the arbitrary region has a boundary with respect to frequency and time such that a bounded range of frequencies varies with respect to time;

isolating a portion of the audio data, the portion corresponding to the selected arbitrary region;

editing the isolated portion of the audio data; and

mixing the edited portion of the audio data into the audio data to create edited audio data.

34. A system comprising:

one or more processors configured to perform operations comprising;

receiving a selection of an arbitrary region within a visual display of audio data, where the arbitrary region has a boundary with respect to frequency and time such that a bounded range of frequencies varies with respect to time;

isolating a portion of the audio data, the portion corresponding to the selected arbitrary region;

editing the isolated portion of the audio data; and

mixing the edited portion of the audio data into the audio data to create edited audio data.