Audio view using 3-dimensional plot

• US 7,880,748 B1
• Filed: 08/17/2005
• Issued: 02/01/2011
• Est. Priority Date: 08/17/2005
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Sevigny, Benoit
• Primary Examiner(s): GOOD JOHNSON, MOTILEWA
• Application Number:
  US11/206,536
• Time in Patent Office: 1,994 Days
• Field of Search: 345/440, 702/68
• US Class Current: 345/619
• CPC Class Codes: G06T 11/206 Drawing of charts or graphs
• CPC Class Codes: G11B 27/036 Insert-editing
• CPC Class Codes: G11B 27/34 Indicating arrangements indicating means incorporated in magazine or cassette G11B23/046 and G11B23/0875; indicating measured values in general G01D

Abstract

Techniques for editing signal data using a three-dimensional visual representation of the signal data are provided. According to one embodiment of the invention, audio signal data is displayed as a surface occupying three dimensions. User input, which indicates a selected area of the surface, is received. The audio signal data is then modified by applying an effect to one or more parts of the audio signal data that correspond to the selected area.

Claims

1. A computer-implemented method for editing signal data, the method comprising steps of:

displaying original signal data as a three-dimensional representation, wherein each dimension of the representation corresponds to an attribute of the original signal data;

receiving user input that indicates a selected portion of the representation;

modifying the original signal data by applying an effect to one or more parts of the original signal data that correspond to the selected portion, thereby producing modified signal data; and

storing the modified signal data on a computer-readable medium;

wherein three different attributes of the original signal data are spatially represented within the three-dimensional representation along three different spatial axes, each axis of the axes following a separate vector that points from a common reference point shared by the three different spatial axes, and each axis-following vector pointing in a different direction for each other axis-following vector;

wherein said steps are performed by one or more computing devices.

2. The method of claim 1, wherein a first dimension of the three dimensions represents a frequency of the signal data, a second dimension of the three dimensions represents an amplitude of the signal data, and a third dimension of the three dimensions represents time.

3. The method of claim 1, further comprising:

rotating the representation to display a view of the representation at a different angle than an angle at which the representation was previously displayed.

4. The method of claim 1, further comprising:

receiving user input that indicates a selection of a particular selection indicator from among a plurality of selection indicators;

wherein a size of the selected area is based on the particular selection indicator.

5. The method of claim 1, wherein the representation is displayed as a surface occupying three dimensions, wherein the selected portion is a selected two-dimensional area of the surface, and further comprising:

in response to user input, moving a two-dimensional selection indicator on the surface, wherein a location of the selection indicator on the surface determines which area of the surface is the selected area; and

after moving the selection indicator, playing at least a part of a sound that is represented by one or more parts of the signal data that correspond to the selected area without playing any sound that is not represented by any part of the signal data that corresponds to the selected area;

wherein the selected area comprises less than all of the surface; and

wherein two dimensions of the selection indicator are aligned with two of the three different spatial axes.

6. The method of claim 1, wherein:

the representation is displayed as a surface occupying three dimensions;

the selected portion is a selected area of the surface,

the selected area represents amplitudes of frequencies that occur within a first range of frequencies over a first period of time;

the surface represents amplitudes of frequencies that occur within a second range of frequencies over a second period of time;

the first range of frequencies is of lesser magnitude than the second range of frequencies; and

the first period of time is of lesser magnitude than the second period of time.

7. The method of claim 1, wherein the representation is displayed as a surface occupying three dimensions, wherein the selected portion is a selected area of the surface, and wherein displaying the representation comprises:

segregating the surface into a grid of sub-areas; and

for each sub-area in the grid, performing steps comprising;

categorizing, into one or more particular categories of sound, particular sounds that are represented by signal data that corresponds to the sub-area; and

displaying the sub-area in a color that is based on one or more categories of sound into which the particular sounds have been categorized.

8. The method of claim 7, wherein the particular categories of sound comprise noise, tone, and transient sound.

9. The method of claim 1, wherein applying the effect to the one or more parts of the signal data that correspond to the selected portion comprises:

applying the effect only to parts of the signal data that correspond to a particular category of sound, wherein the particular category is one of tone and transient sound.

10. The method of claim 1, wherein applying the effect to the one or more parts of the signal data that correspond to the selected portion comprises:

modifying a part of the signal data that corresponds to the selected portion so that the part represents sound from another audio signal in addition to sound that the part already represents.

11. The method of claim 1, wherein applying the effect to the one or more parts of the signal data that correspond to the selected portion comprises:

modifying only parts of the signal data that correspond to a specified fundamental frequency or an integer multiple of the specified fundamental frequency.

12. The method of claim 1, wherein the selected portion comprises a selection mask that a user has defined by separately selecting multiple regions of the representation.

13. A computer-readable non-transitory storage medium storing one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 1.

14. A computer-readable non-transitory storage medium storing one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 2.

15. A computer-readable non-transitory storage medium storing one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 3.

16. A computer-readable non-transitory storage medium storing one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 4.

17. A computer-readable non-transitory storage medium storing one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 5.

18. A computer-readable non-transitory storage medium storing one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 6.

19. A computer-readable non-transitory storage medium storing one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 7.

20. A computer-readable non-transitory storage medium storing one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 8.

21. A computer-readable non-transitory storage medium storing one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 9.

22. A computer-readable non-transitory storage medium storing one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 10.

23. A computer-readable non-transitory storage medium storing one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 11.

24. A computer-readable non-transitory storage medium storing one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 12.

25. A computer-implemented method for visualizing signal data, the method comprising steps of:

categorizing separate portions of the signal data into separate categories of sound of a plurality of categories of sound;

displaying the signal data as a three-dimensional representation;

wherein the dimensions of the representation correspond to dimensions of frequency, amplitude, and time; and

wherein displaying the representation comprises coloring sections of the representation based on categories of sound to which parts of the signal data corresponding to the sections belong;

wherein each category of the plurality of categories is associated with a different color;

wherein said steps are performed by one or more computing devices.

26. The method of claim 25, wherein coloring sections of the representation comprises blending together multiple sound-category-associated colors in proportions that are based on the comparative extents to which sound within a particular section belongs to each of the sound categories, thereby producing a uniform mixed color for that particular section, and filling in the particular section entirely with the uniform mixed color for that particular section.

27. A computer-readable non-transitory storage medium storing one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 25.

28. A computer-readable non-transitory storage medium storing one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 26.