Composite audio waveforms with precision alignment guides

• US 8,271,872 B2
• Filed: 01/04/2006
• Issued: 09/18/2012
• Est. Priority Date: 01/05/2005
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Salvucci, Keith D.
• Primary Examiner(s): Hong, Stephen
• Assistant Examiner(s): Nazar, Ahamed I
• Application Number:
  US11/325,886
• Publication Number:
  US 20060150072A1
• Time in Patent Office: 2,449 Days
• Field of Search: 715/500, 715/201, 715/202, 715/203, 715/255
• US Class Current: 715/255
• CPC Class Codes: G11B 27/034 on discs G11B27/036, G11B27/038 take precedence
• CPC Class Codes: G11B 27/10 Indexing; Addressing; Timing or synchronising; Measuring tape travel

Abstract

A technique for aligning a plurality of media clips is provided. One or more intra-clip points of interest (POIs) are identified in at least a first media clip. When aligning a first point in the first media clip with a second point in a second media clip, the first point may be snapped to the second point, wherein at least one of the first point and second point is an intra-clip POI. When a snap occurs, at least one of a visual or audible indication is generated, such as a “pop” sound, a snap line, or automatically aligning the first point with the second point when the first point is within a specified number of pixels of the second point. Techniques for representing multiple channels of an audio clip as a single waveform and caching waveforms are also provided.

Claims

1. A method comprising the steps of:

generating a first visual representation of a first media clip;

generating a second visual representation of a second media clip;

automatically analyzing the first media clip to identify a first point within the first media clip, wherein the first point is an intra-clip point of interest (POI) that is neither the beginning of the first media clip nor the end of the first media clip;

after identifying the first point, generating a visual indication of the first point;

receiving input that drags one of the first visual representation and the second visual representation relative to the other of the first visual representation and the second visual representation;

while receiving said input, monitoring the distance between the first point of the first visual representation and a second point of the second visual representation;

in response to detecting that the distance falls below a threshold while one of the first visual representation and the second visual representation is being dragged, automatically shifting one of the first or second visual representations to cause the first point and the second point to be aligned;

wherein the method is performed by one or more computer systems.

2. The method of claim 1, wherein the intra-clip POI of the first media clip corresponds to at least one of the following:

the beginning of silence within the first media clip, the end of silence within the first media clip, and peaks of audio intensity within the first media clip.

3. The method of claim 2, wherein the second point of the second visual representation is either an edge of the second visual representation or an intra-clip POI of the second media clip.

4. The method of claim 1, wherein at least one of the visual representations includes an audio waveform.

5. The method of claim 1, wherein automatically shifting includes automatically shifting when the first point of the first visual representation is within a number of pixels of the second point of the second visual representation.

6. The method of claim 1, wherein the visual indication is generated in response to detecting that the first point is within the particular distance from the second point.

7. The method of claim 6, wherein the color of the visual indication is a different color than the colors of objects immediately around the visual indication.

8. The method of claim 1, further comprising:

generating a third visual representation of a third media clip;

loading the third visual representation; and

in response to loading the third media clip, automatically aligning a third point in the third visual representation with the first point of the first visual representation.

9. A method comprising the steps of:

at a first point in time, a media editing application displaying an audio clip waveform by;

reading audio data from storage;

calculating, based on the audio data, an audio clip waveform, wherein the audio clip waveform represents one or more characteristics of the audio data;

causing the audio clip waveform to be displayed on a screen;

caching the audio clip waveform for subsequent use by the media editing application by;

generating, by the media editing application, a digital image that depicts the audio clip waveform as the audio clip waveform appeared at a particular point in time;

durably storing, by the media editing application and separate from the audio data, the digital image to a persistent storage;

at a second point in time, the media editing application displaying the audio clip waveform by;

in response to receiving input that requires redisplay of the audio clip waveform;

reading the digital image from the persistent storage without calculating the audio clip waveform based on the audio data, and

displaying the audio clip waveform as the audio waveform appeared at the particular point in time by causing the digital image to be displayed via a user interface;

after the audio clip waveform is redisplayed based on the digital image, the media editing application allowing edit operations, to be performed on the audio data, using the redisplayed audio clip waveform;

wherein the method is performed by one or more computer systems.

10. The method of claim 9, further comprising:

receiving second input to perform at least one of resizing or cropping of the digital image; and

resizing or cropping the digital image based on the second input.

11. The method of claim 9, wherein the digital image is generated during a first session of the media editing application used to generate the digital image, and the method further comprising:

in response to receiving second input, exiting the media editing application, wherein all processes associated with the media editing application are terminated;

in response to receiving third input, initiating a second session of the media editing application;

receiving fourth input to load the audio waveform;

reading, from the persistent storage, the digital image that represents the audio clip waveform; and

displaying the digital image via the user interface.

34. The method of claim 9, further comprising, while the digital image is displayed, re-calculating the audio clip waveform based on the audio data.

12. One or more storage media storing instructions which, when executed by one or more processors, cause the performance of:

generating a first visual representation of a first media clip;

generating a second visual representation of a second media clip;

analyzing the first media clip to identify a first point within the first media clip, wherein the first point is an intra-clip point of interest (POI) that is neither the beginning of the first media clip nor the end of the first media clip;

after identifying the first point, generating a visual indication of the first point;

receiving input that moves one of the first visual representation and the second visual representation relative to the other of the first visual representation and the second visual representation;

while receiving said input, when the first point of the first visual representation is within a particular distance from a second point of the second visual representation, automatically shifting one of the first or second visual representations to cause the first point and the second point to be aligned.

13. The one or more storage media of claim 12, wherein the intra-clip POI of the first media clip corresponds to at least one of the following:

the beginning of silence within the first media clip, the end of silence within the first media clip, and peaks of audio intensity within the first media clip.

14. The one or more storage media of claim 13, wherein the second point of the second visual representation is either an edge of the second visual representation or an intra-clip POI of the second media clip.

15. The one or more storage media of claim 12, wherein at least one of the visual representations includes an audio waveform.

16. The one or more storage media of claim 12, wherein automatically shifting includes automatically shifting when the first point of the first visual representation is within a number of pixels of the second point of the second visual representation.

17. The one or more storage media of claim 12, wherein the visual indication is generated in response to detecting that the first point is within the particular distance from the second point.

18. The one or more storage media of claim 17, wherein the color of the visual indication is a different color than the colors of objects immediately around the visual indication.

19. The one or more storage media of claim 12, wherein the instructions are instructions which, when executed by the one or more processors, further cause:

generating a third visual representation of a third media clip;

loading the third visual representation; and

in response to loading the third media clip, automatically aligning a third point in the third visual representation with the first point of the first visual representation.

20. One or more storage media storing instructions which, when

executed by one or more processors, cause the performance of:

at a first point in time, a media editing application displaying an audio clip waveform by;

reading audio data from storage;

calculating, based on the audio data, an audio clip waveform, wherein the audio clip waveform represents one or more characteristics of the audio data;

causing the audio clip waveform to be displayed on a screen;

caching the audio clip waveform for subsequent use by the media editing application by;

generating, by the media editing application, a digital image that depicts the audio clip waveform;

durably storing, by the media editing application and separate from the audio data, the digital image to a persistent storage;

at a second point in time, the media editing application displaying the audio clip waveform by;

in response receiving input that requires redisplay of the audio clip waveform;

reading the digital image from the persistent storage without calculating the audio clip waveform based on the audio data, and displaying the audio clip waveform as the audio waveform appeared at the particular point in time by causing the digital image to be displayed via a user interface;

after the audio clip waveform is redisplayed based on the digital image, the media editing application allowing edit operations, to be performed on the audio data, using the redisplayed audio clip waveform.

21. The one or more storage media of claim 20, wherein the instructions are instructions which, when executed by the one or more processors, further cause:

receiving second input to perform at least one of resizing or cropping of the digital image; and

resizing or cropping the digital image based on the second input.

22. The one or more storage media of claim 20, wherein the digital image is generated during a first session of the media editing application used to enerate the digital image, and wherein the instructions are instructions which, when executed by the one or more processors, further cause:

in response to receiving second input, exiting the media editing application, wherein all processes associated with the media editing application are terminated;

in response to receiving third input, initiating a second session of the media editing application;

receiving fourth input to load the audio waveform;

reading, from the persistent storage, the digital image, that represents the audio clip waveform; and

displaying the digital image via the user interface.

35. The one or more storage media of claim 20, wherein the instructions, when executed by the one or more processors, further cause, while the digital image is displayed, re-calculating the audio clip waveform based on the audio data.

23. An apparatus comprising:

one or more processors;

one or more storage media storing instructions which, when executed by the one or more processors, cause the performance of;

generating a first visual representation of a first media clip;

generating a second visual representation of a second media clip;

analyzing the first media clip to identify a first point within the first media clip, wherein the first point is an intra-clip point of interest (POI) that is neither the beginning of the first media clip nor the end of the first media clip;

after identifying the first point, generating a visual indication of the first point;

receiving input that moves one of the first visual representation and the second visual representation relative to the other of the first visual representation and the second visual representation;

while receiving said input, when the first point of the first visual representation is within a particular distance from a second point of the second visual representation, automatically shifting one of the first or second visual representations to cause the first point and the second point to be aligned.

24. The apparatus of claim 23, wherein the intra-clip POI of the first media clip corresponds to at least one of the following:

the beginning of silence within the first media clip, the end of silence within the first media clip, and peaks of audio intensity within the first media clip.

25. The apparatus of claim 24, wherein the second point of the second visual representation is either an edge of the second visual representation or an intra-clip POI of the second media clip.

26. The apparatus of claim 23, wherein at least one of the visual representations includes an audio waveform.

27. The apparatus of claim 23, wherein automatically shifting includes automatically shifting when the first point of the first visual representation is within a number of pixels of the second point of the second visual representation.

28. The apparatus of claim 23, wherein the visual indication is generated in response to detecting that the first point is within the particular distance from the second point.

29. The apparatus of claim 28, wherein the color of the visual indication is a different color than the colors of objects immediately around the visual indication.

30. The apparatus of claim 23, wherein the instructions are instructions which, when executed by the one or more processors, further cause:

generating a third visual representation of a third media clip;

loading the third visual representation; and

in response to loading the third media clip, automatically aligning a third point in the third visual representation with the first point of the first visual representation.

31. An apparatus comprising:

one or more processors;

one or more storage media storing instructions which, when executed by the one or more processors, cause performance of;

at a first point in time, a media editing application displaying an audio clip waveform by;

reading audio data from storage;

calculating, based on the audio data, an audio clip waveform, wherein the audio clip waveform represents one or more characteristics of the audio data;

causing the audio clip waveform to be displayed on a screen;

caching the audio clip waveform for subsequent use by the media editing application by;

generating, by the media editing application, a digital image that depicts the audio clip waveform;

durably storing, by the media editing application and separate from the audio data, the digital image to a persistent storage;

at a second point in time, the media editing application displaying the audio clip waveform by;

in response receiving input that requires redisplay of the audio clip waveform;

reading the digital image from the persistent storage without calculating the audio clip waveform based on the audio data, and displaying the audio clip waveform as the audio waveform appeared at the particular point in time by causing the digital image to be displayed via a user interface;

after the audio clip waveform is redisplayed based on the digital image, the media editing application allowing edit operations, to be performed on the audio data, using the redisplayed audio clip waveform.

32. The apparatus of claim 31, wherein the instructions are instructions which, when executed by the one or more processors, further cause:

receiving second input to perform at least one of resizing or cropping of the digital image; and

resizing or cropping the digital image based on the second input.

33. The apparatus of claim 31, wherein the digital image is generated during a first session of an application used to generate the digital image, and wherein the instructions are instructions which, when executed by the one or more processors, further cause:

in response to receiving second input, exiting the media editing application, wherein all processes associated with the media editing application are terminated;

in response to receiving third input, initiating a second session of the media editing application;

receiving fourth input to load the audio waveform;

reading, from the persistent storage, the digital image that represents the audio clip waveform; and

displaying the digital image via the user interface.

36. The apparatus of claim 31, wherein the instructions, when executed by the one or more processors, further cause, while the digital image is displayed, re-calculating the audio clip waveform based on the audio data.