Modular audio control surface

• US 9,952,739 B2
• Filed: 06/03/2013
• Issued: 04/24/2018
• Est. Priority Date: 03/15/2013
• Status: Active Grant

• Current Assignee: Avid Technology Incorporated
• Original Assignee: Avid Technology Incorporated
• Inventors: Milne, Steven H., Kloiber, Martin, Vercellotti, Paul A., Tremblay, David M., Jones, Edward, Boyer, Robert
• Primary Examiner(s): Nguyen, Duc
• Assistant Examiner(s): ELJAIEK, ALEXANDER L
• Application Number:
  US13/908,052
• Publication Number:
  US 20140277647A1
• Time in Patent Office: 1,786 Days
• Field of Search: 84622, 84625, 84634, 84636, 84723, 84725, 84730, 84746, 345102, 345173, 345174, 345184, 3454401, 345501, 345690, 345 82, 381103, 381104, 381107, 381109, 381119, 381120, 381122, 381123, 381150, 381160, 381 17, 381 18, 381 2, 381 22, 381 23, 381300, 381303, 381340, 381 56, 381 58, 381 59, 381 61, 381 63, 381 731, 381 77, 381 92, 381 941, 381 98, 700 94, 715716, 715732, 715733, 715810
• CPC Class Codes: G06F 16/61 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 3/048 Interaction techniques based on graphical user interfaces [GUI]
• CPC Class Codes: G06F 3/162 Interface to dedicated audio devices, e.g. audio drivers, interface to CODECs
• CPC Class Codes: G06F 3/165 Management of the audio stream, e.g. setting of volume, audio stream path
• CPC Class Codes: G06T 11/206 Drawing of charts or graphs

Abstract

A user-configurable modular audio control surface comprises master modules for controlling global surface properties and channel modules for controlling one or more individual audio channels. The modules are disposed in a two-dimensional spatial arrangement such that any module can occupy a location within the control surface not occupied by another module. The modules are connected to each other and to external platforms hosting media applications and plug-ins via a network. Control surface users can interact with external applications via remote graphical user interfaces displayed on modules within the surface, and can automate multiple external applications using an automation system built into the surface. Automation line graphs and metadata for both internal and external applications are displayed over the corresponding waveform displays that can include audio ahead of a current playback location.

Claims

1. A media control surface comprising:

a plurality of modules including a display module, the display module comprising;

a network interface that connects the display module to a network;

an output display for displaying properties of one or more media processing applications hosted on one or more platforms external to the media control surface, wherein each of the one or more platforms is in communication with the media control surface via the network; and

an automation system capable of automating the one or more media processing applications, wherein automating a given one of the one or more media processing applications includes;

at the automation system, receiving control parameter adjustments via one or more input controls of the media control surface, wherein the control parameter adjustments are applied to the processing of media data by the given one of the one or more media processing applications;

time stamping and storing the control parameter adjustments in storage of the automation system;

in response to receiving an automation playback command subsequent to the storing of the control parameter adjustments;

causing logic within the automation system to retrieve the stored control parameter adjustments from the storage of the automation system;

sending the stored control parameter adjustments to the one or more platforms external to the media control surface in synchrony with playback of the media data; and

wherein the media control surface;

receives over the network data comprising a temporal location of a plurality of clips of the media data being processed by the given media processing application; and

uses logic of the display module of the media control surface to generate from the retrieved control parameter adjustments an automation line graph and to temporally align the automation line graph with the temporal location of the plurality of clips of the media data being processed by the given media processing application, thereby enabling the logic of the display module to superimpose the automation line graph onto a display of the temporal location of the plurality of clips, wherein the superimposition of the automation line graph onto the display of the temporal location of the plurality of clips is independent of automation data received over the network from the given media processing application.

2. The media control surface of claim 1, wherein a user is able to select a parameter of one of the one or more media processing applications, and following the selection, the automation system is capable of recording and playing back automation of the selected parameter, wherein the selected parameter is assigned to an input control of one of the plurality of modules.

3. The media control surface of claim 1, wherein the display module:

receives from the automation system time-based automation information from a media processing application hosted by the control surface;

receives over the network time-based automation information for one or more of the one or more externally hosted media processing applications; and

displays on the output display a graphical representation of the time-based automation information for both the media processing application hosted by the control surface and the one or more externally hosted media processing applications.

4. The media control surface of claim 1, wherein:

a first parameter of a given one of the one or more externally hosted media processing applications is automated by the automation system;

the given one of the one or more externally hosted media processing applications includes a second automation system; and

a second parameter of the given one of the one or more externally hosted media processing applications is automated by the second automation system.

5. The media control surface of claim 4, wherein the display module receives automation information for the first and second parameters, and displays a graphical representation of the automation information for the first and second parameters.

6. The media control surface of claim 1, wherein the media control surface receives over the network a graphical audio waveform representing audio data being processed by one or more of the one or more media processing applications, and displays the graphical audio waveform on the output display.

7. The media control surface of claim 1, wherein the media data being processed by one of the one or more media processing applications corresponds to an audio clip.

8. A media control surface module comprising:

a CPU;

a random access memory connected to the CPU;

a network interface for connecting the module to a network;

a display module including an output display for displaying properties of one or more media processing applications hosted on one or more platforms external to the audio control surface, wherein each of the one or more platforms is in communication with the media control surface via the network; and

an automation system, wherein the automation system is capable of automating the one or more media processing applications, and wherein automating a given one of the media processing applications includes;

at the automation system, receiving control parameter adjustments via one or more input controls of the media control surface, wherein the control parameter adjustments are applied to the processing of media data by the given one of the media processing applications;

time stamping and storing the control parameter adjustments in storage of the automation system; and

in response to receiving an automation playback command subsequent to the storing of the control parameter adjustments;

causing logic within the automation system to retrieve the stored control parameter adjustments from the storage of the automation system;

sending the stored control parameter adjustments to the one or more platforms external to the media control surface in synchrony with playback of the media data; and

wherein the media control surface receives over the network data comprising a temporal location of a plurality of clips of the media data being processed by the given media processing applications, and uses logic of the display module to generate a display of an automation line graph generated from the retrieved control parameter adjustments and to temporally align and superimpose the automation line graph onto a display of the temporal location of the plurality of clips of the media data being processed by the given media processing application, wherein superimposing the automation line graph onto the display of the temporal location of the plurality of clips is independent of automation data received over the network from the given media processing application.

9. The media control surface module of claim 8, wherein during playback of the media data, the display module:

receives data describing the recorded control parameter adjustments; and

displays a graphical representation of the control parameter adjustments.

10. A method of automating one or more media processing applications, the method comprising:

receiving via one or more input controls of a media control surface adjustments of a set of control parameters for controlling media data, wherein the control parameter adjustments apply to parameters of one or more media processing applications hosted on platforms external to the media control surface; and

using an automation system hosted by the media control surface;

time stamping and storing the control parameter adjustments in storage of the automation system; and

in response to receiving an automation playback command subsequent to the storing of the control parameter adjustments;

causing logic within the automation system to retrieve the stored control parameter adjustments from the storage of the automation system;

sending the stored control parameter adjustments to the one or more platforms external to the media control surface in synchrony with playback of the media data; and

wherein the media control surface receives from a given one of the one or more media processing applications data comprising a temporal location of a plurality of clips of the media data being processed by the given media processing application, and uses logic of the media control surface to generate from the retrieved control parameter adjustments an automation line graph and to temporally align the automation line graph with the temporal location of the plurality of clips of the media data being processed by the given media processing application, thereby enabling the media control surface to superimpose the automation line graph onto a display of the temporal location of the plurality of clips independently of automation data received from the given media processing application.

11. The method of claim 10, further comprising during playback of the media data:

receiving data describing the recorded control parameter adjustments; and

displaying a graphical representation of the recorded control parameter adjustments.

12. The method of claim 10, further comprising during playback of the media data:

receiving a graphical audio waveform representing audio data being processed by one of the one or more media processing applications; and

displaying the graphical audio waveform.

13. The method of claim 12, wherein the audio data being processed comprises a clip of an audio track.

14. The method of claim 10, wherein the media data comprises a clip of an audio track.