Modular audio control surface

• US 10,191,607 B2
• Filed: 03/15/2013
• Issued: 01/29/2019
• Est. Priority Date: 03/15/2013
• Status: Active Grant

• Current Assignee: Avid Technology Incorporated
• Original Assignee: Avid Technology Incorporated
• Inventors: Milne, Steven H., Kloiber, Martin, Vercellotti, Paul A., Tremblay, David M., Jones, Edward, Mueller, Karl M., Boyer, Robert
• Primary Examiner(s): Nguyen, Duc
• Assistant Examiner(s): Eljaiek, Alexander L
• Application Number:
  US13/836,456
• Publication Number:
  US 20140281979A1
• Time in Patent Office: 2,146 Days
• Field of Search: 84622, 84625, 84634, 84636, 84723, 84725, 84730, 84746, 345102, 345173, 345174, 345184, 3454401, 345501, 345690, 345 82, 381103, 381104, 381107, 381109, 381119, 381120, 381122, 381123, 381150, 381160, 381 17, 381 18, 381 2, 381 22, 381 23, 381300, 381303, 381340, 381 56, 381 58, 381 59, 381 61, 381 63, 381 731, 381 77, 381 92, 381 941, 381 98, 700 94, 715716, 715732, 715733, 715810
• CPC Class Codes: G06F 16/61 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 3/048 Interaction techniques based on graphical user interfaces [GUI]
• CPC Class Codes: G06F 3/162 Interface to dedicated audio devices, e.g. audio drivers, interface to CODECs
• CPC Class Codes: G06F 3/165 Management of the audio stream, e.g. setting of volume, audio stream path
• CPC Class Codes: G06T 11/206 Drawing of charts or graphs

Abstract

A user-configurable modular audio control surface comprises master modules for controlling global surface properties and channel modules for controlling one or more individual audio channels. The modules are disposed in a two-dimensional spatial arrangement such that any module can occupy a location within the control surface not occupied by another module. The modules are connected to each other and to external platforms hosting media applications and plug-ins via a network. Control surface users can interact with external applications via remote graphical user interfaces displayed on modules within the surface, and can automate multiple external applications using an automation system built into the surface. Automation line graphs and metadata for both internal and external applications are displayed over the corresponding waveform displays that can include audio ahead of a current playback location.

Claims

1. An audio control surface comprising:

a plurality of modules, wherein each module comprises at least one of;

a physical user control comprising one of a switch, knob, and fader;

a touch user control; and

an output display for displaying one or more properties of a media application;

wherein the plurality of modules are disposed in a two-dimensional spatial arrangement of discrete locations, and wherein each of the modules is adapted to occupy one of a set of discrete locations within the control surface not occupied by another module, and wherein each module includes a CPU, random access memory, and a network interface that connects the module to a network; and

wherein a user is able to specify a discrete spatial location for a given module of the plurality of modules within the two-dimensional spatial arrangement of discrete locations by;

on a display of a designated module of the audio control surface, positioning an icon representing the given module on a two-dimensional graphical representation of the audio control surface in accordance with the spatial location of the given module within the two dimensional spatial arrangement of discrete locations; and

in response to a request output by the designated module, adjusting on the given module a physical control comprising one of a switch, a fader, and a knob, thereby causing the given module to output a signal over the network, and wherein the designated module receives the signal from the given module over the network and uses the received signal to tie a network address of the given module to the discrete spatial location of the given module within the two-dimensional spatial arrangement of modules.

2. The audio control surface of claim 1, wherein at least a part of the spatial arrangement conforms to a two-dimensional rectangular grid of discrete locations.

3. The audio control surface of claim 1, wherein at least a part of the spatial arrangement comprises modules disposed along a curved grid of discrete locations.

4. The audio control surface of claim 1, wherein at least one of the plurality of modules comprises a touch screen, and wherein the user controls of the given module comprise a set of virtual controls implemented on the touch screen.

5. The audio control surface of claim 1 wherein one of the plurality of modules comprises a touchscreen interface configured with an expandable touch control for controlling a given functional category, and wherein expanding the touch control with a user-input touch gesture causes a set of virtual touch controls to be displayed, wherein each of the set of virtual controls is assigned to a function related to the given functional category.

6. The audio control surface of claim 1, wherein the given module is assigned to control functionality provided by one of a plurality of media applications, and wherein each of the plurality of media applications is hosted by a different computer-based platform connected to the audio control surface via the network.

7. The audio control surface of claim 1, wherein a user interface of a plug-in operating with a media application hosted on a platform external to the control surface and in communication with the audio control surface via the network is displayed on a display of a second designated module of the control surface.

8. The audio control surface of claim 7, wherein one or more physical controls in physical proximity to the display of the second designated module are assigned to control functions of the plug-in.

9. The audio control surface of claim 1, wherein the audio control surface receives a graphical audio waveform representation of an audio track from a media application external to the audio control surface and in communication with the audio control surface via the network, and displays the graphical audio waveform on a display of a second designated module of the control surface.

10. The audio control surface of claim 9, wherein during playing of the audio track by the media application, the audio control surface receives the graphical audio waveform representation of the audio track corresponding to temporal locations in the audio track in advance of a current playback temporal location in the audio track, and wherein the audio control surface displays the graphical audio waveform corresponding to temporal locations before and after the current playback time of the audio track.

11. The audio control surface of claim 9, wherein the audio control surface receives automation information from the media application and displays a graphical representation of the automation on the display of the designated module of the control surface such that the displayed automation information is temporally aligned with the displayed graphical audio waveform representation of the audio track.

12. The audio control surface of claim 9, wherein the audio control surface receives metadata pertaining to a clip of an audio track that is represented by the graphical audio waveform, and wherein the audio control surface displays the metadata in association with the graphical audio waveform.

13. The audio control surface of claim 12, wherein during playback of the clip, the metadata pertaining to the clip is received by the audio control surface in advance of a current playback time of the clip, and wherein the audio control surface displays the graphical representation of the waveform and its metadata for a temporal interval spanning times before and after the current playback time of the clip.

14. The audio control surface of claim 1, wherein the control surface receives automation information from a media application hosted on a platform external to the control surface, and displays a graphical representation of the automation on the display of a second designated module of the control surface.

15. The audio control surface of claim 14, wherein the automation information pertains to an audio clip, and wherein during playback of the clip by the media application, the graphical representation of the automation information spans temporal locations within the clip before and after a current playback time.

16. The audio control surface of claim 1, wherein the control surface receives metadata pertaining to a clip of an audio track from a media application hosted on a platform external to the control surface, and displays at least one of a graphical and textual representation of the metadata.

17. The audio control surface of claim 16, wherein during playback of the clip by the media application, the displayed representation of the metadata pertains to temporal locations within the clip spanning a current playback time.

18. The audio control surface of claim 1, further comprising an internal automation system, wherein the automation system is capable of automating a media application hosted on a platform external to the audio control surface.

19. The audio control surface of claim 1, further comprising an internal automation system, wherein the automation system is capable of automating a plurality of media applications hosted on one or more platforms external to the audio control surface.

20. The audio control surface of claim 19, wherein a user is able to record and play back automation of a selected function of one of the plurality of media applications, wherein the selected function is assigned to a control of one of the modules of the audio control surface.

21. A module of an audio control surface comprising:

a CPU;

a random access memory connected to the CPU;

a network interface for connecting the module to a network; and

a physical user control comprising one of a switch, a fader, and a knob, wherein a user of the audio control surface is able to use the physical control to control audio-related properties of one or more audio processing applications;

wherein the module is insertable into the audio control surface at any one of a plurality of discrete physical locations within the control surface, wherein the plurality of discrete physical locations are disposed in a two-dimensional arrangement; and

wherein the user is able to specify a discrete spatial location for the module within the two-dimensional arrangement by;

on a display of a second module of the audio control surface, the second module comprising a CPU, a random access memory connected to the CPU of the second module, and a network interface for connecting the second module to the network, positioning an icon representing the first-mentioned module on a two-dimensional graphical representation of the audio control surface in accordance with the spatial location of the first-mentioned module within the two-dimensional arrangement; and

in response to a request output by the second module, adjusting the physical control on the first-mentioned module, thereby causing the first-mentioned module to output a signal over the network, and wherein the second module receives the signal from the first-mentioned module over the network and uses the received signal to tie a network address of the first-mentioned module to the spatial location of the first-mentioned module within the two-dimensional spatial arrangement.

22. The module of claim 21, wherein the two-dimensional arrangement is a rectangular grid of discrete locations.

23. The module of claim 21, wherein the module is adapted to:

receive via the network interface audio track data from a media processing application hosted on a platform external to the audio control surface during playback of the audio track by the media processing application; and

output a representation of the audio track data on the display, wherein the output is based at least in part on a current playback location of the audio track by the media processing application.

24. An audio control surface comprising:

a plurality of hardware modules, wherein each module comprises at least one of;

a physical user control comprising one of a switch, a knob, and a fader, wherein a user of the audio control surface is able to use the physical control to control properties of one or more media processing applications running on a platform external to the audio control surface and in data communication with the audio control surface via a network; and

a touch user control; and

wherein the plurality of hardware modules are disposed in a two-dimensional spatial arrangement of discrete locations, and wherein each of the modules is adapted to occupy one of a set of discrete locations of the two-dimensional spatial arrangement within the control surface not occupied by another module, and wherein each module includes a CPU, random access memory, and a network interface that connects the module to the network; and

wherein the user is able to specify a discrete spatial location for a given module of the plurality of hardware modules within the two-dimensional spatial arrangement by;

on a display of a designated module of the audio control surface, positioning an icon representing the given module on a two-dimensional graphical representation of the control surface in accordance with the spatial location of the given module within the two dimensional spatial arrangement; and

in response to a request output by the designated module, adjusting a physical control of the given module, the physical control comprising one of a switch, a fader, and a knob, thereby causing the given module to output a signal over the network, and wherein the designated module receives the signal from the given module over the network and uses the received signal to tie a network address of the given module to the discrete spatial location of the given module within the two-dimensional spatial arrangement.