Synchronized display and performance mapping of musical performances submitted from remote locations

• US 9,857,934 B2
• Filed: 06/16/2014
• Issued: 01/02/2018
• Est. Priority Date: 06/16/2013
• Status: Active Grant

• Current Assignee: Jammit, Inc.
• Original Assignee: Jammit, Inc.
• Inventors: Humphrey, Scott, Gryner, Frank
• Primary Examiner(s): FLETCHER, MARLON T
• Application Number:
  US14/306,224
• Publication Number:
  US 20150046824A1
• Time in Patent Office: 1,296 Days
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G10H 1/0058 Transmission between separate instruments or between individual components of a musical system G10H1/0083 takes precedence
• CPC Class Codes: G10H 1/368 displaying animated or moving pictures synchronized with the music or audio part
• CPC Class Codes: G10H 2210/091 for performance evaluation, i.e. judging, grading or scoring the musical qualities or faithfulness of a performance, e.g. with respect to pitch, tempo or other timings of a reference performance
• CPC Class Codes: G10H 2220/106 using icons, e.g. selecting, moving or linking icons, on-screen symbols, screen regions or segments representing musical elements or parameters
• CPC Class Codes: G10H 2240/171 Transmission of musical instrument data, control or status information; Transmission, remote access or control of music data for electrophonic musical instruments details about the transmitted data contents G10H2240/011
• CPC Class Codes: G10H 2240/175 for jam sessions or musical collaboration through a network, e.g. for composition, ensemble playing or repeating; Compensation of network or internet delays therefor
• CPC Class Codes: H04L 67/10 in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46

Abstract

Systems and methods are provided for assembling and displaying a visual ensemble of musical performances that were created and uploaded from one or more locations that are remote from a host of the network, a director or other administrator reviewing submissions for selection and assembly, or perhaps merely remote from one or more other submissions received over a computer network. The assembled performances include a plurality of submissions, the submissions including performances created and uploaded at one or more locations remote from the location of the director for the assembly and display over the computer network. Systems and methods are also included for mapping one performance against another performance qualitatively, quantitatively, in real-time, or some combination thereof, enabling a musician, or a reviewer of performances, in the assessment of one performance relative to another performance.

Claims

1. A system for creating an ensemble of a set of remotely-performed and remotely-uploaded musical performances through a computer network, the system comprising:

a processor operably connected to a memory on a non-transitory computer readable medium;

a receiving module operable for receiving audio data from a musical submission and a second musical submission on a non-transitory computer readable medium;

an integration engine embodied in a non-transitory computer readable medium, wherein the integration engine is operable to interface the timing of a musical submission with the timing a second musical submission using a timing reference track that includes a tempo map in the creation of an ensemble; and

a display module operable for displaying the ensemble;

wherein the timing reference is selected from the group consisting of;

a click-track;

a drum track;

a select reference track of a select reference performance, , the select reference performance having a series of sound frequencies, each of the frequencies performed at a particular time in the select reference performance;

a residual audio track representing a multi-track digital audio file of a musical work having the select reference performance removed, the residual audio track having a series of sound frequencies, each of the frequencies performed at a particular time in the residual audio track; and

,

a combination thereof.

2. The system of claim 1, wherein the timing reference used by the integration engine is one track in each of a series of multi-track digital audio files available upon request by each of a plurality of musicians;

each of the series containing a shared musical work, highlighting a select reference performance on a select reference track within the musical work, and including

(i) T reference performances within the musical work, wherein T=Σ

_i; and

, _i=₁, ₂, ₃, . . . , or . . . _T;

(ii) a select reference track of the select reference performance, _i, the select reference performance having a series of sound frequencies, each of the frequencies performed at a particular time in the select reference performance;

(iii) a residual audio track representing the multi-track digital audio file having the select reference performance removed; and

,

(iv) instructions for execution of the timing reference by the processor.

3. The system of claim 1, further comprising

an input device operable to receive video data on a non-transitory computer readable medium;

a video engine embodied in a non-transitory computer readable medium, wherein the video engine is operable to transform input video data to output video data as a video recording of a musical submission for display in a visual ensemble;

an output module embodied in a non-transitory computer readable medium, wherein the output module is operable to transmit the integrated audio data, along with the video data, to an output device, the output device operable to display the visual ensemble.

4. The system of claim 1, further comprising a guidance module on a non-transitory computer readable medium and operable as a guidance system including a timing comparison tool to compare the musical submission to the select reference performance, the guidance system providing a feedback that includes a qualitative or quantitative measure of the timing comparison.

5. The system of claim 1, further comprising a guidance module on a non-transitory computer readable medium and operable as a guidance system including a frequency comparison tool to compare the musical submission to the select reference performance, the guidance system providing a feedback that includes a qualitative or quantitative measure of the frequency comparison.

6. The system of claim 2, further comprising a guidance module on a non-transitory computer readable medium and operable as a guidance system including a frequency comparison tool and a timing comparison tool to compare the musical submission to the select reference performance, the guidance system providing a feedback that includes a qualitative or quantitative measure of the frequency comparison and the timing comparison.

7. The system of claim 2, further comprising a dynamic guidance module on a non-transitory computer readable medium and operable as a dynamic guidance system including a frequency comparison tool and a timing comparison tool to provide a continuous comparison of the musical submission with the select reference performance from the beginning of the performance to the end of the performance, or any section therein, the dynamic guidance system providing a qualitative or quantitative measure of the frequency comparison and the timing comparison.

8. The system of claim 2, further comprising a dynamic guidance module on a non-transitory computer readable medium and operable as a dynamic guidance system, the guidance system including a frequency comparison tool and a timing comparison tool for each of the plurality of musicians to continuously compare their performance of the musical submission with the select reference performance while they perform, the dynamic guidance system providing a real-time feedback that includes a qualitative or quantitative measure of the frequency comparison and the timing comparison, the real-time feedback being no slower than 30 milliseconds from the occurrence of a respective transient or note performed in the musical submission and used for comparison to the select reference performance.