METHOD AND APPARATUS FOR REMOTE REAL TIME COLLABORATIVE MUSIC PERFORMANCE

• US 20030164084A1
• Filed: 03/01/2002
• Published: 09/04/2003
• Est. Priority Date: 03/01/2002
• Status: Active Grant

• Current Assignee: eJamming, Inc.
• Original Assignee: eJamming, Inc.
• Inventors: Glueckman, Alan Jay, Redmann, Willam Gibbens, Kantor, Gail Susan
• Granted Patent: US 6,653,545 B2
• Time in Patent Office: Days
• US Class Current: 84/615
• CPC Class Codes: G10H 1/0058 Transmission between separate instruments or between individual components of a musical system G10H1/0083 takes precedence
• CPC Class Codes: G10H 2240/056 MIDI or other note-oriented file format
• CPC Class Codes: G10H 2240/175 for jam sessions or musical collaboration through a network, e.g. for composition, ensemble playing or repeating; Compensation of network or internet delays therefor
• CPC Class Codes: G10H 2240/271 Serial transmission according to any one of RS-232 standards for serial binary single-ended data and control signals between a DTE and a DCE
• CPC Class Codes: G10H 2240/305 Internet or TCP/IP protocol use for any electrophonic musical instrument data or musical parameter transmission purposes
• CPC Class Codes: G10H 2240/311 MIDI transmission G10H2240/056 takes precedence

Abstract

A method and apparatus are disclosed to permit real time, distributed performance by multiple musicians at remote locations. The latency of the communication channel is transferred to the behavior of the local instrument so that a natural accommodation is made by the musician. This allows musical events that actually occur simultaneously at remote locations to be played together at each location, though not necessarily simultaneously at all locations. This allows locations having low latency connections to retain some of their advantage. The amount of induced latency can be overridden by each musician.

Claims

1. A musical performance station for use by a musician, said station comprising:

a keyboard for the musician to play,

said keyboard generating a local musical event in response to being played by the musician;

a communication channel interface,

said interface providing access through a communication channel to at least one remote musical performance station,

said access to each of the at least one remote musical performance station having an associated latency,

said interface sending the local musical event from the keyboard to the at least one remote musical performance station,

said interface further receiving a remote musical event from the at least one remote musical performance station;

a delay,

said delay having a non-zero local delay value,

said delay receiving the local musical event from the keyboard and holding the local musical event for a first amount of time specified by the local delay value,

said delay further having a remote delay value associated with each of the at least one remote musical performance station,

said delay receiving the remote musical event from the communication channel interface and holding the remote musical event for a second amount of time specified by the remote delay value associated with the remote musical performance station which originated the remote musical event; and

,

a synthesizer for rendering musical events into an audio signal,

said synthesizer receiving the local musical event from the delay when the first amount of time has elapsed, and rendering the local musical event into the audio signal,

said synthesizer receiving the remote musical event from the delay when the second amount of time has elapsed, and rendering the remote musical event into the audio signal.

2. The musical performance station of claim 1, in which the local delay value is set to the greatest latency.

3. The musical performance station of claim 1, in which each remote delay value is set to the greatest latency less the latency associated with the respective remote performance station.

4. The musical performance station of claim 1, in which the local delay value is determined by the musician, and each remote delay value is set to the larger of zero and the local delay value less the latency associated with the respective remote performance station.

5. The musical performance station of claim 4, in which the delay discards the remote musical event when the latency associated with the respective remote performance station exceeds the local delay value.

6. The musical performance station of claim 4, in which the delay discards the remote musical event when the latency associated with the respective remote performance station exceeds the local delay value by more than a threshold value set by the musician.

7. The musical performance station of claims 1, 2, or 3, further comprising:

a groove track,

said groove track having a like groove track on each of the at least one remote musical performance station,

and in which the local musical event represents a command to start the groove track,

said synthesizer rendering the groove track into the audio signal upon receiving the local musical event.

8. The musical performance station of claims 1, 2, or 3, further comprising:

a groove track,

said groove track having a like groove track on each of the at least one remote musical performance station,

and in which the remote musical event represents a command to start the groove track,

said synthesizer rendering the groove track into the audio signal upon receiving the remote musical event.

9. The musical performance station of claim 1, further comprising:

a local clock, and

wherein each of the at least one remote musical performance station has a remote clock,

said local musical event including the value of the local clock at the time the local musical event is generated,

said remote musical event including the value of the respective remote clock at the time the remote musical event is generated,

said remote delay value being calculated as the local delay value less the mean latency associated with the respective remote performance station less the difference between the local clock at the time the remote musical event is received and the value of the respective remote clock contained in the remote musical event plus the difference between the local clock and the respective remote clock.

10. The musical performance station of claim 9, in which the local delay value is set to the greatest latency.

11. The musical performance station of claim 9, in which the local delay value is determined by the musician.

12. The musical performance station of claim 9, 10, or 11, in which the delay discards the remote musical event when the remote delay value is negative.

13. The musical performance station of claim 9, 10, or 11, in which the delay discards the remote musical event when the remote delay value is more negative than a threshold value set by the musician.

14. A system for real time, distributed, musical performance by multiple musicians comprising:

a plurality of the musical performance stations of claim 1, 2, 3, 4, 5, 6, 9, 10, or 11,

wherein the communication channel interface of each of the plurality of musical performance stations separately directs the local musical event to each other one of the plurality of musical performance stations.

15. A system for real time, distributed, musical performance by multiple musicians comprising:

a plurality of the musical performance stations of claim 1, 2, 3, 4, 5, 6, 9, 10, or 11, and

a fanout server,

said fanout server being operatively connected to the communication channel,

wherein the communication channel interface of each of the plurality of musical performance stations directs the local musical event to the fanout server, and

the fanout server forwards each local musical event received from each of the plurality of musical performance stations to each other one of the plurality of musical performance stations.

16. A method for real time, distributed, musical performance by multiple musicians, comprising the steps of:

creating a local musical event,

advancing the local musical event through a communication channel having access to at least one remote location,

said access to each of the at least one remote location having an associated latency,

receiving through the communication channel, from the at least one remote location, a remote musical event,

delaying the local musical event by a non-zero first amount of time,

delaying the remote musical event by a second amount of time associated with the remote location which originated the remote musical event,

playing the local musical event when the first amount of time has elapsed, and

playing the remote musical event when the second amount of time has elapsed.

17. The method of claim 16, wherein the first amount of time equals the greatest latency.

18. The method of claim 16, wherein the second amount of time equals the greatest latency less the latency associated with the remote location which originated the remote musical event.

19. The method of claim 16, in which the first amount of time is determined by the musician, and the second amount of time equals the larger of zero and the first amount of time less the latency associated with the remote location which originated the remote musical event.

20. The method of claim 19, further comprising a step in which the remote musical event is discarded without being played when the latency associated with the remote location which originated the remote musical event exceeds the first amount of time.

21. The method of claim 19, further comprising a step in which the remote musical event is discarded without being played when the latency associated with the remote location which originated the remote musical event exceeds the first amount of time by more than a threshold value set by the musician.

22. The method of claims 16, 17, or 18, in which the local musical event represents a command to start playing a groove track available locally and at each of the at least one remote location.

23. The method of claims 16, 17, or 18, in which the remote musical event represents a command to start playing a groove track available locally and at each of the at least one remote location.

24. The method of claim 16, wherein said remote musical event is augmented with the value of a respective remote clock at the time the remote musical event is generated, and further comprising the steps of:

augmenting the local musical event with the value of a local clock at the time the local musical event is generated, and

calculating said second amount of time as the first amount of time less the mean latency associated with the respective remote location less the difference between the local clock at the time the remote musical event is received and the value of the respective remote clock contained in the remote musical event plus the difference between the local clock and the respective remote clock.

25. The method of claim 24, in which the first amount of time is equal to the greatest latency.

26. The method of claim 24, further comprising the step of:

the first amount of time being set by the musician.

27. The method of claim 24, 25, or 26, further comprising the step of:

discarding the remote musical event when the second amount of time is negative.

28. The method of claim 24, 25, or 26, further comprising the step of:

discarding the remote musical event when the second amount of time is more negative than a threshold value set by the musician.

29. The method claim 16, 17, 18, 19, 20, 21, 24, 25, or 26, wherein the step of advancing the local musical event comprises separately sending the local musical event to each of the at least one remote location.

30. The method claim 16, 17, 18, 19, 20, 21, 24, 25, or 26, wherein the step of advancing the local musical event comprises the steps of:

sending the local musical event to an intermediate location, and

forwarding the local musical event from the intermediate location to each of the at least one remote location.