Collaborative environment for syncronizing audio from remote devices

• US 6,175,872 B1
• Filed: 12/12/1997
• Issued: 01/16/2001
• Est. Priority Date: 12/12/1997
• Status: Expired due to Fees

• Current Assignee: BBNT Solutions LLC (Rtx Corporation)
• Original Assignee: GTE Internetworking Incorporated (Lumen Technologies, Inc.)
• Inventors: Neumann, Eric K., Reider, David J.
• Primary Examiner(s): Matar, Ahmad F.
• Assistant Examiner(s): Barot, Bharat
• Application Number:
  US08/990,122
• Time in Patent Office: 1,131 Days
• Field of Search: 395/559, 348/12-15, 707/100-104, 709/200-203, 709/217-219, 709/223-224, 709/231-233, 709/236-237, 709/248, 710/30-31, 710/33, 370/392-398, 463/40-43
• US Class Current: 709/231
• CPC Class Codes: G10H 1/0066 using a MIDI interface
• CPC Class Codes: H04L 67/131 Protocols for games, networked simulations or virtual reality

Abstract

Apparatus and software systems are disclosed that allow real-time interactive playing of musicians each at remote sites from each other. The remote sites are interconnected over a local communications network or a wide area network like the Internet. The apparatus is connected to music synthesizers that accept streams of MIDI data for playing, or generate streams of MIDI data by playing. The streams of MIDI data are bundled by a MIDI management system into transfer control protocol packets, such that the packets of MIDI data may be transferred over the communications network as or within the transfer control packets. In addition, the networked, remote computer systems are synchronized to a standard clock which is appended to the MIDI data packets, and additionally, the network delay is calculated and also appended to the MIDI data packets. In this manner the local musician can receive the MIDI data packets from the remote sites where musicians are playing and in real-time play with them. The present invention allows other data bases of music to be interrogated and the music therein retrieved for local use, storing and/or editing. The invention allows the received MIDI data from remote sites to be precisely time positioned within the local memory as if on separate tracks like a multi-track tape recorder for later or real-time playing and/or editing.

Claims

1. A method of interactive, synchronized, collaborative real time, playing of MIDI data by MIDI synthesizers and MIDI instruments across an Internet network coupling remote computer systems comprising the steps of:

synchronizing all said remote computer systems to a single clock,

determining a time delay of said Internet network,

creating MIDI data at at least one of the remote computer systems,

time stamping said MIDI data using said clock at the at least one remote computer system,

receiving the MIDI data with said time stamp from the at least one remote computer system, and

applying said time delay to the received MIDI data to synchronize the received MIDI data.

2. The method of claim 1 further comprising the step of:

time ordering said MIDI data in accordance with said time stamping from each remote computer system sending MIDI data.

3. The method of claim 2 further comprising the step of:

sending said time ordered MIDI data to at least one of a synthesizer and a MIDI instrument, the synthesizer and the MIDI instrument playing the MIDI data from all the remote computer systems in correct time synchronization.

10. The method as defined in claim 1 further comprising the steps of:

wrapping said MIDI data having the time stamp with information identifying the MIDI data structure,

creating transfer control packets, and

incorporating said wrapped MIDI data into the transfer control packets for transmission on the network.

11. The method of claim 1 further comprising the steps of:

accessing the MIDI data,

adding information to the MIDI data, said information identifying the MIDI data structure to be sent over said network,

creating transfer control protocol packets,

incorporating said MIDI data into the transfer control packets, and

transmitting said transfer control packets to the network.

12. The method of claim 11 wherein the receiving step includes the substeps of:

receiving the transfer control packets from the network,

identifying the MIDI data structure,

extracting the MIDI data from the transfer control packets, and

creating sound from said MIDI data.

13. The method of claim 11 wherein the receiving step includes the substeps of:

receiving the transfer control packets,

extracting the MIDI data from the transfer control packets, and

storing said MIDI data.

4. Apparatus for interactive, collaborative, real time, playing of MIDI data by MIDI synthesizers and MIDI instruments across a an Internet network coupling remote computer systems, comprising:

means for synchronizing all said remote computer systems to a single clock,

means for determining a time delay of said network,

means for creating MIDI data at at least one of the remote computer systems,

means for time stamping said MIDI data using said clock at the at least one remote computer system,

means for receiving the MIDI data with said time stamp from the at least one remote computer system, and

means for applying said time delay to the received MIDI data to synchronize the received MIDI data.

5. The apparatus defined in claim 4 further comprising:

means for time ordering said MIDI data in accordance with said time stamping from each remote computer system sending MIDI data.

6. The apparatus defined in claim 5 further comprising:

means for sending said time ordered MIDI data to at least one of a synthesizer and a MIDI instrument, the synthesizer and the MIDI instrument playing the MIDI data from all the remote computer systems in correct time synchronization.

7. A method of synchronizing MIDI data packets received from remote computers over a time synchronized internet network, comprising the steps, executed at a local computer, of:

determining a delay time in said network,

receiving MIDI data packets from the remote computers, the MIDI data packets having time stamps indicating a time of creation of said MIDI data packets, and

adding the delay time to the received MIDI data packets to synchronize the received MIDI data packets.

8. Apparatus for synchronizing MIDI data packets received from remote computers over a time synchronized an Internet network, comprising:

means for determining a delay time in said network,

means for receiving MIDI data packets from the remote computers, the MIDI data packets having time stamps indicating a time of creation of said MIDI data packets, and

means for applying the delay time to the received MIDI data packets to synchronize the received MIDI data packets.

9. A method of synchronizing MIDI data received at a computer system over an Internet network coupling the computer system to remote computer systems comprising the steps of:

synchronizing all said computer systems to a single clock,

time stamping MIDI data using said clock as said MIDI data is created at one of the remote computer systems,

determining a time delay of said Internet network,

receiving the MIDI data having said time stamp at the computer system, and applying said time delay to synchronize the received MIDI data.

14. Apparatus for synchronizing MIDI data received at a computer system over an Internet network coupling the computer system to remote computer systems, comprising:

means for synchronizing all said computer systems to a single clock,

means for time stamping MIDI data using said clock as said MIDI data is created at at least one of the remote computer systems,

means for determining a time delay of said network,

means for receiving the MIDI data having said time stamp at the computer system, and

means for applying said time delay to said received MIDI data to synchronize the received MIDI data.

15. The apparatus as defined in claim 14 further comprising:

means for wrapping said MIDI data having the time stamp with information identifying the MIDI data structure,

means for creating transfer control packets, and

means for incorporating said wrapped MIDI data into the transfer control packets for transmission on the network.

16. A networked computer system for playing MIDI data from remote computer systems and from local MIDI apparatuses, the network computer system and the remote computer systems being coupled via a an Internet network, comprising:

means for receiving a network clock providing network time data,

means for synchronizing said networked computer system to the network clock,

means for incorporating the time data with the MIDI data from the local MIDI apparatuses,

means for determining a delay time representing communication delays on said network,

means for incorporating said delay time with said MIDI data from the remote computer systems,

means for organizing said MIDI data from the remote computer systems and from the local MIDI apparatuses into a time ordered sequence using the time data and the delay time, and

means for playing the organized MIDI data.

17. A memory, comprising:

local MIDI data created locally;

transfer control packets including;

remote MIDI data received from the remote sites,

time data including a time of creation of said local and remote MIDI data,

network time delay data including a time delay of the remote MIDI data received from the remote sites, and

information identifying a data structure of the local and remote MIDI data, such that the local and remote MIDI data can be extracted from the data structure for synchronized playing using the time data and the network time delay data.