Tele-jam system and method for real-time musical interaction

• US 6,636,887 B1
• Filed: 06/01/1999
• Issued: 10/21/2003
• Est. Priority Date: 06/02/1998
• Status: Expired due to Term

• Current Assignee: Mark A. Augeri
• Original Assignee: Mark A. Augeri
• Inventors: Augeri, Mark A.
• Primary Examiner(s): Lim, Krisna
• Application Number:
  US09/323,246
• Time in Patent Office: 1,603 Days
• Field of Search: 709/203, 370/276, 370/285
• US Class Current: 709/203
• CPC Class Codes: H04L 65/1101 Session protocols
• CPC Class Codes: H04L 65/4038 with floor control
• CPC Class Codes: H04L 67/10 in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46
• CPC Class Codes: H04L 67/12 specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
• CPC Class Codes: H04L 69/329 in the application layer [OSI layer 7]

Abstract

The invention provides tele-jam a system and methods for establishing ensembles or groups of participants by a variety of original organizational methods such that the participants interact in real-time (quasi-real-time) in a desired format (strictly predefined format, loosely defined format, or spontaneous format). A host system is provided having servers and client electronic apparatuses connected to the host system by electronic data network. A database of characteristics is stored in the host system, the characteristics including musical details, queries, responses triggered by queries for classifying users based on queried, experimentally detected, or previously stored data including network data transfer delay. Data streams are formed from user'"'"'s input data and placed on queues corresponding to the particular classifications, and groups of arranged users. Algorithms in the system act on the supplied and/or detected information with capability for subsequently switching/bridging together. The data streams (audio signals) from each client apparatus of the formed group(s) are combined and the resultant combined data stream (signal) returned to each client apparatus for achieving real-time full-duplex musical interaction.

Claims

1. A tele-jam method comprising:

providing a host system, connecting the host system to a network of multiple interactive user devices and to multiple servers,

compiling data for storing in and retrieving from the host system,

inputting request by user devices, transferring the request to the host system, classifying the request and maintaining a queue of multiple user requests,

matching the request with other user requests to form data signals,

forming a group of matched requests, switching and bridging together and combining the data signals from each user of the group, and

returning the combined data signals to each user device for achieving real-time full-duplex musical interaction.

2. The method of claim 1, wherein the connecting the network is for connecting single or multiple users.

3. The method of claim 1, wherein the connecting the network is an electronic data network selected from a group comprising circuit switched plain old telephone network, packetized internet protocol (TCP/IP or UDP) network over standard telephone lines, and cable or packetized ATM network.

4. The method of claim 1, wherein the classifying further comprises classifying users based on queried, experimentally detected, or previously stored data including network data transfer delay, placing an instance of a data structure corresponding to a user'"'"'s data on a queue corresponding to particular classifications, and wherein arranging users into groups comprises arranging via specialized algorithms acting on the supplied and/or detected information.

5. The method of claim 1, further comprising querying data of requests from users via an interactive voice response system or web page, or stored data or combinations, including musical characteristics of the user request, finding a desired session or format to join, and determining users physical location, and computing data transfer time delays.

6. The method of claim 5, wherein the format is a predefined format.

7. The method of claim 5, wherein the format is a spontaneous format.

8. The method of claim 5, wherein the musical characteristics comprise instrument, tempo, musical key, and skill levels.

9. The method of claim 1, further comprising prompting the users for selecting descriptive musical characteristics.

10. The method of claim 9, wherein the selecting comprises providing characteristics including instrument, musical key, tempo, skill levels, and matching the selected characteristics within a database of stored information within the host system.

11. The method of claim 1, further comprising automatically analyzing a sample of user inputs and detecting musical characteristics of input signals for frequency, timbre, tempo, musical key, and skills.

12. The method of claim 1, further comprising determining data transfer time delays corresponding to physical distance, network delays and network topology including number of routes through which data must pass, calculating number of potential bottlenecks via a lookup of network topology and experimental network delay test, determining practical latency thereby determining effective network proximity or network latency between the host system and user devices, using the determined latency as a constraint in formation of groups of users.

13. The method of claim 12, wherein the calculating the experimental network delay test comprises notifying user device of network delay test, responding to a server ready for network delay test, sending test data UDP packet including originator network address and time stamp corresponding to time sent to the user device which receives a test packet, reading originator address, and sending packet back to the server, the server receiving response test packet from the user device and marking time received, and computing network latency with the marked time difference.

14. The method of claim 13, wherein the forming groups of participants based network delay or latency comprises connecting participants to a server, determining if a group is formed and if all participants possess a connection with an acceptable network delay as determined by network delay test, and proceeding with the switching/bridging.

15. The method of claim 14, further comprising determining excessive network delay for real-time interaction by the network delay test, placing the participants on a queue, analyzing and assigning to another group or server with which the participant has a lower network delay.

16. The method of claim 14, further comprising determining if participant is unmatched due to network delay, selecting list of alternate servers with anticipated acceptable network delay, and broadcasting data object corresponding to the participant in the queue to a list of alternate servers, operating the alternate servers on the data object, matching the alternate server including network delay test (send/receive UDP packet) with the user device, notifying the matching to the original server for dropping the participant from the queue and broadcasting to remaining alternate servers to drop the participant.

17. The method of claim 16, further comprising having the original server respond to the alternate server which first notified it that it has control of the client, notifying the user device with a network address of the server to be connected, proceeding with the alternate server to match processes and call setup to the user device.

18. The method of claim 17, further comprising notifying the user of a successfully completed match via the server thereby ignoring subsequent notifications by other servers with a busy signal to avoid interruption.

19. The method of claim 18, further comprising switching an identifiable group on any given server within acceptable proximity to each other to a remote server, implementing a second network delay test to verify that the participants are within acceptable proximity with respect to network latency, establishing connections with the remote server to individual participants and user devices and performing signal processing and redistribution.

20. The method of claim 14, further comprising classifying users into groups such that all members of a particular group have an acceptable latency for real-time interaction.

21. The method of claim 1, wherein the classifying comprises classifying the user based on type of matching algorithm used and placing user on master queue for corresponding type of matching algorithm.

22. The method of claim 1, wherein the classifying comprises classifying the user based on instrument type and group forming characteristics and placing user on sub-queues corresponding to a matching algorithm type with at least one major characteristic.

23. The method of claim 22, further comprising programming criteria for being placed on the sub-queues into the host system, automatically selecting predominant group forming characteristics determined by statistical analysis of the characteristics of the users on a master queue of a matching algorithm.

24. The method of claim 1, further comprising driving telephony or communications switching hardware and audio processing subsystems, combining incoming signal data from each user device, redistributing combined signal data to each user device in full-duplex real-time or quasi-real-time manner for all participants to hear all other participants in real-time, selectively including or excluding each participant'"'"'s signal in a return signal representing an ensemble, and selectively including or excluding echo cancellation for achieving real-time musical interaction between the participants corresponding to each user device.

25. A method of real-time full-duplex musical interaction comprising:

providing a host system with plural servers, connecting multiple users to the host system, providing a database of user information in the host system,

inputting data requests from users with varied musical characteristics,

allowing a database algorithm to determine queues of user requests, reading for each participant a code corresponding to a predefined format, checking to see if there is an instance of a particular requested format in the process of being filled,

checking vacancy queue to see if there is a current group of the requested format which has a vacancy which matches the participant under evaluation,

initiating a data structure corresponding to the group if there is no partial group existing, and

adding a first member to an appropriate slot in a new instance.

26. The method of claim 25, further comprising picking up members selecting a desired format, completing an instance format by filling remaining slots, placing the instance format on a partially complete queue if an instance is not complete within a programmable number of passes of the necessary queue(s), passing the instance format to the host system for further processing and switching.

27. The method of claim 26, further comprising setting a flag on a process so that the algorithm cannot use the participant until the replacement algorithm has passed over and checked the participant against current vacancies thereby giving priority to current groups with vacancies.

28. The method of claim 26, further comprising a spontaneous formation algorithm for searching queues of unmatched participants, selecting members on a quasi-random basis, scanning the queue with the spontaneous matching algorithm and randomly selecting a first member of the group, initiating a data structure corresponding to the group, adding the first member to the group, randomly picking up defaults from a preprogrammed list for selecting a second member of the group.

29. The method of claim 28, wherein the picking the defaults comprises analyzing arithmetic and statistical numerical values assigned to characteristics of members already selected and characteristics of prospective participants.

30. The method of claim 29, further comprising repeating the selecting and picking until the instance of the group reaches a preset limit for passing the group data structure to a switching module for grouping user inputs together.

31. The method of claim 25, further comprising searching queues of unmatched participants with a loosely predefined format, selecting participants having descriptive attributes that match to a degree specified in the algorithm, and matching a corresponding open slot in the instance of the format.

32. The method of claim 31, wherein the algorithm searches a pool of users choosing a particular format, selecting participants to fill instances of the predefined formats chosen by default settings and programmed comparisons of participant characteristics to specifications of the selected format in combination with comparisons to the characteristics of group members already selected as determined by analysis of the participants signal'"'"'s or by selections previously prompted from the user(s).

33. The method of claim 31, further comprising choosing a predefined format with desired specifications of ensemble, musical keys and characteristics and designating instrument to be played by the participant.

34. The method of claim 33, further comprising placing participants on a queue for the chosen predefined format, searching database and other queues, selecting participants to fill instances of predefined formats by utilizing programmed comparisons of a participant'"'"'s characteristics to the specifications of the chosen format in combination with comparisons to the characteristics of group members already selected.

35. The method of claim 25, further comprising choosing a strict predefined format, searching queues (pools) of unmatched participants, selecting participants with characteristics exactly defined in the strict predefined format and matching a corresponding open slot in an instance of the format for implementing as a data structure.

36. A tele-jam system comprising a host system, servers connected to the host system, client electronic apparatuses connected to the host system by electronic data network, a database of characteristics stored in the host system, the characteristics including musical details, queries, responses triggered by queries for classifying users based on queried, experimentally detected, or previously stored data including network data transfer delay, data streams formed from user'"'"'s input data, queues corresponding to the particular classifications, groups of arranged users, specialized algorithms in the system for acting on the supplied and/or detected information, and means for subsequently switching/bridging together and combining the data streams (audio signals) from each client apparatus of the formed group(s) and returning the resultant combined data stream (signal) to each client apparatus for achieving real-time full-duplex musical interaction.

37. The system of claim 36, wherein the electronic data network is selected from a group consisting of circuit switched plain old telephone network, packetized internet protocol (TCP/IP or UDP) network over standard telephone lines, cable, and packetized ATM network.

38. The system of claim 36, wherein the client apparatus comprises an interactive host-client communication device, a connection for connecting the client apparatus and the servers via POTS, ISDN, IP over PSTN or Cable TV, the communication device having means for simultaneously sending outgoing signals and for receiving incoming signals of other users of the host system for achieving real-time full-duplex musical interaction.

39. The system of claim 38, wherein the client apparatus further comprises an audio-visual display for displaying information from the server and prompting the user for id/password and IVR prompts for the participants matching characteristics and an input device for inputting desired requests.

40. The system of claim 36, further comprising an interactive voice response system connected to the client apparatus for communicating user input requests including musical characteristics of the user, desired session or format to join, and participants physical location and/or computed data transfer time delays to the host system.

41. The system of claim 36, further comprising musical instruments connectable to the client apparatus for allowing participation in real-time full-duplex musical interaction with other users.

42. The system of claim 36, further comprising a processor for detecting musical characteristics of signals input by users and a computing device for analyzing the signals for musical characteristics including frequency, timbre, tempo, musical key.

43. The system of claim 36, further comprising amplifiers connectable to the client apparatus for amplifying signals received from the host system.