Method and apparatus for multiple media digital communication system

• US 8,116,301 B2
• Filed: 07/10/2006
• Issued: 02/14/2012
• Est. Priority Date: 06/09/1993
• Status: Expired due to Fees

• Current Assignee: RPX Corporation
• Original Assignee: RPX Corporation
• Inventors: Richter, Andreas, Reed, Ogden Cartwright Jr.
• Primary Examiner(s): MARCELO, MELVIN C
• Application Number:
  US11/484,200
• Publication Number:
  US 20060274655A1
• Time in Patent Office: 2,045 Days
• Field of Search: 370/352, 715/753
• US Class Current: 370/352
• CPC Class Codes: H04L 12/18 for broadcast or conference , e.g. multicast
• CPC Class Codes: H04L 12/1813 for computer conferences, e.g. chat rooms instant messaging H04L51/04; protocols for multimedia communication H04L65/1101; arrangements for multi-party communication H04L65/403; telephonic conference arrangements H04M3/56; television conference systems H04N7/15
• CPC Class Codes: H04L 12/1827 Network arrangements for conference optimisation or adaptation
• CPC Class Codes: H04L 12/1881 with schedule organisation, e.g. priority, sequence management
• CPC Class Codes: H04L 12/6418 Hybrid transport
• CPC Class Codes: H04L 12/66 Arrangements for connecting between networks having differing types of switching systems, e.g. gateways
• CPC Class Codes: H04L 2012/6464 Priority
• CPC Class Codes: H04L 2012/6481 Speech, voice
• CPC Class Codes: H04L 2012/6483 Video, e.g. MPEG
• CPC Class Codes: H04L 2012/6489 Buffer Management, Threshold setting, Scheduling, Shaping
• CPC Class Codes: H04L 47/15 in relation to multipoint traffic arrangements for broadcast or multicast in data networks H04L12/18
• CPC Class Codes: H04L 47/22 Traffic shaping
• CPC Class Codes: H04L 47/2416 Real-time traffic
• CPC Class Codes: H04L 47/2433 Allocation of priorities to traffic types
• CPC Class Codes: H04L 47/283 in response to processing delays, e.g. caused by jitter or round trip time [RTT]
• CPC Class Codes: H04L 47/50 Queue scheduling
• CPC Class Codes: H04L 47/56 implementing delay-aware scheduling
• CPC Class Codes: H04L 47/6215 Individual queue per QOS, rate or priority
• CPC Class Codes: H04L 47/6225 Fixed service order, e.g. Round Robin
• CPC Class Codes: H04L 47/624 Altering the ordering of packets in an individual queue
• CPC Class Codes: H04L 47/6285 : Provisions for avoiding starvation of low priority queues
• CPC Class Codes: H04L 49/90 : Buffering arrangements
• CPC Class Codes: H04L 65/1101 : Session protocols
• CPC Class Codes: H04L 65/4038 : with floor control
• CPC Class Codes: H04L 65/611 : for multicast or broadcast systems for broadcast or conference H04L12/18; arrangements for broadcast or distribution combined with broadcast H04H20/00; arrangements for broadcast applications with a direct linkage to broadcast information or to broadcast space-time H04H60/00; selective distribution of broadcast services, e.g. multimedia broadcast multicast service [MBMS], H04W4/06
• CPC Class Codes: H04L 65/752 : adapting media to network capabilities
• CPC Class Codes: H04L 65/80 : Responding to QoS
• CPC Class Codes: H04L 9/40 : Network security protocols
• CPC Class Codes: H04M 2203/2011 : Service processing based on information specified by a party before or during a call, e.g. information, tone or routing selection
• CPC Class Codes: H04M 3/436 : Arrangements for screening incoming calls , i.e. evaluating the characteristics of a call before deciding whether to answer it based on the calling party profile H04M3/42059; based on location H04M3/42348; based on presence H04M3/42365; diversion H04M3/54
• CPC Class Codes: H04M 3/563 : User guidance or feature selection
• CPC Class Codes: H04M 3/567 : Multimedia conference systems
• CPC Class Codes: H04M 7/006 : Networks other than PSTN/ISDN providing telephone service, e.g. Voice over Internet Protocol (VoIP) , including next generation networks with a packet-switched transport layer H04L65/00 takes precedence; aspects not specific to the type of network H04M3/00; special services in those networks H04M3/42
• CPC Class Codes: H04N 21/23614 : Multiplexing of additional data and video streams arrangements for simultaneous broadcast of plural pieces of information H04H20/28
• CPC Class Codes: H04N 21/2368 : Multiplexing of audio and video streams
• CPC Class Codes: H04N 21/43072 : of multiple content streams on the same device
• CPC Class Codes: H04N 21/4341 : Demultiplexing of audio and video streams
• CPC Class Codes: H04N 21/4348 : Demultiplexing of additional data and video streams
• CPC Class Codes: H04N 21/439 : Processing of audio elementary streams
• CPC Class Codes: H04N 21/44004 : involving video buffer management, e.g. video decoder buffer or video display buffer
• CPC Class Codes: H04N 21/4788 : communicating with other users, e.g. chatting arrangements for providing for computer conferences, e.g. chat rooms, to substation in data switching networks H04L12/1813; distributed application using peer-to-peer [P2P] networks H04L67/104
• CPC Class Codes: H04N 21/6375 : for requesting retransmission , e.g. of data packets lost or corrupted during transmission from serverARQ protocols H04L1/18; implementation or adaptation of Internet protocol [IP], of transmission control protocol [TCP] or of user datagram protocol [UDP] H04L69/16
• CPC Class Codes: H04N 21/6583 : Acknowledgement
• CPC Class Codes: H04N 7/147 : Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals selecting H04Q
• CPC Class Codes: H04N 7/15 : Conference systems
• CPC Class Codes: H04N 7/52 : Systems for transmission of a pulse code modulated video signal with one or more other pulse code modulated signals, e.g. an audio signal or a synchronizing signal assembling of a multiplex stream by combining a video stream with other content or additional data, remultiplexing of multiplex streams, insertion of stuffing bits into the multiplex stream, assembling of a packetised elementary stream at server side H04N21/236; disassembling of a multiplex stream, remultiplexing of multiplex streams, extraction or processing of Service Information, disassembling of packetised elementary stream at client side H04N21/434

Abstract

The present invention is embodied in a digital communication system where multiple media data sources are time multiplexed into a packetized data stream, each packet having an assigned priority and the packetized data stream transmitted in substantially the order of assigned priority. At both the transmit side, and the receive side, audio packets are given priority processing over video packets, which in turn have priority over text/graphics data packets. Continuous real time audio playback is maintained at the receiver by delaying the playback of received audio in a first in/first out (FIFO) buffer providing a delay at least equal to the predicted average packet delay for the communication system. Optionally, the average system delay is continuously monitored, and the audio playback delay time is adjusted accordingly. Audio playback is slowed or accelerated in order to shrink or grow the difference in time between the sender and receiver. In another aspect of the invention, a conference of three or more callers is created by broadcasting a common packetized data stream to all conference callers.

Claims

1. An apparatus for communicating with remote processing machines over a network comprising a packet switched network, the apparatus comprising:

a processor;

an interface in data communication with said processor; and

at least one connection routine operative to run on said processor and negotiate, with at least one of the remote processing machines a media type selection from a plurality of media types including audio, video and data, and to configure at least one media routine to process media packets received from and to be transmitted to said at least one of the remote processing machines in a bi-directional communication over said network;

wherein the apparatus is configurable to receive one-way multimedia messages from at least one remote processing machine via said interface; and

one or more separate variable delay buffers for each selected media type, each variable delay buffer for varying a delay provided to a selected media type between the apparatus and the remote processing machines to a selected level.

2. The apparatus of claim 1, wherein at least one of said remote processing machines comprises a telephony device.

3. The apparatus of claim 2, wherein said at least one telephony device is coupled to said apparatus directly or indirectly via a circuit-switched network.

4. The apparatus of claim 1, wherein said apparatus is further adapted to assign a priority to each of said media packets, said media packets being transmitted from said apparatus in substantially said order of said assigned priorities.

5. The apparatus machine of claim 4, wherein said apparatus further comprises a queue for holding said plurality of media packets respectively corresponding to each of said plurality of media types prior to transmission, and said transmission of said plurality of media packets in substantially said order of said assigned priority further comprises:

placing successive ones of said plurality of media packets in said queue in order of said assigned priority; and

transmitting the first media packet of said queue.

6. The apparatus of claim 1, wherein said media packets include video packets and audio packets.

7. The apparatus of claim 1, wherein said media packets include video packets, audio packets, and collaborative data packets.

8. The apparatus of claim 7, wherein said video packets are assigned a lower priority than said audio packets.

9. The apparatus of claim 7, wherein said collaborative data packets are assigned a lower priority than said video packets.

10. The apparatus of claim 1, wherein said one-way multimedia message comprises an audio/video call.

11. The apparatus of claim 1, further adapted to receive control signals comprising one or more messages sent from said at least one connection routine to respective connection routines of the remote processing machines.

12. The apparatus of claim 1, wherein said media type selection involves user interaction.

13. A method of operating an apparatus adapted to communicate with remote processing machines over a network comprising a packet switched network, the method comprising:

providing at least one connection routine operative to run on said apparatus;

using at least said at least one routine, negotiating with at least one of the remote processing machines a media type selection from a plurality of media types including audio, video and data, and configuring at least one media routine to process media packets received from and to be transmitted to said at least one of the remote processing machines in a bidirectional communication over said network;

configuring the apparatus to receive one-way multimedia messages from at least one remote processing machine; and

allocating one or more separate variable delay buffers for each selected media type, each variable delay buffer for varying a delay provided to a selected media type between the apparatus and the remote processing machines to a selected level.

14. The method of claim 13, wherein at least one of said remote processing machines comprises a telephony device.

15. The method of claim 14, wherein said at least one telephony device is coupled to said apparatus directly or indirectly via a circuit-switched network.

16. The apparatus of claim 13, wherein said method further comprises assigning a priority to each of said media packets;

and

transmitting said media packets from said apparatus in substantially said order of said assigned priorities.

17. The method of claim 13,wherein further comprising queuing said plurality of media packets respectively corresponding to each of said plurality of media types prior to transmission, assigning a priority to each of said plurality of media packets, and transmitting said plurality of media packets in substantially an order of said assigned priority by:

placing successive ones of said plurality of media packets in a queue in order of said assigned priority; and

transmitting the first media packet of said queue.

18. The method of claim 13, wherein said media packets include video packets and audio packets.

19. The method of claim 13, wherein said media packets include video packets, audio packets, and collaborative data packets.

20. The method of claim 19, wherein said video packets are assigned a lower priority than said audio packets.

21. The method of claim 19, wherein said collaborative data packets are assigned a lower priority than said video packets.

22. The method of claim 13, wherein said configuring to receive one-way multimedia messages comprises configuring to receive at least one audio/video call.

23. The method of claim 13, further comprising receiving control signals comprising one or more messages sent from said at least one connection routine to respective connection routines of the remote processing machines.

24. The method of claim 13, wherein said media type selection involves user interaction.

25. An apparatus for communicating with a remote processing machine over a network comprising at least a packet switched network, the apparatus comprising:

a processor; and

at least one connection routine operative to run on said processor and accept, modify, or deny a communication attempt from the remote processing machine;

wherein a modification of the communication attempt comprises a media type selection using a media type selection protocol that supports the description of media types including audio, video and data;

wherein the media type selection protocol selects a media packet in a first priority range before selecting any media packet in a second priority range and selects the media packet in the second priority range according to a relative priority of the media packet; and

wherein the apparatus is configurable to negotiate the receipt of a one-way multimedia message.

26. The apparatus of claim 25, wherein the one-way multimedia message is a pre-recorded message file.

27. The apparatus of claim 25, wherein receipt of the one-way multimedia message occurs not in real-time.

28. The apparatus of claim 25, wherein the one-way multimedia message comprises audio and video.

29. The apparatus of claim 25, wherein the one-way multimedia message comprises audio and text data.

30. The apparatus of claim 25, wherein the one-way multimedia message comprises audio and graphics data.

31. The apparatus of claim 25, wherein the one-way multimedia message comprises video and text data.

32. The apparatus of claim 25, wherein the one-way multimedia message comprises video and graphics data.

33. The apparatus of claim 25, wherein the apparatus is configurable to negotiate the receipt of multimedia data for temporary storage and playback of one-way multimedia message.

34. The apparatus of claim 25, wherein the one-way multimedia message is delivered as a multimedia message file and is stored in a memory in the apparatus.

35. The apparatus of claim 25, wherein the one-way multimedia message may be played back by the apparatus.

36. Integrated circuit apparatus adapted to communicate with a remote processing machine via a network apparatus, said integrated circuit adapted for direct or indirect data communication with said network apparatus, comprising:

a digital processor;

a routine running on said processor adapted to negotiate with the remote processing machine via the network apparatus a selection of at least one media type using a media type selection protocol that supports the description of a plurality of media types including audio, video and data, and to cause said network apparatus to be configured, according to said selection, to process media data packets received from and to be transmitted to said remote processing machine over a packet switched or circuit switched network using one or more communication channels, said one or more channels in communication with a single connection stream that includes packets of at least one media stream set up for each of said at least one media type; and

one or more separate variable delay buffers for each of said at least one media type, each variable delay buffer for varying a delay provided to a selected media type between the apparatus and the remote processing machines to a selected level.

37. The integrated circuit apparatus as claimed in claim 36, wherein said media data packets include video packets, audio packets, and collaborative data packets.

38. The apparatus as claimed in claim 36, wherein said media type selection is performed by either a user or by a routine running on said remote processing machine.

39. The apparatus as claimed in claim 36, wherein said one or more communication channels providing indeterminate system delays and bandwidth limitations that give rise to indeterminate packet loss, and said routine is configured to operate in an indeterminate packet loss environment.

40. An integrated circuit for use in a telephonic device, the telephonic device being adapted for signal communication with network apparatus adapted to couple into communication a remote processing machine and the telephonic device, comprising:

a processor;

a routine running on said processor for negotiating, with at least one of the remote processing machine and the network apparatus, a selection of at least one media type from a plurality of media types including audio, video and data, said routine configured to cause the network apparatus to process media data packets received from and to be transmitted to said remote processing machine over a packet switched network;

wherein the routine processes all media data packets in a first priority range before processing any media data packets in a second priority range and processes the media data packets in the second priority range according to a relative priority of the media data packets; and

wherein the integrated circuit is configured to be placed into signal communication with the remote processing machine, said network apparatus configured to rebroadcast said selection via a non-packet switched network.

41. A telephonic device adapted for signal communication with network apparatus, said network apparatus adapted to couple into communication a remote processing machine and the telephonic device, comprising:

an integrated circuit comprising;

a processor; and

a routine running on said processor for negotiating, with at least one of the remote processing machine and the network apparatus, a selection of at least one media type from a plurality of media types including audio, video and data, said routine configured to cause the network apparatus to process media data packets received from and to be transmitted to said remote processing machine over a packet switched network;

wherein the routine processes all media data packets in a first priority range before processing any media data packets in a second priority range and processes the media data packets in the second priority range according to a relative priority of the media data packets; and

wherein the telephonic device is configured to be placed into signal communication with the remote processing machine, said network apparatus configured to rebroadcast said selection via a non-packet switched network.

42. Computerized apparatus adapted to communicate with a remote processing machine via a network apparatus, said computerized apparatus being adapted for direct or indirect data communication with said network apparatus, comprising:

an integrated circuit comprising;

a digital processor;

a routine running on said processor adapted to negotiate with the remote processing machine via the network apparatus a selection of at least one media type using a media type selection protocol that supports the description of a plurality of media types including audio, video and data, and to cause said network apparatus to be configured, according to said selection, to process media data packets received from and to be transmitted to said remote processing machine over a packet switched or circuit switched network using one or more communication channels, said one or more channels in communication with a single connection stream that includes packets of at least one media stream set up for each of said at least one media type; and

wherein the routine processes all media data packets in a first priority range before processing any media data packets in a second priority range and processes the media data packets in the second priority range according to a relative priority of the media data packets.