Data processor having controlled scalable input data source and method thereof

• US 6,026,097 A
• Filed: 03/13/1997
• Issued: 02/15/2000
• Est. Priority Date: 03/13/1997
• Status: Expired due to Term

• Current Assignee: Rovi Technologies Corporation (Adeia Inc.)
• Original Assignee: 8X8, Inc.
• Inventors: Voois, Paul Augustine, Joe, Truman, Bednarz, Philip Stanley, Andrews, Barry Dean
• Primary Examiner(s): Hsu, Alpus H.
• Assistant Examiner(s): Qureshi, Afsar M.
• Application Number:
  US08/815,966
• Time in Patent Office: 1,069 Days
• Field of Search: 348/423, 348/14, 348/17, 375/222, 370/465, 370/468, 370/522, 370/537, 370/538, 370/487, 370/490, 370/477, 370/442, 395/200.65, 395/200.62
• US Class Current: 370/468
• CPC Class Codes: H04N 21/2187 Live feed
• CPC Class Codes: H04N 21/2343 involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements video transcoding H04N19/40; media packet handling at the source H04L65/762
• CPC Class Codes: H04N 21/23614 Multiplexing of additional data and video streams arrangements for simultaneous broadcast of plural pieces of information H04H20/28
• CPC Class Codes: H04N 21/2368 Multiplexing of audio and video streams
• CPC Class Codes: H04N 21/2385 Channel allocation H04N21/266 takes precedence; Bandwidth allocation H04N21/24 takes precedence ; allocation of channels according to the instantaneous demands of the users in time-division multiplex systems H04J3/1682; admission control, resource allocation in open networks H04L12/5692; arrangements for maintenance or administration in data switching networks involving bandwidth and capacity management H04L41/0896; negotiating bandwidth in wireless networks H04W28/16
• CPC Class Codes: H04N 21/2402 Monitoring of the downstream path of the transmission network, e.g. bandwidth available traffic monitoring in data switching networks H04L43/00; monitoring data switching networks utilization H04L43/0876
• CPC Class Codes: H04N 21/2662 Controlling the complexity of the video stream, e.g. by scaling the resolution or bitrate of the video stream based on the client capabilities
• CPC Class Codes: H04N 21/4788 communicating with other users, e.g. chatting arrangements for providing for computer conferences, e.g. chat rooms, to substation in data switching networks H04L12/1813; distributed application using peer-to-peer [P2P] networks H04L67/104
• CPC Class Codes: H04N 21/6137 involving transmission via a telephone network, e.g. POTS
• CPC Class Codes: H04N 7/52 Systems for transmission of a pulse code modulated video signal with one or more other pulse code modulated signals, e.g. an audio signal or a synchronizing signal assembling of a multiplex stream by combining a video stream with other content or additional data, remultiplexing of multiplex streams, insertion of stuffing bits into the multiplex stream, assembling of a packetised elementary stream at server side H04N21/236; disassembling of a multiplex stream, remultiplexing of multiplex streams, extraction or processing of Service Information, disassembling of packetised elementary stream at client side H04N21/434

Abstract

A multimedia communication system processes and multiplexes different types of data, including data from an adaptive data rate data source and a nonadaptive data rate data source, to substantially increase data throughput over a communication channel. The system includes: a first data source, responsive to a control signal, including a video image processor constructed to capture images and to present the images as a first type of data at a rate determined as a function of the control signal; at least one additional data source generating at least one additional data signal; a data signal processor that determines an available bandwidth factor for the communication channel, generates the control signal in response to this factor, collects the first type of data at a rate that varies in response to the available channel bandwidth of the modem, and collects the at least one additional type of data at at least one established rate. A communication channel modem transmits the data to a receiving terminal as the data is presented by the data signal processor.

Claims

1. A multimedia communication system for transferring video and other signal types from an image collecting terminal to a modem for transmission over a communication channel to a data recovery terminal, comprising:

a first data source, responsive to a control signal, including a video image processor constructed to capture images and to present the images as a first type of data at a rate determined at least in part as a function of the control signal;

at least one additional data source generating at least one additional data signal;

a data signal processor configured and arranged to determine an available bandwidth factor for the communication channel, to generate the control signal in response to the available bandwidth factor, to collect the first type of data from the first data source at a rate that varies in response to an available channel bandwidth of the modem, and to collect at least one additional type of data from the at least one additional data source at at least one established rate; and

the modem responsive to the data signal processor and arranged to transmit the first and at least one additional types of data to the data recovery terminal.

2. A multimedia communication system, according to claim 1, wherein the data signal processor generates the control signal as a function of a sum of an amount of the first type of data transmitted during a previous time interval and the available channel bandwidth factor.

3. A multimedia communication system, according to claim 1, further including a prediction filter responsive to the control signal and coupled to the first data source.

4. A multimedia communication system, according to claim 3, wherein the prediction filter generates an output that is received by the first data source to adjust the rate at which the images are presented.

5. A multimedia communication system, according to claim 3, wherein the prediction filter is constructed and arranged to determine an average available bandwidth over a plurality of update periods as a function of the control signal.

6. A multimedia communication system, according to claim 3, wherein the prediction filter is implemented using a first order infinite impulse response filter.

7. A multimedia communication system, according to claim 1, wherein the data signal processor is implemented using at least one of the following:

a programmable digital signal processor chip, a general-purpose processor, a reduced instruction set computer, and an application-specific integrated circuit.

8. A multimedia communication system, according to claim 1, wherein the data signal processor is implemented using a first data buffer arranged to receive data from the first data source, and at least one additional data buffer arranged to receive data from the at least one additional data source.

9. A multimedia communication system, according to claim 8, further including a modem data buffer configured to receive the first and at least one additional types of data from the data signal processor.

10. A multimedia communication system, according to claim 1, further including a modem data buffer configured to receive the first and at least one additional types of data from the data signal processor.

11. A multimedia communication system, according to claim 1, wherein the video image processor is implemented using a video camera.

12. A multimedia communication system, according to claim 1, wherein the data signal processor is further configured and arranged to generate a set of data for transmission over the communication channel as a function of the first and at least one additional types of data and as a function of one preselected format of a set of preselected formats, the preselected formats specifying amounts of data to be collected from the data sources and a sequence in which data is to be collected from the data sources.

13. A multimedia communication system, according to claim 12, wherein at least one of the preselected formats specifies the first data source last in the sequence and the amount of data collected from the first data source varies in response to the amount of data collected from the at least one additional data source.

14. A multimedia communication system, according to claim 12, wherein the preselected formats are stored as entries in a table.

15. A multimedia communication system, according to claim 12, wherein the preselected formats are assigned priority values.

16. A multimedia communication system, according to claim 15, wherein the data signal processor selects the one of the set of preselected formats at least in part as a function of the priority values of the preselected formats.

17. A multimedia communication system, according to claim 15, wherein the priority values of the preselected formats are determined as a function of at least one of the following:

a quantity of data sources referenced by the preselected formats, a priority ranking of the data sources, and quantities of data drawn from the referenced data sources.

18. A multimedia communication system, according to claim 15, wherein the data signal processor selects the preselected format as a function of at least one of the following:

a quantity of data sources referenced by the preselected formats for which a corresponding type of data is available for transmission, and quantities of data drawn from the referenced data sources.

19. A multimedia communication system, according to claim 1, wherein the data signal processor collects the first and at least one additional types of data using a linked list.

39. A multimedia communication system, according to claim 1, wherein the control signal indicates an amount of retransmission occurring in the communication channel.

40. A multimedia communication system, according to claim 39, wherein the video image processor responds to the control signal by encoding video data at a rate that is a function of the amount of channel bandwidth used by retransmission.

20. A multimedia communication terminal, comprising:

a first data source, responsive to a control signal, including a video image processor constructed to capture images and to present the images as a first type of data at a rate determined at least in part as a function of the control signal;

at least one additional data source generating at least one additional data signal of at least one additional type; and

a data signal processor configured and arranged to determine an available bandwidth factor for the communication channel, to generate the control signal in response to the available bandwidth factor, to collect the first type of data from the first data source at a rate that varies in response to the available channel bandwidth of the modem, and to collect the at least one additional type of data from the at least one additional data source at least one established rate, the first and at least one additional types of data being collected for transmission from the terminal.

21. A multimedia communication terminal, according to claim 20, wherein the video image processor includes a video camera.

22. A multimedia communication terminal, according to claim 20, wherein one of the at least one additional data source is arranged to provide audio data at an established rate.

23. A multimedia communication terminal, according to claim 20, wherein the data signal processor generates the control signal as a function of an amount of the first type of data transmitted during a previous time interval and the available channel bandwidth factor.

24. A multimedia communication terminal, according to claim 20, further including a prediction filter, responsive to the control signal, for averaging the control signal over a plurality of update periods.

25. A multimedia communication terminal, according to claim 24, wherein the prediction filter is implemented using a first order infinite impulse response filter.

26. A multimedia communication terminal, according to claim 24, wherein the prediction filter is implemented as part of the data signal processor.

27. A multimedia communication terminal, according to claim 26, wherein the data signal processor is implemented using at least one of the following:

a programmable digital signal processor chip, a general-purpose processor, a reduced instruction set computer, and an application-specific integrated circuit.

28. A multimedia communication terminal, according to claim 27, wherein the data signal processor is implemented using a first data buffer arranged to receive data from the first data source, and at least one additional data buffer arranged to receive data from the at least one additional data source.

29. A multimedia communication terminal, according to claim 28, wherein the data signal processor stores the first and at least one additional types of collected data in an output data buffer before the collected data is transmitted.

30. A multimedia communication terminal, according to claim 29, wherein the output data buffer is implemented as part of the data signal processor.

31. A multimedia communication terminal, according to claim 20, wherein one of the at least one additional data signal represents audio information.

32. A multimedia communication terminal, according to claim 31, wherein one of the at least one additional data source generates a packet of the one of the at least one additional type of data every 30 milliseconds.

33. A multimedia communication terminal, according to claim 20, wherein the data signal processor is further configured to fit packets of the first type of data and of the at least one additional type of data into predetermined packet forms, the packet forms being selected according to the type and amount of data collected by the data signal processor over a period of data collection.

34. A multimedia communication terminal, according to claim 33, wherein the predetermined packet forms are stored as entries in a table.

35. A multimedia communication terminal, according to claim 33, wherein the data signal processor selects the packet forms at least in part as a function of priority values assigned to the packet forms, the priority values being determined as a function of at least one of the following:

a quantity of data sources referenced by the packet form, a priority ranking of the data sources, and quantities of data drawn from the referenced data sources.

36. A multimedia communication terminal, according to claim 33, wherein the data signal processor selects the packet forms as a function of at least one of the following:

a quantity of the data sources referenced by each of the packet forms for which a corresponding type of data is available for transmission, and quantities of data drawn from the referenced data sources.

37. A multimedia communication terminal, according to claim 20, wherein the data signal processor collects the first and at least one additional types of data using a linked list.

41. A multimedia communication terminal, according to claim 20, wherein the control signal indicates an amount of retransmission occurring in the communication channel.

42. A multimedia communication terminal, according to claim 41, wherein the video image processor responds to the control signal by encoding video data at a rate that is a function of the amount of channel bandwidth used by retransmission.

38. A method for formatting multiple types of data, including data representing images, for transmission over a communication channel, the method comprising:

receiving a first type of data using a first data source at a data rate determined at least in part by a control signal and at least one additional type of data using at least one additional data source at at least one established data rate;

using a data processor to determine an available bandwidth factor for the communication channel;

using the data processor to generate the control signal in response to the available bandwidth factor;

varying the data rate of the first type of data in response to the control signal by filtering the control signal using a prediction filter; and

formatting the first and at least one additional types of data for presentation to a output port according to one of a plurality of preselected formats by using a multiplexer arrangement to construct a data packet using data drawn from one or more of the first and at least one additional data sources.

43. A method for a multimedia communication system, according to claim 38, wherein the control signal indicates an amount of retransmission occurring in the communication channel.

44. A method for a multimedia communication system, according to claim 43, further including responding to the control signal by encoding video data at a rate that is a function of the amount of channel bandwidth used by retransmission.