Method and apparatus including system architecture for multimedia communications

• US 6,356,945 B1
• Filed: 08/08/1997
• Issued: 03/12/2002
• Est. Priority Date: 09/20/1991
• Status: Expired due to Term

• Current Assignee: Steven M Shaw, Venson M. Shaw
• Original Assignee: Steven M Shaw, Venson M. Shaw
• Inventors: Shaw, Venson M., Shaw, Steven M.
• Primary Examiner(s): Feild, Joseph H.
• Application Number:
  US08/907,690
• Time in Patent Office: 1,677 Days
• Field of Search: 370/229-237, 345/302, 725/116, 725/119, 386/101, 348/400.1, 348/14.12, 709/219, 709/231-235, 375/240, 375/240.01, 375/240.02, 375/240.12, 375/240.16, 707/500.1
• US Class Current: 709/231
• CPC Class Codes: G06F 9/4887 involving deadlines, e.g. rate based, periodic
• CPC Class Codes: G06T 1/60 Memory management
• CPC Class Codes: G09G 2340/02 Handling of images in compressed format, e.g. JPEG, MPEG
• CPC Class Codes: G09G 3/3611 Control of matrices with row and column drivers
• CPC Class Codes: H04L 47/10 Flow control; Congestion control
• CPC Class Codes: H04L 65/1101 Session protocols
• CPC Class Codes: H04L 65/612 for unicast
• CPC Class Codes: H04L 65/762 at the source reformatting of additional data in video distribution servers H04N21/2355
• CPC Class Codes: H04L 69/18 Multiprotocol handlers, e.g. single devices capable of handling multiple protocols
• CPC Class Codes: H04L 69/32 Architecture of open systems interconnection [OSI] 7-layer type protocol stacks, e.g. the interfaces between the data link level and the physical level
• CPC Class Codes: H04L 9/40 Network security protocols
• CPC Class Codes: H04N 1/00204 with a digital computer or a digital computer system, e.g. an internet server programmed control between transmitter and receiver or between image input and image output device H04N1/32561
• CPC Class Codes: H04N 1/00281 with a telecommunication apparatus, e.g. a switched network of teleprinters for the distribution of text-based information, a selective call terminal details of transmission H04N1/00095; establishing a communication with one of a facsimile machine or another apparatus sharing a single line H04N1/32704; interfacing cordless telephone terminals with an accessory to increase the functionality of user interface H04M1/72409
• CPC Class Codes: H04N 1/00318 with a multi-media apparatus
• CPC Class Codes: H04N 1/00392 Other manual input means, e.g. digitisers or writing tablets
• CPC Class Codes: H04N 1/32502 in systems having a plurality of input or output devices
• CPC Class Codes: H04N 19/107 between spatial and temporal predictive coding, e.g. picture refresh
• CPC Class Codes: H04N 19/132 Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
• CPC Class Codes: H04N 19/164 Feedback from the receiver or from the transmission channel
• CPC Class Codes: H04N 19/172 the region being a picture, frame or field
• CPC Class Codes: H04N 19/176 : the region being a block, e.g. a macroblock
• CPC Class Codes: H04N 19/423 : characterised by memory arrangements H04N19/433 takes precedence
• CPC Class Codes: H04N 19/436 : using parallelised computational arrangements
• CPC Class Codes: H04N 19/46 : Embedding additional information in the video signal during the compression process H04N19/517, H04N19/68, H04N19/70 take precedence
• CPC Class Codes: H04N 19/467 : characterised by the embedded information being invisible, e.g. watermarking
• CPC Class Codes: H04N 19/51 : Motion estimation or motion compensation
• CPC Class Codes: H04N 19/587 : involving temporal sub-sampling or interpolation, e.g. decimation or subsequent interpolation of pictures in a video sequence
• CPC Class Codes: H04N 19/59 : involving spatial sub-sampling or interpolation, e.g. alteration of picture size or resolution
• CPC Class Codes: H04N 19/61 : in combination with predictive coding
• CPC Class Codes: H04N 19/89 : involving methods or arrangements for detection of transmission errors at the decoder
• CPC Class Codes: H04N 21/23614 : Multiplexing of additional data and video streams arrangements for simultaneous broadcast of plural pieces of information H04H20/28
• CPC Class Codes: H04N 21/426 : Internal components of the client ; Characteristics thereof H04N5/44 takes precedence
• CPC Class Codes: H04N 21/4348 : Demultiplexing of additional data and video streams
• CPC Class Codes: H04N 21/43615 : Interfacing a Home Network, e.g. for connecting the client to a plurality of peripherals home Audio Video Interoperability [HAVI] data switching networks H04L12/2805
• CPC Class Codes: H04N 21/438 : Interfacing the downstream path of the transmission network originating from a server, e.g. retrieving encoded video stream packets from an IP network
• CPC Class Codes: H04N 21/4382 : Demodulation or channel decoding, e.g. QPSK demodulation analog front ends or means for connecting modulators, demodulators or transceivers to a transmission line H04L27/0002
• CPC Class Codes: H04N 2201/0081 : Image reader H04N2201/0091 - H04N2201/0094 take precedence
• CPC Class Codes: H04N 2201/0082 : Image hardcopy reproducer H04N2201/0091 - H04N2201/0094 take precedence
• CPC Class Codes: H04N 2201/0084 : Digital still camera
• CPC Class Codes: H04N 2201/0086 : Image transceiver H04N2201/0093 and H04N2201/0094 take precedence
• CPC Class Codes: H04N 2201/0087 : Image storage device
• CPC Class Codes: H04N 2201/0089 : Image display device
• CPC Class Codes: H04N 2201/33357 : Compression mode

Abstract

A system architecture for multimedia communications comprises a bandwidth controller for continuously moderating the run-time status and/or condition changes of the telecommunications network and dynamically adjusting the corresponding bandwidth requirement for accommodating the change of status or condition prior to transmitting the video and/or audio information. The system further includes a reconfiguration memory for standardizing and re configuring the video and/or audio information according to a selective internal file format which is universally compatible with any coding algorithms received from, or supplied to the telecommunications network, the reconfiguration memory performing scalable internal data reformatting among incompatibly received or transmitted video and/or audio information.

Claims

1. A server-based controller, wherein a plurality of client-server entities are connected together through a telecommunications network, a server provides video and/or audio information to a selective one or plurality of its clients, said server-based controller operating a plurality of video and/or audio information production devices based upon video and/or audio information supplied to, or received from a telecommunications network, comprising:

an input/output means for receiving or transmitting video and/or audio information from or to a telecommunications network;

a monitor means connecting to said input/output device for moderating external run-time status or condition of said telecommunications network; and

accommodation means for dynamically controlling or adjusting corresponding transmission bandwidth requirement for said video and/or audio information, wherein said accommodation means determines said transmission requirement according to said external network status or condition, said accommodation means does not determine audio/video transmission ratio according to internal content of the transmission, said accommodation means further dynamically adjust and output a single bit stream for transmission.

2. The server-based controller in accordance with claim 1, further including a reconfiguration means for conforming said video and/or audio information according to a selective internal file format, said reconfiguration means further performing data reformatting for incompatibly received or transmitted video and/or audio information.

3. The server-based controller in accordance with claim 2, further including a memory device for storing video and/or audio information received from or supplied to said telecommunications network or said information production devices conforming to said internal file format, comprising:

processor, memory control or memory management means for transforming an external file format to a generic internal file format, said means further processing said reduced internal file format and exchanging and translating said file internal format to selective external file format.

4. The server-based controller in accordance with claim 1, further including interface means for communication between said controller and said video and/or audio information production devices, said interface means receiving information from said video and/or audio information production devices or transmitting information to said video and/or audio production devices.

5. The server-based controller in accordance with claim 1, further including a processor means connected to said input/output device for processing video and/or audio information supplied to, or received from said input/output means.

6. The server-based controller in accordance with claim 5, further including a motion estimation means, a motion compensation means or a frame differentiator means connecting to said processor.

7. The server-based controller in accordance with claim 5, further including data interchange means for providing video and/or audio data interchange among incompatible codecs or transceivers.

8. The server-based controller in accordance with claim 5, wherein said processor further includes a decoder or an encoder.

9. The server-based controller in accordance with claim 1, wherein said input/output device further includes a channel means for receiving or transmitting audio and/or video information between a source controller and a destination controller of a telecomunications network, comprising:

a signaling or control channel means for transmitting, receiving, or interpreting command, control, and communications message between said source controller and said destination controller;

wherein said means is either in-band or out-of-band, said means can be used as an auxiliary channel for transmitting audio and/or video information when it is not in use; and

scheduling means for said channel means for performing real time conferencing, store and forward, broadcasting, or distribution of said audio and/or video information.

10. The server-based controller in accordance with claim 1, further including a segmentation means connected to said input/output means, wherein said means does not use unused bandwidth to superimpose and accompany additional analog graphics overlay and underlay information, said means decomposing said transmitting audio and/or video information into a selective plurality of overlay and underlay information according to external network condition, said segmentation means includes a means for producing a single or plurality of graphics overlay, a means for producing a single or plurality of text overlay, a means for producing a single or plurality of motion object overlay, a means for producing a single or plurality of still background underlay and a means for producing a single or plurality of audio overlay, a selective one or plurality of said overlays or underlay are transmitted to said video and/or audio information production devices or said telecommunications network.

11. The server-based controller in accordance with claim 10, further including a bandwidth controller for choosing bandwidth or quality of said video information supplied said telecommunications network or bandwidth or quality of said audio information supplied to said telecommunications network according to external network condition.

12. The server-based controller in accordance with claim 11, wherein said bandwidth controller comprising means for automatically choosing bandwidth of said video information or quality of said audio information supplied to the telecommunication network based upon external status or condition of said telecommunications network, said means does not choose transmission ratio according to internal content of the transmission.

13. The server-based controller in accordance with claim 11, wherein said bandwidth controller means includes a means for simulating and annealing randomly distributed noise or distorted audio and/or video information to improve the transmission quality of said telecommunications network or audio and/or video information production device according to external network condition.

14. The server-based controller in accordance with claim 11, further including interpretation means for performing video, audio, and/or graphics animation for improving, supplementing, or compensating quality of audio and/or video information for presentation in an audio/video production device or transmission in a telecommunications network according to external network condition, comprising:

preparation means for preparing a plurality of predetermined information sequence to correspond an anticipative bandwidth or bit rate with a particular external network condition or an external application/user requirement; and

means for storing, retrieving, or transmitting said sequence.

15. The server-based controller in accordance with claim 14, further including a means for automatically selecting a predetermined audio, graphics, and/or video sequence for a particular network condition or a particular application/program requirement, said means further switching to another predetermined sequence when change of requirement or change of network condition take place during a run-time session.

16. The server-based controller in accordance with claim 10, further including a reconstruction means for reassembling, approximating, simulating, or annealing audio, graphics, video, text overlay or underlay for reconstructing or presenting audio and/or video information at a receiver.

17. The server-based controller in accordance with claim 1, further including a video display, a microphone or at least one speaker associated with said audio and/or video information production devices whereby a video and/or audio conference session can be held, said controller directing transmission bandwidth for said audio/video information according to external network condition, said controller does not direct transmission ratio according to internal content of transmission.

18. The server-based controller in accordance with claim 1, further comprising telecommunications network means for wired or wireless data network, telephone networks or interconnections ;

and/or a single or plurality of video and/or audio production means for capturing, storing, retrieving, transmitting, switching, routing, relaying or receiving video and/or audio information.

19. The server-based controller in accordance with claim 1, further performing audio/video on demand service, comprising:

an encoder for storing, accessing, or retrieving program or applications comprising audio and/or video information residing at a customer premise, a central office, a switch, a router, a network, or a database;

receiver means comprising a decoder for receiving or reviewing said applications or program from a remote server to selective one or plurality of local terminal residing at a customer premise, a central office, a switch, a router, a network, or a database; and

means for preparing, transmitting, receiving, or interpreting signaling, command, control and/or communications message between said server and said receiver, said means further receiving or analyzing a customer'"'"'s request or an individual subject of interest;

assessing said network condition;

directing transmission bandwidth for audio/video information according to said external network condition, and providing recommendations to said receiver.

20. The server-based controller in accordance with claim 1, wherein serving as an adjunct to improving feature or performance of its host switching equipment or network, said adjunct reside at a customer'"'"'s premise or next to said switching equipment or network, make ease or speed up multimedia application or service development, deployment or delivery, comprising:

interface means for exchanging bandwidth, protocol, line condition, status, command, control, signaling, or data information between said adjunct and said switching equipment or network;

control means for said switching equipment or network accessing, transmitting, storing, searching, or retrieving multimedia data information from said adjunct; and

disseminating means for said switching equipment disseminating multimedia application or services through a telecommunications network.

21. The server-based controller in accordance with claim 1, further including a media switching system or a set-top controller means for a selective group of audio, video, telephonic, and/or computing apparatus to collaborate, share, exchange, or complement capabilities with one another, comprising:

means for enabling a selective subgroup of said apparatus to be in receive-only, transmit-only, or transmit-and-receive mode;

means for assigning an unique address/identifier for each of said enabled apparatus;

channel means for establishing, maintaining, and terminating a physical or virtual path between a source apparatus and a destination apparatus wherein a multimedia information can be routed from said source to said destination;

signaling means for performing signaling, wherein status, command, control, or communications message can be exchanged between said source and said destination;

input means for receiving media data from a video source including video camera, television, VCR, camcorder, or digital storage, or audio source including stereo, television, microphone, or CD-Audio;

conversion means for digitizing said media data from analog to digital form;

storage means for media data storage;

remote control programming or user interface means; and

host means for executing user, application, or computing/communications tasks.

22. The server-based controller for transmission bandwidth management comprising:

means for decomposing a multimedia information a combination of media objects including a selective plurality of compressed motion video object, still image object, digital coded animated bit map or vector graphic object, digital audio object, and/or text object;

means for selecting an appropriate quality level for said media objects according to relative priority as determined by user, application or network requirement; and

detection means for detecting external network condition and dynamically adjusting transmission bandwidth through selection of compression ratio, frame rate, or display resolution for said multimedia information;

said means does not determine audio/video transmission ratio according to internal content of the transmission.

23. The server-based controller in accordance with claim 22, wherein said controller directing transmission bandwidth according to external network condition, said controller further including a regulator means for automatically reducing media traffic through selective reducing quality level of less prioritized media objects, limiting access of media types, or statistically rerouting congested portion for traffic redistribution.

24. The server-based controller in accordance with claim 22, wherein said controller directing transmission bandwidth according to external network condition, said controller further including a prediction means for recording, accumulating, or analyzing past or present traffic history for determining future communications pattern or possible network condition for preventing traffic congestion.

25. The server-based controller in accordance with claim 22, further comprising look ahead means for predetermining a selective media profile including an appropriate frame rate, display resolution, and compression ratio for directing transmission bandwidth according to said predetermined external network condition, wherein said look ahead means does not perform bus arbitration/synchronization, said look ahead means predict forthcoming media profile or execution steps according to external network condition or bandwidth shortage, said means further direct exception handling when prediction fails.

26. The server-based controller, in accordance with claim 22 for transmission bandwidth management comprising a signaling channel means for transmitting status, command, or control messages between a source controller and a destination controller.