Private multimedia network

• US 20050254440A1
• Filed: 05/05/2004
• Published: 11/17/2005
• Est. Priority Date: 05/05/2004
• Status: Abandoned Application

• Current Assignee: John D. Sorrell
• Original Assignee: John D. Sorrell
• Inventors: Sorrell, John D.
• Application Number:
  US10/838,365
• Publication Number:
  US 20050254440A1
• Time in Patent Office: Days
• US Class Current: 370/264
• CPC Class Codes: H04L 65/1101 Session protocols
• CPC Class Codes: H04L 65/4038 with floor control
• CPC Class Codes: H04L 65/611 for multicast or broadcast systems for broadcast or conference H04L12/18; arrangements for broadcast or distribution combined with broadcast H04H20/00; arrangements for broadcast applications with a direct linkage to broadcast information or to broadcast space-time H04H60/00; selective distribution of broadcast services, e.g. multimedia broadcast multicast service [MBMS], H04W4/06
• CPC Class Codes: H04Q 2213/13248 Multimedia
• CPC Class Codes: H04Q 2213/13332 Broadband, CATV, dynamic bandwidth allocation
• CPC Class Codes: H04Q 2213/13337 Picturephone, videotelephony
• CPC Class Codes: H04Q 2213/1336 Synchronisation
• CPC Class Codes: H04Q 2213/13383 Hierarchy of switches, main and subexchange, e.g. satellite exchange
• CPC Class Codes: H04Q 2213/13384 Inter-PBX traffic, PBX networks, e.g. corporate networks
• CPC Class Codes: H04Q 2213/13389 LAN, internet

Abstract

Private Multimedia Network (PMN) complements, and is an improved alternative to digital videoconferencing and multimedia delivery systems. PMN'"'"'s desktop and meeting room delivery system is designed to support the exponential growth of enterprise team-based initiatives. PMN provides “one-stop-shopping” for the full multimedia rubric. It delivers user-friendly control and cost/effective TV and broadcast quality videoconferencing and other multimedia services to organizations with “critical mass” campuses and building complexes. Though digital systems dominate the videoconferencing marketplace, PMN'"'"'s hybrid digital/analog architecture has no digital peer in breadth or quality of service within or between campuses. The novel architecture leverages advances in analog video short-haul technology, digital long-haul technology, and telephony audio and control technology to deliver four-level multimedia services: 1) premise; 2) campus; 3) multi-site; and 4) ubiquitous (any site with ITU compatible multimedia equipment (e.g., videoconferencing) and communication links). On balance, the price/performance afforded by PMN'"'"'s centralized Telco-based control and audio delivery combined with its decentralized broadcast quality video distribution raise videoconferencing and other multimedia services to a new level of ubiquity. Just as telephones and PC LANs, PMN delivers expensive Boardroom and mobile cart videoconferencing capabilities to every desktop via existing multimedia wall plates. The key phases for this invention are: Lip-synchronization across differing network communication links and protocols; Ubiquitous multimedia service; Cost/effective room and desktop deployment; Telco control and audio; Broadcast quality video; Isochronous Quality; Centralized control and distributed operation; and Interoperable architecture.

Claims

1. An interoperable multimedia architecture that combines hardware, software, and communications technologies that together provide intuitive system management, synchronize diverse real-time and non-real-time transmission means, and deliver isochronous NTSC TV quality or better video and lip-synchronized audio and video to and between a plurality of enterprise end-user terminals and devices that are participating in multimedia sessions between a plurality of enterprise and foreign site participants.

2. The method of claim 1, wherein the Private Multimedia Network architecture and components thereof conform to industry multimedia standards (e.g., ITU, IETF, ISO, MPEGIF), thereby providing worldwide industry interoperability between both Enterprise (inside the Enterprise) and Foreign (outside the Enterprise) multimedia products. Products that adhere to these standards allow users to participate in multimedia sessions (e.g., videoconferencing) regardless of their platform. The ITU has developed the H, G, and T series standards and the IETF has developed Real-Time Protocol (RTP), Real-Time Control Protocol (RTCP) and Resource Reservation Protocol (RSVP). For example, ITU multimedia standards include:

Transport Protocols (TCP, UDP, RTP), Transport Media (ISDN, LAN, WAN, Internet, ADSL, VPN), ISDN (H.320), and LAN, Internet, VPN, and ADSL (H.323).

3. The claim 1 architecture comprises the following definitions for terms, end-user venues, site types, connectivity, and service levels:

(a) “

administrator session”

is defined as a set of management control and system management actives to govern PMN operation. Administrator sessions are 2-levels with corresponding security clearances;

a) policy setting to insure that resources are obtained, protected, and used effectively and efficiently in the accomplishment of the organization'"'"'s goals (e.g., budget and service conformance reporting, “

transfer priced”

services, long distance call restrictions, logon scenarios requiring not only USER IDs and passwords, but also project codes;

Priority overrides for emergencies and senior management imperatives. security policy);

b) system management (e.g., system configuration and control table maintenance, and use of Network Manager to monitor and manage the PMN Network.;

(b) “

business rule book”

is defined as the central repository for PMN business rules. It is structured (table driven) to capture and maintain relevant enterprise management control rules that govern the deployment, use, billing, and security of resources. For example, it identifies the multimedia devices that individuals are allowed to use (e.g., only equipment in their cubicle or office). Are there resources that an individual cannot use (e.g., Continuous Presence Multiviewer Switches)? At log-on, must an individual enter a User ID, password, and project code? Is there a priority system, and levels of “

shut-down”

for emergencies (e.g., President overrides other end-users)? Can an individual be denied access because of budget overrun (billed for system use)?

(c) “

camera consolidation sites”

are defined as consolidation points for real-time baseband collection of surveillance camera video from dispersed proximity camera locations that are re-transmitted at the site using broadband real-time service (e.g., fiber, wireless) to enterprise premise PMN Premise Switching Centers for distribution throughout the enterprise;

(d) “

codec”

, an abbreviation of “

coder/decoder”

, is defined as a device or program capable of performing transformations on data streams or signals. Codecs can both put the stream or signal into an encoded form (e.g,, digital for transmission, storage or encryption) and retrieve, or decode that form for viewing or manipulation in a format more appropriate for the intended operation (e.g., analog). Codecs used for videoconferencing and streaming media solutions often service “

non-real-time”

communication links. These multimedia data streams contain both audio and video data, and some form of metadata that permits lip-synchronization of audio and video between end-point codecs. Videoconferencing vendors (e.g., Tandberg, Polycom) have developed sophisticated algorithms and methodologies to overcome intra-stream, inter-stream, and end-point to end-point latency, and network jitter (jerky video and audio loss) to achieve end-point to end-point audio and video synchronization;

(e) “

codec farm”

is defined as a collection of rack mounted or shelf stacked codecs located at a enterprise site PMN Premise Switching Center;

(f) “

continuous presence video”

(CPV) is defined as a split screen display of multiparty session participants (3 or more) that is continuously shown on participant terminals. Local Call continuous presence engines are video Multiviewers at each site, working in combination with Cross Point Switches, that enable the simultaneous display of multiple video sources in real time;

e.g., 4 inputs to 1 output with 4 quadrant display. Just as video Multiviewers, MCUs are Long Distance Call continuous presence engines. Multiviewers are intra-site and MCUs are inter-site continuous presence engines;

For Local Calls, Multiviewers are needed for 3 or more participants;

For Long Distance Calls, MCUs are required for 3 or more sites, and Multiviewers are required for sites with greater than 1 participant. For example, multiparty collaboration between 2 sites requires no MCU. However, sites with more than 1 participant require Multiviewers;

(g) “

cross-point switches”

(e.g., PESA, Ademco, Extron) also referred to as “

matrix switchers”

are defined as Multimedia Switches that route multiple inputs to multiple outputs route any input to any output, or multiple outputs, at any time. Just as with telephone systems, the switch manages the movement of multimedia information both between nodes within the premise (end-user-terminals and devices), and between nodes in the premise with external nodes (enterprise and foreign sites). Other sites are connected by either interlocked Cross-Point Switches (real-time trunk lines connecting input and output nodes on both site'"'"'s Cross-Point Switches), or via non-real-time codecs and multipoint control units (e.g., IP or ISDN). Under Network Management System (NMS) management, cross-point switches control dynamic switching of shared PMN resource (e.g., Continuous Presence Engines, Audio Mixers, Data Recorders, Codecs, On-demand Servers, and end-user appliances) during multimedia sessions. Internally, the switcher consists of a series of distribution amplifiers and switchers, housed in a single enclosure and controlled by remote or front panel controllers. They are capable of routing a variety of NTSC/PAL and broadcast quality audio/video signal types, including;

composite video, S-video, HDTV/component video, RGsB/RGBS/RGBHV video, stereo audio (balance/unbalanced). To meet differing application requirements, input/output configurations can be symmetrical or unbalanced (different number of inputs and output. Switchers are modular and can be coupled (inputs and outputs) to increase input or output capacity (e.g., increase end-user or surveillance camera input connections with “

blocked”

service) or provide limited service to multiple locations (e.g., Homeland Security Backup Center). “

Blocked”

service is a shared output path that can only be used by one input at a time (NMS manages resource sharing).

Multimedia nodes are predefined. As shown by example in FIG. 5, the low-end nodes are reserved for shared resources (e.g., trunk lines, on-demand servers, audio and video bridges, codecs, and data recorders (use for Home Land Security and not shown on FIG. 5), and the upper nodes are used for transceivers that connect via the Multimedia LAN to end-user nodes. Multimedia Switch capacity is determined by end-user node requirements and shared resource input and output requirements;

(h) “

device manager”

is defined as the Network Management System (NMS) function that manages the operation of PMN devices through a network of Premise Control Units. Device Manager uses the existing enterprise telephone and IP networks to deliver remote control commands to Premise Control Units. Premise Control Units are configured and controlled by a variety of protocols (e.g., ARP, UDP, TCP, TFTP, ICMP, HTTP, SNMP, DHCP and Telnet and even telephone call to wake-up the device (e.g., Telephone Hybrid)). Control commands conform to vendor hardware APIs and use device appropriate network interface (e.g., RJ45, DB-25), and serial interfaces (e.g., RS232, RS422, RS485). There are many products in the marketplace (e.g., Lantroniox, Digi Connectware) that satisfy PMN Control Unit requirements. The following premise hardware is controlled by Premise Control Units;

end-user appliances, PMN Premise Switching Centers (on-demand servers, Multimedia Switches, Surveillance Cameras and Data Recorders, Codec Farms, and Telephone Hybrids), and Multipoint Controls;

(i) “

end-user locations”

are defined as desktops, meeting rooms, executive suites and boardrooms, Emergency Response Center Amphitheaters, and other enterprise premise and campus venues, and foreign end-users location, which are outside the enterprise;

(j) “

end-user terminals”

are defined as enterprise end-user audio/video appliances (e.g., telephone, display, camera) used by end-users to request and receive multimedia services. telephone audio (input and output), video output (e.g., TVs, video projectors, and PC internal or external video TV tuners and monitors), and video input (e.g., document and video cameras). Existing telephones, and typically existing video displays are used. An optional embodiment is a mix of telephone out-of-band audio means and in-band audio and video means. In this configuration end-user terminals include, but are not limited to;

TVs, PCs, video projectors (audio and video output);

document and video cameras (video input);

TV tuners and monitors (video output);

amplified speakers (audio output); and

telephones (audio input and output). End-user terminals do not have dedicated or embedded codecs, multipoint control units, gatekeepers, and gateways. PCs are optional;

(k) “

enterprise”

is defined as one or more public or private sector organizations that operate as a single entity in their use of the Private Multimedia Network (PMN), and “

foreign”

is defined as an organization or entity that is outside the “

enterprise”

;

(l) “

enterprise campus sites”

are defined as locations within the premise complex that are served by interlocked Multimedia Switches (i.e., Cross-point Switches) connected by Multimedia LANs that deliver full multimedia service (services between nodes on interlocked cross-point switches are referred to as “

Local Calls”

);

(m) “

enterprise geographically dispersed sites”

are defined as locations that are linked by non-real-time codecs (e.g., IP, ISDN) that deliver full multimedia service (services between these sites are referred to as “

Long Distance Calls”

);

(n) “

enterprise premise sites”

are defined as the physical locations for end-users and Private Multimedia Network equipment (as shown in FIG. 2, End-user Participant Nodes, Multimedia LANs, PMN Premise Switching Center);

(o) “

enterprise proximity sites”

are defined as locations served by interlocked Multimedia Switches (i.e., Cross-point Switches) connected by real-time trunk lines (e.g., fiber, wireless) that deliver full multimedia service (services between nodes on interlocked cross-point switches are referred to as “

Local Calls”

);

(p) “

foreign sites”

are defined as locations outside the enterprise that collaborate (videoconference) or provide or receive other multimedia services (camera video) to/from enterprise sites, and receive services that are constrained by limitations imposed by their multimedia system (e.g., codec capabilities) and enterprise management control policy, which are enforced by the PMN control system (services between these sites are referred to as “

Long Distance Calls”

);

(q) “

host-directed-video”

(HDV) is defined as host directed switching of multimedia participant displays to a specific participant, similar to the chair of a meeting giving a participant the floor. The new speaker continues to view the last speaker. The host uses the telephone keypad or Internet-based human/computer interface to signal the Session Manager to switch screens. The telephone keypad is the preferred embodiment. VSV and HDV/PDV are the preferred embodiments;

(r) “

long-haul”

, also referred to as “

Long Distance Call”

, is defined as between served by a “

non-real-time”

transmission means;

(s) “

multimedia session”

is defined a set of multimedia service delivery processes and nested sub-processes with a discrete beginning and end, governed by business rules that control and coordinate activities and resource acquisition, use, and cost across time and place to produce specified outputs. The content delivered by such a process includes information that supports collaboration, decision-making, learning, and appeals to multiple senses, such as text, sound, video, graphics, and possibly, in future, tactile and olfactory feedback. Constrained by the scope and governance of multimedia service types, end-users specify schedule, participants, services, resources, and session operational rules. Enterprise management specifies management control rules and resource deployment. Brief multimedia sessions include videoconferencing, voicemail, media on demand, and advertising. Medium-length multimedia sessions include distance learning, telemedicine, emergency response collaboration, broadcasting, and administration. Long multimedia sessions include emergency response and security monitoring;

(t) “

multipoint control unit”

, often shortened to “

MCU”

, is defined as a device or program that eestablishes multimedia calls between three or more end-point codecs for converged voice, video and data conferences. MCUs essentially creates a point-to-point videoconference with each endpoint within a given conference, and uses sophisticated software and hardware to combine these inputs into a shared environment very like a physical meeting space. They also overcome latency and jitter to achieve end-point-to-end-point synchronization. Often referred to as a bridge, an MCU can provide audio-only services or any combination of audio, video and data, depending on the capabilities of each participant'"'"'s end-point codec. Though some MCUs transcode between protocols (e.g., IP, ISDN) and provide gatekeeper services, the preferred embodiment is to use gateways to transcode between protocols and gatekeepers to address mapping and bandwidth management. The determinant of degree of device specialization is determined by price/performance, which changes over time;

(u) “

multipoint control unit farms”

are located in the PMN Control Center and are composed of rack mounted or shelf stacked Multipoint Control Unit (e.g., RadVision) bridging and switching devices managed by the Network Management System (NMS) software;

(v) “

multiviewers”

are defined as Multimedia Switches that, just as Multipoint Control Units (MCUs), combine multiple video inputs into a single, full motion, full color, “

continuous presence”

windowed output. They deliver a variety of NTSC/PAL and broadcast quality signals. Windowing can be fixed and programmable, including a built-in generator of source identification and border colors, real-time clock and date. Multiviewers occupy fixed positions on Cross Point Switches, and are either preset to perform specified functions or function controlled (e.g., RS232, RS422) by the Network Management System (NMS);

(w) “

network management system”

(NMS) is defined as the means by which the PMN Control Center controls system setup, operation, and maintenance at the direction of end-users, under the governance of the enterprise management control structure. NMS orchestrates execution of system sub-functions that perform defined management and operational control duties that span the PMN life cycle;

System Startup;

Session Manager provides a structured dialogue that collects management control information from the System Administrator, which establishes business rules, and configures the PMN architecture, resources, and services to conform to enterprise requirements (refer to “

multimedia session” and

“

session manager”

). For example, the Telco/multimedia devices that individuals are allowed to use (e.g., only equipment in cubicles, office);

equipment that most employees cannot use (e.g., Multiviewers for “

Hollywood Squares”

(continuous presence, conferences). Log-on scenarios requiring not only USER IDs and passwords, but also project codes;

Priority overrides for emergencies and senior management imperatives. PMN'"'"'s table driven structure facilitates customization and ongoing maintenance.

Operational Control;

Session Manager facilitates end-user session management, under the governance of enterprise management control constraints;

     1) Before sessions, Scheduler together with Resource Manager book participants, determine and record required services, and reserve resources;

     2) During sessions, Session Manager together with Device and Resource Manager, startup (including security screening and resource acquisition), operate (including orchestration, interrupt handlers, and control of information flow) sessions;

     3) End-of-sessions, Session Manager tears-down session (including releasing resources). Refer to “

Scheduler”

, “

Resource Manager”

, and “

Device Manager”

.

NMS is event-driven software. It provides real time service to events and multi-user changes of state. NMS handles all device interrupts;

interrupts are specific to devices, system interrupts (e.g., scheduler, resource manager), and end-user signaling. To this end, the following are examples of NMS interrupt handlers;

codec control, video switch control, on demand server control, media control, video bridge control, audio bridge control, audio mixer control, continuous presence (video matrix Multiviewers) multimedia switch control, scheduler control, active speaker control, host and participant signaling control, etc.

NMS'"'"'s multimedia control system is built upon an audio conferencing platform that is a scalable, open, interoperable architecture that conforms to industry-standards (i.e., Signal Computing Systems Architecture, Scbus.) Refer to “

Telco Audio Bridge”

. NMS'"'"' Human/Computer Interface (HCI) is intuitive, uses the telephone, and mirrors the telephony conference paradigm. For example, to establish a conference call without scheduling, the meeting host uses the telephone to;

Dial the PMN Control Center for videoconferencing service;

Dial the 1^st part and connect;

Hit the plunger; and

continue the process until all parties are connected; and

hit the plunger twice to start the conference. To simplify the calling process, PMN uses enterprise telephone numbers;

therefore, there is only one telephone book. For more complex multimedia sessions and services (e.g., scheduling, selection of meeting formats, special resources, help assistance), NMS Scheduler provides Interactive Voice Response (IVR) to guide the end-user through the process. For example;

videoconferencing session formats include;

     1) Point-To-Point or Multi-Point (Continuous Presence or Voice or Host Switched Video (Participant Request for Floor));

     2) Listener Control (e.g., permission to audit);

     3) Open or Closed Door Meetings (e.g., permission to join, entry rules when started, ad hoc initiations);

     4) Local and Far-end Camera Control;

     5) Special Equipment Requirements (e.g., document camera);

     6) Special Software (e.g., PowerPoint, Whiteboard, Internet). Telephony-based HCI is the preferred embodiment for Systems Operation. However, there are situations requiring complex schedules and resources that would be better served by the IP version of the HCI. The IP version is the preferred embodiment for System Startup and Management. System Administrators are more adept at using Internet-based system;

(x) “

non-real-time”

is defined as communication means (e.g., bridges, switches, communication links, transmission) that are not immediate and are beset by latency and jitter. Non-real-time communication link and transmission means include IP, Ethernet, ISDN, and MPEG. Enterprise geographically dispersed sites have “

non-real-time”

communication means. Most private sector enterprise inter-site communication links are “

non-real-time”

;

(y) “

participant-directed-video”

(PDV) is defined as signaling that requests the host to give a participant the floor. The host uses the telephone keypad or Internet-based human/computer interface to signal the meeting host (refer to 2 (n) for further discussion). The telephone keypad is the preferred embodiment. VSV and HDV/PDV are the preferred embodiments;

(z) “

phone book”

is defined as a central directory of enterprise PMN end-users, and the scope of services and resources they can select. The Phone Book is structured for dynamic update. To facilitate intuitive operations and maintenance, end-users are given their enterprise phone numbers or extensions. Speed dial service is also provided to reduce keystrokes. Resources are related to end-users (owners) as well as physical location (e.g., premise and room) and relationship to other resources (e.g., hardwiring of resources to the Multimedia Switch, or either pool or direct relationship between Codecs and Telephone Hybrids). Ownership implies control (e.g., desktop telephone). Shared Resources are owned by the enterprise. The Phone Book reflects the standard relationships between people and resources. However, PMN allows ad hoc, temporal relationships to be created (e.g., scheduling a future conference using a different location (not the participant'"'"'s office) and a different telephone. Generally, the new location and telephone and other end-user resources required for the session will be recorded in the Phone Book. However, if they are not and the Business Rule Book permits it, they can be placed in a temporal section of the Phone/Resource book. A section of the Phone Book is also set-aside for frequently called numbers outside the enterprise (Foreign entities that frequently engage in PMN collaboration sessions with Enterprise staff);

(aa) “

PMN control center”

is defined as a central office that performs 3 major functions for enterprise sites;

PMN system control, Telephony-based audio server, and Inter-site multiparty video communications middleman. The Control Center can be located inside or outside (e.g., service bureau) the Enterprise. Just as Telco Central Office switching equipment joins subscribers'"'"' lines for connecting subscribers to each other and controls end-to-end connectivity, the PMN Control Center components provides similar services;

     1) Internet and telephony-based end-user management and operational control;

     2) telephony-based audio bridging services;

     3) multipoint video and audio switching and bridging services via Multipoint Control Units (MCUs) and site codecs; and

     4) Network Management System software and hardware (Network Manager (e.g., gatekeepers, gateways, QOS, Public Network and Internet connectivity), Resource Manager, Devise Manager, Session Manager, and Scheduler);

(bb) “

real-time”

is defined as communication means (e.g., bridges, switches, communication links, and transmission) that are immediate and provide synchronized audio and video. Real-time communication link and transmission means include telephony, wireless and fiber analog and broadband digital, twisted pair analog. Enterprise premise, campus, and proximity sites have “

real-time”

communication means. Public sector Municipal fiber loop inter-site connectivity is an example of proximity sites (e.g., school districts and government facilities);

(cc) “

resource manager”

is defined as the Network Management System (NMS) function that manages PMN resources. Resource Manager keeps track (maintains a calendar and resource inventory) of the location and disposition of all system hardware and communication facilities. Most importantly, it manages “

shared”

devices (e.g., Multimedia Switches, Premise Codec Farms, Telephone Hybrid Farms, On-demand Servers). For example, if an enterprise configures a PMN system with “

blocked ports”

on Cross Point Switches, Resource Manager has to keep track of session using the “

gateway”

path (“

nailed-down”

circuit) between switch modules (granular switch), and between differing inputs and outputs in an unbalanced switch (e.g., 160 input versus 128 outputs). Blocking is used to allow more ports to be connected to a switch than can be serviced simultaneously. It is based on the assumption that the system will rarely be fully utilized. Other shared resources require similar “

share”

management. If resources are not available, the end-user is given a “

busy”

signal.

During session scheduling, Resource Manager determines all resources required during the session, and reserves them (including end-to-end circuits) from the start date and time to projected session end. This includes both immediate and future scheduled sessions. During session startup, Resource Manager determines that all resources needed to support the session are available;

(dd) “

scheduler”

is defined as the Network Management System (NMS) function responsible for scheduling and maintaining the schedule for Multimedia Sessions. Scheduler, combined with Resource Manager, Phone and Business Rules Books, and real-time telephony-based (Telco Client) and Internet-based (IP Client) tools allow end-users to schedule sessions and participants, and reserve resources (e.g., devices, end-to-end circuits, and meeting rooms). Sessions can be “

immediate”

or “

future”

, and “

recurring”

or “

one-time”

. Session participants can either be confirmed by the Scheduler, or by the session host (end-user). The Scheduler uses Telco IVR and IP resources, as appropriate. For “

future scheduled”

meetings, Scheduler sends the host and participants a follow-up email outlining facts about the meeting;

(ee) “

session manager”

is defined as the Network Management System (NMS) function responsible for session start-up, operation, and teardown services. At session start-up, session manager insures that all scheduled and changed resources are available via Resource Manager, participants are properly identified, directed, and mentored (off-line and online), and that meeting protocol (e.g., “

open” and

“

closed”

door meetings) and security is adhered to;

During sessions, continuing mentoring, as needed, and together with NMS and other NMS functions (e.g., Device Manager) responsible for servicing interrupt requests (e.g., telephone keypad signals to Session Manager and between participants), managing changes of state (e.g., new participants), and overall orchestration of session resource use and conformance to meeting protocol;

At tear-down, responsible for servicing requests for time extension, enforcing requested, timed, and emergency shut-down procedures, release of session resources via Resource Manager, and creating accounting records for the billing of session services rendered to end-users. In the preferred embodiment, end-user billing is facilitated by telephony control (telephone keypad and IVR) and telephony-based audio, which is central to session management and service delivery;

(ff) “

short-haul”

, also referred to as “

Local Call”

, is defined as transmission within a premise or campus;

or between sites served by “

real-time”

communication means (e.g., transcontinental “

proximity”

sites served by broadband fiber);

(gg) “

synchronization”

is defined as end-to-end isochronous video and lip synchronized audio and video, with differences imperceptible to the human eye and ear, between a plurality of Enterprise end-user terminals participating in multimedia sessions via “

real-time”

or a mix of “

real-time” and

“

non-real-time”

communication means;

(hh) “

telco audio bridge”

also referred to as “

Telephony Server”

is defined as the PMN central office that delivers telephony services to enterprise sites. The computer-based server is composed of cards that fit into computer chassis that serve as a PBX and Network Management System (NMS) control software. A Private Branch Exchange (PBX) is a privately owned, mini version of a telephone company'"'"'s central office (CO) switch. The advantage of a PBX is the efficiency and cost gains of sharing a specific number of telephone lines among a large group of users. The real-time, multi-party cards (e.g., NMS) support over 500 seats (multiple cards are placed in a single chassis), over 100 ports, and digital trucking. Multiple cards can be placed in a computer chassis. Though not designed for video conferencing, commercial telephony bridges contain programmable API call control features that facilitate implementation of collaboration applications (e.g., videoconferencing, distance learning);

     1) create and delete a conference;

     2) active talker status (capability to determine which participant is talking at a given time);

     3) coaching mode (the ability to selectively control which conference members can hear chosen participants without the knowledge of other conference members);

     4) echo cancellation (prevents disturbing feedback and echoes);

     5) data logging (recording full-duplex conference calls);

     6) IVR (Interactive Voice Response for management and operational control end-user dialogue-based system settings);

     7) Call control setup and tear-down;

     8) Real-time faxing, and analog and IP voice;

     9) T1 and E1 interfaces;

(ii) “

voice switched video”

(VSV) is defined as automatic switching of multimedia session participant displays to show the video of the predominant speaker. The predominant speaker continues to view the last predominant speaker. Network Management System (NMS) uses the telephony-based audio bridge to determine the predominant speaker. VSV and HDV/PDV are the preferred embodiments;

4. The system of claim 1 wherein a network of diverse real-time communication means and components are integrated and configured to work in tandem to deliver lip synchronized audio and video end-to-end to and between a plurality of end-user terminals, and between enterprise terminals and foreign site codecs comprises:

(a) PMN architecture compliance with industry standards facilitates interoperability between differing communication means and vendors (refer to claim

     2).

(b) differing in-band and out-of-band real-time audio and video communication means are inherently isochronous (two-way without delay) and lip-synchronized because differences are imperceptible to the human eye and ear. For example;

surveillance camera baseband transmission to consolidation sites (analog twisted pair), and analog and broadband digital trunk lines between Cross-Point Switches (fiber, wireless, twisted pair);

Multimedia LANs within sites (analog/twisted pair);

telephony-based multimedia audio between all enterprise sites.

5. The system of claim 4 wherein end-users use existing enterprise telephones for input and output audio, Telco service provider telephone service and enterprise PBXs, as needed, and PMN Telco Audio Bridge to deliver lip-synchronized, real-time, point-to-point and multipoint audio service to a plurality of end-user terminal locations coupled to the same or interlocked Cross Point Switches.

6. The system of claim 4 wherein a real-time network of Cross Point Switches that interconnect end-user terminals via Multimedia LANs terminated by analog signal extender transceivers deliver lip-synchronized point-to-point and multipoint service to a plurality end-user terminals coupled to the same or interlocked Cross Point Switches comprising the following configuration:

(a) enterprise site end-user terminals comprised of real-time NTSC, PAL, or broadcast quality video, as appropriate, AV (audio/video) appliances.

(b) end-user terminals are coupled to input and output ports on real-time analog signal extender transceivers that transmit signals 1,000 feet or more (e.g., Extron) over Multimedia LANs that provide full duplex service with imperceptible loss of signal quality end-to-end. The transceiver preferred embodiment is video only, with audio provided by telephony-based service.

(c) real-time Multimedia LANs, dark twisted pair wireline full duplex network, is terminated by analog signal extender transceivers at the end-user node hub and PMN head-in-hub ends. The preferred embodiment is for the PMN Switching Center head-in-hub to be located either at the site LAN or telephone head-in and use dark wire pairs in existing LAN or telephone wiring sheaths;

(d) PMN Switching Center head-in hub transceivers coupled to real-time analog Cross Point Switches (e.g., PESA) on video, and optionally audio, input and output nodes. The preferred embodiment is video only connectivity with audio provided by telephony-based service.

(e) Multiviewers coupled to Cross Point Switches provide multipoint continuous presence service, and audio mixers coupled to Cross Point Switches provide bridged CD quality audio-audio option for Executive Board Rooms and Homeland Security Amphitheaters.

(f) interlocked Cross Point Switch trunk lines provide real-time lip-synchronized communication means between enterprise premise, campus, and proximity sites.

7. The system of claim 4 wherein diverse non-real-time communication means and components are integrated and configured to work in tandem to overcome inherent video quality and lip-synchronization audio and video problems between a plurality of end-points (codecs) participating in multiparty multimedia sessions between a plurality of enterprise and foreign sites comprises the following configuration:

(a) PMN architecture compliance with industry standards (e.g., ITU, IETF, ISO, MPEGIF) facilitates interoperability between differing communication means and vendors (refer to claim 2);

(b) PMN Switching Center codecs, which interconnect enterprise geographically dispersed sites, are the end-points for “

non-real-time”

communication links;

(c) high-end codecs, typically used in executive conference rooms, boardrooms, and Homeland Security amphitheaters, have overcome video quality and lip-synchronization audio and video problems (e.g., jitter, latency). Manufacturers of these products provide sophisticated technology to overcome communication link anomalies by delivering end-to-end multi-site digital audio and video communication link synchronization for Long Distance Calls;

(d) PMN Switching Center Codec Farms, which convert inter-site signals to digital for transmission and back to analog at delivery, provide point-to-point and limited multipoint communication link services (e.g., voice switched video) between sites connected by a network of codecs;

Sites connected by codecs and Multipoint Control Units provide “

Continuous Presence”

multipoint service to a plurality of end-users. The preferred embodiment is multipoint service provided by placing PMN Control Center Multipoint Control Units at the center of transmissions between site codecs. This is the preferred embodiment. The architecture also supports codecs with embedded Multipoint Control Units, which deliver multipoint service between sites without PMN Control Center intervention;

(e) PMN Control Center, Multipoint Control Unit Farms serve as the multipoint intermediary between a plurality of codecs engaged in collaboration and other multimedia services. Multipoint Control Unit vendors provide sophisticated technology to overcome latency by delivering end-to-end, end-point multi-site audio and video communication link synchronization;

(f) PMN Control Center Network Manager combined with enterprise network management facilities (e.g., QOS methodologies, gatekeepers, gateways, firewalls and proxies) overcome inherent non-real-time network anomalies.

8. The system of claim 4 wherein real-time communication means are aligned and combined with non-real-time communication means to deliver lip-synchronized audio and video to and between a plurality of end-user terminals and devices within and between a plurality of sites participating in multimedia sessions configured as follows:

(a) PMN Premise Switching Center Cross-Point Switches that control the flow of multimedia (video and audio) information throughout the system comprising;

i. cross-point switches control the flow of multimedia information between Local Call sites via Multimedia LANs and trunk lines connecting interlocked Cross-point Switches as described in claims 5 and 6;

ii. cross-point switches control the flow of multimedia information between Long Distance Call sites (i.e., between enterprise geographically dispersed enterprise sites, and between enterprise and foreign sites) via PMN Premise Switching Center codecs and PMN Control Center multipoint control units (MCUs) as describe in claim 7;

9. The method of claim 8 wherein end-user terminal and codec video and audio (the preferred embodiment is video-only) are aligned and combined comprising the following steps:

(a) couple codec analog video and audio, input and output ports to Cross Point switch input and output ports to combine real-time and non-real-time synchronized communication means;

i. Codec output is aligned synchronized video and audio;

ii. Codec input is unaligned synchronized video and audio;

(b) for Long Distance Calls, all end-user terminal participant video, and optional audio, must flow through the PMN Premise Switching Center Codec Farm for real-time/non-real-time alignment;

i. for Voice Switched Video (VSV), the active speaker is determined by NMS test of the Telco Audio Bridge, and the Multimedia Switch with the “

active speaker”

sends the speaker'"'"'s video to the codec for broadcast to participant terminals except the active speaker, which receives the last active speaker'"'"'s video;

ii. for Host Directed Video (HDV) and Participant Requested Video (PRV) (host acknowledged), which are triggered by telephone keypad signaling, the Multimedia Switch with the “

selected speaker”

sends the speaker'"'"'s video to the codec for broadcast to participant terminals except the selected speaker, which receives the last selected speaker'"'"'s video;

iii. for a multiparty site with Continuous Presence Video (CPV), the Multimedia Switch sends the Multiviewer “

Hollywood Squares”

windowed video to the Codec;

iv. for a point-to-point site the Multimedia Switch sends the participant'"'"'s video to the Codec.

(c) once aligned, there is no loss of alignment or signal quality between end-user terminals and Cross-Point switches because the communication link is real-time;

(d) As shown in FIG. 5, in addition to analog signal extender transceivers and codecs, other communication links and devices that can be coupled to Cross Point switches and shared in both Local and Long Distance Calls include;

analog and broadband digital trunk lines (e.g., fiber, wireless, twisted pair transport for surveillance cameras, inter-site communication, and other Enterprise facilities), “

Continuous Presence”

Multiviewers (e.g., Zandar), and CD-Quality Audio Mixers on the output side; and

analog and broadband digital trunk lines, “

Continuous Presence”

Multiviewers, CD-Quality Audio Mixers, Video Servers (e.g., education courseware), and cable TV on the input side.

10. The method of claim 8 wherein end-user terminal telephony and codec audio are aligned and combined comprising the following steps:

(a) couple codec analog audio input and output ports to corresponding ports on Telephone Hybrids;

(b) couple Telephone Hybrid voice ports to Telco service provider voice ports to exchange (send and receive) voice audio signals with the Telco Audio Bridge.

(c) for Long Distance Calls all end-user terminal participant audio must flow through the PMN Premise Switching Center Codec Farm for alignment;

i. input is unaligned, bridged (as needed), synchronized telephone handset audio output flowing to Telco service provider, to Telco Audio Bridge, to Codec;

ii. output is aligned, bridged (as needed), synchronized Codec output flowing to Telephone Hybrid, to Telco service provider, to Telco Audio Bridge to end-user terminal telephone headset speakers;

(d) once aligned, there is no loss of alignment or signal quality between end-user terminals and Cross-Point switches because the communication link is real-time;

11. The system of claim 1, wherein the Network Management System (NMS) is the intuitive means by which end-users, under enterprise management control governance, perform system launch, operational control, and system management tasks that match business needs comprising:

12. The method of claim 11, wherein NMS provides the following system launch services:

(a) Session Manager provides a structured dialogue with a designated member of enterprise senior management or their designee that facilitates collection of management control business rules that are recorded in the Business Rule Book. This function has a senior management security level, and entries must be approved by senior management;

(b) Session Manager provides a structured dialogue with the enterprise System Administrator to define the PMN system control tables (e.g., Phone Book, Resources, Networks, Session Processes,) and configure the system for integration with other enterprise systems.

13. The method of claim 11, wherein NMS provides enterprise end-users with the following operational control services:

(a) Before sessions, Scheduler together with Resource Manager books participants, determines and records required services, and reserves resources;

(b) During sessions, Session Manager together with Device and Resource Manager, startup (including security screening and resource acquisition), operate (including process orchestration, interrupt handlers, and device and information flow control) sessions;

(c) End sessions, Session Manager tears-down session (including releasing resources).

14. The method of claim 11, wherein NMS provides enterprise System Administrators with the following ongoing system management services:

(a) Session Manager provides System Administrators with structured dialogues to facilitate ongoing system management of system control tables;

(b) Network Manager provides System Administrators with real-time network monitoring reports, including alarms;

(c) Resource Manager provides System Administrators with real-time resource utilization reports, including alarms. (d) Session Manager facilitates Administrator maintenance of system tables (e.g., configuration, end-user profile, business rules) using appropriate security clearances, and use of Network Manager to monitor and manage the PMN Network, including gateways, firewalls and proxies, support of public, Internet, and Intranets;

15. The method of claim 11, wherein NMS provides enterprise end-users and system administrators with an intuitive human/computer interface (HCI) and session control that matches business needs comprising:

(a) existing enterprise telephones for audio and session control;

(b) Telephone and audio conferencing paradigms, which are well understood and require minimal training (IVR provided for more complex tasks);

(c) Telephone key pads used for session scheduling and dynamic change of state signaling;

(d) Single toll free number dialing to request services (also supports speed dialing) and enterprise telephone book mirroring (use same enterprise telephone book telephone numbers);

(e) Meeting schedule can be immediate or future;

Confirmed by scheduler or meeting host;

Recurring or one-time;

(f) IVR-based Socratic scheduling questions reduce data entry;

point-to-point (2 participants) or multipoint (greater than 2 participants);

if multipoint, continuous presence (scarce resource), or voice or host directed switching;

if host directed, (e.g., participant request for floor);

(g) Virtual meeting rooms that mirror the format, controls, and rituals followed by organizations in physical face-to-face collaboration and education (e.g., listener protocol, open and closed door meeting entry, entry after meeting start, signaling the host to get the floor, requesting more time, inviting participants during meeting);

(h) Telephone keypad operational appliance and device control (e.g., surveillance camera selection, and PAN, tilt, and zoom control)

(i) Use of special software (e.g., Microsoft PowerPoint, Whiteboard, and Internet).

(j) Accounting records and billing of session resources and services rendered;

(k) Supports Internet-based HCI for complex tasks (e.g., complex scheduling, system launch and administration).

16. The method of claim 11, wherein NMS is event-driven software that provides real-time services to events and multi-user changes of state.

17. The method of claim 1, wherein isochronous NTSC TV quality or better video is transmitted to and between end-user terminals and devices comprises:

(a) End-user appliances (e.g., video cameras, TV displays) that range from NTSC quality to boardroom and broadcast quality, codecs, and MCUs are the determinants of end-to-end quality. Real-time communication means are neutral;

they deliver the same quality signal they are given.

(b) For Long Distance Calls PMN uses non-real-time codecs and Multipoint Control Units (MCUs) are the same or better quality than the codecs used in Boardrooms. They deliver a minimum of NTSC between end-user terminal and foreign site codecs;

(c) For Local Calls PMN uses real-time analog signal extender transceivers and Premise Switching Center switches (e.g., Cross Point, Multiviewer) that transmit video between end-user terminals connected to the same or interlocked Multimedia Switches via Multimedia LANs without loss of signal quality.

18. A multimedia architecture that manages, shares, and eliminates use of expensive dedicated capital-intensive resources, thereby significantly reducing Enterprise capital outlay per end-user. Many of these resources are typically deployed at point of service (e.g., desktops, meeting rooms).

19. The method of claim 18 wherein PMN Control Center optimizes resource management by providing the following services:

(a) End-user Human/Computer Interface (Telephony-based Interactive Voice Response (IVR), telephone keypad, Internet);

(b) Scheduler (schedule (immediate and future) and reserve resources required to book a session),

(c) Resource Manager (e.g., managing “

nailed-down”

circuit resources),

(d) Session Manager (insuring all resources are available and that session protocol is followed and billing is accurate),

(e) Device Manager (controlling operation of PMN Control Center and Enterprise Premise hardware during session setup, operation, and teardown).

20. The method of claim 18, wherein resource deployment is optimized by centralizing resources at PMN Control Center and sharing the following resources with all sites:

(a) Multipoint Control Units (MCUs),

(b) Telco Audio Bridge (computer hardware, conference bridge computer cards, NMS software),

(c) Network Manager (gatekeepers, gateways).

21. The method of claim 18, wherein resource deployment is optimized by centralizing resources at each enterprise premise site and sharing them with end-users at the site comprise:

(a) PMN Premise Switching Center (Codec Farm, Telephone Hybrid Farm;

(b) Multimedia Switches (Transceiver Hub, Cross Point Switches, Continuous Presence Multiviewers, Audio Mixers, On-Demand Servers, Trunk Lines);

(c) Multimedia LAN (use existing LAN and telephone wire lines);

(d) End-user Appliances (uses existing telephones, and Audio/Video devices (e.g., computers, monitors and TVs).

22. The method of claim 18, wherein use of dynamic real-time switching methodologies (e.g., VSV, HDV/PRV) eliminate the need for PMN Control Center MCUs.

23. The method of claim 18, wherein use of telephony-based audio and control eliminate the need for expensive multimedia resources that are commonly used at desktops or in meeting rooms:

(a) telephony-based audio eliminates the need for audio mixers, transceiver audio, end-user appliance speakers, cross point switch audio, computers. Therefore, telephony-based architecture supports deployment of multimedia (e.g., videoconferencing) in environments where there are no computers (e.g., hotel rooms, meeting rooms).

(b) telephony-based audio and control combined with real-time end-to-end video (e.g., Municipal fiber loops) in addition to audio equipment claimed in (a), they eliminate the need for inter-site Enterprise (e.g., codecs, telephone hybrids) and PMN Control Center (e.g., Multimedia Control Units, gatekeepers, gateways) resources. If collaboration with Foreign sites (outside the Enterprise) is required, codec and Multimedia Control unit resources can be centralized at the Central PMN Control Center rather than deploying codecs at each site. In this configuration, since outside collaboration is generally much less than Enterprise collaboration, codec and MCU resource requirements will be significantly reduced.

(c) telephony-based voice switched video eliminates the need for “

Continuous Presence”

Multiviewers that are not only expensive, but become a bottleneck for local (within premise or campus) and long-distance (between sites) multiparty collaboration (videoconferencing).