HYDRA: high-performance data recording architecture for streaming media

• US 20050283818A1
• Filed: 06/21/2005
• Published: 12/22/2005
• Est. Priority Date: 06/22/2004
• Status: Active Grant

• Current Assignee: University of Southern California
• Original Assignee: University of Southern California
• Inventors: Desai, Dwipal A., Zimmermann, Roger, Fu, Kun
• Granted Patent: US 8,370,888 B2
• Time in Patent Office: Days
• US Class Current: 725/134
• CPC Class Codes: H04N 21/2187 Live feed
• CPC Class Codes: H04N 21/23103 using load balancing strategies, e.g. by placing or distributing content on different disks, different memories or different servers storage management G06F3/0604; allocation of resources considering the load in multiprogramming arrangements G06F9/505; techniques for rebalancing the load in a distributed system G06F9/5083; access to distributed or replicated servers, e.g. load balancing, in data networks H04L67/1001
• CPC Class Codes: H04N 21/23116 involving data replication, e.g. over plural servers synchronization of replicated data G06F11/1658; error detection or correction by means of data replication G06F11/2053; replication in distributed file systems G06F16/10; replication in distributed file systems G06F16/27; replication or mirroring of data in data networks H04L67/1095
• CPC Class Codes: H04N 21/2312 Data placement on disk arrays data placement in general G06F3/0604
• CPC Class Codes: H04N 21/2396 characterized by admission policies admission control, resource allocation in open networks H04L12/5692; arrangements for network security using user profiles for access control H04L63/102; access security in wireless networks H04W12/08

Abstract

A data stream recorder system, for multi-stream recording and retrieval of utilizes a number of gateways, each for sending and receiving packets containing streaming multimedia content data at real-time rates via a packet data network. A session manager communicates via the network with source client devices and receiver client devices, to establish and control recording and retrieval sessions. The manager assigns sessions to the gateways for the sending and receiving of the packets to and from client devices. Content is distributed across storage devices associated in storage nodes. Each of the gateways receives packets containing content data at real time rates during a recording session and distributes the received packets from the session across all of the storage nodes. A scheduler of each respective storage node distributes content data from packets distributed to the respective storage node, across all of the digital storage devices of the respective storage node.

Claims

1. A data stream recorder system for real-time multi-stream recording, retrieval and control of multimedia content via a packet data network, the recorder comprising:

a plurality of gateways, each for sending and receiving packets containing streaming multimedia content data at real-time rates via the packet data network;

a session manager for communicating via the network with source client devices and receiver client devices to establish and control sessions through the packet data network and to assign sessions to the gateways for the sending and receiving of the packets to and from client devices; and

a plurality of storage nodes for storing multimedia content data from packets received via sessions through the gateways and for providing stored multimedia content data to the gateways for communication via sessions through the gateways, each respective storage node comprising;

(a) a plurality of digital storage devices, for storage and retrieval of distributed portions of multimedia content data for a plurality of sessions; and

(b) a scheduler for controlling distribution of content data from packets received at the respective node for storage across the digital storage devices of the respective storage node,

wherein;

each of the gateways receiving packets containing content data at real time rates during a recording session distributes the received packets across all of the storage nodes, and

each scheduler of a respective storage node distributes content data from packets distributed to the respective storage node across all of the digital storage devices of the respective storage node.

2. The system of claim 1, wherein the session manager is configured to monitor resource utilization at each of the gateways and assign different sessions for sending or receiving different streams to respective gateways, based on the current workload and resources available at each of the gateways.

3. The system of claim 1, wherein the session manager implements a statistical admission control with respect to each new session, to assign each new station to one of the gateways in such a manner as to accommodate variable bit rate (VBR) streams and multizone disk drives as the digital storage devices.

4. The system of claim 1, wherein the gateways and storage nodes are configured to receive, store, retrieve and send streaming multimedia content data of multiple different digital media types.

5. The system of claim 4, wherein the different digital media types include three or more of the following media types:

CD-quality audio, NTSC quality MPEG encoded video, HD quality MPEG encoded video, digital video (DV), DVCPRO50, DVCPROHD, and high definition camera (HDCAM) video.

6. The system of claim 1, wherein:

each respective storage node further comprises means for aggregating content data from received packets into data blocks for storage on the digital storage devices of the respective storage node;

the means aggregates content data sequentially from the received packets into the data blocks, and

the scheduler of each respective storage node randomly assigns the data blocks in sequence to the digital storage devices of the respective storage node.

7. The system of claim 6, wherein:

the scheduler of each respective storage node randomly assigns content data from the received packets to the digital storage devices of the respective storage node; and

said means aggregates content data assigned to each digital storage device of the respective storage node into a data block for storage on the digital storage device.

8. The system of claim 6, wherein the scheduler of each respective storage node distributes blocks of data and controls the storage devices to randomly place blocks within the surface of a storage disk in each of the digital storage devices of the respective storage node.

9. The system of claim 6, wherein each gateway receiving streaming multimedia content data for recording in the system via an assigned session randomly distributes blocks of packets received via the assigned session to each of the storage nodes for processing and storage of the streaming multimedia content data from those packets.

10. The system of claim 6, wherein the scheduler of each respective storage node provides deadline-driven data reading and writing across all of the digital storage devices of the respective storage node.

11. The system of claim 10, wherein:

a deadline for each block is formed from a time stamp of the last packet contained in the block plus an offset;

for deadline-driven retrieval, the offset is the time when the retrieval request was admitted into the system, and the deadline-driven retrieval allows data blocks for an outgoing stream to be retrieved in order of their deadlines and if some deadlines are identical then in scan order; and

for deadline-driven writing to a storage device, the offset is determined from the delay that a block is allowed to stay in a main memory buffer until it must output to a digital storage device.

12. The system of claim 1, further comprising a shared pool of memory buffers for assembly of incoming packets into the data blocks and the partitioning of retrieved data blocks into outgoing packets.

13. A method of receiving, recording, and distributing multimedia content at real-time rates via a packet data network, comprising steps of:

establishing a plurality of first sessions between multiple gateways and source client devices, via the packet data network;

establishing a plurality of second sessions between the gateways and receiver client devices via the packet data network;

receiving a stream of packets containing real-time streaming multimedia content data via each first session through the packet data network;

distributing the packets from each received stream from one of the gateways to a plurality of storage nodes;

distributing payload content data from packets of each stream distributed to each respective storage node across a plurality of digital storage devices associated with the respective storage node; and

for each second session, retrieving the stored content from the storage devices, and transmitting packets containing the retrieved content as a real-time outgoing stream via one of the gateways and one of the second session through the packet data network to at least one of the receiver client devices.

14. The method of claim 13, wherein for at least one of the second sessions, the transmitting of the packets containing the retrieved content as the real-time outgoing stream via one of the gateways occurs while reception of the content is ongoing via one of the first sessions from one of the source client devices.

15. The method of claim 13, wherein the establishment of the first and second sessions balances load across the gateways.

16. The method of claim 13, wherein the content received via the first sessions and the content transmitted via the second sessions comprises three or more of the following different types of media:

CD-quality audio, NTSC quality MPEG encoded video, HD quality MPEG encoded video, digital video (DV), DVCPRO50, DVCPROHD, and high definition camera (HDCAM) video.

17. A method of receiving, recording, and distributing multimedia content at real-time rates via a packet data network, comprising steps of:

receiving requests to establish sessions for incoming and outgoing streaming media communications, via the packet data network;

assigning each requested incoming session to one of a plurality of gateways, for establishing a first communication session through the packet network with a source client device;

assigning each requested outgoing session to one of a plurality of the gateways, for establishing a second communication session through the packet network with a receiver client device;

receiving a stream of packets containing real-time streaming multimedia content data via each respective first communication session through the packet data network, at the gateway assigned the respective first session;

distributing the packets from each received stream from the gateway assigned the respective first session to a plurality of storage nodes;

distributing payload content data from packets of each stream distributed to each respective storage node across a plurality of digital storage devices associated with the respective storage node; and

for each respective second session, retrieving the stored content from the storage devices, and transmitting packets containing the retrieved content as a real-time outgoing stream via the gateway assigned the respective second session through the packet data network to at least one of the receiver client devices.

18. The method of claim 17, wherein for at least one of the second communication sessions, the transmitting of the packets containing the retrieved content as the real-time outgoing stream via one of the gateways occurs while reception of the content is ongoing via one of the first communication sessions from one of the source client devices.

19. The method of claim 17, wherein the assigning steps balance load across the gateways.

20. The method of claim 17, wherein the content received via the first communication sessions and the content transmitted via the second communication sessions comprises three or more of the following different types of media:

CD-quality audio, NTSC quality MPEG encoded video, HD quality MPEG encoded video, digital video (DV), DVCPRO50, DVCPROHD, and high definition camera (HDCAM) video.