Methods and systems that use information about encrypted data packets to determine an order for sending the data packets

• US 7,966,488 B2
• Filed: 10/20/2004
• Issued: 06/21/2011
• Est. Priority Date: 01/30/2004
• Status: Expired due to Fees

• Current Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Original Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Inventors: Apostolopoulos, John G.
• Primary Examiner(s): Zand; Kambiz
• Assistant Examiner(s): RAHIM, MONJUR
• Application Number:
  US10/970,620
• Publication Number:
  US 20050169175A1
• Time in Patent Office: 2,435 Days
• Field of Search: 370/338, 370/413, 370/473, 380/210, 709/203, 709/227, 709/238, 709/245, 709/250, 713/167, 713/193, 713/160, 726/12
• US Class Current: 713/160
• CPC Class Codes: H04L 47/10 Flow control; Congestion control
• CPC Class Codes: H04L 47/19 at layers above the network layer network arrangements for networked applications for scheduling or organising the servicing of application requests H04L67/60
• CPC Class Codes: H04L 47/283 in response to processing delays, e.g. caused by jitter or round trip time [RTT]
• CPC Class Codes: H04L 63/0823 using certificates cryptographic mechanisms or cryptographic arrangements for entity authentication involving certificates H04L9/3263
• CPC Class Codes: H04L 65/1101 Session protocols
• CPC Class Codes: H04L 65/70 Media network packetisation
• CPC Class Codes: H04L 65/752 adapting media to network capabilities
• CPC Class Codes: H04L 65/756 adapting media to device capabilities
• CPC Class Codes: H04L 65/765 intermediate
• CPC Class Codes: H04L 65/80 Responding to QoS
• CPC Class Codes: H04L 69/22 Parsing or analysis of headers
• CPC Class Codes: H04L 9/40 Network security protocols
• CPC Class Codes: H04N 19/89 involving methods or arrangements for detection of transmission errors at the decoder
• CPC Class Codes: H04N 21/23439 for generating different versions
• CPC Class Codes: H04N 21/2389 Multiplex stream processing, e.g. multiplex stream encrypting
• CPC Class Codes: H04N 21/262 Content or additional data distribution scheduling, e.g. sending additional data at off-peak times, updating software modules, calculating the carousel transmission frequency, delaying a video stream transmission, generating play-lists scheduling strategies for dispatcher in multiprogramming arrangements G06F9/4881; arrangements for scheduling broadcast services or broadcast-related services H04H60/06; flow control in packet networks H04L47/10; establishing a time schedule or organising the servicing of application requests H04L67/62
• CPC Class Codes: H04N 21/26216 involving the channel capacity, e.g. network bandwidth admission control, resource allocation in open networks H04L12/5692; flow control in packet networks H04L47/10; establishing a schedule or organising the servicing of application requests taking into account QoS H04L67/61
• CPC Class Codes: H04N 21/2662 Controlling the complexity of the video stream, e.g. by scaling the resolution or bitrate of the video stream based on the client capabilities
• CPC Class Codes: H04N 21/4385 Multiplex stream processing, e.g. multiplex stream decrypting

Abstract

Methods and systems thereof for streaming media data are described. Media data comprising a plurality of data packets is accessed. The media data includes encrypted data. Information that quantifies a characteristic of each data packet in the plurality of data packets is also accessed. That information is used to determine an order for sending the data packets over a network. The order is determined without decrypting the encrypted data.

Claims

1. A method of streaming data, said method comprising:

receiving media data comprising a plurality of data packets from a storage element, wherein said media data comprises encrypted data;

receiving a plurality of scheduling cues that quantify characteristics of individual data packets in said plurality of data packets from said storage element, wherein each scheduling cue of said plurality of scheduling cues is uniquely associated with a particular data packet of said plurality of data packets; and

using said plurality scheduling cues to determine a distortion reducing order for sending said plurality of data packets in a network that reduces distortion at a receiver, wherein said order is determined without decrypting said encrypted data and without requiring analysis of said data packets in real time.

2. The method of claim 1 further comprising:

determining that a particular data packet did not arrive at a node of said network; and

using a scheduling cue associated with said particular data packet to determine whether to resend said particular data packet.

3. The method of claim 1 further comprising:

using said plurality scheduling cues to determine whether or not to send a particular data packet.

4. The method of claim 1 wherein said plurality scheduling cues comprises information selected from the group consisting of:

information identifying the beginning and end of said data packet'"'"'s data;

header information for said data packet;

information identifying the length of said data packet;

information identifying a transmission time of said data packet;

information identifying nominal presentation time for said data packet;

information quantifying for said data packet an amount of distortion predicted to occur should said data packet not be sent or received;

coding dependencies between said data packets;

information identifying for said data packet how many data packets are dependent on said data packet;

information identifying which of said data packets provide error resiliency;

information identifying which of said data packets provide redundancy; and

information identifying a deadline for delivering said data packet.

5. The method of claim 1 further comprising:

accessing second information that characterizes network performance; and

using said second information with said plurality scheduling cues to determine said order for sending said data packets.

6. The method of claim 5 wherein said second information comprises information selected from the group consisting of:

bandwidth available along a path in said network;

bottleneck link capacity along said path;

data packet delivery rate;

data packet loss rate;

data packet received pattern;

data packet loss pattern;

information identifying which of said data packets were received at a node along said path;

information identifying which of said data packets did not arrive at a node along said path;

information quantifying time needed to traverse said path; and

information quantifying delays associated with said path.

7. The method of claim 1 wherein said data packets constitute data that are predictively encoded.

8. The method of claim 1 wherein said data packets constitute data that are scalably encoded.

9. The method of claim 1 wherein said data and said scheduling cue are stored in a single file.

10. The method of claim 1 wherein said plurality scheduling cues are encrypted.

11. The method of claim 10 wherein said media data are encrypted using a first encryption algorithm and said plurality scheduling cues are encrypted using a second encryption algorithm that is different from said first encryption algorithm.

12. The method of claim 1 wherein said order is determined without knowledge of the content of said data packets.

13. A method of scheduling the transmission of data packets comprising data, said method comprising:

generating a distortion monitoring transmission schedule for sending a plurality of data packets over a network, wherein said distortion monitoring transmission schedule maintains distortion at or below a specified threshold, wherein said data packets comprise encrypted data and wherein said distortion monitoring transmission schedule is generated without decrypting said encrypted data and without requiring analysis of said data packets in real time;

receiving a scheduling cue from a storage element that quantifies for a data packet an effect that would result if said data packet was not sent or received;

evaluating said distortion monitoring transmission schedule using said scheduling cue to measure an effect associated with said distortion monitoring transmission schedule; and

determining whether to transmit any of said data packets using said distortion monitoring transmission schedule considering said measure of said effect.

14. The method of claim 13 wherein said measure of said effect comprises an amount of distortion associated with a display of said data, wherein said scheduling cue comprises for said data packet an amount of distortion that would result if said data packet was not sent or received.

15. The method of claim 14 wherein said evaluating comprises:

determining said total distortion assuming loss of a single data packet from said plurality of data packets.

16. The method of claim 14 wherein said evaluating comprises:

determining said total distortion assuming loss of more than one data packet, said total distortion determined assuming loss of data packets in different numbers and combinations.

17. The method of claim 13 wherein said plurality of data packets comprises any data packets not yet transmitted, any data packets transmitted but known to be lost during transmission, and any data packets transmitted but whose status is unknown.

18. The method of claim 13 further comprising:

transmitting some portion of said plurality of said data packets according to said distortion monitoring transmission schedule.

19. The method of claim 13 further comprising:

repeating said generating, accessing, evaluating and selecting steps periodically to determine a newer schedule for transmitting said plurality of data packets.

20. The method of claim 13 said scheduling cue comprises information selected from the group consisting of:

information identifying the beginning and end of said data packet'"'"'s data;

header information for said data packet;

information identifying the length of said data packet;

information identifying a transmission time of said data packet;

information identifying nominal presentation time for said data packet;

information quantifying for said data packet an amount of distortion predicted to occur should said data packet not be sent or received;

coding dependencies between said data packets;

information identifying for said data packet how many data packets are dependent on said data packet;

information identifying which of said data packets provide error resiliency;

information identifying which of said data packets provide redundancy; and

information identifying a deadline for delivering said data packet.

21. The method of claim 13 further comprising evaluating said distortion monitoring transmission schedule also using second information that characterizes network performance.

22. The method of claim 21 wherein said second information is selected from the group consisting of:

bandwidth available along a path in said network;

bottleneck link capacity along said path;

data packet delivery rate;

data packet loss rate;

data packet received pattern;

data packet loss pattern;

information identifying which of said data packets were received at a node along said path;

information identifying which of said data packets did not arrive at a node along said path;

information quantifying time needed to traverse said path; and

information quantifying delays associated with said path.

23. The method of claim 13 wherein said data packets constitute data that are predictively encoded.

24. The method of claim 13 wherein said data packets are scalably encoded.

25. The method of claim 13 wherein said data and said scheduling cue are stored in a single file.

26. The method of claim 13 wherein said scheduling cue is encrypted.

27. The method of claim 13 wherein said distortion monitoring transmission schedule is selected without knowledge of the content of said data packets.

28. The method of claim 13 further comprising:

generating a plurality of distortion monitoring transmission schedules that list different orders for sending said plurality of data packets over said network;

evaluating said distortion monitoring transmission schedules using said scheduling cue to measure said effect for each distortion monitoring transmission schedule evaluated; and

selecting one of said distortion monitoring transmission schedules considering said measure, wherein said one of said distortion monitoring transmission schedules are selected without decrypting said encrypted data.

29. A system for streaming data in a network, said system comprising:

a streaming element for streaming a plurality of data packets over said network according to a distortion reducing that reduces distortion at a receiver, said distortion reducing order is determined using a plurality of scheduling cues that quantify characteristics of individual data packets in said plurality of data packets, said data packets comprising encrypted data, wherein said plurality of data packets and said plurality of scheduling cues are received from a storage element, wherein each scheduling cue of said plurality of scheduling cues is uniquely associated with a particular data packet of said plurality of data packets, and wherein said distortion reducing order is determined and said data packets are streamed without decrypting said encrypted data and without requiring analysis of said data packets in real time.

30. The system of claim 29 wherein said streaming element is also for determining said distortion reducing order for streaming said data packets.

31. The system of claim 29 wherein said streaming element receives said distortion reducing order for streaming said data packets from another node of said network.

32. The system of claim 29 further comprising:

a storage element coupled to said streaming element, said storage element for storing said data and for storing said plurality of scheduling cues;

an encoder element coupled to said storage element, said encoder element for encoding said data and for generating said plurality of scheduling cues; and

an encrypter element coupled to said storage element, said encrypter element for encrypting said data.

33. The system of claim 29 further comprising:

a storage element coupled to said streaming element, said storage element for storing said data and for storing said plurality of scheduling cues;

an encoder element coupled to said storage element, said encoder element for encoding said data;

an encrypter element coupled to said storage element, said encrypter element for encrypting said data; and

a cue generator element coupled to said storage element, said cue generator element for generating said plurality of scheduling cues.

34. The system of claim 29 adapted to receive an indication that a particular data packet did not arrive at a node of said network, wherein said streaming element determines whether to resend said particular data packet.

35. The system of claim 29 wherein said streaming element is also for determining whether or not to send a particular data packet according to said distortion reducing order.

36. The system of claim 29 wherein said plurality of scheduling cues comprises information selected from the group consisting of:

information identifying the beginning and end of said data packet'"'"'s data;

header information for said data packet;

information identifying the length of said data packet;

information identifying a transmission time of said data packet;

information identifying nominal presentation time for said data packet;

information quantifying for said data packet an amount of distortion predicted to occur should said data packet not be sent or received;

coding dependencies between said data packets;

information identifying for said data packet how many data packets are dependent on said data packet;

information identifying which of said data packets provide error resiliency;

information identifying which of said data packets provide redundancy; and

information identifying a deadline for delivering said data packet.

37. The system of claim 29 wherein said distortion reducing order for sending said data packets is also determined using second information that characterizes network performance, wherein said second information is selected from the group consisting of:

bandwidth available along a path in said network;

bottleneck link capacity along said path;

data packet delivery rate;

data packet loss rate;

data packet received pattern;

data packet loss pattern;

information identifying which of said data packets were received at a node along said path;

information identifying which of said data packets did not arrive at a node along said path;

information quantifying time needed to traverse said path; and

information quantifying delays associated with said path.

38. The system of claim 29 wherein a scheduling cue of said plurality of scheduling cues describes for said data packet an amount of distortion that would result if said data packet was not sent or received, wherein said streaming element generates transmission schedules that list different distortion reducing orders for sending said plurality of data packets over said path, evaluates said transmission schedules using second information that characterizes network performance along said path to determine a total distortion associated with transmission schedules that are evaluated, and selects a transmission schedule that substantially minimizes said total distortion.

39. The system of claim 29 wherein said plurality of data packets comprises any data packets not yet transmitted, any data packets transmitted but known to be lost during transmission, and any data packets transmitted but whose status is unknown.

40. The system of claim 29 wherein said streaming element also uses said plurality of scheduling cues to determine whether to allocate forward error correction information (FEC) among said data packets and how to allocate said FEC information among said data packets.

41. The system of claim 29 wherein said streaming element also uses said scheduling cue to determine whether to switch to streaming another plurality of data packets constituting said data, said switch made according to network performance characteristics.

42. The system of claim 29 wherein said plurality of scheduling cues are encrypted.

43. The system of claim 29 wherein said distortion reducing order is determined without knowledge of the content of said data packets.

44. A method of streaming data, said method comprising:

receiving media data comprising a plurality of data packets from a storage element;

receiving a plurality of scheduling cues that quantify characteristics of individual data packets in said plurality of data packets from said storage element, wherein each scheduling cue of said plurality of scheduling cues is uniquely associated with a particular data packet of said plurality of data packets, wherein said scheduling cue is encrypted; and

using said plurality scheduling cues to determine distortion reducing order for sending said plurality of data packets in a network that reducing distortion at a receiver, wherein said distortion reducing order is determined without decrypting said encrypted data and without requiring analysis of said data packets in real time.

45. The method of claim 44 wherein said plurality of scheduling cues comprises information selected from the group consisting of:

information identifying the beginning and end of said data packet'"'"'s data;

header information for said data packet;

information identifying the length of said data packet;

information identifying a transmission time of said data packet;

information identifying nominal presentation time for said data packet;

information quantifying for said data packet an amount of distortion predicted to occur should said data packet not be sent or received;

coding dependencies between said data packets;

information identifying for said data packet how many data packets are dependent on said data packet;

information identifying which of said data packets provide error resiliency;

information identifying which of said data packets provide redundancy; and

information identifying a deadline for delivering said data packet.