Caching media data using content-sensitive identifiers

• US 6,813,690 B1
• Filed: 10/16/2001
• Issued: 11/02/2004
• Est. Priority Date: 06/12/2001
• Status: Active Grant

• Current Assignee: Network Appliance Incorporated (NetApp, Inc.)
• Original Assignee: Network Appliance Incorporated (NetApp, Inc.)
• Inventors: Merrick, Jeffrey Dale, Wagner, J. Christopher, Lango, Jason, Roussos, Konstantinos, Tsai, Robert
• Primary Examiner(s): NGUYEN, THAN VINH
• Application Number:
  US09/981,668
• Time in Patent Office: 1,113 Days
• Field of Search: 711/118, 711/113, 711/137, 707/103.R, 707/103.Y, 707/10
• US Class Current: 711/118
• CPC Class Codes: G06F 12/0866 for peripheral storage systems, e.g. disk cache
• CPC Class Codes: G06F 2212/264 Remote server
• CPC Class Codes: G06F 2212/463 File
• CPC Class Codes: G06F 2212/464 Multimedia object, e.g. image, video
• CPC Class Codes: H04L 67/561 Adding application-functional data or data for application control, e.g. adding metadata
• CPC Class Codes: H04L 67/5682 Policies or rules for updating, deleting or replacing the stored data
• CPC Class Codes: Y10S 707/99944 Object-oriented database structure

Abstract

Techniques for caching media data, including streaming media data, using content-sensitive identifiers. The content-sensitive identifiers enable a caching proxy or a caching server to unambiguously determine the version or contents of media data cached by the caching proxy for a particular data pointer or data reference (e.g., a URL) such that an appropriate version of the media data can be served to a requesting client system in an efficient and economical manner.

Claims

1. A method of storing media data in a cache, comprising:

receiving information associated with a data pointer, the information comprising media data associated with the data pointer and stored by a media data server and first information comprising information identifying a version of the media data associated with the data pointer;

storing the media data in the cache as a plurality of objects, each object in the plurality of objects storing a portion of the media data; and

generating an object identifier for each object in the plurality of objects based upon a portion of the first information and the portion of the media data stored by each object, wherein the object identifier for an object identifies the portion of the media data stored by the object and the version of the media data.

2. The method of claim 1 wherein the data pointer is a Uniform Resource Locator (URL).

3. The method of claim 1 wherein the media data comprises streaming media data.

4. The method of claim 1 wherein:

the first information comprises information related to properties of the media data; and

storing the media data in the cache as the plurality of objects comprises storing the plurality of objects in the cache such that each object in the plurality of objects is individually accessible from the cache using the object identifier generated for the object.

5. The method of claim 4 further comprising:

storing mapping information that maps each object in the plurality of objects to an object identifier generated for the object.

6. The method of claim 5 wherein the mapping information stores information mapping the object identifier generated for-an object stored in the cache to an object handle of the object.

7. The method of claim 5 wherein:

each object in the plurality of objects is a file in a plurality of files;

generating an object identifier for each object comprises generating a filename for each file in the plurality of files, wherein the filename for a file identifies the portion of the media data stored by the file and the version of the media data;

storing the media data in the cache as a plurality of objects comprises storing the plurality of files in the cache such that each file in the plurality of files is individually accessible from the cache using a filename generated for the file; and

storing the mapping information comprises storing information that maps each file in the plurality of files to a filename generated for the file.

8. The method of claim 5 wherein storing the media data in the cache as the plurality of objects comprises:

partitioning the media data into a plurality of portions such that each portion comprises media data of a pre-determined time duration;

storing each portion in the plurality of portions in an object from the plurality of objects; and

generating an object number for each object in the plurality of objects based upon the temporal position of the portion of the media data stored by the object in the media data.

9. The method of claim 8 wherein:

the first information comprises version information identifying the version of the media data; and

generating an object identifier for each object in the plurality of objects comprises generating an object identifier for each object in the plurality of objects using the version information and the object number generated for the object.

10. The method of claim 9 wherein the first information comprises encoding information for the media data and wherein generating an object identifier for each object in the plurality of objects comprises using the encoding information to generate the object identifier.

11. The method of claim 9 wherein generating an object identifier for each object in the plurality of objects using the version information and the object number generated for the object comprises:

generating a first value based upon the version information;

generating an object string using the first value and the object number generated for the object; and

generating the object identifier for the object using the object string.

12. The method of claim 11 wherein:

generating the first value comprises applying a first hashing technique to the version information to generate the first value;

generating the object string using the first value and the object number generated for the object comprises concatenating the first value and the object number to generate the object string; and

generating the object identifier for the object using the object string comprises applying a second hashing technique to the object string to generate the object identifier for the object.

13. The method of claim 8 wherein:

the first information comprises version information identifying the version of the media data, second information identifying authorship of the media data, third information identifying a title of the media data, and fourth information identifying a time duration of the media data; and

generating an object identifier for each object in the plurality of objects comprises generating an object identifier for each object in the plurality of objects using the version information, the second information, the third information, the fourth information, and the object number generated for the object.

14. The method of claim 5 further comprising:

receiving a data request from a first system requesting media data associated with the data pointer;

in response to receiving the data request, reading the first information associated with the data pointer from the media data server storing the requested media data for the data pointer;

generating object identifiers for the data request based upon the requested media data and a portion of the first information read from the media data server, the object identifiers identifying a version of the requested media data;

searching the mapping information to determine if the object identifiers generated for the data request are included in the mapping information;

retrieving, from the cache, a set of objects corresponding to the object identifiers for the data request that are included in the mapping information; and

communicating media data stored by the set of objects retrieved from the cache to the first system.

15. A method of communicating media data from a cache in response to a data request, the method comprising:

storing media data in the cache as a plurality of objects, each object in the plurality of objects storing a portion of the Media data;

storing a first set of object identifiers corresponding to the plurality of objects, wherein an object identifier for an object identifies the portion of the media data stored by the object and a version of the media data;

receiving a data request from a first system requesting media data associated with a data pointer;

reading media data description information associated with the data pointer from a media data server storing the requested media data for the data pointer, the media data description information comprising information related to properties of the media data requested by the data request;

generating a second set of object identifiers for the data request based upon the requested media data and a portion of the media data description information, the object identifiers in the second set identifying the requested media data and a version of the requested media data associated with the data pointer;

determining a third set of object identifiers comprising one or more object identifiers from the second set that are included in the first set of object identifiers;

retrieving, from the cache, a set of objects corresponding to the object identifiers included in the third set of object identifiers; and

communicating media data stored by the set of objects retrieved from the cache to the first system.

16. The method of claim 15 wherein generating the second set of object identifiers for the data request comprises:

determining a plurality of object numbers for the requested media data; and

generating the second set of object identifiers for the data request using the portion of the media data description information and the plurality of object numbers.

17. The method of claim 16 wherein:

the media data description information comprises first information identifying a version of the media data stored by the media data server and requested by the data request; and

generating the second set of object identifiers for the data request comprises generating an object identifier for each object number in the plurality of object numbers using the first information and the object number.

18. The method of claim 17 wherein the media data description information comprises encoding information for the requested media data and generating the second set of object identifiers for the data request comprises using the encoding information to generate the object identifiers.

19. The method of claim 17 wherein generating an object identifier for each object number in the plurality of object numbers using the first information and the object number comprises:

|

generating a first value based upon a portion of the media data description information including the first information;

generating an object string using the first value and the object number; and

generating the object identifier for the object using the object string.

20. The method of claim 16 wherein:

the media data description information comprises first information identifying a version of the media data stored by the media data server and requested by the data request, second information identifying authorship of the media data requested by the data request, third information identifying a title of the media data requested by the data request, and fourth information identifying a time duration of the media data requested by the data request; and

generating an object identifier for each object number in the plurality of object numbers comprises generating the object identifier using the first information, the second information, the third information, the fourth information, and the object number.

21. A system for caching media data stored by a media data server system for a data pointer, the system comprising:

a processor; and

a memory coupled to the processor, the memory configured to store a plurality of code modules for execution by the processor;

the plurality of code modules comprising;

a code module for receiving information associated with a data pointer, the information comprising media data associated with the data pointer and stored by a media data server and first information comprising information identifying a version of the media data associated with the data pointer;

a code module for storing the media data in the memory as a plurality of objects, each object in the plurality of objects storing a portion of the media data; and

a code module for generating an object identifier for each object in the plurality of objects based upon a portion of the first information and the portion of the media data stored by each object, wherein the object identifier for an object identifies the portion of the media data stored by the object and the version of the media data.

22. The system of claim 21 wherein the data pointer is a Uniform Resource Locator (URL) and the media data comprises streaming media data.

23. The system of claim 21 wherein:

the first information comprises information related to properties of the media data; and

the code module for storing the media data in the memory as the plurality of objects comprises a code module for storing the plurality of objects in the memory such that each object in the plurality of objects is individually accessible from the memory using the object identifier generated for the object.

24. The system of claim 23 wherein the plurality of code modules stored in the memory further comprises:

a code module for storing mapping information that maps each object in the plurality of objects to an object identifier generated for the object.

25. The system of claim 24 wherein the mapping information stores information mapping the object identifier generated for an object stored in the memory to an object handle of the object.

26. The system of claim 24 wherein:

each object in the plurality of objects is a file in a plurality of files;

the code module for generating an object identifier for each object comprises a code module for generating a filename for each file in the plurality of files, wherein the filename for a file identifies the portion of the media data stored by the file and the version of the media data;

the code module for storing the media data in the memory as a plurality of objects comprises a code module for storing the plurality of files in the memory such that each file in the plurality of files is individually accessible from the memory using a filename generated for the file; and

the code module for storing the mapping information comprises a code module for storing information that maps each file in;

the plurality of files to a filename generated for the file.

27. The system of claim 24 wherein the code module for storing the media data in the memory as the plurality of objects comprises:

a code module for partitioning the media data into a plurality of portions such that each portion comprises media data of a pre-determined time duration;

a code module for storing each portion in the plurality of portions in an object from the plurality of objects; and

a code module for generating an object number for each object in the plurality of objects based upon the temporal position of the portion of the media data stored by the object in the media data.

28. The system of claim 27 wherein:

the first information comprises version information identifying the version of the media data; and

the code module for generating an object identifier for each object in the plurality of objects comprises a code module for generating an object identifier for each object in the plurality of objects using the version information and the object number generated for the object.

29. The system of claim 28 wherein the first information comprises encoding information for the media data and wherein the code module for generating an object identifier for each object in the plurality of objects comprises a code module for using the encoding information to generate the object identifier.

30. The system of claim 28 wherein the code module for generating an object identifier for each object in the plurality of objects using the version information and the object number generated for the object comprises:

a code module for generating a first value based upon the version information;

a code module for generating an object string using the first value and the object number generated for the object; and

a code module for generating the object identifier for the object using the object string.

31. The system of claim 30 wherein:

the code module for generating the first value comprises a code module for applying a first hashing technique to the version information to generate the first value;

the code module for generating the object string using the first value and the object number generated for the object comprises a code module for concatenating the first value and the object number to generate the object string; and

the code module for generating the object identifier for the object using the object string comprises a code module for applying a second hashing technique to the object string to generate the object identifier for the object.

32. The system of claim 27 wherein:

the first information comprises first information identifying the version of the media data, second information identifying authorship of the media data, third information identifying a title of the media data, and fourth information identifying a time duration of the media data; and

the code module for generating an object identifier for each object in the plurality of objects comprises a code module for generating an object identifier for each object in the plurality of objects using the version information, the second information, the third information, the fourth information, and the object number generated for the object.

33. The system of claim 24 wherein the plurality of code modules stored in the memory further comprises:

a code module for receiving a data request from a first system requesting media data associated with the data pointer;

a code module for reading the first information associated with the data pointer from the media data server storing the requested media data for the data pointer;

a code module for generating object identifiers for the data request based upon the requested media data and a portion of the first information read from the media data server, the object identifiers identifying a version of the requested media data;

a code module for searching the mapping information to determine if the object identifiers generated for the data request are included in the mapping information;

a code module for retrieving, from the cache, a set of objects corresponding to the object identifiers for the data request that are;

included in the mapping information; and

a code module for communicating media data stored by the set of objects retrieved from the cache to the first system.

34. A system for communicating media data comprising:

a processor;

a cache; and

a memory coupled to the processor, the memory configured to store a plurality of code modules for execution by the processor, the plurality of code modules comprising;

a code module for storing media data in the cache as a plurality of objects, each object in the plurality of objects storing a portion of the media data;

a code module for storing a first set of object identifiers corresponding to the plurality of objects, wherein an object identifier for an object identifies the portion of the media data stored by the object and a version of the media data;

a code module for receiving a data request from a first system requesting media data associated with a data pointer;

a code module for reading media data description information associated with the data pointer from a media data server storing the requested media data for the data pointer, the media data description information comprising information related to properties of the media data requested by the data request;

a code module for generating a second set of object identifiers for the data request based upon the requested media data and a portion of the media data description information, the object identifiers in the second set identifying the requested media data and a version of the requested media data associated with the data pointer;

a code module for determining a third set of object identifiers comprising one or more object identifiers from the second set that are included in the first set of object identifiers;

a code module for retrieving, from the cache, a set of objects corresponding to the object identifiers included in the third set of object identifiers; and

a code module for communicating media data stored by the set of objects retrieved from the cache to the first system.

35. The system of claim 34 wherein the code module for generating the second set of object identifiers for the data request comprises:

a code module for determining a plurality of object numbers for the requested media data; and

a code module for generating the second set of object identifiers for the data request using the portion of the media data description information and the plurality of object numbers.

36. The system of claim 35 wherein:

the media data description information comprises first information identifying a version of the media data stored by the media data server and requested by the data request; and

the code module for generating the second set of object identifiers for the data request comprises a code module for generating an object identifier for each object number in the plurality of object numbers using the first information and the object number.

37. The system of claim 36 wherein the media data description information comprises encoding information for the requested media data and wherein the code module for generating the second set of object identifiers for the data request comprises a code module for using the encoding information to generate the object identifiers.

38. The system of claim 36 wherein the code module for generating an object identifier for each object number in the plurality of object numbers using the first information and the object number comprises:

a code module for generating a first value based upon a portion of the media data description information including the first information;

a code module for generating an object string using the first value and the object number; and

a code module for generating the object identifier for the object using the object string.

39. The system of claim 35 wherein:

the media data description information comprises first information identifying a version of the media data stored by the media data server and requested by the data request, second information identifying authorship of the media data requested by the data request, third information identifying a title of the media data requested by the data request, and fourth information identifying a time duration of the media data requested by the data request; and

the code module for generating an object identifier for each object number in the plurality of object numbers comprises a code module for generating the object identifier using the first information, the second information, the third information, the fourth information, and the object number.

40. A computer program product stored on a computer-readable storage medium for storing media data in a cache, the computer program product comprising:

code for receiving information associated with a data pointer, the information comprising media data associated with the data pointer and stored by a media data server and first information comprising information identifying a version of the media data associated with the data pointer;

code for storing the media data in the cache as a plurality of objects, each object in the plurality of objects storing a portion of the media data; and

code for generating an object identifier for,each object in the plurality of objects based upon a portion of the first information ;

associated with the data pointer and the portion of the media data stored by each object, wherein the object identifier for an object identifies the portion of the media data stored by the object and the version of the media data.

41. The computer program product of claim 40 wherein the data pointer is a Uniform Resource Locator (URL) and the media data comprises streaming media data.

42. The computer program product of claim 40 wherein:

wherein the first information comprises information related to properties of the media data;

wherein the code for storing the media data in the cache as the plurality of objects comprises code for storing the plurality of objects in the cache such that each object in the plurality of objects is individually accessible from the cache using the object identifier generated for the object; and

the computer program product further comprises code for storing mapping information that maps each object in the plurality of objects to an object identifier generated for the object.

43. The computer program product of claim 42 wherein:

each object in the plurality of objects is a file in a plurality of files;

the code for generating an object identifier for each object comprises code for generating a filename for each file in the plurality of files, wherein the filename for a file identifies the portion of the media data stored by the file and the version of the media data;

the code for storing the media data in the cache as a plurality of objects comprises code for storing the plurality of files in the cache such that each file in the plurality of files is individually accessible from the cache using a filename generated for the file; and

the code for storing the mapping information comprises code for storing information that maps each filename generated for a file to the file.

44. The computer program product of claim 42 wherein the code for storing the media data in the cache as the plurality of objects comprises:

code for partitioning the media data into a plurality of portions such that each portion comprises media data of a predetermined time duration;

code for storing each portion in the plurality of portions in an object from the plurality of objects; and

code for generating an object number for each object in the plurality of objects based upon the temporal position of the portion of the media data stored by the object in the media data.

45. The computer program product of claim 44 wherein:

the first information comprises version information identifying the version of the media data; and

the code for generating an object identifier for each object in the plurality of objects comprises code for generating an object identifier for each object in the plurality of objects using the version information and the object number generated for the object.

46. The computer program product of claim 45 wherein the first information comprises encoding information for the media data and wherein the code for generating an object identifier for each object in the plurality of objects comprises code for using the encoding information to generate the object identifier.

47. The computer program product of claim 42 further comprising:

code for receiving a data request from a first system requesting media data associated with the data pointer;

code for reading the first information associated with the data pointer from the media data server storing the requested media data for the data pointer;

code for generating object identifiers for the data request based upon the requested media data and a portion of the first information read from the media data server, the object identifiers identifying a version of the requested media data;

code for searching the mapping information to determine if the object identifiers generated for the data request are included in the mapping information;

code for retrieving, from the cache, a set of objects corresponding to the object identifiers for the data request that are included in the mapping information; and

code for communicating media data stored by the set of objects retrieved from the cache to the first system.

48. A computer program product stored on a computer-readable storage medium for communicating media data from a cache in response to a data request, the computer program product comprising:

code for storing media data in the cache as a plurality of objects, each object in the plurality of objects storing a portion of the media data;

code for storing a first set of object identifiers corresponding to the plurality of objects, wherein an object identifier for an object identifies the portion of the media data stored by the object and a version of the media data;

code for receiving a data request from a first system requesting media data associated with a data pointer;

code for reading media data description information associated with the data pointer from a media data server storing the requested media data for the data pointer, the media data description information comprising information related to properties of the media data requested by the data request;

code for generating a second set of object identifiers for the data request based upon the requested media data and a portion of the media data description information, the object identifiers in the second set identifying the requested media data and a version of the requested media data associated with the data pointer;

code for determining a third set of object identifiers comprising one or more object identifiers from the second set that are included in the first set of object identifiers;

code for retrieving, from the cache, a set of objects corresponding to the object identifiers included in the third set of object identifiers; and

code for communicating media data stored by the set of objects retrieved from the cache to the first system.

49. The computer program product of claim 48 wherein the code for generating the second set of object identifiers for the data request comprises:

code for determining a plurality of object numbers for the requested media data; and

code for generating the second set of object identifiers for the data request using the portion of the media data description information and the plurality of object numbers.

50. The computer program product of claim 49 wherein:

the media data description information comprises first information identifying a version of the media data stored by the media data server and requested by the data request; and

the code for generating the second set of object identifiers for the data request comprises code for generating an object identifier for each object number in the plurality of object numbers using the first information and the object number.

51. The computer program product of claim 50 wherein the media data description information comprises encoding information for the requested media data and the code for generating the object identifiers for the data request comprises code for using the encoding information to generate the object identifiers.

52. A system for storing media data in a cache, comprising:

means for receiving information associated with a data pointer, the information comprising media data associated with the data pointer and stored by a media data server and first information comprising information identifying a version of the media data associated with the data pointer;

means for storing the media data in the cache as a plurality of objects, each object in the plurality of objects storing a portion of the media data; and

means for generating an object identifier for each object in the plurality of objects based upon a portion of the first information associated with the data pointer and the portion of the media data stored by each object, wherein the object identifier for an object identifies the portion of the media data stored by the object and the version of the media data.

53. A system for communicating media data from a cache in response to a data request, the system comprising:

means for storing media data in the cache as a plurality of objects, each object in the plurality of objects storing a portion of the media data;

means for storing a first set of object identifiers corresponding to the plurality of objects, wherein an object identifier for an object identifies the portion of the media data stored by the object and a version of the media data;

means for receiving a data request from a first system requesting media data associated with a data pointer;

means for reading media data description information associated with the data pointer from a media data server storing the requested media data for the data pointer, the media data description information comprising information related to properties of the media data requested by the data request;

means for generating a second set of object identifiers for the data request based upon the requested media data and a portion of the media data description information, the object identifiers in the second set identifying the requested media data and a version of the requested media data associated with the data pointer;

means for determining a third'"'"'set of object identifiers comprising one or more object identifiers from the second set that are included in the first set of object identifiers;

means for retrieving, from the cache, a set of objects corresponding to the object identifiers included in the third set of object identifiers; and

means for communicating media data stored by the set of objects retrieved from the cache to the first system.

54. A computer implemented method of storing information, the method comprising:

storing a first set of objects, the first set of objects storing a version of media data associated with a data pointer at a first time, each object storing a portion of the version of the media data;

storing a first set of object identifiers corresponding to the first set of objects, wherein an object identifier in the first set corresponding to an object in the first set of objects identifies the portion of the media data stored by the object and version of the stored media data;

storing a second set of objects, the second set of objects storing a second version of media data associated with the data pointer at a second time different from the first time, each object in the second set storing a portion of the second version of the media data; and

storing a second set of object identifiers corresponding to the second set of objects, wherein an object identifier in the second set corresponding to an object in the second set of objects identifies the portion of the media data stored by the object and version of the stored media data.

55. The method of claim 54 further comprising:

communicating the version of the media data stored by the first set of objects to a first system; and

communicating the second version of the media data stored by the second set of objects to a second system while the version of the media data is being communicated to the first system.