Methods and apparatus for managing the storage of content in a file system

• US 20060294164A1
• Filed: 06/23/2005
• Published: 12/28/2006
• Est. Priority Date: 06/23/2005
• Status: Active Grant

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: EMC Corporation (Dell Technologies Inc.)
• Inventors: Todd, Stephen, Armangau, Philippe
• Granted Patent: US 7,805,470 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/137 Hash-based content-based indexing of textual data G06F16/31

Abstract

One embodiment of the invention is directed to the containerization of content units that are accessed using an identifier. Applicants have appreciated that file systems sometimes impose limits on the number of files that may be stored therein that are too restrictive. Thus, in one embodiment, multiple content units may be stored in a single file, called a container file, in a file system. Each content unit may have an identifier associated with it. When an accessing entity requests access to a previously-stored content unit and provides the identifier for the content unit, the identifier may be used to locate the container file in which the content unit is stored.

Claims

1. A method of managing content units in a file system, the method comprising acts of:

receiving a first request to store a first content unit, the first content unit having a first content address that is computed based, at least in part, on the content of the first content unit;

in response to the first request, storing the first content unit in a file in the file system;

receiving a second request to store the second content unit, the second content unit having a second address that is computed based, at least in part, on the content of the second content unit; and

in response to the second request, storing the second content unit in the file.

2. The method of claim 1, wherein the file system includes a time-based directory structure having a plurality of directories, wherein each of the plurality of directories corresponds to a period of time, and wherein the file is stored in one of the plurality of directories that corresponds to a period of time during which the first and second request requests were received.

3. The method of claim 2, where the first request is received from a storage system that stores the first content unit, and wherein the first content address includes a timestamp that indicates a time related to the time of storage of the first content unit on the storage system.

4. The method of claim 3, further comprising acts of:

in response to the first request, determining one the plurality of directories that corresponds to a period of time that includes the time indicated in the timestamp; and

storing, in the one the plurality of directories that corresponds to a period of time that includes the time indicated in the timestamp, an indication of the file that stores first the content unit.

5. The method of claim 4, further comprising an act of:

in response to the first request, associating an identifier with the first content unit.

6. The method of claim 5, where the identifier is the first content address.

7. The method of claim 5, further comprising an act of:

returning the identifier as acknowledgement of receipt of the first request.

8. The method of claim 5, further comprising acts of:

receiving a third request to access the first content unit, the request identifying the first content unit using the identifier;

locating the indication of the file that stores first the content unit using the identifier;

locating the file that stores the first content unit based on the indication;

locating the first content unit in the file; and

returning the first content unit in response to the third request.

9. The method of claim 3, wherein the file corresponds to a period of time that includes the times at which the first and second requests were received.

10. The method of claim 3, wherein the file corresponds to a period of time that is subsumed by the period time of the one of the plurality of directories in which the file is stored.

11. The method of claim 1, wherein the first request is a request to replicate the first content unit from a storage system that stores the first content unit.

12. The method of claim 2, further comprising an act of:

creating an entry for the first content unit in an index file, the entry identifying the file that stores the first content unit and wherein the index file is stored in one of the plurality of directories that corresponds to an index time for the first content unit.

13. The method of claim 12, wherein the first request is a request to replicate the first content unit from a storage system that stores the first content unit and wherein the index time is a time related to the time of storage of the first content unit on the storage system.

14. The method of claim 12, wherein the index time is a time related to the time of receipt of the first request.

15. The method of claim 12, wherein the index time is a time derived from the content address of the first content unit.

16. The method of claim 15, wherein the time derived from the content address of the first content unit is derived from a portion of the content address that is based on the content of the first content unit.

17. The method of claim 1, further comprising acts of:

receiving a third request to store a third content unit, the third content unit having a same content unit as the first content unit;

determining that the first and third content units have the same content; and

in response to determining that the first and third content units have the same content, declining to store the third content unit.

18. The method of claim 17, further comprising acts of:

receiving a fourth request to access the third content unit; and

in response to the fourth request, returning the first content unit.

19. At least one computer readable medium encoded with instructions that, when executed on a computer system, perform a method of managing content units in a file system, the method comprising acts of:

receiving a first request to store a first content unit, the first content unit having a first content address that is computed based, at least in part, on the content of the first content unit;

in response to the first request, storing the first content unit in a file in the file system;

receiving a second request to store the second content unit, the second content unit having a second address that is computed based, at least in part, on the content of the second content unit; and

in response to the second request, storing the second content unit in the file.

20. The at least one computer readable medium of claim 19, wherein the file system includes a time-based directory structure having a plurality of directories, wherein each of the plurality of directories corresponds to a period of time, and wherein the file is stored in one of the plurality of directories that corresponds to a period of time during which the first and second request requests were received.

21. The at least one computer readable medium of claim 20, where the first request is received from a storage system that stores the first content unit, and wherein the first content address includes a timestamp that indicates a time related to the time of storage of the first content unit on the storage system.

22. The at least one computer readable medium of claim 21, wherein the method further comprises acts of:

in response to the first request, determining one the plurality of directories that corresponds to a period of time that includes the time indicated in the timestamp; and

storing, in the one the plurality of directories that corresponds to a period of time that includes the time indicated in the timestamp, an indication of the file that stores first the content unit.

23. The at least one computer readable medium of claim 22, wherein the method further comprises an act of:

in response to the first request, associating an identifier with the first content unit.

24. The at least one computer readable medium of claim 23, where the identifier is the first content address.

25. The at least one computer readable medium of claim 23, wherein the method further comprises an act of:

returning the identifier as acknowledgement of receipt of the first request.

26. The at least one computer readable medium of claim 23, wherein the method further comprises acts of:

receiving a third request to access the first content unit, the request identifying the first content unit using the identifier;

locating the indication of the file that stores first the content unit using the identifier;

locating the file that stores the first content unit based on the indication;

locating the first content unit in the file; and

returning the first content unit in response to the third request.

27. The at least one computer readable medium of claim 21, wherein the file corresponds to a period of time that includes the times at which the first and second requests were received.

28. The at least one computer readable medium of claim 21, wherein the file corresponds to a period of time that is subsumed by the period time of the one of the plurality of directories in which the file is stored.

29. The at least one computer readable medium of claim 19, wherein the first request is a request to replicate the first content unit from a storage system that stores the first content unit.

35. The at least one computer readable medium of claim 19, wherein the method further comprises acts of:

receiving a third request to store a third content unit, the third content unit having a same content unit as the first content unit;

determining that the first and third content units have the same content; and

in response to determining that the first and third content units have the same content, declining to store the third content unit.

36. The at least one computer readable medium of claim 35, wherein the method further comprising acts of:

receiving a fourth request to access the third content unit; and

in response to the fourth request, returning the first content unit.

30. The at least one computer readable medium of clam 20, wherein the method further comprises an act of:

creating an entry for the first content unit in an index file, the entry identifying the file that stores the first content unit and wherein the index file is stored in one of the plurality of directories that corresponds to an index time for the first content unit.

31. The at least one computer readable medium of claim 30, wherein the first request is a request to replicate the first content unit from a storage system that stores the first content unit and wherein the index time is a time related to the time of storage of the first content unit on the storage system.

32. The at least one computer readable medium of claim 30, wherein the index time is a time related to the time of receipt of the first request.

33. The at least one computer readable medium of claim 30, wherein the index time is a time derived from the content address of the first content unit.

34. The at least one computer readable medium of claim 33, wherein the time derived from the content address of the first content unit is derived from a portion of the content address that is based on the content of the first content unit.

37. A computer that manages content units in a file system comprising:

at least one input; and

at least one controller coupled to the at least one input that;

receives, via the input, a first request to store a first content unit, the first content unit having a first content address that is computed based, at least in part, on the content of the first content unit;

in response to the first request, stores the first content unit in a file in the file system;

receives, via the input, a second request to store the second content unit, the second content unit having a second address that is computed based, at least in part, on the content of the second content unit; and

in response to the second request, stores the second content unit in the file.

38. The computer of claim 37, wherein the file system includes a time-based directory structure having a plurality of directories, wherein each of the plurality of directories corresponds to a period of time, and wherein the file is stored in one of the plurality of directories that corresponds to a period of time during which the first and second request requests were received.

39. The computer of claim 38, where the first request is received from a storage system that stores the first content unit, and wherein the first content address includes a timestamp that indicates a time related to the time of storage of the first content unit on the storage system.

40. The computer of claim 39, wherein the at least one controller:

in response to the first request, determines one the plurality of directories that corresponds to a period of time that includes the time indicated in the timestamp; and

stores, in the one the plurality of directories that corresponds to a period of time that includes the time indicated in the timestamp, an indication of the file that stores first the content unit.

41. The computer of claim 40, wherein the at least one controller:

in response to the first request, associates an identifier with the first content unit.

42. The computer of claim 41, where the identifier is the first content address.

43. The computer of claim 41, wherein the at least one controller:

returns the identifier as acknowledgement of receipt of the first request.

44. The computer of claim 41, wherein the at least one controller:

receives, via the input, a third request to access the first content unit, the request identifying the first content unit using the identifier;

locates the indication of the file that stores first the content unit using the identifier;

locates the file that stores the first content unit based on the indication;

locates the first content unit in the file; and

returns the first content unit in response to the third request.

45. The computer of claim 39, wherein the file corresponds to a period of time that includes the times at which the first and second requests were received.

46. The computer of claim 39, wherein the file corresponds to a period of time that is subsumed by the period time of the one of the plurality of directories in which the file is stored.

47. The computer of claim 37, wherein the first request is a request to replicate the first content unit from a storage system that stores the first content unit.

53. The computer of claim 37, wherein the at least one controller:

receives a third request to store a third content unit, the third content unit having a same content unit as the first content unit;

determines that the first and third content units have the same content; and

in response to determining that the first and third content units have the same content, declines to store the third content unit.

54. The computer of claim 53, wherein the at least one controller:

receives, via the input, a fourth request to access the third content unit; and

in response to the fourth request, returns the first content unit.

55. The computer of claim 37, wherein the at least one controller further comprises means for:

receiving, via the input, a first request to store a first content unit, the first content unit having a first content address that is computed based, at least in part, on the content of the first content unit;

in response to the first request, storing the first content unit in a file in the file system;

receiving, via the input, a second request to store the second content unit, the second content unit having a second address that is computed based, at least in part, on the content of the second content unit; and

in response to the second request, storing the second content unit in the file.

48. The computer of clam 38, wherein the at least one controller:

creates an entry for the first content unit in an index file, the entry identifying the file that stores the first content unit and wherein the index file is stored in one of the plurality of directories that corresponds to an index time for the first content unit.

49. The computer of claim 48, wherein the first request is a request to replicate the first content unit from a storage system that stores the first content unit and wherein the index time is a time related to the time of storage of the first content unit on the storage system.

50. The computer of claim 48, wherein the index time is a time related to the time of receipt of the first request.

51. The computer of claim 48, wherein the index time is a time derived from the content address of the first content unit.

52. The computer of claim 51, wherein the time derived from the content address of the first content unit is derived from a portion of the content address that is based on the content of the first content unit.