Methods and systems for single instance storage of asset parts
First Claim
1. A method for maintaining single instance storage of a plurality of asset parts in a data storage system comprising a plurality of storage nodes, the method comprising:
- using at least one processor for;
maintaining a journal associated with the plurality of the asset parts, the journal comprising entries having generation numbers that are larger than generation numbers of previous entries;
creating a first entry to the journal corresponding to a request to add or remove a reference from an asset to an asset part, the first entry further comprising a first generation number assigned thereto;
performing a confirmation cycle, wherein the confirmation cycle comprisescomparing the first generation number to a second generation number that is associated with the asset part, andwhen the first generation number is larger than the second generation number, causing a reference count associated with the asset part to be changed; and
wherein each of the plurality of asset parts comprises asset part content and asset part metadata, the asset part metadata being associated with a stripe, wherein the asset part content and the stripe are replicated across at least two of the plurality of storage nodes, forming at least one replicated asset part content and at least one replicated stripe having a replicated journal, and wherein creating the first entry further comprises creating the first entry in the replicated journal.
3 Assignments
0 Petitions
Accused Products
Abstract
Methods and systems are disclosed that relate to single instance storage of asset parts utilizing a back reference count associated with each asset part. Entries in a journal associated with the asset parts reflect the intention to increment or decrement an asset part'"'"'s back reference count. The journal may be used to maintain an accurate and reliable reference count for asset parts replicated across multiple disk drives in a distributed storage system, where the replication is structured to reduce the risk of data loss in the event of a disk drive failure. The journal entries may comprise generation numbers used to maintain a back reference count associated with an asset part and to verify a back reference count associated with an asset part.
73 Citations
18 Claims
-
1. A method for maintaining single instance storage of a plurality of asset parts in a data storage system comprising a plurality of storage nodes, the method comprising:
using at least one processor for; maintaining a journal associated with the plurality of the asset parts, the journal comprising entries having generation numbers that are larger than generation numbers of previous entries; creating a first entry to the journal corresponding to a request to add or remove a reference from an asset to an asset part, the first entry further comprising a first generation number assigned thereto; performing a confirmation cycle, wherein the confirmation cycle comprises comparing the first generation number to a second generation number that is associated with the asset part, and when the first generation number is larger than the second generation number, causing a reference count associated with the asset part to be changed; and wherein each of the plurality of asset parts comprises asset part content and asset part metadata, the asset part metadata being associated with a stripe, wherein the asset part content and the stripe are replicated across at least two of the plurality of storage nodes, forming at least one replicated asset part content and at least one replicated stripe having a replicated journal, and wherein creating the first entry further comprises creating the first entry in the replicated journal. - View Dependent Claims (2, 3, 4, 5, 6)
-
7. A data storage system for maintaining single instance storage of a plurality of asset parts, the data storage system comprising:
-
a plurality of disk drives storing assets and a plurality of asset parts;
at least one processor configured tomaintain a journal associated with the plurality of the asset parts, the journal comprising entries having generation numbers that are larger than generation numbers of previous entries; create a first entry to the journal corresponding to a request to add or remove a reference from an asset to an asset part, the first entry further comprising a first generation number assigned thereto; and perform a confirmation cycle, wherein the confirmation cycle comprises comparing the first generation number to a second generation number that is associated with the asset part, and, when the first generation number is larger than the second generation number, causing a reference count associated with the asset part to be changed; and a plurality of storage nodes, wherein each of the plurality of asset parts comprises asset part content and asset part metadata, the asset part metadata being associated with a stripe, wherein the asset part content and the stripe are replicated across at least two of the plurality of storage nodes, forming at least one replicated asset part content and at least one replicated stripe having a replicated journal, and wherein creating the first entry further comprises creating the first entry in the replicated journal. - View Dependent Claims (8, 9, 10, 11, 12)
-
-
13. A non-transitory computer readable medium comprising executable instructions maintaining single instance storage of a plurality of asset parts in a data storage system comprising a plurality of storage nodes, the instructions, when executed by a computer system, causing the computer system to:
-
maintain a journal associated with the plurality of asset parts, the journal comprising entries having generation numbers that are larger than generation numbers of previous entries; create a first entry to the journal corresponding to a request to add or remove a reference from an asset to an asset part, the first entry further comprising a first generation number assigned thereto; and perform a confirmation cycle, wherein the confirmation cycle comprises comparing the first generation number to a second generation number that is associated with the asset part, and when the first generation number is larger than the second generation number, causing a reference count associated with the asset part to be changed; and wherein each of the plurality of asset parts comprises asset part content and asset part metadata, the asset part metadata being associated with a stripe, wherein the asset part content and the stripe are replicated across at least two of the plurality of storage nodes, forming at least one replicated asset part content and at least one replicated stripe having a replicated journal, and wherein creating the first entry further comprises creating the first entry in the replicated journal. - View Dependent Claims (14, 15, 16, 17, 18)
-
Specification