Efficient data storage system
First Claim
Patent Images
1. A method for storing data comprising:
- receiving a data stream comprising a set of one or more substreams, wherein each substream includes a plurality of data segments from a single data source;
determining whether one of the plurality of data segments of a substream of the set has been stored previously in a low latency memory;
in the event that the data segment is determined to have been stored previously, returning an identifier for the previously stored data segments;
in the event that the data segment is not determined to have been stored, confirming whether the data segment has been stored; and
in the event that the data segment is confirmed to not have been stored, storing the data segment, comprising;
receiving the data segment in a receiving container in memory, wherein the receiving container receives additional data segments from the substream; and
writing the receiver container to a segment container in a segment database.
11 Assignments
0 Petitions
Accused Products
Abstract
A system and method are disclosed for providing efficient data storage. A plurality of data segments is received in a data stream. The system determines whether a data segment has been stored previously in a low latency memory. In the event that the data segment is determined to have been stored previously, an identifier for the previously stored data segment is returned.
268 Citations
29 Claims
-
1. A method for storing data comprising:
-
receiving a data stream comprising a set of one or more substreams, wherein each substream includes a plurality of data segments from a single data source; determining whether one of the plurality of data segments of a substream of the set has been stored previously in a low latency memory; in the event that the data segment is determined to have been stored previously, returning an identifier for the previously stored data segments; in the event that the data segment is not determined to have been stored, confirming whether the data segment has been stored; and in the event that the data segment is confirmed to not have been stored, storing the data segment, comprising; receiving the data segment in a receiving container in memory, wherein the receiving container receives additional data segments from the substream; and writing the receiver container to a segment container in a segment database. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27)
-
-
28. A data storage device comprising:
-
an interface adapted to receive an input data stream comprising a set of one or more substreams, wherein each substream includes a plurality of data segments from a single data source; a segment redundancy check engine adapted to determine using a low latency memory whether one of the plurality of data segments of a substream of the set has been stored previously; and
in the event that the data segment is determined to have been stored previously, return an identifier for the previously stored data segment, and in the event that the data segment is not determined to have been stored, confirm whether the data segment has been stored;a receiving container in memory adapted to receive the data segment and additional data segments from the substream in the event that the data segment is confirmed to not have been stored; and a segment database comprising a segment container, wherein the receiver container is written to the receiver container in the event that the data segment is confirmed to not have been stored.
-
-
29. A computer program product for storing data, the computer program product being embodied in a computer readable medium and comprising computer instructions for:
-
receiving a data stream comprising a set of one or more substreams, wherein each substream includes a plurality of data segments from a single data source; determining whether one of the plurality of data segments of a substream of the set has been stored previously; in the event that the data segment is determined to have been stored previously, returning an identifier for the previously stored data segment; in the event that the data segment is not determined to have been stored, confirming whether the data segment has been stored; and in the event that the data segment is confirmed to not have been stored, storing the data segment;
comprising;receiving the data segment in a receiving container in memory, wherein the receiving container receives additional data segments from the substream; and writing the receiver container to a segment container in a segment database.
-
Specification