METHODS AND SYSTEMS FOR OBJECT LEVEL DE-DUPLICATION FOR SOLID STATE DEVICES

• US 20120137054A1
• Filed: 11/23/2011
• Published: 05/31/2012
• Est. Priority Date: 11/24/2010
• Status: Active Grant

• Current Assignee: JP Morgan Chase Bank, N.A. (JP Morgan Chase & Co)
• Original Assignee: STEC Incorporated (Western Digital Corporation)
• Inventors: Sadri, Mohammad R., Choudhuri, Siddharth
• Granted Patent: US 10,162,553 B2
• Time in Patent Office: Days
• US Class Current: 711/103
• CPC Class Codes: G06F 11/1453 using de-duplication of the data
• CPC Class Codes: G06F 3/0608 Saving storage space on storage systems
• CPC Class Codes: G06F 3/0641 De-duplication techniques
• CPC Class Codes: G06F 3/0688 Non-volatile semiconductor memory arrays

Abstract

In one aspect, the present disclosure relates to a method of de-duplicating data in a solid state storage device. The method can include receiving a block of data to be written to a solid state storage device, wherein the block of data comprises header portion and a payload, wherein the header portion comprises context information; and determining whether the payload should be de-duplicated prior to storage, based on the context information stored within the header portion; if the payload is determined to be de-duplicated, de-duplicating the payload; and storing the de-duplicated payload to the solid state storage device.

Claims

1. A method of de-duplicating data in a solid state storage device (SSD) comprising:

receiving a block of data to be written to a solid state storage device, wherein the block of data comprises a header portion and a payload, wherein the header portion comprises context information; and

determining whether the payload should be de-duplicated prior to storage, based on the context information stored within the header portion;

if the payload is determined to be de-duplicated,

de-duplicating the payload; and

storing the de-duplicated payload to the solid state storage device.

2. The method of claim 1, wherein the context information comprises metadata passed down to the SSD from application layers.

3. The method of claim 2, wherein the metadata comprises an indication of whether the payload comprises random data.

4. The method of claim 2, wherein the metadata comprises a sensitivity level.

5. The method claim 1, wherein payload data comprises a plurality of sub-objects.

6. The method of claim 5 comprising de-duplicating based on a correlation between sub-objects of other payloads.

7. The method of claim 1, wherein the SSD performs de-duplication.

8. The method of claim 1, comprising before storing, separating out the header from payload and storing each in separate locations on the SSD.

9. The method of claim 1, wherein de-duplication occurs during garbage collection.

10. A method for storing data objects comprising:

receiving a data object;

analyzing the data file to determine context information about the data object; and

storing the context information in a header of the data object.

11. The method of claim 10, comprising:

separating the data object into a plurality of sub-objects; and

creating a separate header for each for the plurality of sub-objects.

12. The method of claim 10, wherein the context information comprises a sensitivity level.

13. The method of claim 10, wherein the context information comprises an indication of whether the payload comprises random data.

14. A solid state device for de-duplicating data prior to storage comprising

a host interface for communicating to a host device;

a flash memory for storing data;

a de-duplication engine comprising logic stored in a computer readable medium, the logic configured to;

receive a block of data to be written to a solid state storage device, wherein the block of data comprises a header portion and a payload, wherein the header portion comprises context information; and

determine whether the payload should be de-duplicated prior to storage, based on the context information stored within the header portion;

if the payload is determined to be de-duplicated,

de-duplicate the payload; and

store the de-duplicated payload to the solid state storage device.

15. The solid state device of claim 14, wherein the context information comprises metadata passed down to the SSD from application layers.

16. The solid state device of claim 14, wherein the metadata comprises an indication of whether the payload comprises random data.

17. The solid state device of claim 14, wherein the metadata comprises a sensitivity level.

18. The solid state device of claim 14, wherein payload data comprises a plurality of sub-objects.