METHOD AND SYSTEM FOR EFFICIENT DATA TRANSMISSION WITH SERVER SIDE DE-DUPLICATION

• US 20100188273A1
• Filed: 03/31/2010
• Published: 07/29/2010
• Est. Priority Date: 11/18/2008
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: International Business Machines Corporation
• Inventors: Sheinin, Vadim, He, Dake
• Granted Patent: US 8,138,954 B2
• Time in Patent Office: Days
• US Class Current: 341/87
• CPC Class Codes: H03M 7/30 Compression speech analysis-synthesis for redundancy reduction G10L19/00; for image communication H04N; Expansion; Suppression of unnecessary data, e.g. redundancy reduction

Abstract

The invention provides a method and system for reducing redundant data blocks. The method includes reducing redundant data blocks by decoding a first data block from a first plurality of data blocks and a bitstream.

Claims

1. A method comprising:

reducing redundant data blocks by decoding a first data block from a first plurality of data blocks and a bitstream.

2. The method of claim 1, wherein each block in the first plurality of data blocks has a length equal to the first length.

3. The method of claim 1, wherein the decoding is performed with a Slepian-Wolf decoder.

4. The method of claim 1, wherein upon decoding being successful for the complete first length, de-duplication is performed on the first data block.

5. The method of claim 1, wherein upon the decoding being unsuccessful for the complete first length, requesting further information of the first data block from a client.

6. The method of claim 4, further comprising:

encoding the first data block having the first length into another bitstream having one of the second length and a third length;

transmitting the other bitstream to the server; and

reducing redundant data blocks by decoding the first data block from a second plurality of data blocks and the other bitstream, where each block in the second plurality of data blocks has a length equal to the first length.

7. The method of claim 1, wherein de-duplication is performed by the decoding.

8. The method of claim 1, wherein the decoding is performed with a variable length for a predetermined collision rate.

9. The method of claim 1, wherein a transmission rate for the transmitting is variable to meet a predetermined collision rate.

10. A system for reducing redundant data blocks, comprising:

a client device including an encoder module configured to encode a data block into a bitstream;

a server device including a decoder module configured to decode the bitstream using a plurality of previously stored data blocks on the server device; and

a de-duplication module coupled to the decoder module configured to deduplicate successful decoded portions of the data block.

11. The system of claim 10, further comprising a data storage device coupled to the server device.

12. The system of claim 10, further comprising at least another client device.

13. The system of claim 10, wherein the encoder module performs a Slepian-Wolf encoding.

14. The system of claim 10, wherein the decoder module performs a Slepian-Wolf decoding.

15. A computer program product comprising a computer usable medium including a computer readable program, wherein the computer readable program when executed on a computer causes the computer to:

encode a first data block having a first length into a bitstream having a second length;

transmit the bitstream to a server; and

reduce redundant data blocks by decoding the first data block from a first plurality of data blocks and the bitstream where each block in the first plurality of data blocks has a length equal to the first length.

16. The computer program product of claim 15, wherein the decoding is performed with a Slepian-Wolf decoder.

17. The computer program product of claim 15, wherein upon decoding being successful for the complete first length, de-duplication is performed on the first data block.

18. The computer program product of claim 15, wherein upon decoding being unsuccessful for the complete first length, requesting further information of the first data block from a client.

19. The computer program product of claim 15, further causing the computer to:

encode the first data block having the first length into another bitstream having one of the second length and a third length;

transmit the other bitstream to the server; and

reduce redundant data blocks by decoding the first data block from a second plurality of data blocks and the other bitstream, where each block in the second plurality of data blocks has a length equal to the first length.

20. The computer program product of claim 15, wherein a transmission rate for the transmitting is variable to meet a predetermined collision rate.

21. A method comprising:

encoding a first data block having a first length into a bitstream having a second length using a Slepian-Wolf encoding process;

transmitting the bitstream to a server device; and

reducing redundant data blocks before de-duplication using a Slepian-Wolf decoding process by decoding the first data block from a first plurality of data blocks and the bitstream, where each block in the first plurality of data blocks has a length equal to the first length.

22. The method of claim 21, wherein upon the decoding being unsuccessful for the complete first length, requesting further information of the first data block from a client.

23. The method of claim 22, further comprising:

encoding the first data block having the first length into another bitstream having one of the second length and a third length;

transmitting the other bitstream to the server device; and

reducing redundant data blocks by decoding the first data block from a second plurality of data blocks and the other bitstream, where each block in the second plurality of data blocks has a length equal to the first length.

24. A system for reducing transmission of redundant data blocks, comprising:

a client device including a Slepian-Wolf encoder module configured to encode a data block into a bitstream;

a server device including a Slepian-Wolf decoder module configured to decode the bitstream using a plurality of previously stored data blocks on the server device; and

a de-duplication module coupled to the decoder module configured to deduplicate successful decoded portions of the data block.

25. The system of claim 24, wherein the Slepian-Wolf decoder is configured to reduce redundant data blocks before de-duplication.