STORAGE ACCESS INTERFACE TO AN ENCODED STORAGE SYSTEM

• US 20190260388A1
• Filed: 06/20/2017
• Published: 08/22/2019
• Est. Priority Date: 06/20/2016
• Status: Active Grant

• Current Assignee: Anacode Labs, Inc.
• Original Assignee: Anacode Labs, Inc.
• Inventors: WEGENER, ALBERT W
• Granted Patent: US 10,972,125 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 16/00 Information retrieval; Database structures therefor; File system structures therefor
• CPC Class Codes: G06F 16/116 Details of conversion of file system types or formats
• CPC Class Codes: G06F 3/0608 Saving storage space on storage systems
• CPC Class Codes: G06F 3/0661 Format or protocol conversion arrangements
• CPC Class Codes: G06F 3/0673 Single storage device
• CPC Class Codes: H03M 7/30 Compression speech analysis-synthesis for redundancy reduction G10L19/00; for image communication H04N; Expansion; Suppression of unnecessary data, e.g. redundancy reduction
• CPC Class Codes: H03M 7/3059 Digital compression and data reduction techniques where the original information is represented by a subset or similar information, e.g. lossy compression
• CPC Class Codes: H03M 7/3086 employing a sliding window, e.g. LZ77
• CPC Class Codes: H03M 7/40 Conversion to or from variable length codes, e.g. Shannon-Fano code, Huffman code, Morse code
• CPC Class Codes: H03M 7/42 using table look-up for the coding or decoding process, e.g. using read-only memory H03M7/4006 takes precedence
• CPC Class Codes: H03M 7/6011 Encoder aspects

Abstract

A combination of a block-oriented encoder and decoder with a modified dataset identifier that is associated with an encoded block size are used to perform block-based encoding and decoding operations. The encoding process may generate optional metadata that includes an array of encoded block sizes to support random access into the stream or group of encoded blocks during the decoding process. The modified dataset identifier associates the original dataset identifier with the block size used by the encoder.

Claims

1. A computer implemented method that encodes an input dataset associated with an input dataset identifier, comprising:

receiving an input data set having a data set identifier from a source system;

determining a block size for encoding the data set;

sending the data set to an encoding engine which applies a lossless encoding process to a plurality of blocks of block size N of the data set to generate a plurality of encoded blocks; and

transmitting to the source system, a modified dataset identifier based on the input dataset identifier and associated with the plurality of encoded data blocks.

2. The method of claim 1, wherein the encoding engine generates an array of encoded block sizes, where each encoded block size is associated with a corresponding encoded block, and further including:

associating the plurality of encoded data blocks with the array of encoded block sizes.

3. The method of claim 1, including generating the modified data set identifier by modifying the input dataset identifier to include the block size N.

4. The method of claim 1, including associating the block size N with a metadata field of the modified data set identifier.

5. A computer implemented method that decodes an encoded dataset associated with an encoded dataset identifier, the encoded dataset being a losslessly encoded version of an input dataset, comprising:

receiving a command from a destination system to return requested data of the input dataset, the command including the encoded data set identifier;

sending a plurality of encoded data blocks associated with the encoded dataset identifier to a decoding engine to generate a plurality of decoded blocks of block size N; and

transmitting data from the plurality of decoded data blocks representing the requested. data of the input data set to the destination system.

6. The method of claim 5, wherein the command includes a range specifier, and further including:

selecting the plurality of encoded data blocks based on the range specifier; and

wherein the transmitting includes selecting the data from the plurality of decoded data blocks based on the range specifier.

7. The method of claim 5, wherein the command includes a range specifier, and blocks in the plurality of encoded data blocks have variant block sizes of N or less, further including:

storing an array of offsets based on the variant block sizes that identify locations of encoded data blocks in the plurality of encoded data blocks;

selecting the plurality of encoded data blocks by comparing the range specifier to the array of offsets to determine a start block and an end block for the plurality encoded blocks; and

selecting the data from the plurality of decoded data blocks based on the range specifier.

8. A computer system that encodes an input dataset associated with an input dataset identifier, comprising:

a processor, memory and a communication interface, including instructions executable by the processor to

receive an input data set having a data set identifier from a source system;

determine a block size for encoding the data set;

send the data set to an encoding engine which applies a lossless encoding process to a plurality of blocks of block size N of the data set to generate a plurality of encoded blocks; and

transmit to the source system, a modified dataset identifier based on the input dataset identifier and associated with the plurality of encoded data blocks.

9. The system of claim 8, wherein the encoding engine generates an array of encoded block sizes, where each encoded block size is associated with a corresponding encoded block, and further including instructions executable by the processor to

associate the plurality of encoded data blocks with the array of encoded block sizes.

10. The system of claim 8, including instructions executable by the processor to generate the modified data set identifier by

modifying the input dataset identifier to include the block size N.

11. The system of claim 8, including instructions executable by the processor to associate the block size N with a metadata field of the modified data set identifier.

12. A computer system that decodes an encoded dataset associated with an encoded dataset identifier, the encoded dataset being a losslessly encoded version of an input dataset, comprising:

a processor, memory and a communication interface, including instructions executable by the processor to

receive a command from a destination system to return requested data of the input dataset, the command including the encoded data set identifier;

send a plurality of encoded data blocks associated with the encoded dataset identifier to a decoding engine to generate a plurality of decoded blocks of block size N; and

transmit data from the plurality of decoded data blocks representing the requested data of the input data set to the destination system.

13. The system of claim 12, wherein the command includes a range specifier, and further including instructions executable by the processor to

select the plurality of encoded data blocks based on the range specifier;

and

wherein the instructions to transmit includes selecting the data from the plurality of decoded data blocks based on the range specifier.

14. The system of claim 12, wherein the command includes a range specifier, and blocks in the plurality of encoded data blocks have variant block sizes of N or less, further including instructions executable by the processor to:

store an array of offsets based on the variant block sizes that identify locations of encoded data blocks in the plurality of encoded data blocks;

select the plurality of encoded data blocks by comparing the range specifier to the array of offsets to determine a start block and an end block for the plurality encoded blocks; and

select the data from the plurality of decoded data blocks based on the range specifier.

15. A computer program product comprising a non-transitory computer readable medium for a method that encodes an input dataset associated with an input dataset identifier, comprising:

instructions executable by a computer stored in the computer readable medium to receive an input data set having a data set identifier from a source system;

determine a block size for encoding the data set;

send the data set to an encoding engine which applies a lossless encoding process to a plurality of blocks of block size N of the data set to generate a plurality of encoded blocks; and

transmit to the source system, a modified dataset identifier based on the input dataset identifier and associated with the plurality of encoded data blocks.

16. The computer program product of claim 15, wherein the encoding engine generates an array of encoded block sizes, where each encoded block size is associated with a corresponding encoded block, and further including instructions executable by the computer stored in the computer readable medium to

associate the plurality of encoded data blocks with the array of encoded block sizes.

17. The computer program product of claim 15, including instructions executable by the computer stored in the computer readable medium to generate the modified data set identifier by

modifying the input dataset identifier to include the block size N.

18. The computer program product of claim 15, including instructions executable by the computer stored in the computer readable medium to associate the block size N with a metadata field of the modified data set identifier.

19. A computer program product comprising a non-transitory computer readable medium for a method that decodes an encoded dataset associated with an encoded dataset identifier, the encoded dataset being a losslessly encoded version of an input dataset, comprising:

instructions executable by a computer stored in the computer readable medium to receive a command from a destination system to return requested data of the input dataset, the command including the encoded data set identifier;

send a plurality of encoded data blocks associated with the encoded dataset identifier to a decoding engine to generate a plurality of decoded blocks of block size N; and

transmit data from the plurality of decoded data blocks representing the requested data of the input data set to the destination system.

20. The computer program product of claim 19, wherein the command includes a range specifier, and further including instructions executable by the computer stored in the computer readable medium to

select the plurality of encoded data blocks based on the range specifier;

and

wherein the instructions to transmit includes selecting the data from the plurality of decoded data blocks based on the range specifier.

21. The computer program product of claim 19, wherein the command includes a range specifier, and blocks in the plurality of encoded data blocks have variant block sizes of N or less, further including instructions executable by the computer stored in the computer readable medium to:

store an array of offsets based on the variant block sizes that identify locations of encoded data blocks in the plurality of encoded data blocks;

select the plurality of encoded data blocks by comparing the range specifier to the array of offsets to determine a start block and an end block for the plurality encoded blocks; and

select the data from the plurality of decoded data blocks based on the range specifier.