Apparatus, system and method for secure, recoverable, adaptably compressed file transfer
First Claim
Patent Images
1. A system for managing and transmitting a file having a degree of compressibility and a plurality of buffers over a communication channel having a physical bandwidth comprising:
- a) a file transfer server at a first location operable to transmit a file;
b) a file transfer client at a second location operable to receive a file; and
c) a feedback loop for compressing the file for transmission, including;
1.) means for compressing a first buffer to a first level of compressibility;
2.) means for evaluating the throughput of the first buffer when the first buffer is transmitted from the first location to the second location;
3.) means for setting the compression level of a second buffer based on the throughput of the first buffer and the first level of compressibility.
1 Assignment
0 Petitions
Accused Products
Abstract
The present invention provides a system for handling and transmitting a file over a communication channel wherein the file may be adaptably compressed to improve throughput. The invention also provides for a method of recovery in the event of a communication failure. The file may be encrypted while it is being transmitted. The compression and transmission may occur while the file is being written, so that the receiving location receives the data in near real time.
161 Citations
78 Claims
-
1. A system for managing and transmitting a file having a degree of compressibility and a plurality of buffers over a communication channel having a physical bandwidth comprising:
-
a) a file transfer server at a first location operable to transmit a file; b) a file transfer client at a second location operable to receive a file; and c) a feedback loop for compressing the file for transmission, including; 1.) means for compressing a first buffer to a first level of compressibility; 2.) means for evaluating the throughput of the first buffer when the first buffer is transmitted from the first location to the second location; 3.) means for setting the compression level of a second buffer based on the throughput of the first buffer and the first level of compressibility. - View Dependent Claims (2, 3)
-
-
4. A system for managing and transmitting a file having a degree of compressibility and a plurality of buffers over a communication channel having a physical bandwidth comprising:
-
a) a file transfer server at a first location; b) a file transfer client at a second location; c) a throughput measuring mechanism at the first location which measures throughput for each transmitted block of data; and d) an adaptive compressor at the first location, wherein the adaptive compressor accepts as an input the throughput from previously transmitted data blocks. - View Dependent Claims (5, 6, 7)
-
-
8. A system for managing and transmitting a file having a degree of compressibility and a plurality of buffers over a communication channel comprising:
-
a.) a communications channel having a physical bandwidth; b.) a transmitting processor; c.) a receiving processor; d.) a means to determine transmission throughput; e.) a means for compressing the file by compressing a first buffer to a first degree of compressibility, compressing a second buffer to a second degree of compressibility, transmitting the first and second buffers and determining a relative throughput, based on the relative throughput, compressing a third buffer to a third degree of compressibilty; f.) a means for compressing the buffers of the file in a stream in real time while the file is being written; g.) a means for transmission of the file; h.) a means for maintaining a received location of the transmission within the file; i.) a means, responsive to an transmission interruption and subsequent resumption of transmission, for communicating the location of the data last received prior to the transmission interruption; j.) a means for encrypting the file in a stream before and during transmission; and k.) a means for decrypting the file after receipt of transmission.
-
-
9. A method of compressing a source file, having a plurality of buffers, for transmission over a communication channel, comprising:
-
a.) selecting a first buffer of the source file; b.) compressing the first buffer to a first compression level; c.) marshalling the first buffer onto the communication channel; d.) transmitting the first buffer; e.) de-compressing the first buffer; f.) writing the first buffer to a destination file; g.) determine a first throughput which was achieved for steps (a) through (f); h.) selecting a second compression level of a second buffer based on the first throughput wherein there is a difference between the first and second compression levels indicating a direction of change in compression level; and i.) repeating steps (a) through (h) for each of the buffers in turn until all of the buffers of the source file have been transmitted-- and have been written to the destination file. - View Dependent Claims (10, 11)
-
-
12. A system for handling and transmitting a first file having a sequential plurality of buffers where each buffer is either the first buffer, the second buffer, the last buffer or a buffer that has a previous and a next buffer, over a communication channel comprising:
-
a.) a means for reading while writing data to the first file; b.) a means for compressing each buffer of the first file at one of multiple levels of compression; c.) a means for queueing the first file; d.) a means for marshalling the first file; e.) a means for sequentially transmitting the buffers that constitute the first file from a first location to a second location and for each transmitted buffer determining the throughput achieved; f.) a means for unmarshalling the first file; g.) a means for decompressing the first file; and h.) a means for writing the first file to a second file; wherein the means for compressing b) for each buffer that is not the first buffer nor the second buffer, selects a compression level based on the throughput achieved in the transmission of the two preceding buffers. - View Dependent Claims (13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 36)
-
-
24. A system for handling and transmitting a first file having a sequential plurality of buffers where each buffer is either the first buffer, the second buffer, the last buffer or a buffer that has a previous and a next buffer, over a communication channel comprising:
-
a.) a means for reading while writing data to the first file; b.) a means for compressing each buffer of the first file at one of multiple levels of compression; c.) a means for queueing the first file; d.) a means for marshalling the first file; e.) a means for sequentially transmitting the buffers that constitute the first file from a first location to a second location and for each transmitted buffer determining the throughput achieved; f.) a first means for unmarshalling the first file; g.) a means for re-queueing the first file; h.) a means for re-marshalling the first file; i.) a remote procedure call for sending the first file within the second location; j.) a second means for unmarshalling the first file; k.) a means for decompressing the first file; and l.) a means for writing the first file to a second file; wherein the means for compressing b) for each buffer that is not the first buffer nor the second buffer, selects a compression level based on the throughput achieved in the transmission of the two preceding buffers. - View Dependent Claims (25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 37)
-
-
38. A method for handling and transmitting a first file consisting of a sequence of buffers over a communication channel comprising the steps of:
-
a.) reading while writing data to the first file; b.) compressing each buffer of the first file at one of multiple levels of compression; c.) queueing the first file; d.) marshalling the first file; e.) sequentially transmitting the buffers of the first file from a first location to a second location while determining the throughput achieved for each transmitted buffer; f.) unmarshalling the first file; g.) decompressing the first file; and h.) writing the first file to a second file; wherein the step of compressing b) comprises selecting a compression level based on the achieved throughput of previously transmitted buffers. - View Dependent Claims (39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50)
-
-
51. A method for handling and transmitting a first file consisting of a sequence of buffers over a communication channel comprising the steps of:
-
a.) reading while writing data to the first file; b.) compressing each buffer of the first file at one of multiple levels of compression; c.) queueing the first file; d.) marshalling the first file; e.) sequentially transmitting the buffers of the first file from a first location to a second location while determining the throughput achieved for each transmitted buffer; f.) unmarshalling the first file; g.) re-queueing the first file; h.) re-marshalling the first file; i.) sending via a remote procedure call the first file within the second location; j.) unmarshalling the first file a second time; k.) decompressing the first file; and l.) writing the first file to a second file; wherein the step of compressing b) comprises selecting a compression level based on the achieved throughput of previously transmitted buffers. - View Dependent Claims (52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64)
-
-
65. A system for acquiring and transmitting well log data, comprising:
-
a) a log data acquisition and processing software module operable to receive well log data; b) a primary file transfer utility program connected to the log data acquisition and processing software module and having a compression module, wherein the file transfer server comprises; i) a means for periodically adjusting the compression level based on a most recently used compression level and a throughput obtained using the most recently used compression level; and c) a remote file transfer utility program connected to the primary file transfer utility program. - View Dependent Claims (66, 67, 68, 69, 70, 71)
-
-
72. A method of transmitting a file over a communications channel from a first location to a second location, comprising:
-
a) compressing a first buffer of the file to a first compression level; b) transmitting the first buffer from the first location to the second location; c) compressing a second buffer of the file to a second compression level; d) transmitting the second buffer from the first location to the second location; e) evaluating the relative throughput of the first and second buffers from the first location to the second location; f) setting a third compression level as a function of the throughput of the first and second buffers; g) compressing a third buffer to the third compression level; and h) transmitting the third buffer from the first location to the second location. - View Dependent Claims (73, 74, 75, 76, 77)
-
-
78. A method of transmitting a file composed of multiple buffers, comprising:
-
a) transmitting a sequence of buffers having a first buffer, wherein, except for the first buffer, for each buffer there is a preceding buffer; b) prior to transmitting a buffer, adjusting the compression level with respect to the compression level used for the previous buffer; c) for each buffer determining its throughput; d) if the throughput improves with the compression level adjustment, adjusting the compression level in step b) in the same direction as used for the preceding buffer and if the throughput degrades with the compression level adjustment, adjusting the compression level in step b) in the opposite direction as used for the preceding buffer.
-
Specification