Systems and Methods for Securely Transmitting Large Data Files

• US 20180357431A1
• Filed: 07/26/2018
• Published: 12/13/2018
• Est. Priority Date: 04/21/2017
• Status: Active Grant

• Current Assignee: Ntropy, LLC
• Original Assignee: infOsci, LLC
• Inventors: ELLINGSON, John, HANSEN, Robert M.
• Granted Patent: US 10,546,139 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 16/178 Techniques for file synchronisation in file systems
• CPC Class Codes: G06F 21/606 by securing the transmission between two devices or processes
• CPC Class Codes: H04L 63/0272 Virtual private networks
• CPC Class Codes: H04L 63/0428 wherein the data content is protected, e.g. by encrypting or encapsulating the payload
• CPC Class Codes: H04L 63/08 for authentication of entities cryptographic mechanisms or cryptographic arrangements for entity authentication H04L9/32
• CPC Class Codes: H04L 63/102 Entity profiles
• CPC Class Codes: H04L 67/06 specially adapted for file transfer, e.g. file transfer protocol [FTP]
• CPC Class Codes: H04W 12/033 of the user plane, e.g. user's traffic

Abstract

In methods, systems, and computing devices configured to implement methods of conveying a data file from a first computing device to a second computing device. A referential complex dataset (RCD) is stored in memory of a first computing device and a second computing device. The first computing device may compare bit strings within a data file to bit strings within the RCD to identify a matching bit string in the RCD, generate a set of rules for locating the matched bit string in the RCD, and transmit the rule set to the second computing device. The second computing device may receive the rule set, sequentially use each rule in the rule set to identify corresponding bit strings in the RCD in memory, and copy the identified bit strings into a memory to replicate the data file.

Claims

1. A method of conveying a data file from a first computing device to a second computing device, comprising:

storing a referential complex dataset (RCD) in memory of the first computing device and in memory of the second computing device;

performing by a processor of the first computing device the following operations sequentially on the data file until the entire data file has been processed;

comparing bit strings within the data file to bit strings within the RCD to identify a matching bit string in the RCD;

generating a rule for locating the matched bit string in the RCD; and

storing the generated rule sequentially in a rule set; and

transmitting the rule set to the second computing device.

2. The method of claim 1, wherein storing the RCD in memory of the first computing device and in memory of the second computing device comprises:

generating, by the processor of the first computing device, a rule for changing the RCD stored in the memories of both the first computing device and the second computing device;

applying, by the processor of the first computing device, the generated rule to the RCD stored in memory of the first computing device to generate a changed RCD; and

transmitting the rule to the second computing device in a format that will enable a processor of the second computing device to apply the rule to the RCD stored in the memory of the second computing device to generate the changed RCD in the second computing device.

3. The method of claim 1, wherein storing the RCD in memory of the first computing device and in memory of the second computing device comprises:

generating the RCD by the processor of the first computing device; and

transmitting the RCD to the second computing device for storage.

4. The method of claim 3, wherein generating the RCD by the processor of the first computing device comprises:

analyzing one or more data files representative of the data file to be transmitted to identify long bit strings that appear at least a threshold number of times;

storing in the RCD each long bit string that appears at least a threshold number of times; and

generating an index for locating each long bit string stored in the RCD, and

wherein generating a rule for locating the matched bit string in the RCD and storing the generated rule sequentially in a rule set comprises storing the index for the matched bit string in the rule set.

5. The method of claim 4, further comprising:

generating metadata corresponding to each long bit string stored in the RCD that provides information useful for comparing data files to the RCD to identify matching long bit strings; and

storing the generated metadata in association with the corresponding long bit string,

wherein comparing bit strings within the data file to bit strings within the RCD to identify a matching bit string in the RCD comprises using metadata associated with a matched bit string in the RCD to determine whether a longer bit string including the same matched bits exists in the RCD.

6. The method of claim 4, further comprising:

storing a subscriber RCD in memory of the first computing device and in memory of the second computing device;

performing by the processor of the first computing device the following operations sequentially on the rule set until the entire rule set has been processed;

comparing a bit string of each rule in the rule set to bit strings within the subscriber RCD to identify a matching bit string in the subscriber RCD;

generating a secondary rule for locating the matched bit string in the subscriber RCD; and

storing the generated secondary rule in a secondary rule set,

wherein transmitting the rule set to the second computing device comprises transmitting the secondary rule set to the second computing device.

7. The method of claim 6, further comprising:

receiving the secondary rule set in the second computing device;

sequentially using each secondary rule in the secondary rule set to identify corresponding bit strings in the subscriber RCD in memory;

using the identified corresponding bit strings in the subscriber RCD as rule to identify corresponding bit strings in the RCD in memory; and

copying the identified bit strings into a memory to replicate the data file.

8. The method of claim 4, wherein the data file to be transmitted is a multimedia file and analyzing one or more data files representative of the data file comprises analyzing one or more multimedia files having a similarity to the multimedia file to be transmitted.

9. The method of claim 1, further comprising:

receiving the rule set in the second computing device;

sequentially using each rule in the rule set to identify corresponding bit strings in the RCD in memory; and

copying the identified bit strings into a memory to replicate the data file.

10. The method of claim 9, further comprising:

processing the replicated data file using an application compatible with the data file processed by the first computing device.

11. A method of receiving a data file in a first computing device from a second computing device, comprising:

storing a referential complex dataset (RCD) in memory of the first computing device that matches an RCD stored in the second computing device;

receiving a rule set from the second computing device;

sequentially using each rule in the rule set to identify corresponding bit strings in the RCD in memory; and

copying the identified bit strings into a memory to replicate the data file.

12. The method of claim 11, further comprising:

processing the replicated data file using an application compatible with the data file.

13. The method of claim 11, further comprising:

storing a subscriber RCD in memory of the first computing device;

receiving a secondary rule set from the second computing device;

sequentially using each secondary rule in the secondary rule set to identify corresponding bit strings in the subscriber RCD in memory;

using the identified corresponding bit strings in the subscriber RCD as rule to identify corresponding bit strings in the RCD in memory; and

copying the identified bit strings into a memory to replicate the data file.

14. A computing device, comprising:

a memory; and

a processor coupled to the memory and configured with processor-executable instructions to perform operations comprising;

storing in the memory a referential complex dataset (RCD) that is the same as an RCD stored in a memory of a second computing device;

performing the following operations sequentially on the data file until the entire data file has been processed;

comparing bit strings within the data file to bit strings within the RCD to identify a matching bit string in the RCD;

generating a rule for locating the matched bit string in the RCD; and

storing the generated rule sequentially in a rule set; and

transmitting the rule set to the second computing device.

15. The computing device of claim 14, wherein the processor is configured with processor-executable instructions to perform operations such that storing in the memory the RCD that is the same as the RCD stored in the memory of the second computing device comprises:

generating a rule for changing the RCD stored in memories of both the first computing device and the second computing device;

applying the generated rule to the RCD stored in the memory to generate a changed RCD; and

transmitting the rule to the second computing device in a format that will enable a processor of the second computing device to apply the rule to the RCD to generate the changed RCD in the second computing device.

16. The computing device of claim 14, wherein the processor is configured with processor-executable instructions to perform operations such that storing in the memory the RCD that is the same as the RCD stored in the memory of the second computing device comprises:

generating the RCD by the processor of the first computing device; and

transmitting the RCD to the second computing device for storage.

17. The computing device of claim 16, wherein the processor is configured with processor-executable instructions to perform operations such that generating the RCD by the processor of the first computing device comprises:

analyzing one or more data files representative of the data file to be transmitted to identify long bit strings that appear at least a threshold number of times;

storing in the RCD each long bit string that appear at least a threshold number of times; and

generating an index for locating each long bit string stored in the RCD, and

wherein generating a rule for locating the matched bit string in the RCD and storing the generated rule sequentially in a rule set comprises storing the index for the matched bit string in the rule set.

18. The computing device of claim 17, wherein the processor is configured with processor-executable instructions to perform operations further comprising:

generating metadata corresponding to each long bit string stored in the RCD that provides information useful for comparing data files to the RCD to identify matching long bit strings; and

storing the generated metadata in association with the corresponding long bit string,

wherein comparing bit strings within the data file to bit strings within the RCD to identify a matching bit string in the RCD comprises using metadata associated with a matched bit string in the RCD to determine whether a longer bit string including the same matched bits exists in the RCD.

19. The computing device of claim 17, wherein the processor is configured with processor-executable instructions to perform operations further comprising:

storing a subscriber RCD in the memory of the first computing device;

providing the subscriber RCD to the second computing device for storage in the memory of the second computing device;

performing by the processor of the first computing device the following operations sequentially on the rule set until the entire rule set has been processed;

comparing a bit string of each rule in the rule set to bit strings within the subscriber RCD to identify a matching bit string in the subscriber RCD;

generating a secondary rule for locating the matched bit string in the subscriber RCD; and

storing the generated secondary rule in a secondary rule set,

wherein transmitting the rule set to the second computing device comprises transmitting the secondary rule set to the second computing device.

20. The computing device of claim 17, wherein the data file to be transmitted is a multimedia file and analyzing one or more data files representative of the data file comprises analyzing one or more multimedia files having a similarity to the multimedia file to be transmitted.

21. A computing device, comprising:

a memory; and

a processor coupled to the memory and configured with processor-executable instructions to perform operations comprising;

storing a referential complex dataset (RCD) in memory of the computing device that matches an RCD stored in a second computing device;

receiving a rule set from the second computing device;

sequentially using each rule in the rule set to identify corresponding bit strings in the RCD in memory; and

copying the identified bit strings into a memory to replicate the data file.

22. The computing device of claim 21, further comprising:

processing the replicated data file using an application compatible with the data file.

23. The computing device of claim 21, further comprising:

storing a subscriber RCD in memory of the first computing device;

receiving a secondary rule set from the second computing device;

sequentially using each secondary rule in the secondary rule set to identify corresponding bit strings in the subscriber RCD in memory;

using the identified corresponding bit strings in the subscriber RCD as rule to identify corresponding bit strings in the RCD in memory; and

copying the identified bit strings into a memory to replicate the data file.

24. A non-transitory processor-readable storage medium having stored thereon processor-executable instructions configured to cause a processor of a computing device to perform operations comprising:

storing in a memory a referential complex dataset (RCD) that is the same as an RCD stored in a memory of a second computing device;

performing the following operations sequentially on the data file until the entire data file has been processed;

comparing bit strings within the data file to bit strings within the RCD to identify a matching bit string in the RCD;

generating a rule for locating the matched bit string in the RCD; and

storing the generated rule sequentially in a rule set; and

transmitting the rule set to the second computing device.

25. The non-transitory processor-readable storage medium of claim 24, wherein the stored processor-executable instructions are configured to cause the processor of the computing device to perform operations such that storing in the memory the RCD that is the same as the RCD stored in the memory of the second computing device comprises:

generating a rule for changing the RCD stored in memories of both the first computing device and the second computing device;

applying the generated rule to the RCD stored in the memory to generate a changed RCD; and

transmitting the rule to the second computing device in a format that will enable a processor of the second computing device to apply the rule to the RCD to generate the changed RCD in the second computing device.

26. The non-transitory processor-readable storage medium of claim 24, wherein the stored processor-executable instructions are configured to cause the processor of the computing device to perform operations such that storing in the memory the RCD that is the same as the RCD stored in the memory of the second computing device:

generating the RCD; and

transmitting the RCD to the second computing device for storage.

27. The non-transitory processor-readable storage medium of claim 26, wherein the stored processor-executable instructions are configured to cause the processor of the computing device to perform operations such that generating the RCD comprises:

analyzing one or more data files representative of the data file to be transmitted to identify long bit strings that appear at least a threshold number of times;

storing in the RCD each long bit string that appear at least a threshold number of times; and

generating an index for locating each long bit string stored in the RCD, and

wherein generating a rule for locating the matched bit string in the RCD and storing the generated rule sequentially in a rule set comprises storing the index for the matched bit string in the rule set.

28. The non-transitory processor-readable storage medium of claim 27, wherein the stored processor-executable instructions are configured to cause the processor of the computing device to perform operations further comprising:

generating metadata corresponding to each long bit string stored in the RCD that provides information useful for comparing data files to the RCD to identify matching long bit strings; and

storing the generated metadata in association with the corresponding long bit string,

wherein comparing bit strings within the data file to bit strings within the RCD to identify a matching bit string in the RCD comprises using metadata associated with a matched bit string in the RCD to determine whether a longer bit string including the same matched bits exists in the RCD.

29. The non-transitory processor-readable storage medium of claim 27, wherein the stored processor-executable instructions are configured to cause the processor of the computing device to perform operations further comprising:

storing a subscriber RCD in the memory of the first computing device;

providing the subscriber RCD to the second computing device for storage in the memory of the second computing device;

performing by the processor of the first computing device the following operations sequentially on the rule set until the entire rule set has been processed;

comparing a bit string of each rule in the rule set to bit strings within the subscriber RCD to identify a matching bit string in the subscriber RCD;

generating a secondary rule for locating the matched bit string in the subscriber RCD; and

storing the generated secondary rule in a secondary rule set,

wherein transmitting the rule set to the second computing device comprises transmitting the secondary rule set to the second computing device.

30. The non-transitory processor-readable storage medium of claim 27, wherein the data file to be transmitted is a multimedia file and analyzing one or more data files representative of the data file comprises analyzing one or more multimedia files having a similarity to the multimedia file to be transmitted.

31. A non-transitory processor-readable storage medium having stored thereon processor-executable instructions configured to cause a processor of a computing device to perform operations comprising:

storing a referential complex dataset (RCD) in memory of the computing device that matches an RCD stored in a second computing device;

receiving a rule set from the second computing device;

sequentially using each rule in the rule set to identify corresponding bit strings in the RCD in memory; and

copying the identified bit strings into a memory to replicate the data file.

32. The non-transitory processor-readable storage medium of claim 31, wherein the stored processor-executable instructions are configured to cause the processor of the computing device to perform operations further comprising:

processing the replicated data file using an application compatible with the data file.

33. The non-transitory processor-readable storage medium of claim 31, wherein the stored processor-executable instructions are configured to cause the processor of the computing device to perform operations further comprising:

storing a subscriber RCD in memory of the computing device;

receiving a secondary rule set from the second computing device;

sequentially using each secondary rule in the secondary rule set to identify corresponding bit strings in the subscriber RCD in memory;

using the identified corresponding bit strings in the subscriber RCD as rule to identify corresponding bit strings in the RCD in memory; and

copying the identified bit strings into a memory to replicate the data file.