Stopping functions for grouping and differentiating files based on content

• US 8,874,578 B2
• Filed: 12/30/2009
• Issued: 10/28/2014
• Est. Priority Date: 07/16/2009
• Status: Expired due to Fees

• Current Assignee: Micro Focus Software, Inc. (Open Text Corporation)
• Original Assignee: Novell Incorporated (Open Text Corporation)
• Inventors: Isaacson, Scott A., Teerlink, Craig N., Nazeer, Nadeem A.
• Primary Examiner(s): Beausoliel, Jr., Robert
• Assistant Examiner(s): BROMELL, ALEXANDRIA Y
• Application Number:
  US12/649,688
• Publication Number:
  US 20110016101A1
• Time in Patent Office: 1,763 Days
• Field of Search: 707/737, 707/755
• US Class Current: 707/737
• CPC Class Codes: G06F 16/1744 using compression, e.g. sparse files
• CPC Class Codes: G06F 16/285 Clustering or classification
• CPC Class Codes: G06T 9/005 Statistical coding, e.g. Huffman, run length coding
• CPC Class Codes: H04L 2209/30 Compression, e.g. Merkle-Damgard construction
• CPC Class Codes: H04L 2209/34 Encoding or coding, e.g. Huffman coding or error correction
• CPC Class Codes: H04L 9/0894 Escrow, recovery or storing of secret information, e.g. secret key escrow or cryptographic key storage
• CPC Class Codes: H04N 19/00 Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
• CPC Class Codes: Y10S 707/972 Partitioning

Abstract

Methods and apparatus teach a digital spectrum of a data file. The digital spectrum is used to map a file'"'"'s position in multi-dimensional space. This position relative to another file'"'"'s position reveals closest neighbors. Certain of the closest neighbors are grouped together, while others are differentiated. Grouping ceases upon application of a stopping function so that rightly sized, optimum numbers of file groups are obtained. Embodiments of stopping functions relate to curve types in a mapping of numbers of groups per sequential rounds of grouping, recognizing whether groups have overlapping file members or not, and/or determining whether groups meet predetermined numbers of members, to name a few. Properly grouped files can then be further acted upon.

Claims

1. In a computing system environment, a method of differentiating files stored on one or more computing devices, comprising:

over a sequence of rounds, grouping together files compressed according to an original relationship of highly occurring patterns in all bits of binary data of the uncompressed data of the files without any prior classification scheme nor metadata analysis of said files, the original relationship including a distance value between said compressed files in a multi-dimensional space;

determining a tolerance value for the sequence of rounds based on a minimum and a maximum said distance value; and

applying a stopping function to the grouping together.

2. The method of claim 1, further including determining a relationship between numbers of groups per each round of the sequence of rounds.

3. The method of claim 2, wherein the relationship between the numbers of groups per said each round includes a curve and the applying the stopping function occurs relative to an identified type of said curve.

4. The method of claim 2, further including determining a decrease in the numbers of groups per said each round.

5. The method of claim 4, wherein the applying the stopping function occurs when the decrease is about 5% to 50% relative to an earlier round of the sequence of rounds.

6. The method of claim 2, further including determining a local minimum in the numbers of groups per said each round.

7. The method of claim 6, wherein the applying the stopping function occurs upon reaching the local minimum.

8. The method of claim 1, further including determining a total number of rounds of the sequence of rounds, wherein the applying the stopping function occurs at about one fourth of the determined total number of rounds.

9. The method of claim 1, further including applying a linear or exponential stepping function.

10. In a computing system environment, a method of differentiating files stored on one or more computing devices, comprising:

over a sequence of rounds, grouping together files compressed according to an original relationship of highly occurring patterns in all bits of binary data of uncompressed data of the files without any prior classification scheme nor metadata analysis of said files, the original relationship including a distance value between said compressed files in a multi-dimensional space;

determining a tolerance value for the sequence of rounds based on a minimum and a maximum said distance value; and

applying a stopping function to the grouping together to obtain rightly sized pluralities of file groups.

11. The method of claim 10, wherein the applying the stopping function occurs when individual groups of the pluralities of file groups have no overlapping members of files.

12. The method of claim 11, further including redefining at least two said individual groups initially having overlapping members of files to be mutually exclusive in members of files.

13. The method of claim 10, further including applying the stopping function upon reaching a predetermined decrease in numbers of groups from an original number of groups of the pluralities of file groups.

14. In a computing system environment, a method of differentiating files stored on one or more computing devices, comprising:

over a sequence of rounds, grouping together files compressed according to an original relationship of highly occurring patterns in all bits of binary data of uncompressed data of the files without any prior classification scheme nor metadata analysis of said files, the original relationship including a distance value between said compressed files in a multi-dimensional space;

determining a tolerance value for the sequence of rounds based on a minimum and a maximum said distance value; and

applying a stopping function to the grouping together to obtain rightly sized pluralities of file groups having members of files below a predetermined numerical threshold.

15. The method of claim 14, further including determining a size of said members of files for each of said file groups.

16. The method of claim 14, further including applying the stopping function upon reaching the predetermined numerical threshold.

17. The method of claim 14, further including applying the stopping function upon said members of files in each of said file groups reaching about one fourth or less of a maximum number of original file groups.

18. The method of claim 14, furthering including dismissing any of said file groups having members of files above the predetermined numerical threshold.