Grouping and Differentiating Files Based on Content

• US 20110016138A1
• Filed: 12/15/2009
• Published: 01/20/2011
• Est. Priority Date: 07/16/2009
• Status: Active Grant

• Current Assignee: Micro Focus Software, Inc. (Open Text Corporation)
• Original Assignee: Novell Incorporated (Open Text Corporation)
• Inventors: Teerlink, Craig N.
• Granted Patent: US 9,053,120 B2
• Time in Patent Office: Days
• US Class Current: 707/758
• CPC Class Codes: G06F 16/1744 using compression, e.g. sparse files
• CPC Class Codes: G06F 16/285 Clustering or classification
• CPC Class Codes: G06T 9/005 Statistical coding, e.g. Huffman, run length coding
• CPC Class Codes: H04L 2209/30 Compression, e.g. Merkle-Damgard construction
• CPC Class Codes: H04L 2209/34 Encoding or coding, e.g. Huffman coding or error correction
• CPC Class Codes: H04L 9/0894 Escrow, recovery or storing of secret information, e.g. secret key escrow or cryptographic key storage
• CPC Class Codes: H04N 19/00 Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
• CPC Class Codes: Y10S 707/972 Partitioning

Abstract

Methods and apparatus teach a digital spectrum of a file. The digital spectrum is used to map a file'"'"'s position. This position relative to another file'"'"'s position reveals distances between the files. Representatively, files have a plurality of symbols representing an underlying data stream of original bits of data. The number of occurrences of each symbol in each file is compared to like symbols in other files. This can occur via algorithms, mapping, or both. In certain instances, comparison reveals a difference in counts between the symbols of the files. This difference is then squared, added together, and a square root taken. Comparing “distance values” reveals file adjacency, grouping, or the like. Also, normalizing, weighting, filtering functions and/or other statistical computations are applied in certain instances.

Claims

1. In a computing system environment, a method of differentiating files stored on one or more computing devices, each file having a plurality of symbols representing an underlying data stream of original bits of data, comprising:

determining a number of occurrences of each said symbol in said each file; and

computing a distance between said each file and every other file based on the determined number of occurrences.

2. The method of claim 1, further including normalizing the determined number of occurrences for said each symbol by comparing to a total number of symbols in a source file.

3. The method of claim 1, further including weighting the determined number of occurrences for said each symbol according to a bit length of said original bits of data represented by the symbols.

4. The method of claim 3, wherein the weighting further includes multiplying the determined number of occurrences for said each symbol by a corresponding said bit length.

5. The method of claim 1, further including grouping together closest neighbors of the files.

6. The method of claim 1, further including sorting into an ordered list the computed said distances between said each file and every other file.

7. The method of claim 1, further including normalizing and weighting the determined number of occurrences for said each symbol.

8. The method of claim 7, wherein the normalizing further includes comparing the determined number of occurrences for said each symbol to a total number of symbols in a source file.

9. The method of claim 7, wherein the weighting further includes multiplying the determined number of occurrences for said each symbol by a bit length of said original bits of data represented by the symbols.

10. In a computing system environment, a method of differentiating files stored on one or more computing devices, each file having a plurality of symbols representing an underlying data stream of original bits of data, comprising computing a distance between said each file and every other file based on said symbols.

11. The method of claim 10, further including determining a number of occurrences of each said symbol in said each file.

12. The method of claim 10, wherein the computing the distance further includes determining for like symbols a difference in said determined number of occurrences.

13. The method of claim 12, further including squaring the determined difference for said like symbols.

14. The method of claim 13, further including adding together the squared determined differences.

15. The method of claim 14, further including taking a square root of the added together squared determined differences.

16. In a computing system environment, a method of determining closest files stored on one or more computing devices, each file having a plurality of symbols representing an underlying data stream of original bits of data, comprising:

computing a distance value between said each file and every other file; and

concluding a closest two files based on the computed distance value.

17. The method of claim 16, wherein the computing the distance value further includes determining a number of occurrences of each said symbol in said each file.

18. The method of claim 17, further including normalizing the determined number of occurrences for said each symbol.

19. The method of claim 17, further including weighting the determined number of occurrences for said each symbol.

20. The method of claim 17, further including considering only a portion of the determined number of occurrences when said computing the distance.