NORMALIZING NON-NUMERIC FEATURES OF FILES

• US 20160154793A1
• Filed: 12/13/2015
• Published: 06/02/2016
• Est. Priority Date: 11/28/2014
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Li, Chang Sheng, MENG, FAN JING, Stern, Edith Helen, Wang, Han, Xu, Jing Min, Yang, Lin, ZHUO, XUEJUN
• Granted Patent: US 10,078,667 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 16/2455 Query execution
• CPC Class Codes: G06F 16/24578 using ranking
• CPC Class Codes: G06F 16/248 Presentation of query results
• CPC Class Codes: H04L 41/0806 for initial configuration or provisioning, e.g. plug-and-play

Abstract

Embodiments include method, computer program products and apparatuses for normalizing non-numeric features of files and corresponding apparatus Aspects include segmenting at least one pair of positive instances of a non-numeric feature of a file into a number of tokens and comparing the tokens in the at least one pair of positive instances to obtain matching tokens. Aspects also include calculating weights of their matching the file, for the matching tokens, and storing the tokens and their weights in a token base.

Claims

1. A computer-implemented method for normalizing non-numeric features of files, comprising:

segmenting at least one pair of positive instances of a non-numeric feature of a file into a number of tokens;

comparing the tokens in the at least one pair of positive instances to obtain matching tokens; and

for each of the matching tokens, calculating weights of their matching the file, and storing the tokens and their weights in a token base.

2. The computer-implemented method of claim 1, further comprising:

forming token sequences from the matching tokens as segments of the positive instances; and

for each of the token sequences, calculating weights of their matching the file, and storing the token sequences and their weights in the token base.

3. The computer-implemented method of claim 2, further comprising:

in response to receiving a test instance of the non-numeric feature of a test file, segmenting the test instance into a number of tokens and token sequences;

calculating a score that the test instance matches the file by matching the tokens and the token sequences in the test instance with the tokens and token sequences in the token base, as a normalized numeric feature of the non-numeric feature of the test file.

4. The computer-implemented method of claim 1, wherein segmenting the positive instance into a number of tokens is performed by using structural characteristics of the non-numeric feature.

5. The computer-implemented method of claim 1, wherein comparing the tokens in the at least one pair of positive instances to obtain matching tokens comprises:

calculating the maximum matching score between each token in a positive instance with the tokens in another positive instance;

selecting the tokens of which the maximum matching scores are greater than a given threshold, to get the matching tokens.

6. The computer-implemented method of claim 1, wherein for a matching token or token sequence, calculating a weight of its matches the file comprises:

according to at least one of the number of appearances of the matching token or token sequence in the group of positive instance, the token length of the token or the token sequence, and the token position of the token or the token sequence in the positive instance, calculating a weight of its matching the file, wherein the greater the number of appearances is, the greater the token length is or the more backward the token position is, the greater the weight is.

7. The computer-implemented method of claim 3, wherein calculating a score that the test instance matches the file by matching the tokens and the token sequences in the test instance with the tokens and the token sequences in the token base comprises:

obtaining different combinations formed by tokens and token sequences of the test instance;

comparing each token and token sequence in each combination with the tokens and token sequences in the token base, to get the most similar token or token sequence in the token base, and a similarity and the weight thereof;

for each combination, according to the similarities and weights of the most similar tokens or token sequences of its tokens and token sequences, calculating a weight of the combination; and

obtaining the maximum weight in the different combinations, as the score that the test instance matches the file.

8. The computer-implemented method of claim 3, further comprising:

determining whether the test file is the give file according to the normalized numeric features of the test file.

9. The computer-implemented method of claim 8, wherein the determining whether the test file is the give file according to the normalized numeric feature of the test file comprises:

providing the normalized numeric feature of the test file to a classifier to determine whether the test file is the file.