METHOD AND APPARATUS FOR PROCESSING TEXT INFORMATION

• US 20160321353A1
• Filed: 06/08/2016
• Published: 11/03/2016
• Est. Priority Date: 01/06/2014
• Status: Active Grant

• Current Assignee: Tencent Technology Shenzhen Company Limited (Tencent Holdings Limited)
• Original Assignee: Tencent Technology Shenzhen Company Limited (Tencent Holdings Limited)
• Inventors: CHEN, Shengrong
• Granted Patent: US 10,387,460 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/313 Selection or weighting of terms for indexing
• CPC Class Codes: G06F 16/328 Management therefor
• CPC Class Codes: G06F 40/103 Formatting, i.e. changing of presentation of documents automatic justification G06F40/189; automatic line break hyphenation G06F40/191
• CPC Class Codes: G06F 40/126 Character encoding
• CPC Class Codes: G06F 40/216 using statistical methods
• CPC Class Codes: G06Q 10/107 Computer-aided management of electronic mailing [e-mailing]

Abstract

The present application relates to a A method for processing text information is provided, the method including: obtaining text information and extracting text characters contained in the text information; extracting, from the text characters, target characters satisfying a predetermined rule; calculating a filtering index of the text information according to the target characters; and when the filtering index meets a predetermined condition, executing an operation corresponding to the predetermined condition on the text information. In addition, an apparatus for processing text information is further provided. The method and apparatus for processing text information can improve the accuracy and efficiency of filtering out junk text information.

Claims

1. A method for processing text information, comprising:

obtaining text information and extracting text characters contained in the text information;

extracting, from the text characters, target characters satisfying a predetermined rule;

calculating a filtering index of the text information according to the target characters; and

when the filtering index meets a predetermined condition, executing an operation corresponding to the predetermined condition on the text information.

2. The method according to claim 1, wherein the step of extracting, from the text characters, target characters satisfying a predetermined rule comprises:

matching the extracted text characters one by one with interfering information characters recorded in a preset interfering character dictionary table, and using characters that are in the text characters and match the interfering information characters as the target characters.

3. The method according to claim 1, wherein the step of calculating a filtering index of the text information according to the target characters comprises:

separately converting the text characters and the target characters into characters in a given encoding form; and

calculating a ratio of the total number of bytes of the converted target characters to the total number of bytes of remaining characters in the text characters excluding the target characters, and using the ratio as the filtering index of the text information.

4. The method according to claim 1, wherein the step of, when the filtering index meets a predetermined condition, executing an operation corresponding to the predetermined condition on the text information comprises:

when the filtering index falls within a preset numerical range, determining that the text information is junk text information, and executing an operation corresponding to the predetermined condition on the text information.

5. The method according to claim 4, wherein the method further comprises:

when the filtering index does not fall within the preset numerical range, analyzing the filtering index to determine whether the filtering index is greater than a maximum value of the preset numerical range or the filtering index is less than a minimum value of the preset numerical range, and assigning a corresponding weight value to the filtering index according to an analysis result and a preset weight table;

analyzing the text characters to determine whether the text characters contain characters matching preset target keywords and analyzing the situation of the characters that are contained in the text characters and match the target keywords, and assigning a corresponding weight value to an analysis result according to the preset weight table;

obtaining a user identifier of a sender of the text information, matching the user identifier with user identifiers in a preset blacklist and whitelist, and assigning a corresponding weight value to a matching result according to the preset weight table; and

obtaining respective base values of the filtering index, the analysis result, and the matching result, and performing a weighting operation on the respective base values, the weight value corresponding to the filtering index, the weight value corresponding to the analysis result, and the weight value corresponding to the matching result; and

when a value obtained after the weighting operation is greater than a preset numerical value, determining that the text information is the junk text information, and executing the corresponding operation on the text information.

6. The method according to claim 1, wherein the step of, when the filtering index meets a predetermined condition, executing an operation corresponding to the predetermined condition on the text information comprises:

analyzing a magnitude relationship between the filtering index and a preset numerical range, and assigning a corresponding weight value to the filtering index according to an analysis result and a preset weight table;

analyzing the text characters to determine whether the text characters contain characters matching preset target keywords and analyzing the situation of the characters that are contained in the text characters and match the target keywords, and assigning a corresponding weight value to an analysis result according to the preset weight table;

obtaining a user identifier of a sender of the text information, matching the user identifier with user identifiers in a preset blacklist and whitelist, and assigning a corresponding weight value to a matching result according to the preset weight table; and

obtaining respective base values of the filtering index, the analysis result, and the matching result, and performing a weighting operation on the respective base values, the weight value corresponding to the filtering index, the weight value corresponding to the analysis result, and the weight value corresponding to the matching result; and

when a value obtained after the weighting operation is greater than a preset numerical value, determining that the text information is the junk text information, and executing the corresponding operation on the text information.

7. The method according to claim 1, wherein the operation corresponding to the predetermined condition comprises:

intercepting or deleting the text information, or giving a prompt in a preset prompting manner.

8. The method according to claim 3, wherein the given encoding form is an 8-bit Unicode transformation format (UTF-8) encoding form.

14. The apparatus according to claim 2, wherein the processing module comprises:

a first weight value assignment unit, configured to;

when the filtering index does not fall within a preset numerical range, analyze the filtering index to determine whether the filtering index is greater than a maximum value of the preset numerical range or the filtering index is less than a minimum value of the preset numerical range, and assign a corresponding weight value to the filtering index according to an analysis result and a preset weight table;

a second weight value assignment unit, configured to analyze the text characters to determine whether the text characters contain characters matching preset target keywords and analyze the situation of the characters that are contained in the text characters and match the target keywords, and assign a corresponding weight value to an analysis result according to the preset weight table;

a third weight value assignment unit, configured to obtain a user identifier of a sender of the text information, match the user identifier with user identifiers in a preset blacklist and whitelist, and assign a corresponding weight value to a matching result according to the preset weight table; and

an operational unit, configured to obtain respective base values of the filtering index, the analysis result, and the matching result, and perform a weighting operation on the respective base values, the weight value corresponding to the filtering index, the weight value corresponding to the analysis result, and the weight value corresponding to the matching result; and

a determining and execution unit, configured to, when a value obtained after the weighting operation is greater than a preset numerical value, determines that the text information is the junk text information, and executes the corresponding operation on the text information.

9. An apparatus for processing text information, comprising:

a text character extraction module, configured to obtain text information and extract text characters contained in the text information;

a target character extraction module, configured to extract, from the text characters extracted by the text character extraction module, target characters satisfying a predetermined rule;

a calculation module, configured to calculate a filtering index of the text information according to the target characters extracted by the target character extraction module; and

a processing module, configured to, when the filtering index calculated by the calculation module meets a predetermined condition, execute an operation corresponding to the predetermined condition on the text information.

10. The apparatus according to claim 9, wherein the target character extraction module is configured to match the extracted text characters one by one with interfering information characters recorded in a preset interfering character dictionary table, and use characters that are in the text characters and match the interfering information characters as the target characters.

11. The apparatus according to claim 9, wherein the calculation module comprises:

a conversion unit, configured to separately convert the text characters and the target characters into characters in a given encoding form; and

a calculation unit, configured to calculate a ratio of the total number of bytes of the converted target characters to the total number of bytes of remaining characters in the text characters excluding the target characters, and use the ratio as the filtering index of the text information.

12. The apparatus according to claim 9, wherein the processing module comprises:

a determining and execution unit, configured to when the filtering index falls within a preset numerical range, determine that the text information is junk text information, and execute an operation corresponding to the predetermined condition on the text information.

13. The apparatus according to claim 9, wherein the processing module further comprises:

a first weight value assignment unit, configured to analyze a magnitude relationship between the filtering index and the preset numerical range, and assign a corresponding weight value to the filtering index according to an analysis result and a preset weight table;

a second weight value assignment unit, configured to analyze the text characters to determine whether the text characters contain characters matching preset target keywords and analyze the situation of the characters that are contained in the text characters and match the target keywords, and assign a corresponding weight value to an analysis result according to the preset weight table;

a third weight value assignment unit, configured to obtain a user identifier of a sender of the text information, match the user identifier with user identifiers in a preset blacklist and whitelist, and assign a corresponding weight value to a matching result according to the preset weight table; and

an operational unit, configured to obtain respective base values of the filtering index, the analysis result, and the matching result, and perform a weighting operation on the respective base values, the weight value corresponding to the filtering index, the weight value corresponding to the analysis result, and the weight value corresponding to the matching result, wherein

when a value obtained after the weighting operation is greater than a preset numerical value, the determining and execution unit determines that the text information is the junk text information, and executes the corresponding operation on the text information.

15. The apparatus according to claim 9, wherein the operation corresponding to the predetermined condition comprises:

intercepting or deleting the text information, or giving a prompt in a preset prompting manner.

16. The apparatus according to claim 11, wherein the given encoding form is an 8-bit Unicode transformation format (UTF-8) encoding form.