System and method for keyword spotting using multiple character encoding schemes

• US 8,990,238 B2
• Filed: 04/26/2012
• Issued: 03/24/2015
• Est. Priority Date: 04/27/2011
• Status: Active Grant

• Current Assignee: Cognyte Technologies Israel Ltd. (Cognyte Software Ltd.)
• Original Assignee: Verint Systems Limited (Verint Systems Incorporated)
• Inventors: Goldfarb, Eithan
• Primary Examiner(s): Chau, Dung K
• Application Number:
  US13/457,373
• Publication Number:
  US 20120303648A1
• Time in Patent Office: 1,062 Days
• Field of Search: None
• US Class Current: 707/760
• CPC Class Codes: G06F 40/126 Character encoding
• CPC Class Codes: G06F 40/289 Phrasal analysis, e.g. finite state techniques or chunking

Abstract

Methods and systems for finding search phrases in a body of data that is encoded using any of multiple possible character encoding schemes. An analytics system accepts an input search phrase for searching in a certain body of data. The system identifies two or more candidate character encoding schemes, which may have been used for encoding the body of data. Having determined the candidate encoding schemes, the system translates the input search phrase into multiple encoding-specific search phrases that represent the input search phrase in the respective candidate encoding schemes. The system then searches the body of data for occurrences of the input search phrase using the multiple encoding-specific search phrases.

Claims

1. A method, comprising:

accepting an input search phrase to be located in a body of data;

identifying multiple candidate character encoding schemes using one or more characteristics of the input search phrase;

translating the input search phrase into multiple encoding-specific search phrases, each encoding-specific search phrase representing the input search phrase in a different, respective candidate character encoding scheme; and

identifying one or more occurrences of the input search phrase in the body of data by searching the body of data using each of the multiple encoding-specific search phrases.

2. The method according to claim 1, wherein the input search phrase comprises at least one wildcard operator, and wherein translating the input search phrase comprises representing the wildcard operator in each of the encoding-specific search phrases using a respective encoding-specific wildcard operator that is equivalent to the wildcard operator in the respective candidate character encoding scheme.

3. The method according to claim 1, wherein the body of data comprises network traffic that is received from a communication network.

4. The method according to claim 1, wherein the identifying multiple candidate character encoding schemes comprises identifying a language in which the input search phrase or the body of data is written, and selecting the candidate character encoding schemes based on the identified language.

5. The method according to claim 1, wherein the translating input search phrase comprises producing at least one encoding-specific search phrase that matches left-to-right text and at least one other encoding-specific search phrase that matches right-to-left text.

6. The method according to claim 1, further comprising reporting the identified occurrences of the input search phrase to an operator.

7. The method according to claim 1, wherein the identifying one or more occurrences of the input search phrase comprises searching the body of data using only a partial subset of the candidate character encoding schemes, selected based on a characteristic of the body of data.

8. The method according to claim 7, further comprising selecting the partial subset based on one or more escape sequences found in the body of data.

9. Apparatus, comprising:

an interface, which is configured to accept an input search phrase to be located in a body of data; and

a processor, which is configured to identify multiple candidate character encoding schemes using one or more characteristics of the input search phrase, to translate the input search phrase into multiple encoding-specific search phrases, each encoding-specific search phrase representing the input search phrase in a different, respective candidate character encoding scheme, and to identify one or more occurrences of the input search phrase in the body of data by searching the body of data using each of the multiple encoding-specific search phrases.

10. The apparatus according to claim 9, wherein the input search phrase comprises at least one wildcard operator, and wherein the processor is configured to represent the wildcard operator in each of the encoding-specific search phrases using a respective encoding-specific wildcard operator that is equivalent to the wildcard operator in the respective candidate character encoding scheme.

11. The apparatus according to claim 9, wherein the body of data comprises network traffic that is received from a communication network.

12. The apparatus according to claim 9, wherein the processor is configured to identify a language in which the input search phrase or the body of data is written, and to select the candidate character encoding schemes based on the identified language.

13. The apparatus according to claim 9, wherein the processor is configured to produce at least one encoding-specific search phrase that matches left-to-right text and at least one other encoding-specific search phrase that matches right-to-left text.

14. The apparatus according to claim 9, wherein the processor is configured to report the identified occurrences of the input search phrase to an operator.

15. The apparatus according to claim 9, wherein the processor is configured to search the body of data using only a partial subset of the candidate character encoding schemes, selected based on a characteristic of the body of data.

16. The apparatus according to claim 15, wherein the processor is configured to select the partial subset based on one or more escape sequences found in the body of data.