Methods and apparatus for creation of parsing rules

• US 20050022207A1
• Filed: 07/25/2003
• Published: 01/27/2005
• Est. Priority Date: 07/25/2003
• Status: Active Grant

• Current Assignee: Meta Platforms, Inc. (f/k/a Facebook, Inc.)
• Original Assignee: International Business Machines Corporation
• Inventors: Grabarnik, Genady, Thoenen, David H., Perng, Chang-shing, Salahshour, Abdolreza, Ma, Sheng
• Granted Patent: US 7,343,604 B2
• Time in Patent Office: Days
• US Class Current: 719/313
• CPC Class Codes: G06F 8/427 Parsing

Abstract

Techniques for parsing rule creation are provided. A technique for constructing one or more message parsing rules may comprise the following steps. First, message data representing past messages, for example, associated with a network, an application and/or a system being analyzed, is obtained. For example, this may involve reading the past or historical message data from messages logs or having a system point to the message data in existing data storage. Parsing rules are then generated by a process from one or more existing rule templates and/or based on user selection and classification of at least a portion of a message. For example, the user may choose a message part and demonstratively classify the part, for example, as a positive or negative example. The generated rules may then be stored for access by a rule-based parsing system such as a message adaptation system. Prior to generation of the one or more parsing rules, a message structure may be established upon which generation of the rules may be based.

Claims

1. A method of constructing one or more message parsing rules in accordance with a user and a machine, comprising the steps of:

obtaining message data representing past messages; and

generating one or more message parsing rules by a process based on at least one of one or more existing rule templates and user selection and classification of at least a portion of a message, wherein the one or more parsing rules are storable for access by a rule-based parsing system.

2. The method of claim 1, wherein the past messages are associated with one of a network, an application and a system being analyzed.

3. The method of claim 2, wherein the message data is obtained by at least one of:

(i) reading past message data from one or more messages logs; and

(ii) one of the network, the application and the system pointing to message data in existing data storage.

4. The method of claim 1, wherein the rule-based parsing system comprises a message adaptation system.

5. The method of claim 1, further comprising the step of establishing a message structure prior to the generating step.

6. The method of claim 5, wherein when one or more existing rule templates are available, the step of establishing a message structure comprises the steps of:

creating a message skeleton;

matching the one or more rule templates against the message skeleton; and

providing potential matches to the user for validation and choice of a proper message structure.

7. The method of claim 6, wherein when the message structure is found to be insufficient, templates are built by an iterative process between the user and the machine based on user selection of at least a portion of the message.

8. The method of claim 7, wherein the iterative process further comprises demonstrative classification of the selected portion as one of a positive example and a negative example.

9. The method of claim 6, wherein the message skeleton comprises information relating to one or more of a message start, a message end, and a separator between fields.

10. The method of claim 1, wherein classification comprises user demonstration of at least one of a positive example and a negative example.

11. The method of claim 10, wherein classification further comprises the steps of:

the machine parsing message data sequentially until an unparseable message is encountered;

the machine displaying the unparseable message to the user;

the user selecting at least a portion of the unparseable message and marking the selected portion as one of a positive example and a negative example; and

the machine learning based on the example and creating one or more candidate rules.

12. The method of claim 11, further comprising the step of the machine revising the one or more candidate rules based on feedback from the user.

13. The method of claim 1, wherein each of the one or more generated parsing rules comprises a regular expression of a portion of a message.

14. The method of claim 1, wherein each of the one or more generated parsing rules comprises a transformation rule of a portion of a message.

15. The method of claim 14, wherein the transformation rule comprises a string constant.

16. The method of claim 14, wherein the transformation rule comprises a permutation of one or more input tokens.

17. Apparatus for constructing one or more message parsing rules, comprising:

a memory; and

at least one machine-based processor coupled to the memory and operative to;

(i) obtain message data representing past messages; and

(ii) generate one or more message parsing rules by a process based on at least one of one or more existing rule templates and user selection and classification of at least a portion of a message, wherein the one or more parsing rules are storable for access by a rule-based parsing system.

18. The apparatus of claim 17, wherein the rule-based parsing system comprises a message adaptation system.

19. The apparatus of claim 17, wherein the processor is further operative to establish a message structure prior to the generating operation.

20. The apparatus of claim 19, wherein when one or more existing rule templates are available, the operation of establishing a message structure comprises:

creating a message skeleton;

matching the one or more rule templates against the message skeleton; and

providing potential matches to the user for validation and choice of a proper message structure.

21. The apparatus of claim 17, wherein classification comprises user demonstration of at least one of a positive example and a negative example.

22. The apparatus of claim 21, wherein classification further comprises the operations of:

the machine parsing message data sequentially until an unparseable message is encountered;

the machine displaying the unparseable message to the user;

the user selecting at least a portion of the unparseable message and marking the selected portion as one of a positive example and a negative example; and

the machine learning based on the example and creating one or more candidate rules.

23. An article of manufacture for constructing one or more message parsing rules in accordance with a user and a machine, comprising a machine readable medium containing one or more programs which when executed implement the steps of:

obtaining message data representing past messages; and

generating one or more message parsing rules by a process based on at least one of one or more existing rule templates and user selection and classification of at least a portion of a message, wherein the one or more parsing rules are storable for access by a rule-based parsing system.