DYNAMICALLY PROCESSING AN EVENT USING AN EXTENSIBLE DATA MODEL

• US 20110246528A1
• Filed: 09/08/2010
• Published: 10/06/2011
• Est. Priority Date: 03/31/2010
• Status: Active Grant

• Current Assignee: Cloudera Incorporated
• Original Assignee: Cloudera Incorporated
• Inventors: Hsieh, Jonathan Ming-Cyn, Robinson, Henry Noel
• Granted Patent: US 8,874,526 B2
• Time in Patent Office: Days
• US Class Current: 707/791
• CPC Class Codes: G06F 16/211 Schema design and management
• CPC Class Codes: G06F 16/24568 Data stream processing; Continuous queries
• CPC Class Codes: H04L 67/01 Protocols
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]

Abstract

Systems and methods of dynamically processing an event using an extensible data model are disclosed. One embodiment includes, specifying attributes of the event in a data model; the data model being extensible to add properties to the event as the dataset is streamed from the source to the sink.

Claims

1. A method for dynamically processing an event including a dataset that is streamed from a source to a sink via nodes, the method, comprising:

specifying attributes of the event in a data model;

the data model being extensible to add properties to the event as the dataset is streamed from the source to the sink.

2. The method of claim 1, further comprising:

appending additional data fields to the data model in response to additional sources being added to the event.

3. The method of claim 1, wherein, the attributes include a priority.

4. The method of claim 1, wherein, the priority is specified by a logging system.

5. The method of claim 1, wherein, the attributes include a source machine.

6. The method of claim 1, wherein, the attributes include a body.

7. The method of claim 1, wherein, the attributes include a timestamp.

8. The method of claim 1, wherein, the attributes include a metadata table.

9. The method of claim 7, further comprising, using the timestamp to bucket the dataset based on time.

10. The method of claim 1, further comprising, grouping the dataset based on project identifier or person.

11. A method for dynamically processing an event including a dataset that is streamed from a source to a sink via nodes, the method, comprising:

specifying attributes of the event in a data model;

annotating the event described by the data model with key-value attribute pairs in a metadata table to enable analytic features to be performed on the dataset.

12. The method of claim 11, wherein, the metadata table is used to specify routing of the event to multiple or different storage locations.

13. The method of claim 11, wherein, the metadata table is used to specify outputting of the dataset at the sink in multiple or different formats.

14. The method of claim 11, wherein, the analytic features include feature extraction.

15. The method of claim 11, wherein, the analytic features include filtering.

16. The method of claim 11, wherein, the analytic features include transformation.

17. A system for dynamically processing an event including a dataset that is streamed from a source to a sink via nodes, the method, comprising:

means for, specifying attributes of the event in a data model;

the data model being extensible to add properties to the event as the dataset is streamed from the source to the sink;

means for, annotating the event described by the data model with key-value attribute pairs in a metadata table to enable analytic features to be performed on the dataset.

18. The system of claim 17, wherein, the metadata table is used to specify routing of the event to multiple or different storage locations.

19. The system of claim 17, wherein, the metadata table is used to specify outputting of the dataset at the sink in multiple or different formats.