×

Methods and systems for high throughput information refinement

  • US 8,090,873 B1
  • Filed: 03/14/2005
  • Issued: 01/03/2012
  • Est. Priority Date: 03/14/2005
  • Status: Active Grant
First Claim
Patent Images

1. A method for processing messages, the method comprising the steps of:

  • a computer system;

    receiving, from a plurality of different data sources in a data network, a plurality of data messages each having a data type associated therewith and comprising a payload, wherein the plurality of data messages comprises different data formats;

    for each of the data messages, determining a classification of the data message by parsing out information identifying the data type;

    re-formatting each of the data messages, each of the re-formatted messages having a uniform data structure comprising an identifier, the classification, and the payload;

    selecting a message service queue for each of the reformatted messages from a plurality of message service queues each dedicated to storing messages of a particular data type according to the classifications of the reformatted messages such that each of the selected service queues store a subset of the reformatted messages of a single data type;

    with a parsing processor, monitoring the plurality of message service queues; and

    with a parsing processor, selecting one of the message service queues based on the monitoring and then retrieving and parsing a next one of the reformatted messages from the selected one of the message service queues in accordance with a target output data model,wherein the parsing processor comprises a plurality of parsers each operable to parse data messages having a different data type and wherein the parsing processor parses the next one of the reformatted messages using one of the plurality of parsers that is configured for parsing the data type associated with the selected service queue and is dynamically selected and allocated during the parsing step,wherein the parsing processor, during the parsing step, selects one of the parsers to use for parsing the next one of the reformatted messages based on the data type and at least one parsing rule applied to one or more characteristics of the next one of the reformatted messages and wherein the plurality of parsers includes at least two parsers adapted for parsing the data type associated with the selected service queue, andwherein the parsing includes extracting a subset of information in the payload of the next one of the reformatted messages defined in the target output data model, whereby throughput of the parsing processor is enhanced by extracting only select information from each of the data messages.

View all claims
  • 2 Assignments
Timeline View
Assignment View
    ×
    ×