Data federation methods and system

• US 20050021502A1
• Filed: 05/20/2004
• Published: 01/27/2005
• Est. Priority Date: 05/23/2003
• Status: Abandoned Application

• Current Assignee: Snapbridge Software, Inc.
• Original Assignee: Snapbridge Software, Inc.
• Inventors: Oliver, Jason, Schwartz, David, Chen, Benjamin, Lindsey, William, MacDonald, Angus
• Application Number:
  US10/850,826
• Publication Number:
  US 20050021502A1
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/2471 Distributed queries
• CPC Class Codes: G06F 16/84 Mapping; Conversion

Abstract

A method is provided for processing tree like data structures in a streaming manner. An initial context of name/value bindings is set up. A tree of objects is constructed. Each element in the tree of objects is represented as a function object that accepts a context parameter and a target parameter that it can send a stream of start, content, and end events to represent tree output. The parse tree of objects is examined for element names that are recognized as commands. The commands are converted to special function objects that implement command'"'"'s semantics. Other elements, that are not recognized as commands, are mapped to a default function object.

Claims

1. A method for optimizing distributed computing for tree like data structures represented via mark-up languages, comprising:

receiving an input;

determining if a plurality of execution process can be performed against the input;

determining if the input and the plurality of execution process can be split into components that can be run in parallel on different processors;

splitting the input into components;

executing a process on each of the components with one or more processors to produce a plurality of results,

2. The method of claim 1, further comprising:

determining if executing the process modifies any data that is used in the processing of another component.

3. The method of claim 1, further comprising:

recording in at least one of;

a memory, a file or in a database the cost, in terms of computing resources, of the step of executing a process.

4. The method of claim 1, further comprising:

reassembling the plurality of results into a new tree like data structure.

5. The method of claim 1, wherein the input includes anything that can be represented as a tree like data structure.

6. The method of claim 1, wherein the input includes XML.

7. The method of claim 1, further comprising:

using a processing language that retrieves, constructs, filters and transforms the input.

8. The method of claim 7, wherein the processing language is represented as XML.

9. The method claim 1, further comprising:

wherein the processing language of has the same data model as the data itself.

10. The method of claim 1, wherein the processing language is built from atomic functions that can retrieve, construct, filter and/or transform tree like data structures.

11. The method of claim 10, wherein the processing language is built by retrieving, constructing, filtering and/or transforming tree like data structures represented by XML.

12. The method of claim 1, wherein atomic functions are pipelined together to provide parsing, filtering and transformation of XML data sets.

13. The method of claim 7, wherein the processing language includes XML documents that can be stored and manipulated in an XML database or computer file system.

14. The method of claim 1, wherein the mark-up language is anything that can be represented as a tree like data structure.

15. The method of claim 1, wherein the mark-up language is XML.

16. The method of claim 1, wherein logic resources for determining if the input and the plurality of execution process can be split into components that can be run in parallel on different processors.

17. The method of claim 1, wherein the inputs are selected from XML documents, soap web services, relational databases, flatfiles, and anything that can be represented as a tree like data structure.

18. The method of claim 1, further comprising:

determining a cost of execution for the input.

19. The method of claim 1, wherein the components are branches or nodes in the tree like data structures.

20. A method of caching via lexical analysis, comprising:

converting a parse tree of a command to be executed to a character string representation;

converting runtime parameters used by the command to a character string representation of parameter names and values;

concatenating character strings together;

processing the character string to generate a number;

looking up the number in an association table to determine if the number has been previously recorded;

returning a value associated with the number in the association table if the number is present;

executing the command and storing the result, and associating the result with the number in the association table before returning the result.

21. The method of claim 20, wherein the execution step retrieves, constructs, filters, and/or transforms tree like data structures.

22. A method of optimizing a processing of template based transformation languages, comprising:

parsing a transformation script into logical templates;

for each template constructing a node address expression that specifies types of source nodes from tree like data structures that can be processed by that template;

identifying other templates that can cause that template to be invoked;

modifying the node address expression by adding predicates that eliminate any source node type which is not available from the invoking templates.

examining each source node against the types matched by each template

23. The method of claim 22, wherein constructing the node address includes extracting the node match parameter from the template.

24. The method of claim 22, wherein the step of identifying other templates includes:

constructing a list all the templates;

identifying each command in each template which invokes other templates;

identifying the node address expression the invoking command uses to select the list of nodes it constructs before invoking the target template; and

determining if the expression in the invoking command may address some of the same nodes as the target template.

25. A method for processing tree like data structures in a streaming manner, comprising:

setting up an initial context of name/value bindings

constructing a tree of objects, each element in the tree of objects being represented as a function object that accepts a context parameter and a target parameter it can send a stream of start, content, and end events to represent tree output

examining the parse tree of objects for element names that are recognized as commands;

converting the commands to special function objects that implement command'"'"'s semantics.

mapping other elements not recognized as commands to a default function object.

26. The method of claim 25, wherein the default function object sends an event to a target parameter.

27. The method of claim 26, wherein the target parameter represents a start element with the same name as the parsed element and calls the function of each child element with its original context and event target parameters.

28. The method of claim 25, wherein the step of examining the parse tree of objects includes:

identifying a root node of the tree;

listing direct children of the node; and

for each child node in the list,

obtaining the node'"'"'s element name property if available; and

examine the node'"'"'s list of children if available.

29. The method of claim 25, wherein the step of converting the commands includes:

replacing the command object with a special function object at a same location in the tree.

30. The method of claim 25, wherein the step of mapping other elements includes:

replacing the other element object with the default function object at the same location in the tree.

31. The method of claim 25, further comprising:

analyzing incoming data and apportioning pieces of the data for processing.

32. The method of claim 31, further comprising:

inspecting the data.

33. The method of claim 32, further comprising:

parsing the data and executing parsed data in real time.

34. The method of claim 33, further comprising:

leveraging, streaming and parsing the data.

35. The method of claim 34, further comprising:

using functional data processing language to provide that all commands return as a form of XML that is processed in optimized pieces.

36. The method of claim 35, further comprising:

utilizing downstream commands to generate results before a system command has finished.

37. The method of claim 25, wherein the language is for processing tree like data structures.

38. The method of claim 25, wherein the language is represented in XML 3.

39. A method of reversing transformation of tree like data structures, comprising:

(1) transforming a transformation script into a second script by replacing every command in the transformation script that copies a source leaf node to an output with a command that outputs a record of a source node'"'"'s positional address and an output node'"'"'s positional address;

(2) transform an input source into a transformation script which produces a literal copy of the input source;

(3) transforming an input of the input source through the transformation script produced in step 1; and

(4) transforming outputs of steps 2 and 3 by replacing every command that constructs a leaf node with an appropriate node copying command for those nodes that were produced by copying.

40. A method for providing a non-repudiation audit trail, comprising:

receiving a soap request through software;

auditing the soap request by determining when the soap request was first seen and where it is from;

performing a security step to determine the person sending the soap request;

determining if a response requires transformation; and

forwarding.

41. The method of claim 40, wherein sending and receiving is performed to guarantee that a transaction was successful.