Optimized streaming evaluation of XML queries

• US 8,250,062 B2
• Filed: 11/09/2007
• Issued: 08/21/2012
• Est. Priority Date: 11/09/2007
• Status: Active Grant

• Current Assignee: Oracle International Corporation (Oracle Corporation)
• Original Assignee: Oracle International Corporation (Oracle Corporation)
• Inventors: Zhang, Ning, Chandrasekar, Sivasankaran, Agarwal, Nipun, Idicula, Sam
• Primary Examiner(s): Ruiz, Angelica
• Application Number:
  US11/938,017
• Publication Number:
  US 20090125495A1
• Time in Patent Office: 1,747 Days
• Field of Search: 707 1-206
• US Class Current: 707/713
• CPC Class Codes: G06F 16/8373 Query execution
• CPC Class Codes: G06F 40/143 Markup, e.g. Standard Generalized Markup Language [SGML] or Document Type Definition [DTD]
• CPC Class Codes: G06F 40/149 Adaptation of the text data for streaming purposes, e.g. Efficient XML Interchange [EXI] format

Abstract

A database system may perform a streaming evaluation of an XPath expression by utilizing an XPath evaluation component in tandem with an XML event-streaming component. For a more optimal filtered streaming evaluation, the XML event-streaming component may provide an interface whereby the evaluation component sends certain criteria to the event-streaming component when requesting an XML event. The criteria may be based on a next unmatched step in the XPath expression. In response to the request for an XML event, the event-streaming component may only return events that match the criteria. The evaluation component may be, for example, a compiled state machine for the XPath expression. The criteria may be pre-compiled for each possible state in the state machine. The event-streaming component may also utilize the criteria along with schema information to skip parsing of certain segments of XML data.

Claims

1. A computer-implemented method comprising the steps of:

(1) during evaluation of at least a first XPath expression comprising a plurality of steps, an XPath evaluation component identifying a next unmatched step in the plurality of steps;

(2) the XPath evaluation component sending an event request to an XML event-streaming component;

wherein the event request includes one or more criteria that specify a characteristic of at least a first XML event that will satisfy the next unmatched step;

wherein the XML event-streaming component is separate from the XPath evaluation component;

(3) in response to the event request, the XML event-streaming component streaming an XML event to the XPath evaluation component;

wherein the XML event-streaming component determines the XML event to stream to the XPath evaluation component based on the specified characteristic of at least the first XML event that will satisfy the next unmatched step;

(4) the XPath evaluation component matching the XML event to said next unmatched step;

(5) repeating steps 1-4 with respect to at least a new next unmatched step in the first XPath expression, until the XPath evaluation component determines that there are no remaining unmatched steps in the first XPath expression; and

(6) outputting an XPath result based at least upon the XML event streamed in the last iteration of step 3;

wherein the method is performed by one or more computing devices.

2. The method of claim 1 wherein the XML evaluation component is a state machine compiled for at least the XPath expression.

3. The method of claim 2 wherein the state machine maintains state information indicating one or more current states based on previously received XML events, wherein the next unmatched step corresponds to state transitions away from the one or more current states.

4. The method of claim 3 wherein, for each state in the state machine, the state machine stores pre-compiled criteria, wherein the criteria sent from the XPath evaluation component comprises the pre-compiled criteria for the one or more current states, wherein the pre-compiled criteria was generated when the state machine was compiled.

5. The method of claim 3 wherein:

matching the XML event to said next unmatched step comprises changing the state information to indicate a new one or more current states;

determining that there are no remaining unmatched steps in the XPath expression comprises determining that a current state is an accepting state; and

outputting an XPath result comprises;

the XPath evaluation component receiving one or more events while in the accepting state;

outputting XML for all XML events received while in the accepting state; and

receiving a closing XML event that causes the state machine to regress to a non-accepting state.

6. The method of claim 3 wherein the state machine further comprises gamma transitions operating in reverse of the state transitions and based on events that signal the end of an element.

7. The method of claim 2 wherein the state machine is a non-deterministic finite automaton.

8. The method of claim 1 further comprising the steps of:

inputting one or more XML data sources into the XML event-streaming component; and

generating the XML event based on an element, attribute, or value the XML event-streaming component parses from the one or more XML data sources.

9. The method of claim 8 wherein the one or more XML data sources is formatted as binary-encoded XML, further comprising the step of:

prior to generating the XML event, decoding a portion of an XML data source with the XML event-streaming component, wherein the XML event is generated based on an element, attribute, or value in said portion.

10. The method of claim 8, wherein the one or more XML data sources is based on an XML schema, further comprising the steps of, prior to generating an XML event:

at the XML event-streaming component, determining that, based on the one or more criteria and the XML schema, no XML event matching the one or more criteria may be generated from a portion of an XML data source; and

skipping the parsing of said portion.

11. The method of claim 1,

wherein the XML event is an XML SAX event comprising at least one element, attribute, or value;

wherein the one or more criteria comprise at least the name of an element or attribute, a comparison operation based on a value, or a structural relationship.

12. The method of claim 1, further comprising the steps of, prior to the step of sending one or more criteria from the XPath evaluation component:

receiving an XML Query;

normalizing the XML Query to generate the XPath expression; and

initiating evaluation of the XPath expression by the XPath evaluation component.

13. The method of claim 1, wherein the step of outputting the XPath result comprises:

the XML evaluation component receiving one or more XML events;

outputting XML for the one or more XML events; and

receiving an XML event indicating the end of a particular element, wherein the beginning of the particular element was indicated by the XML event streamed in the last iteration of the step of streaming an XML event from the XML event-streaming component.

14. The method of claim 1, wherein the one or more criteria are further based upon a next unmatched step in one or more additional XPath expressions.

15. The method of claim 14, wherein the XPath expression and the one or more additional XPath expressions are normalized from a same XML Query.

16. The method of claim 1 wherein the evaluation component comprises a compiled representation of the XPath expression, wherein the compiled representation comprises, for each step of the XPath expression, pre-compiled criteria, wherein the one or more criteria sent from the XPath evaluation component comprises the pre-compiled criteria for the next unmatched step.

17. A method for evaluating an XPath expression comprising the computer-implemented steps of:

compiling a state machine based on an XPath expression, the state machine comprising states and transitions that reflect the steps of the XPath expression;

evaluating the XPath expression using the state machine, wherein the step of evaluating comprises, while executing the state machine;

receiving one or more XML events from an XML event-streaming component;

maintaining state information based on the content and ordering of the one or more XML events, wherein the state information indicates a current state in the state machine;

based at least upon transitions from the current state to subsequent states, determining one or more criteria, wherein the one or more criteria describe characteristics of any XML event that will transition the state machine to at least one of the subsequent states;

sending the one or more criteria to the XML event-streaming component;

requesting, from the XML event-streaming component, a next XML event that meets the one or more criteria;

in response to said requesting, receiving an XML event from the XML streaming component that meets the one or more criteria; and

generating an XPath result based on the evaluation;

wherein the method is performed by one or more computing devices.

19. The method of claim 17, wherein the state machine further comprises gamma transitions operating in reverse of the transitions and based on events that signal the end of an element.

20. The method of claim 17,

wherein the state machine is a non-deterministic finite automaton;

wherein the XML event is an XML SAX event comprising at least one element, attribute, or value;

wherein the one or more criteria comprise at least the name of an element or attribute, a comparison operation based on a value, or a structural relationship.

21. The method of claim 17, further comprising:

receiving an XML Query; and

normalizing the XML Query to generate the XPath expression.

18. A computer-implemented method for evaluating an XML query, comprising the steps of:

compiling a state machine based on one or more XPath expressions;

wherein the state machine comprises a first set of states, a set of transitions, and a set of conditions;

wherein each transition in the set of transitions indicates a transformation from a state in the first set of states to a state in a second set of states;

wherein each condition in the set of conditions describes, for a distinct transition in the set of transitions, one or more criteria under which input received by the state machine will result in the distinct transition; and

while executing the state machine in the first set of states;

sending data from the state machine to an XML event-streaming component indicating the set of conditions;

parsing an XML data source with the XML event-streaming component until the XML streaming engine generates an event that meets the one or more criteria of at least one condition in the set of conditions;

sending input from the XML event-streaming component to the state machine indicating the event; and

based on the event, transitioning the state machine to a third set of states, wherein the third set of states comprises an accepting state, wherein the state machine outputs the XML event as an XPath result;

wherein the method is performed by one or more computing devices.

22. The method of claim 18 wherein the XML data source is formatted as binary-encoded XML, further comprising the step of:

prior to generating the event, decoding a portion of the XML data source with the XML event-streaming component, wherein the event is generated based on an element, attribute, or value in said portion.

23. The method of claim 18,

wherein the XML data source is formatted as binary-encoded XML;

wherein at the time the set of conditions is sent to the XML event-streaming component, the XML event-streaming component is positioned at a first point the XML data source;

wherein the method further comprises;

skipping decoding of a portion of the XML data source based on the set of conditions, the portion being located after the first in the XML data source;

resuming decoding of the XML data source at a second point after the skipped portion;

wherein the XML event is generated based on XML data that was decoded after the skipped portion.

24. The method of claim 18,

wherein the XML data source is formatted as binary-encoded XML;

wherein the method further comprises skipping decoding of a portion of the XML data source based on the set of conditions and an XML schema associated with the XML data source.

25. The method of claim 18, wherein the XML data source is based on an XML schema, further comprising the steps of, prior to generating an XML event:

at the XML event-streaming component, determining that, based on the set of conditions and the XML schema, no XML event matching the one or more criteria may be generated from a portion of the XML data source; and

skipping the parsing of said portion.

26. The method of claim 18, wherein the state machine further comprises transitions operating in reverse of the state transitions and based on events that signal the end of an element.

27. The method of claim 18,

wherein the state machine is a non-deterministic finite automaton;

wherein the XML event is an XML SAX event comprising at least one element, attribute, or value;

wherein the set of conditions comprises at least the name of an element or attribute, a comparison operation based on a value, or a structural relationship.

28. The method of claim 18, further comprising:

receiving an XML Query; and

normalizing the XML Query to generate the XPath expression.

29. One or more non-transitory computer-readable storage media storing instructions that, when executed by one or more computing devices, cause:

(1) during evaluation of at least a first XPath expression comprising a plurality of steps, an XPath evaluation component identifying a next unmatched step in the plurality of steps;

(2) the XPath evaluation component sending an event request to an XML event-streaming component;

wherein the event request includes one or more criteria that specify a characteristic of at least a first XML event that will satisfy the next unmatched step;

wherein the XML event-streaming component is separate from the XPath evaluation component;

(3) in response to the event request, the XML event-streaming component streaming an XML event to the XPath evaluation component;

wherein the XML event-streaming component determines the XML event to stream to the XPath evaluation component based on the specified characteristic of at least the first XML event that will satisfy the next unmatched step;

(4) the XPath evaluation component matching the XML event to said next unmatched step;

(5) repeating steps 1-4 with respect to at least a new next unmatched step in the first XPath expression, until the XPath evaluation component determines that there are no remaining unmatched steps in the first XPath expression; and

(6) outputting an XPath result based at least upon the XML event streamed in the last iteration of step 3;

wherein the method is performed by one or more computing devices.

30. The one or more non-transitory computer-readable storage media of claim 29 wherein the XML evaluation component is a state machine compiled for at least the XPath expression.

31. The one or more non-transitory computer-readable storage media of claim 30 wherein the state machine maintains state information indicating one or more current states based on previously received XML events, wherein the next unmatched step corresponds to state transitions away from the one or more current states.

32. The one or more non-transitory computer-readable storage media of claim 31 wherein, for each state in the state machine, the state machine stores pre-compiled criteria, wherein the criteria sent from the XPath evaluation component comprises the pre-compiled criteria for the one or more current states, wherein the pre-compiled criteria was generated when the state machine was compiled.

33. The one or more non-transitory computer-readable storage media of claim 31 wherein:

matching the XML event to said next unmatched step comprises changing the state information to indicate a new one or more current states;

determining that there are no remaining unmatched steps in the XPath expression comprises determining that a current state is an accepting state; and

outputting an XPath result comprises;

the XPath evaluation component receiving one or more events while in the accepting state;

outputting XML for all XML events received while in the accepting state; and

receiving a closing XML event that causes the state machine to regress to a non- accepting state.

34. The one or more non-transitory computer-readable storage media of claim 31 wherein the state machine further comprises gamma transitions operating in reverse of the state transitions and based on events that signal the end of an element.

35. The one or more non-transitory computer-readable storage media of claim 30 wherein the state machine is a non-deterministic finite automaton.

36. The one or more non-transitory computer-readable storage media of claim 29, wherein the instructions, when executed by the one or more computing devices, further cause:

inputting one or more XML data sources into the XML event-streaming component; and

generating the XML event based on an element, attribute, or value the XML event-streaming component parses from the one or more XML data sources.

37. The one or more non-transitory computer-readable storage media of claim 36 wherein the one or more XML data sources is formatted as binary-encoded XML, wherein the instructions, when executed by the one or more computing devices, further cause:

prior to generating the XML event, decoding a portion of an XML data source with the XML event-streaming component, wherein the XML event is generated based on an element, attribute, or value in said portion.

38. The one or more non-transitory computer-readable storage media of claim 36, wherein the one or more XML data sources is based on an XML schema, wherein the instructions, when executed by the one or more computing devices, further cause, prior to generating an XML event:

at the XML event-streaming component, determining that, based on the one or more criteria and the XML schema, no XML event matching the one or more criteria may be generated from a portion of an XML data source; and

skipping the parsing of said portion.

39. The one or more non-transitory computer-readable storage media of claim 29,

wherein the XML event is an XML SAX event comprising at least one element, attribute, or value;

wherein the one or more criteria comprise at least the name of an element or attribute, a comparison operation based on a value, or a structural relationship.

40. The one or more non-transitory computer-readable storage media of claim 29, wherein the instructions, when executed by the one or more computing devices, further cause, prior to the step of sending one or more criteria from the XPath evaluation component:

receiving an XML Query;

normalizing the XML Query to generate the XPath expression; and

initiating evaluation of the XPath expression by the XPath evaluation component.

41. The one or more non-transitory computer-readable storage media of claim 29, wherein the step of outputting the XPath result comprises:

the XML evaluation component receiving one or more XML events;

outputting XML for the one or more XML events; and

receiving an XML event indicating the end of a particular element, wherein the beginning of the particular element was indicated by the XML event streamed in the last iteration of the step of streaming an XML event from the XML event-streaming component.

42. The one or more non-transitory computer-readable storage media of claim 29, wherein the one or more criteria are further based upon a next unmatched step in one or more additional XPath expressions.

43. The one or more non-transitory computer-readable storage media of claim 42, wherein the XPath expression and the one or more additional XPath expressions are normalized from a same XML Query.

44. The one or more non-transitory computer-readable storage media of claim 29 wherein the evaluation component comprises a compiled representation of the XPath expression, wherein the compiled representation comprises, for each step of the XPath expression, pre-compiled criteria, wherein the one or more criteria sent from the XPath evaluation component comprises the pre-compiled criteria for the next unmatched step.

45. One or more non-transitory computer-readable storage media storing instructions that, when executed by one or more computing devices, cause:

compiling a state machine based on an XPath expression, the state machine comprising states and transitions that reflect the steps of the XPath expression;

evaluating the XPath expression using the state machine, wherein the step of evaluating comprises, while executing the state machine;

receiving one or more XML events from an XML event-streaming component;

maintaining state information based on the content and ordering of the one or more XML events, wherein the state information indicates a current state in the state machine;

based at least upon transitions from the current state to subsequent states, determining one or more criteria, wherein the one or more criteria describe characteristics of any XML event that will transition the state machine to at least one of the subsequent states;

sending the one or more criteria to the XML event-streaming component;

requesting, from the XML event-streaming component, a next XML event that meets the one or more criteria;

in response to said requesting, receiving an XML event from the XML streaming component that meets the one or more criteria; and

generating an XPath result based on the evaluation;

wherein the method is performed by one or more computing devices.

47. The one or more non-transitory computer-readable storage media of claim 45, wherein the state machine further comprises gamma transitions operating in reverse of the transitions and based on events that signal the end of an element.

48. The one or more non-transitory computer-readable storage media of claim 45,

wherein the state machine is a non-deterministic finite automaton;

wherein the XML event is an XML SAX event comprising at least one element, attribute, or value;

wherein the one or more criteria comprise at least the name of an element or attribute, a comparison operation based on a value, or a structural relationship.

49. The one or more non-transitory computer-readable storage media of claim 45, wherein the instructions, when executed by the one or more computing devices, further cause:

receiving an XML Query; and

normalizing the XML Query to generate the XPath expression.

46. One or more non-transitory computer-readable storage media storing instructions that, when executed by one or more computing devices, cause:

compiling a state machine based on one or more XPath expressions;

wherein the state machine comprises a first set of states, a set of transitions, and a set of conditions;

wherein each transition in the set of transitions indicates a transformation from a state in the first set of states to a state in a second set of states;

wherein each condition in the set of conditions describes, for a distinct transition in the set of transitions, one or more criteria under which input received by the state machine will result in the distinct transition; and

while executing the state machine in the first set of states;

sending data from the state machine to an XML event-streaming component indicating the set of conditions;

parsing an XML data source with the XML event-streaming component until the XML streaming engine generates an event that meets the one or more criteria of at least one condition in the set of conditions;

sending input from the XML event-streaming component to the state machine indicating the event; and

based on the event, transitioning the state machine to a third set of states, wherein the third set of states comprises an accepting state, wherein the state machine outputs the XML event as an XPath result;

wherein the method is performed by one or more computing devices

50. The one or more non-transitory computer-readable storage media of claim 46, wherein the XML data source is formatted as binary-encoded XML, wherein the instructions, when executed by the one or more computing devices, further cause:

prior to generating the event, decoding a portion of the XML data source with the XML event-streaming component, wherein the event is generated based on an element, attribute, or value in said portion.

51. The one or more non-transitory computer-readable storage media of claim 46,

wherein the XML data source is formatted as binary-encoded XML;

wherein at the time the set of conditions is sent to the XML event-streaming component, the XML event-streaming component is positioned at a first point the XML data source;

wherein the wherein the instructions, when executed by the one or more computing devices, further cause;

skipping decoding of a portion of the XML data source based on the set of conditions, the portion being located after the first in the XML data source;

resuming decoding of the XML data source at a second point after the skipped portion;

wherein the XML event is generated based on XML data that was decoded after the skipped portion.

52. The one or more non-transitory computer-readable storage media of claim 46,

wherein the XML data source is formatted as binary-encoded XML;

wherein the instructions, when executed by the one or more computing devices, further cause skipping decoding of a portion of the XML data source based on the set of conditions and an XML schema associated with the XML data source.

53. The one or more non-transitory computer-readable storage media of claim 46,

wherein the XML data source is based on an XML schema, wherein the instructions, when executed by the one or more computing devices, further cause, prior to generating an XML event:

at the XML event-streaming component, determining that, based on the set of conditions and the XML schema, no XML event matching the one or more criteria may be generated from a portion of the XML data source; and

skipping the parsing of said portion.

54. The one or more non-transitory computer-readable storage media of claim 46, wherein the state machine further comprises transitions operating in reverse of the state transitions and based on events that signal the end of an element.

55. The one or more non-transitory computer-readable storage media of claim 46,

wherein the state machine is a non-deterministic finite automaton;

wherein the XML event is an XML SAX event comprising at least one element, attribute, or value;

wherein the set of conditions comprises at least the name of an element or attribute, a comparison operation based on a value, or a structural relationship.

56. The one or more non-transitory computer-readable storage media of claim 46,

wherein the instructions, when executed by the one or more computing devices, further cause:

receiving an XML Query; and

normalizing the XML Query to generate the XPath expression.