TECHNIQUES FOR FAST AND SCALABLE XML GENERATION AND AGGREGATION OVER BINARY XML

• US 20110184969A1
• Filed: 01/22/2010
• Published: 07/28/2011
• Est. Priority Date: 01/22/2010
• Status: Active Grant

• Current Assignee: Oracle International Corporation (Oracle Corporation)
• Original Assignee: Oracle International Corporation (Oracle Corporation)
• Inventors: Mane, Sandeep, Khaladkar, Bhushan, Idicula, Sam, Agarwal, Nipun
• Granted Patent: US 8,417,714 B2
• Time in Patent Office: Days
• US Class Current: 707/760
• CPC Class Codes: G06F 16/8373 Query execution

Abstract

Techniques for fast and scalable generation and aggregation of XML data are described. In an example embodiment, an XML query that requests data from XML documents is received. The XML query is evaluated to determine one or more XML results. For each particular XML result, evaluating the XML query comprises: instantiating a particular data structure that represents the particular XML result, where the particular data structure is encoded in accordance with tags specified in the XML query but does not store the tags; and storing, in the particular data structure, one or more locators that respectively point to one or more fragments in the XML documents, where the particular data structure stores the one or more locators but does not store the one or more fragments. On demand, in response to a request indicating the particular XML result, a serialized representation of the particular XML result is generated based at least on the particular data structure.

Claims

1. A computer-implemented method comprising:

receiving an extended markup language (XML) query that requests data from one or more XML documents;

evaluating the XML query to determine one or more XML results of the XML query, wherein for a particular XML result, of the one or more XML results, evaluating the XML query comprises;

instantiating, in volatile memory, a particular data structure that represents the particular XML result, wherein the particular data structure is encoded in accordance with one or more tags that are specified in the XML query, but the particular data structure does not store the one or more tags; and

storing, in the particular data structure, one or more locators that respectively point to one or more fragments in the one or more XML documents, wherein the particular data structure stores the one or more locators but does not store the one or more fragments pointed to by the one or more locators;

in response to a request indicating the particular XML result, generating a serialized representation of the particular XML result based at least on the particular data structure;

wherein the method is performed by one or more computing devices.

2. The method of claim 1, wherein generating the serialized representation of the particular XML result comprises:

generating the one or more tags that correspond to the particular data structure;

storing the one or more tags into the serialized representation of the particular XML result;

using the one or more locators to extract the one or more fragments as entire units from one or more persistent storage devices that store the one or more XML documents; and

embedding the one or more fragments into the serialized representation of the particular XML result.

3. The method of claim 1, wherein:

for the particular XML result, evaluating the XML query further comprises extracting one or more content values from the one or more XML documents and storing the one or more content values in the particular data structure; and

generating the serialized representation of the particular XML result further comprises retrieving the one or more content values from the particular data structure and storing the one or more content values in the serialized representation of the particular XML result.

4. The method of claim 1, wherein evaluating the XML query further comprises:

executing the XML query wherein, for the particular XML result, executing the XML query comprises storing the one or more locators in the particular data structure.

5. The method of claim 1, wherein evaluating the XML query further comprises evaluating the XML query without generating any serialized representations of the one or more XML results of the XML query.

6. The method of claim 1, wherein evaluating the XML query further comprises:

compiling the XML query, wherein compiling the XML query comprises;

generating, and storing in volatile memory, an encoding that maps the one or more tags that are specified in the XML query to one or more fields in the particular data structure; and

instantiating, in volatile memory, a template structure that includes the one or more fields in accordance with the encoding;

wherein instantiating the particular data structure in volatile memory comprises generating the particular data structure by making a copy of the template structure.

7. The method of claim 6, wherein generating the encoding comprises:

determining one or more token identifiers that respectively identify the one or more tags that are specified in the XML query; and

mapping, in the encoding, the one or more token identifiers to the one or more fields in the particular data structure.

8. The method of claim 7, wherein the one or more token identifiers uniquely identify the one or more tags only within the scope of the one or more XML results of the XML query.

9. The method of claim 1, wherein:

the one or more XML results include at least two different XML results that are structured differently from each other, wherein the XML query respectively specifies at least two different sets of tags for said at least two different XML results; and

evaluating the XML query further comprises;

instantiating, in volatile memory, at least two different data structures that respectively represent said at least two different XML results, wherein said at least two different data structures are respectively encoded in accordance with said at least two different sets of tags; and

storing, in said at least two different data structures, one or more content values or locators that are respectively associated with said at least two different XML results.

10. The method of claim 1, further comprising:

generating and storing a Large Object (LOB) identifier to represent a serialized final result of the XML query without materializing the serialized final result of the XML query, wherein the LOB identifier is associated with those one or more data structures that respectively represent the one or more XML results in volatile memory;

at a programming interface that is operable to access LOBs, receiving the request indicating the particular XML result, wherein the request includes the LOB identifier; and

based on the LOB identifier, the programming interface locating the particular data structure in volatile memory and generating the serialized representation of the particular XML result based at least on the particular data structure.

11. The method of claim 1, wherein:

the XML query specifies an operation that requires aggregation of the one or more XML results; and

evaluating the XML query further comprises;

compressing multiple locators, from the one or more locators, into a single locator list and storing the single locator list in the particular data structure, wherein the multiple locators respectively point to multiple fragments in a same particular XML document of the one or more XML documents.

12. The method of claim 1, wherein the one or more XML documents are stored on one or more persistent storage devices in accordance with a storage representation that utilizes element identifiers that do not depend on namespace context.

13. The method of claim 1, wherein the XML query is an XQuery query.

14. The method of claim 1, wherein the method is performed by a database server that is executing on the one or more computing devices.

15. One or more storage media storing one or more sequences of instructions which, when executed by one or more processors, cause the one or more processors to perform steps comprising:

receiving an extended markup language (XML) query that requests data from one or more XML documents;

evaluating the XML query to determine one or more XML results of the XML query, wherein for a particular XML result, of the one or more XML results, evaluating the XML query comprises;

instantiating, in volatile memory, a particular data structure that represents the particular XML result, wherein the particular data structure is encoded in accordance with one or more tags that are specified in the XML query, but the particular data structure does not store the one or more tags; and

storing, in the particular data structure, one or more locators that respectively point to one or more fragments in the one or more XML documents, wherein the particular data structure stores the one or more locators but does not store the one or more fragments pointed to by the one or more locators;

in response to a request indicating the particular XML result, generating a serialized representation of the particular XML result based at least on the particular data structure.

16. The one or more storage media of claim 15, wherein the instructions that cause generating the serialized representation of the particular XML result comprise instructions which, when executed by the one or more processors, cause the one or more processors to perform:

generating the one or more tags that correspond to the particular data structure;

storing the one or more tags into the serialized representation of the particular XML result;

using the one or more locators to extract the one or more fragments as entire units from one or more persistent storage devices that store the one or more XML documents; and

embedding the one or more fragments into the serialized representation of the particular XML result.

17. The one or more storage media of claim 15, wherein:

for the particular XML result, the instructions that cause evaluating the XML query further comprise instructions which, when executed by the one or more processors, cause the one or more processors to perform extracting one or more content values from the one or more XML documents and storing the one or more content values in the particular data structure; and

the instructions that cause generating the serialized representation of the particular XML result further comprise instructions which, when executed by the one or more processors, cause the one or more processors to perform retrieving the one or more content values from the particular data structure and storing the one or more content values in the serialized representation of the particular XML result.

18. The one or more storage media of claim 15, wherein the instructions that cause evaluating the XML query further comprise instructions which, when executed by the one or more processors, cause the one or more processors to perform:

executing the XML query wherein, for the particular XML result, executing the XML query comprises storing the one or more locators in the particular data structure.

19. The one or more storage media of claim 15, wherein the instructions that cause evaluating the XML query further comprise instructions which, when executed by the one or more processors, cause the one or more processors to perform evaluating the XML query without generating any serialized representations of the one or more XML results of the XML query.

20. The one or more storage media of claim 15, wherein the instructions that cause evaluating the XML query further comprise instructions which, when executed by the one or more processors, cause the one or more processors to perform:

compiling the XML query, wherein compiling the XML query comprises;

generating, and storing in volatile memory, an encoding that maps the one or more tags that are specified in the XML query to one or more fields in the particular data structure; and

instantiating, in volatile memory, a template structure that includes the one or more fields in accordance with the encoding;

wherein the instructions that cause instantiating the particular data structure in volatile memory comprise instructions which, when executed by the one or more processors, cause the one or more processors to perform generating the particular data structure by making a copy of the template structure.

21. The one or more storage media of claim 20, wherein the instructions that cause generating the encoding comprise instructions which, when executed by the one or more processors, cause the one or more processors to perform:

determining one or more token identifiers that respectively identify the one or more tags that are specified in the XML query; and

mapping, in the encoding, the one or more token identifiers to the one or more fields in the particular data structure.

22. The one or more storage media of claim 21, wherein the one or more token identifiers uniquely identify the one or more tags only within the scope of the one or more XML results of the XML query.

23. The one or more storage media of claim 15, wherein:

the one or more XML results include at least two different XML results that are structured differently from each other, wherein the XML query respectively specifies at least two different sets of tags for said at least two different XML results; and

the instructions that cause evaluating the XML query further comprise instructions which, when executed by the one or more processors, cause the one or more processors to perform;

instantiating, in volatile memory, at least two different data structures that respectively represent said at least two different XML results, wherein said at least two different data structures are respectively encoded in accordance with said at least two different sets of tags; and

storing, in said at least two different data structures, one or more content values or locators that are respectively associated with said at least two different XML results.

24. The one or more storage media of claim 15, wherein the one or more sequences of instructions further comprise instructions which, when executed by the one or more processors, cause the one or more processors to perform:

generating and storing a Large Object (LOB) identifier to represent a serialized final result of the XML query without materializing the serialized final result of the XML query, wherein the LOB identifier is associated with those one or more data structures that respectively represent the one or more XML results in volatile memory;

at a programming interface that is operable to access LOBs, receiving the request indicating the particular XML result, wherein the request includes the LOB identifier; and

based on the LOB identifier, the programming interface locating the particular data structure in volatile memory and generating the serialized representation of the particular XML result based at least on the particular data structure.

25. The one or more storage media of claim 15, wherein:

the XML query specifies an operation that requires aggregation of the one or more XML results; and

the instructions that cause evaluating the XML query further comprise instructions which, when executed by the one or more processors, cause the one or more processors to perform;

compressing multiple locators, from the one or more locators, into a single locator list and storing the single locator list in the particular data structure, wherein the multiple locators respectively point to multiple fragments in a same particular XML document of the one or more XML documents.

26. The one or more storage media of claim 15, wherein the one or more XML documents are stored on one or more persistent storage devices in accordance with a storage representation that utilizes element identifiers that do not depend on namespace context.

27. The one or more storage media of claim 15, wherein the XML query is an XQuery query.

28. The one or more storage media of claim 15, wherein the one or more sequences of instructions are included in the executable instructions of a database server.