TECHNIQUES FOR EFFICIENT LOADING OF BINARY XML DATA

• US 20080098001A1
• Filed: 05/02/2007
• Published: 04/24/2008
• Est. Priority Date: 10/20/2006
• Status: Active Grant

• Current Assignee: Oracle International Corporation (Oracle Corporation)
• Original Assignee: Oracle International Corporation (Oracle Corporation)
• Inventors: Gupta, Nitin, Chandrasekar, Sivasankaran, Murthy, Ravi, Agarwal, Nipun
• Granted Patent: US 8,010,889 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/258 Data format conversion from or to a database
• CPC Class Codes: G06F 16/80 of semi-structured data, e.g. markup language structured data such as SGML, XML or HTML content-based retrieval of web data G06F16/95
• CPC Class Codes: H03M 7/30 Compression speech analysis-synthesis for redundancy reduction G10L19/00; for image communication H04N; Expansion; Suppression of unnecessary data, e.g. redundancy reduction

Abstract

Various techniques are described hereafter for improving the efficiency of binary XML encoding and loading operations. In particular, techniques are described for incrementally encoding XML in response to amount-based requests. After encoding enough binary XML to satisfy an amount-based request, the encoder stops encoding the XML until a subsequent request is received. The incremental encoding may take place on the client-side or the server-side. Techniques are also described for reducing the character set conversion operations by having a parser convert tokens in text XML into one character set while converting non-token text in the text XML into another character set. Techniques are also described for generating self-contained binary XML documents, and for improving remap operations by providing a binary XML document on a chunk-by-chunk basis.

Claims

1. A method for encoding XML, comprising the computer-implemented steps of:

receiving a request for a specified amount of binary encoded XML;

in response to the request, encoding text XML on a per-node basis until sufficient binary XML has been generated to satisfy the request;

responding to the request with said amount of binary XML; and

once the request has been satisfied, ceasing to encode the text XML until receiving a subsequent request for binary encoded XML.

2. The method of claim 1 wherein:

the step of encoding text XML on a per-node basis includes encoding nodes defined in the text XML to produce a first amount of binary XML data;

the first amount binary XML data exceeds the specified amount by a second amount; and

the method further includes;

storing the second amount of binary XML in a carry-over buffer until receiving the subsequent request for binary XML; and

using said second amount of binary XML to satisfy said subsequent request for binary XML.

3. The method of claim 2 wherein the subsequent request specifies a second specified amount that is greater than said second amount, the method further comprising:

determining a difference between the second specified amount the second amount;

encoding the text XML to produce a third amount of binary XML that is greater than said difference; and

using a portion of the third amount of binary XML, along with the second amount binary XML, to satisfy said subsequent request.

4. The method of claim 1 wherein the step of encoding text XML is performed at a database server that stores the binary XML in an XML repository.

5. The method of claim 1 wherein:

the step of encoding text XML is performed at a client, and

the request is from a database server that manages an XML repository.

6. The method of claim 5 wherein the database server repeatedly sends the client requests for specific amounts of binary XML, and loads the binary XML that the database server receives in response to the requests directly into a structure within the database repository without buffering a copy of the binary XML.

24. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 1.

25. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 2.

26. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 3.

27. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 4.

28. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 5.

29. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 6.

7. A method for encoding binary XML, comprising:

detecting, by a parser that is parsing text XML, a node that qualifies as a simple property;

sending to an encoder an indication that the node is a simple property; and

in response to receiving the indication from the parser, the encoder encoding the node by generating a simple property opcode and a set of one or more operands associated with the simple property opcode.

8. The method of claim 7 wherein the indication is a composite event that represents a plurality of other parser events.

9. The method of claim 7 wherein the one or more operands include (a) an operand indicating a token identifier of the node, (b) an operand that indicates a length of the data associated with the simple property, and (c) data associated with the node.

30. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 7.

31. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 8.

32. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 9.

10. A method for encoding XML, comprising:

receiving, at a parser, text XML to be converted to binary XML;

wherein the text XML includes

(a) character strings that correspond to tokens, and

(b) character strings that do not correspond to tokens;

wherein characters of the text XML are from a first character set;

the parser converting the character strings that correspond to tokens into characters of a second character set; and

the parser converting the character strings that do not correspond to tokens into characters of a third character set;

wherein the first, second and third character sets are three different character sets.

11. The method of claim 10 wherein:

the binary XML is to be stored in a database; and

the second character set is a character set used, within said database, to store tokens that belong to token-id mappings that are used to encode the XML to binary XML.

12. The method of claim 10 wherein:

the binary XML is to be stored in a database; and

the third character set is a default character set used to store text within tables of a database.

33. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 10.

34. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 11.

35. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 12.

13. A method of converting an XML document to a binary XML document, comprising:

converting information from the text XML document to binary XML, wherein the binary XML includes a series of opcodes and operands associated with the opcodes;

in the binary XML generated from the text XML document, representing tokens from the text XML with identifiers; and

including in the binary XML document, (a) the binary XML generated from the text XML document, and (b) an identifier-to-token mapping that allows a recipient of the binary XML document to reproduce the text XML document from the binary XML without resort to translation information separate from the binary XML document.

14. The method of claim 13 further comprising:

storing said binary XML document in a repository; and

storing, within said repository, another binary XML document that does not include any identifier-to-token mapping.

15. The method of claim 14 wherein:

the binary XML document and the other binary XML document are both documents that conform to a particular XML schema;

the other binary XML document was encoded using an identifier-to-token mapping associated with the particular XML schema; and

the identifier-to-token mapping stored in the binary XML document differs from the identifier-to-token mapping associated with the particular XML schema.

16. The method of claim 13 further comprising, prior to storing the binary XML document in a repository, remapping identifiers contained in the binary XML document based on an identifier-to-token mapping associated with an XML schema to which the text XML document conforms.

17. The method of claim 16 further comprising, after remapping identifiers contained in the binary XML document, storing the binary XML document in the repository without including in the binary XML document any identifier-to-token mapping information.

18. The method of claim 13 wherein the step of converting information from the text XML document to binary XML is performed on a client based on an identifier-to-token mapping generated by the client in response to the client being unable to retrieve identifier-to-token mapping information from a server.

19. The method of claim 13 further comprising setting a self-contained flag in the binary XML document to indicate that the binary XML document includes identifier-to-token mapping information.

36. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 13.

37. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 14.

38. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 15.

39. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 16.

40. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 17.

41. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 18.

42. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 19.

20. A method of storing an XML document in a repository, the method comprising:

on a client, receiving the XML document as text XML;

at the client, converting the text XML to binary XML based, at least in part, on a set of one or more client-created token-to-identifier mappings;

after converting the text XML to binary XML, providing the XML document to a server in a series of chunks; and

including with each chunk only those client-created token-to-identifier mappings that (a) were used to generate identifiers that are contained in the chunk, and (b) were not included in any chunks that preceded the chunk in the series of chunks.

21. The method of claim 20 further comprising:

constructing, at the server, a hash table based on the client-created token-to-identifier mappings contained in the chunks; and

using the hash table to remap the identifiers that were generated based on the client-created token-to-identifier mappings.

22. The method of claim 20 further comprising:

the client determining that a particular chunk does not contain any identifiers that were generated based on client-created token-to-identifier mappings; and

in response to determining that a particular chunk does not contain any identifiers that were generated based on client-created token-to-identifier mappings, setting a remap flag associated with the particular chunk to indicate that the particular chunk does not contain any identifiers that need to be remapped.

23. The method of claim 22 further comprising:

performing a remap operation on the binary XML document prior to storing the binary XML document in a repository; and

during the remap operation, analyzing the binary XML contained in only those chunks whose remap flag indicates that the chunk contains identifiers that need to be remapped.

43. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 20.

44. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 21.

45. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 22.

46. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 23.