Data format for processing information

• US 9,465,852 B1
• Filed: 11/13/2007
• Issued: 10/11/2016
• Est. Priority Date: 08/02/2007
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Lusk, Andrew J., Jonker, Todd V., Suver, Chris A.
• Primary Examiner(s): Hershley, Mark E
• Application Number:
  US11/939,477
• Time in Patent Office: 3,255 Days
• Field of Search: 707/809
• US Class Current: 1/1
• CPC Class Codes: G06F 16/2365 Ensuring data consistency and integrity
• CPC Class Codes: G06F 16/24 Querying
• CPC Class Codes: G06F 16/25 Integrating or interfacing systems involving database management systems
• CPC Class Codes: G06F 16/258 Data format conversion from or to a database
• CPC Class Codes: H04L 67/01 Protocols
• CPC Class Codes: H04L 69/02 Protocol performance
• CPC Class Codes: H04L 69/04 Protocols for data compression, e.g. ROHC

Abstract

A method and a system are provided for encoding and processing digital information. The digital information is encoded according to binary encoding formats corresponding to primitive data types. The primitive data types comprise scalar data types including Boolean, integer, float, decimal, time stamp, string, symbol, binary large object, and character large object data types. The primitive data types also comprise composite data types including structure, list, and S-expression data types. The binary-encoded digital information is stored in a message with a predetermined format for transmission. No metadata is included in the message.

Claims

1. A method of communicating digital information, the method comprising:

identifying a plurality of data objects corresponding to a plurality of primitive data types represented by the digital information, each of the plurality of data objects corresponding to at least one of the plurality of primitive data types, the plurality of primitive data types including scalar data types and composite data types, each primitive data type corresponding to a distinct data type of a predefined set of data types and having a corresponding associated NULL value and an associated annotation data type, wherein a first data object of the plurality of data objects corresponds to a first primitive data type of the plurality of primitive data types, wherein a second data object of the plurality of data objects corresponds to a second primitive data type of the plurality of primitive data types, wherein the first data object is different from the second data object, wherein the first primitive data type is different from the second primitive data type and wherein a binary encoding format associated with the first primitive data type is different from a binary encoding format associated with the second primitive data type;

generating a binary-encoded multi-record message comprising a plurality of records, each of the plurality of records corresponding to one of the plurality of data objects, wherein generating the multi-record message comprises generating a binary-encoded record of the plurality of records by encoding at least one data object of the plurality of data objects according to a binary encoding format distinct to an associated primitive data type that corresponds to the data object; and

transmitting the binary-encoded multi-record message to a target computing device.

2. The method of claim 1, wherein the scalar data types include at least one of a Boolean data type, an integer data type, a real data type, a decimal data type, a timestamp data type, a string data type, a symbol data type, a binary large object data type, or a character large object data type.

3. The method of claim 1, wherein the composite data types include at least one of a struct data type, a list data type, or an S-expression data type.

4. The method of claim 3, wherein the struct data type comprises struct data members having one of the plurality of primitive data types, each struct data member being associated with a struct symbol data item, each symbol data item having a symbol data type, the symbol data type being included in the scalar data types.

5. The method of claim 4, wherein each struct symbol data item is associated with a symbol reference number in a message symbol table.

6. The method of claim 1, wherein each record of the plurality of records includes a descriptor indicating the primitive data type of the data object encoded within the record.

7. A non-transitory computer-readable medium having computer-executable components for communication of digital information encoded thereon, the computer-executable components comprising:

a message manager component, running on a source computing device, operable to encode digital information, wherein the message manager component;

identifies a plurality of data objects corresponding to a plurality of primitive data types represented by the digital information, each of the plurality of data objects corresponding to at least one of the plurality of primitive data types, the plurality of primitive data types including at least one of scalar data types or composite data types, each primitive data type corresponding to a distinct data type of a predefined set of data types and having a corresponding associated NULL value and an associated annotation data type, wherein a first data object of the plurality of data objects corresponds to a first primitive data type of the plurality of primitive data types, wherein a second data object of the plurality of data objects corresponds to a second primitive data type of the plurality of primitive data types, wherein the first data object is different from the second data object, wherein the first primitive data type is different from the second primitive data type and wherein a binary encoding format associated with the first primitive data type is different from a binary encoding format associated with the second primitive data type;

encodes individual data objects of the plurality of data objects according to a binary format distinct to an associated primitive data type that corresponds to the individual data object to generate a plurality of binary-encoded records;

generates a binary-encoded multi-record message comprising the plurality of binary-encoded records; and

transmits the binary-encoded multi-record message to a target computing device.

8. The non-transitory computer-readable medium of claim 7, wherein the scalar data types include a Boolean data type, an integer data type, a real data type, a decimal data type, a timestamp data type, a string data type, a symbol data type, a binary large object data type, or a character large object data type.

9. The non-transitory computer-readable medium of claim 7, wherein the composite data types include a struct data type, a list data type, or an S-expression data type.

10. The non-transitory computer-readable medium of claim 7, wherein the message manager component is further operable to parse the digital information into the plurality of data objects and assign one of the plurality of primitive data types to individual data objects of the plurality of data objects.

11. The non-transitory computer-readable medium of claim 10, wherein the message manager component is further operable to generate a symbol table based on at least some of the plurality of data objects.

12. The non-transitory computer-readable medium of claim 7, wherein each of the plurality of binary-encoded records includes a descriptor indicating the primitive data type of the data object encoded within the binary-encoded record.

13. A method of communicating digital information, the method comprising:

receiving a binary-encoded multi-record message including a plurality of binary-encoded data objects, wherein individual data objects are encoded within the multi-record message according to a binary format distinct to a primitive data type that corresponds to the individual data object, wherein individual data objects correspond to one of a plurality of primitive data types each corresponding to a distinct data type in accordance with a predefined classification of data types, wherein the plurality of binary-encoded data objects include a first data object and a second data object, wherein the first data object corresponds to a first primitive data type of the plurality of primitive data types, wherein the second data object corresponds to a second primitive data type of the plurality of primitive data types, wherein the first data object is different from the second data object, wherein the first primitive data type is different from the second primitive data type and wherein a binary encoding format associated with the first primitive data type is different from a binary encoding format associated with the second primitive data type, the plurality of primitive data types including scalar data types and composite data types;

in response to receiving the message;

identifying the at least two different primitive data types used to encode individual data objects of the binary-encoded multi-record message;

decoding the binary-encoded multi-record message into the plurality of binary-encoded data objects based at least in part on the at least two different primitive data types used to encode the plurality of binary-encoded data objects; and

processing the plurality of binary-encoded data objects.

14. The method of claim 13, wherein the scalar data types include a Boolean data type, an integer data type, a real data type, a decimal data type, a timestamp data type a string data type, a symbol data type, a binary large object data type, or a character large object data type.

15. The method of claim 13, wherein the composite data types include a struct data type, a list data type, or an S-expression data type.

16. The method of claim 13, wherein decoding the binary-encoded multi-record message into the plurality of binary-encoded data object comprises using a symbol table previously generated from the binary-encoded multi-record message by a sender of the binary-encoded multi-record message.

17. The method of claim 13, wherein each of the plurality of binary-encoded data objects includes a descriptor indicating the primitive data type used to encode the binary-encoded data object.

18. A method of communicating digital information, the method comprising:

receiving a binary-encoded multi-record message including a plurality of binary-encoded data objects, wherein individual data objects are encoded within the multi-record message according to a binary format distinct to a primitive data type that corresponds to the individual data object, wherein individual data objects correspond to one of a plurality of primitive data types each corresponding to a distinct data type in accordance with a predefined classification of data types, wherein the plurality of binary-encoded data objects include a first data object and a second data object, wherein the first data object corresponds to a first primitive data type of the plurality of primitive data types, wherein the second data object corresponds to a second primitive data type of the plurality of primitive data types, wherein the first data object is different from the second data object, wherein the first primitive data type is different from the second primitive data type and wherein a binary encoding format associated with the first primitive data type is different from a binary encoding format associated with the second primitive data type, the plurality of primitive data types including scalar data types and composite data types, each primitive data type having a corresponding associated NULL value and an associated annotation data type;

in response to receiving the message;

identifying a format conversation Web Service operable to convert the multi-record message to a supported data format;

forwarding the multi-record message to the format conversion Web Service;

receiving at least one of the plurality of data objects in the supported data format from the format conversion Web Service; and

processing the at least one data object.

19. The method of claim 18, wherein the scalar data types include a Boolean data type, an integer data type, a real data type, a decimal data type, a timestamp data type a string data type, a symbol data type, a binary large object data type, or a character large object data type.

20. The method of claim 18, wherein the composite data types include a struct data type, a list data type, or an S-expression data type.

21. The method of claim 18, wherein forwarding the multi-record message to the format conversion Web service comprises forwarding the multi-record message and a symbol table previously generated from the plurality of data objects by a sender of the multi-record message to the format conversion Web service.

22. The method of claim 18, wherein each of the plurality of binary-encoded data objects included within the multi-record message includes a descriptor indicating the primitive data type used to encode the binary-encoded data object.