Techniques for aggregating data from multiple sources

• US 8,745,096 B1
• Filed: 03/31/2010
• Issued: 06/03/2014
• Est. Priority Date: 03/31/2010
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Dykhno, Elena, Noble, Matthew R., Whitney, David J.
• Primary Examiner(s): PENG, HUAWEN A
• Application Number:
  US12/751,018
• Time in Patent Office: 1,525 Days
• Field of Search: 707/809, 707/810, 707/811
• US Class Current: 707/811
• CPC Class Codes: G06F 16/258 Data format conversion from or to a database
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques

Abstract

Systems and methods for providing information responsive to a request. In an embodiment, a request from a requestor is received. A plurality of eligible information sources that are capable of providing data at least partially responsive to the information request are identified. For each identified information source, a data request according to a schema accepted by the identified information source is prepared and responses from the eligible information sources are received. The responses according to a common schema are homogenized and information from the homogenized responses are provided to the requestor.

Claims

1. A computer-implemented method of providing information to a requestor of content, comprising:

under control of one or more computer systems configured with executable instructions,

receiving a request for the content from the requestor, the request specifying at least one characteristic;

identifying a plurality of web services, each of the web services being operable to provide at least a subset of a set of data responsive to the request according to said at least one characteristic;

requesting at least a portion of the data from each of the identified web services;

receiving responses from the web services, the received responses being organized according to original schemas, at least two of the original schemas being different; and

combining, based at least in part on a mapping between the original schemas and a common schema, information from the received responses into a new document that includes the combined information, wherein the common schema of the new document is based at least in part on the received request, the combining of information from the received responses comprising;

organizing the information from the received responses according to a preliminary schema; and

reorganizing the information from the received responses from the preliminary schema according to a destination language schema.

2. The computer-implemented method of claim 1, wherein the preliminary schema defines a plurality of elements, wherein a particular element includes an attribute, and wherein organizing the information from the received responses according to the preliminary schema comprises:

creating an instance of the particular element; and

assigning a value for the attribute of the instance, the value indicative of how information from the instance should be organized according to the destination language schema.

3. The computer-implemented method of claim 2, wherein reorganizing the information from the received responses comprises:

identifying instances of the elements having attributes assigned common values; and

collecting information from the identified instances into a common structure of the destination language schema.

4. The computer-implemented method of claim 1, wherein identifying the web services comprises locating the web services according to a map that associates request types with web services that are applicable to the request types.

5. A computer-implemented method of providing content, comprising:

under control of one or more computer systems configured with executable instructions,

receiving a request from a requestor, the request characterizing desired information;

identifying a plurality of information sources that are each operable to provide at least a subset of a set of data responsive to the request;

requesting from each of the identified information sources a response that is in accordance with the request from the requestor;

receiving responses from the information sources, the responses being organized according to a plurality of original schemas, at least two of the original schemas being different; and

combining, based at least in part on a mapping from the original schemas to a common schema, information from the responses into a new document that includes the combined information based at least in part on the received request, the combining of information comprising;

organizing the information from the responses according to a preliminary schema; and

converting the information organized according to the preliminary schema to information stored according to one or more destination language objects.

6. The computer-implemented method of claim 5, wherein the preliminary schema defines a plurality of elements having attributes, wherein said organizing the information includes assigning at least one value to at least one attribute of at least one instance of the elements, and wherein converting the organized information is performed based at least in part on said at least one value.

7. The computer-implemented method of claim 5, wherein the request from the requestor is of a particular type and wherein identifying the plurality of information sources includes referencing a map to identify the information sources based at least in part on the particular type.

8. The computer-implemented method of claim 5, wherein the common schema employs one or more elements and wherein the combined information includes a plurality of instances of a particular element, the particular element having an attribute, each of the plurality of instances having a corresponding value for the attribute, and wherein combining information includes collecting a subset of the instances sharing a common value for the attribute into a destination language object.

9. A computer-implemented method of providing information, comprising:

under control of one or more computer systems configured with executable instructions,

receiving a content request from a requestor;

identifying a plurality of eligible information sources that are capable of providing data at least partially responsive to the information request;

for each identified information source, preparing a data request according to a schema accepted by the identified information source and sending the prepared data request to the identified information source;

receiving responses to the prepared data requests from the eligible information sources, at least one of the responses differing from a common schema;

homogenizing the responses according to the common schema; and

combining information from the homogenized responses into a new document that includes the combined information based at least in part on the received content request by preparing at least one destination language object that includes at least some of the information from the homogenized responses.

10. The computer-implemented method of claim 9, wherein the common schema employs a plurality of elements, wherein homogenizing the responses includes assigning attributes to a plurality of the elements, and wherein said at least one destination language object includes information from a plurality of elements that are assigned a common attribute.

11. The computer-implemented method of claim 9, wherein, for at least one identified information source, preparing the data request includes converting the information request into a format acceptable by said at least one identified information source.

12. The computer-implemented method of claim 9, wherein the information sources are web services.

13. A computer-readable storage medium having stored thereon instructions for causing at least one computer system to provide information to a requestor, the instructions comprising:

instructions that cause said at least one computer system to receive an information request from the requestor;

instructions that cause said at least one computer system to identify a plurality of eligible information sources that are capable of providing data at least partially responsive to the information request;

instructions that cause said at least one computer system to, for each identified information source, prepare a data request according to a schema accepted by the identified information source;

instructions that cause said at least one computer system to, send the prepared data requests to the identified information sources;

instructions that cause said at least one computer system to receive responses from the eligible information sources;

instructions that cause said at least one computer system to homogenize the responses according to a common schema; and

instructions that cause said at least one computer system to combine information from the homogenized responses into a new document that includes the combined information based at least in part on the received information request, the instructions that cause the at least one computer system to combine information including instructions that cause the at least one computer system to prepare at least one destination language object that includes at least some of the information from the homogenized responses.

14. The computer-readable storage medium of claim 13, wherein the common schema employs a plurality of elements, wherein the instructions that cause said at least one computer system to homogenize the responses includes the instructions that cause said at least one computer system to assign attributes to a plurality of the elements, and wherein said at least one destination language object includes information from a plurality of elements that are assigned a common attribute.

15. The computer-readable storage medium of claim 13, wherein, for at least one identified information source, the instructions that cause said at least one computer system to prepare the data request include the instructions that cause said at least one computer system to convert the information request into a format acceptable by said at least one identified information source.

16. The computer-readable storage medium of claim 13, wherein the information sources are web services.

17. A system for providing information, comprising:

a communication device operable to receive a request from a requestor and provide a response responsive to the request; and

at least one processor operable to;

identify a plurality of information sources that are each operable to provide at least a subset of a set of data responsive to the request;

request from each of the identified information sources a response that is in accordance with the request from the requestor;

receive responses from the information sources, the responses being organized according to a plurality of original schemas, at least two of the original schemas being different;

combine, based at least in part on a mapping from the original schemas to a common schema, information from the responses into a new document that includes the combined information based at least in part on the received request;

organize the information from the responses according to a preliminary schema; and

convert the information organized according to the preliminary schema to information stored according to one or more destination language objects.

18. The system of claim 17, wherein the preliminary schema defines a plurality of elements having attributes, wherein said at least one processor is operable to assign at least one value to at least one attribute of at least one instance of the elements, and wherein said at least one processor is operable to convert the organized information is performed based at least in part on said at least one value.

19. The system of claim 17, wherein the request from the requestor is of a particular type and wherein said at least one processor is operable to identify the information sources based at least in part on the particular type.

20. The system of claim 17, wherein the request is a web service request and the response responsive to the request is a web service response.