Archiving and querying data updates associated with an electronic catalog system

• US 9,292,521 B1
• Filed: 10/20/2011
• Issued: 03/22/2016
• Est. Priority Date: 10/20/2011
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Goo, Almann T., Selberg, Erik W., Suver, Chris A.
• Primary Examiner(s): Hershley, Mark E
• Application Number:
  US13/278,017
• Time in Patent Office: 1,615 Days
• Field of Search: 707/667
• US Class Current: 1/1
• CPC Class Codes: G06F 16/113 Details of archiving lifecycle management in storage systems G06F3/0649; backup systems G06F11/1446
• CPC Class Codes: G06F 16/185 Hierarchical storage management [HSM] systems, e.g. file migration or policies thereof details of archiving G06F16/11
• CPC Class Codes: G06F 16/2358 Change logging, detection, and notification replication G06F16/27
• CPC Class Codes: G06F 16/2455 Query execution
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation
• CPC Class Codes: G06F 16/9538 Presentation of query results

Abstract

Systems and methods are provided for archiving and querying data updates received or generated by an update processing system associated with an electronic catalog. In some embodiments, item data updates associated with items represented in an electronic catalog are formed into new objects consisting of collections of records of item data updates. The new objects may have keys and versions associated with each record within them. The use of these objects allows item data updates, which are generally on the order of a few kilobytes, to be efficiently stored in archive storage systems, and to be queried without affecting the performance of the system that hosts the electronic catalog.

Claims

1. A computer-implemented method for archiving updates associated with item data, said method executed on a computing environment, the method comprising:

under control of one or more configured computing devices,

receiving, from an update processing system that processes updates submitted by catalog update providers, a plurality of update messages that specify updates to item data included in an electronic catalog, wherein the item data is accessed through an electronic catalog system that hosts the electronic catalog, and wherein the update messages comprise a change in at least one of a price, description, sale status, delivery status or viewed status of an individual item data;

creating an object from the plurality of update messages received, wherein the object includes a subset of the update messages associated with a common criterion;

compressing the object;

transmitting the object to an item data update store;

creating a first pointer to the object and a plurality of second pointers to individual update messages included in the object; and

providing the first pointer and at least one second pointer to an entity to enable the entity to access and separately decompress at least one update message included in the object, wherein the at least one second pointer references the at least one update message within the object.

2. The method of claim 1, wherein creating an object from the plurality of update messages comprises appending successive update messages from the plurality of update messages received based on at least one predetermined criterion.

3. The method of claim 2, wherein the predetermined criterion is a time associated with the receipt of an update message.

4. The method of claim 2, wherein the predetermined criterion is an interval of time between the receipt of a first and a second update message.

5. The method of claim 2, wherein the predetermined criterion is a size associated with the created object.

6. The method of claim 1, wherein the object has a size of at least half the size of the object before compressing.

7. The method of claim 1, wherein creating the first pointer to the object comprises creating a locator value corresponding to a location of the object on the item data update store.

8. The method of claim 1, wherein the entity is an operator of the update processing system.

9. A computer system for archiving updates associated with item data, the computer system comprising:

a message receiving module configured to receive, from an update processing system, a plurality of update messages associated with item data, wherein the item data is accessed through an electronic catalog system and wherein an update message associated with item data comprises a change in at least one of a price, description, sale status, delivery status or viewed status of an individual item data;

a batch creation module configured to create a batched version of the plurality of update messages received and configured to compress the batched version, wherein the batch creation module comprises computer hardware and wherein the plurality of update messages are associated with a common criterion;

a transmitting module configured to transmit the batched version to an item data update store;

a pointer creation module configured to create a first pointer to the batched version and a plurality of second pointers to individual update messages included in the batched version; and

a publishing module configured to provide an indication of the first pointer and at least one second pointer to an entity to enable the entity to access and separately decompress at least one update message included in the batched version, wherein the at least one second pointer references the at least one update message within the batched version.

10. The computer system of claim 9, wherein the batch creation module configured to create a batched version of the plurality of update messages is configured to append successive update messages from the plurality of update messages received based on at least one predetermined criterion.

11. The computer system of claim 10, wherein the predetermined criterion is at least one of the price, description, sale status, delivery status and viewed status of the item data.

12. The computer system of claim 9, wherein the entity is an entity not associated with the electronic catalog system.

13. A computer-implemented method for making item data updates available for query, said method executed on a computing environment, the method comprising:

under control of one or more configured computing devices,

generating a first image of an item data store;

receiving a plurality of updates associated with item data from an update processing system, wherein the item data is accessed through an electronic catalog system and wherein an update associated with item data comprises a change in at least one of a price, description, sale status, delivery status or viewed status of an individual item data;

generating a second image comprising the first image and the plurality of updates received;

creating a batched version of a plurality of second images, wherein the plurality of second images are associated with a common criterion;

compressing the batched version;

transmitting the batched version to an item data update store; and

creating a first pointer to the batched version and a plurality of second pointers to individual updates of the plurality of updates included in the batched version to enable querying and separate decompression of at least one update of the plurality of updates, wherein at least one second pointer references the at least one update within the batched version.

14. The method of claim 13, wherein creating a batched version of the plurality of second images comprises appending successive second images until the batched version reaches a predetermined size.

15. The method of claim 13, wherein the compressed batched version has a size of 1/10th the size of the batched version before compressing.

16. A computer readable, non-transitory storage medium having computer executable modules for making item data updates available for querying, the computer executable modules comprising:

an update receiving module configured to receive a plurality of updates associated with item data from an update processing system, wherein the item data is accessed through an electronic catalog system and wherein an update associated with item data comprises a change in at least one of a price, description, sale status, delivery status or viewed status of an individual item data;

an image generating module configured to generate a first image of an item data store and a second image comprising the first image and the plurality of updates received;

a batch creation module configured to create and compress a batched version of a plurality of second images, wherein the batch creation module comprises computer hardware and wherein the plurality of second images are associated with a common criterion;

a transmitting module configured to transmit the batched version to an item data update store; and

a pointer creation module configured to create a first pointer to the batched version and a plurality of second pointers to individual updates of the plurality of updates included in the batched version to enable querying and individual decompression of at least one update of the plurality of updates, wherein at least one second pointer references the at least one update within the batched version.

17. The computer-readable medium of claim 16, wherein the item data and the plurality of updates each comprise a version number.

18. The computer-readable medium of claim 17, wherein the batch creation module is further configured to compare the version associated with item data with the version associated with the update associated with the item data.

19. The computer-readable medium of claim 18, wherein the batch creation module is further configured to delete the item data or the update with the lower version.