Validating and non-validating secondary indexes for a table in a non-relational data store

• US 10,013,449 B1
• Filed: 09/18/2015
• Issued: 07/03/2018
• Est. Priority Date: 09/18/2015
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Xiao, Wei, Muniswamy Reddy, Kiran Kumar, Das, Pejus Manoj, Morley, Adam Douglas, Ly, Shirley Xue Yi
• Primary Examiner(s): Vy, Hung T
• Application Number:
  US14/859,062
• Time in Patent Office: 1,019 Days
• Field of Search: 707687, 707690
• CPC Class Codes: G06F 16/22 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/2365 Ensuring data consistency and integrity

Abstract

A non-relational data store may implement validating and non-validating secondary indexes for a table. Operations at a table for a given item may be performed when indexing the item to create a secondary index or when updates to the given item are received. Attribute values of a given item may be validated with respect to an indexing schema for the secondary index. For a non-validating secondary index, validation errors detected for the attribute values may be ignored so that the operation at the table may be performed. For a validating secondary index, validation errors detected for the attribute values may result in denying performance of the operation. In some embodiments, a secondary index from may be changed from validating to non-validating, or non-validating to validating.

Claims

1. A system, comprising:

one or more storage nodes maintaining a table in a non-relational data store;

one or more other storage nodes maintaining a secondary index for the table in the non-relational data store according to an indexing schema; and

one or more processors and memory to implement a storage engine for the table, configured to;

receive an operation for a given item in the table, wherein the given item is eligible to be included in the secondary index;

validate one or more key values of the given item according to respective key validation parameters in the indexing schema;

detect an invalid value for at least one of the key values according to the validation;

identify the secondary index as a non-validating secondary index; and

perform the operation with respect to the given item even though the invalid value was detected for at least one of the key values of the given item.

2. The system of claim 1, wherein the respective key validation parameters specify a respective data type for the key value, and wherein to detect the invalid value for the at least one key value, the storage engine is configured to determine that a given data type for the at least one key value does not match the respective data type in the key validation parameter for the at least one key value.

3. The system of claim 1, wherein the operation is a secondary index creation operation to create the secondary index that indexes a plurality of different items in the table including the given item.

4. The system of claim 1, wherein the non-relational data store is a network-based storage service, wherein the system further implements a network-based interface for the network-based storage service, and wherein a request to configure the secondary index as non-validating is received via the network-based interface for network-based storage service prior to the attempt to perform the operation.

5. A method, comprising:

performing, by one or more computing devices;

evaluating a given item for performance of an operation at a table with respect to the given item stored in the table, wherein the given item is eligible to be included in a secondary index for the table, wherein the table and the secondary index are maintained in a non-relational data store, and wherein the evaluating comprises;

validating one or more attribute values of the given item according to an indexing schema for the secondary index;

detecting an invalid value for at least one of the attribute values according to the validation;

identifying that the secondary index is a non-validating index; and

performing the operation with respect to the given item even though the invalid value was detected for at least one of the attribute values of the given item.

6. The method of claim 5, further comprising:

wherein the operation is an operation to store the given item in the table;

wherein the evaluating is performed in response to receiving a request to store the given item in the table, wherein one or more other item attributes of the item that are key values according to an indexing schema for the table are validated without error.

7. The method of claim 5, wherein the indexing schema includes one or more validation parameters that specify a respective data type for the one or more attribute values, and wherein detecting the invalid value for the at least one attribute value comprises determining that a given data type for the at least one attribute value does not match the respective data type in the validation parameter for the at least one attribute value.

8. The method of claim 5, further comprising:

prior to evaluating the given item, receiving a request to configure the secondary index as non-validating.

9. The method of claim 5, further comprising:

receiving a request to reconfigure the secondary index as validating such that a subsequent evaluation for performance of an operation at the table with respect to another item denies performance of the operation with respect to the other item in response to detecting an invalid value for one or more attributes of the other item according to the indexing schema.

10. The method of claim 5, wherein one or more other validating secondary indexes are maintained for the table.

11. The method of claim 10, wherein ignoring the detection of the invalid value is further performed in response to determining that the one or more attribute values are valid with respect to an indexing schema for the one or more other validating secondary indexes in addition to determining that the secondary index is a non-validating index.

12. The method of claim 5, wherein the operation is a secondary index creation operation to create the secondary index that indexes a plurality of different items in the table including the given item.

13. The method of claim 5, wherein the non-relational data store is a distributed data store and wherein the secondary index is stored at one or more storage nodes that are different than one or more other storage nodes that store the table.

14. A non-transitory, computer-readable storage medium, storing program instructions that when executed by one or more computing devices cause the one or more computing devices to implement:

receive an operation at a table with respect to a given item stored in the table, wherein the given item is eligible to be included in a secondary index for the table, wherein the secondary index and the table are maintained in a non-relational data store;

validating one or more attribute values of the given item according to an indexing schema for the secondary index;

detecting an invalid value for at least one of the attribute values according to the validation;

identifying that the secondary index is a non-validating index; and

performing the operation with respect to the given item even though the invalid value was detected for at least one of the attribute values of the given item.

15. The non-transitory, computer-readable storage medium of claim 14, wherein the program instructions cause the one or more computing devices to further implement:

prior to attempting to perform the operation, receiving a request to update the one or more attribute values of the given item at the table, wherein the operation is an operation to update the one or more attribute values of the given item.

16. The non-transitory, computer-readable storage medium of claim 14, wherein the indexing schema includes a validation parameter that specifies a data size limitation for the one or more attribute values, and wherein, in detecting the invalid value for the at least one attribute, the program instructions cause the one or more computing devices to implement determining that a data size of the one or more attribute values exceeds the data size limitation specified in the validation parameter.

17. The non-transitory, computer-readable storage medium of claim 14, wherein the program instructions cause the one or more computing devices to further implement:

prior to attempting to perform the operation, receiving a request to configure the secondary index as non-validating.

18. The non-transitory, computer-readable storage medium of claim 14, wherein the program instructions cause the one or more computing devices to further implement:

receiving a request to reconfigure the secondary index as validating such that a subsequent attempt to perform an operation with respect to another item denies performance of the operation with respect to the other item in response to detecting an invalid value for one or more attributes of the other item according to the indexing schema.

19. The non-transitory, computer-readable storage medium of claim 14, wherein one or more other non-validating secondary indexes are maintained for the table.

20. The non-transitory, computer-readable storage medium of claim 14, wherein the non-relational data store is a network-based storage service and wherein the secondary index is stored at one or more storage nodes that are different than one or more other storage nodes that store the table.