Testing data silo

• US 8,874,593 B2
• Filed: 10/20/2011
• Issued: 10/28/2014
• Est. Priority Date: 07/01/2011
• Status: Active Grant

• Current Assignee: Salesforce.com, Inc.
• Original Assignee: Salesforce.com, Inc.
• Inventors: Unger, Richard, Wong, Simon
• Primary Examiner(s): Chen, Susan
• Application Number:
  US13/277,905
• Publication Number:
  US 20130007523A1
• Time in Patent Office: 1,104 Days
• Field of Search: 707/755, 707/635, 707/772, 707/793, 700/83
• US Class Current: 707/754
• CPC Class Codes: G06F 11/3664 Environments for testing or debugging software
• CPC Class Codes: G06F 11/3672 Test management

Abstract

In accordance with disclosed embodiments, there are provided methods, systems, and apparatuses for creating a data silo and testing with a data silo including, for example, initiating a test sequence against a production database within a host organization, wherein the test sequence specifies: (a) new data for insertion into the production database during the test sequence and (b) one or more queries for execution against the production database during the test sequence. Such a method further includes inserting the new data into the production database; recording one or more row IDs corresponding to the inserted new data; injecting a test context filter into the one or more queries based on the one or more row IDs; and executing the one or more queries against the production database.

Claims

1. A method comprising:

maintaining production data within a production database within a system of a host organization,

the system having a processor and memory therein;

initiating a test sequence against the production database within the system of the host organization, wherein the test sequence specifies;

a) new data for insertion into the production database during the test sequence, and

b) one or more queries limited only to the new data for execution against the production database during the test sequence, wherein the one or more queries cause a lock on data elements of the new data as specified by the one or more queries within the production database which in turn will trigger a data access failure for other production transactions requesting access to the locked data elements of the new data;

inserting the new data into the production database;

recording one or more row IDs corresponding to the inserted new data;

injecting a test context filter into the one or more queries based on the one or more row IDs; and

executing the one or more queries having the test context filter injected therein against the production database, wherein the scope of the one or more queries is restricted to only the new data inserted into the production database during the test sequence via the test context filter preventing the new data introduced into to the production database during the test sequence from being discoverable to the production transactions due to the new data being associated with the test context and not a production context.

2. The method of claim 1:

wherein recording the one or more row IDs associates the new data with a test context; and

wherein the text context corresponds to the one or more row IDs identifying one or more new rows in the production database as part of inserting the new data.

3. The method of claim 1:

wherein a first plurality of elements stored within the production database associated with the test context are queryable within the scope of the one or more queries of the test sequence; and

wherein a second plurality of elements stored within the production database which are not associated with the test context are filtered out of the scope of the one or more queries of the test sequence via the test context filter injected into the one or more queries.

4. The method of claim 1, wherein injecting the test context filter into the one or more queries comprises at least one of:

associating data with a test context indicator;

adding a test context indicator to a new table name created pursuant to the inserting the new data into the production database;

adding a test context indicator to a new object name created pursuant to the inserting the new data into the production database;

adding a test context indicator to a new record key created pursuant to the inserting the new data into the production database;

adding a test context indicator to a field or column name created pursuant to the inserting the new data into the production database; and

adding a test context indicator to an account created pursuant to the inserting the new data into the production database.

5. The method of claim 1, wherein inserting the new data into the production database comprises inserting the new data into the production database without committing the new data to the production database.

6. The method of claim 5, further comprising:

executing a roll-back operation against the production database for the new data inserted into the production database during the test sequence, wherein the roll-back operation is initiated responsive to the test sequence terminating.

7. The method of claim 1, further comprising:

receiving the test sequence at the host organization from a client device;

wherein initiating the test sequence against the production database comprises initiating the received test sequence on behalf of the client device responsive to receiving the test sequence; and

wherein the received test sequence comprises a listing of instructions specifying the new data to be inserted into the production database during the test sequence, specifying the one or more queries to be executed against the production database during the test sequence, and specifying one or more additional test operations to be performed by the host organization.

8. The method of claim 1, further comprising:

monitoring, via one or more observers of a Transaction Entity Object Observer of the host organization, all objects created within the production database; and

wherein recording the one or more row IDs for the inserted new data comprises the Transaction Entity Object Observer associating the new data with a test context based on the new data being associated with the test sequence.

9. The method of claim 1, further comprising:

monitoring, via a query optimizer of the host organization, all queries executed against the production database; and

determining, via the query optimizer, that the one or more queries require the test context based on the one or more queries being associated with the test sequence; and

wherein injecting the test context filter into the one or more queries comprises the query optimizer injecting the test context filter into the one or more queries based on the determination.

10. The method of claim 1, wherein injecting the test context filter into the one or more queries comprises a query optimizer of the host organization performing at least one of the following operations for each of the one or more queries:

narrowing a query scope specified by the query to a plurality of elements in the production database associated with a test context;

adding a test context indicator to a table name specified by the query;

adding a test context indicator to an object name specified by the query;

adding a test context indicator to a record key specified by the query;

adding a test context indicator to a field or column name specified by the query;

adding a test context indicator to an account specified by the query;

adding or pre-pending a pre-query to limit a query scope to only elements associated with the test context; and

adding or pre-pending a clause into the query requiring that query results match the test context filter.

11. The method of claim 1, wherein the test sequence further specifies one or more test operations selected from the group of test operations comprising:

computing a result based at least in part on the one or more queries against the production database during the test sequence;

implementing a trigger for an element of the new data inserted into the production database, wherein the trigger causes the production database to compute or derive non-specified data responsive to the new data being inserted into the production database and wherein the trigger further causes the non-specified data to be inserted into the production database;

reading the non-specified data from the production database;

determining whether the trigger was implemented successfully based upon the reading of the non-specified data from the production database and based further upon comparing the non-specified data against an expected result or a threshold.

12. The method of claim 1, wherein the test sequence further specifies new cache data to be cached by the host organization external to the production database;

marking the new cache data with the test context;

inserting the new cache data into a global caching layer of the host organization;

injecting the test context filter into all cache reads initiated on behalf of the test sequence; and

removing the new cache data from the global caching layer responsive to the test sequence terminating.

13. The method of claim 12, wherein marking the new cache data with the test context comprises modifying one or more keys identifying the new cache data in the global caching layer with a test context indicator.

14. The method of claim 1:

wherein the production database is communicably interfaced with a plurality of client devices through the host organization and operates within a production environment of the host organization; and

wherein the production data stored within the production database comprises live production data stored on behalf of the plurality of client devices and accessible to the plurality of client devices due to the live production data being associated with the production context and not the test context.

15. The method of claim 1, wherein:

the production database of the host organization comprises a multi-tenant database system of the host organization; and

wherein the method further comprises;

receiving the test sequence at the host organization as one of a plurality of requests received from a plurality of customer organizations, wherein each customer organization is an entity selected from the group consisting of;

a separate and distinct remote organization, an organizational group within the host organization, a business partner of the host organization, or a customer organization that subscribes to cloud computing services provided by the host organization.

16. The method of claim 15, wherein the multi-tenant database system comprises elements of hardware and software that are shared by a plurality of separate and distinct customer organizations, each of the separate and distinct customer organizations being remotely located from the host organization having the multi-tenant database system operating therein.

17. A non-transitory computer readable storage medium having instructions stored thereon that, when executed by a system in a host organization having a processor and memory therein, the instructions cause the system to perform operations comprising:

maintaining production data within a production database within the system of the host organization;

initiating a test sequence against the production database within the system of the host organization, wherein the test sequence specifies;

a) new data for insertion into the production database during the test sequence, and

b) one or more queries limited only to the new data for execution against the production database during the test sequence, wherein the one or more queries cause a lock on data elements of the new data as specified by the one or more queries within the production database which in turn will trigger a data access failure for other production transactions requesting access to the locked data elements of the new data;

inserting the new data into the production database;

recording one or more row IDs corresponding to the inserted new data;

injecting a test context filter into the one or more queries based on the one or more row IDs; and

executing the one or more queries having the test context filter injected therein against the production database, wherein the scope of the one or more queries is restricted to only the new data inserted into the production database during the test sequence via the test context filter preventing the new data introduced into to the production database during the test sequence from being discoverable to the production transactions due to the new data being associated with the test context and not a production context.

18. The non-transitory computer readable storage medium of claim 17:

wherein a first plurality of elements stored within the production database associated with the test context are queryable within the scope of the one or more queries of the test sequence; and

wherein a second plurality of elements stored within the production database which are not associated with the test context are filtered out of the scope of the one or more queries of the test sequence via the test context filter injected into the one or more queries.

19. The non-transitory computer readable storage medium of claim 17:

wherein inserting the new data into the production database comprises inserting the new data into the production database without committing the new data to the production database; and

wherein the operations further comprise executing a roll-back operation against the production database for the new data inserted into the production database during the test sequence, the roll-back operation initiated responsive to the test sequence terminating.

20. A system having a processor and memory therein, comprising:

the processor to execute instructions;

a production database to maintain production data;

a request interface to receive a test sequence from a client device wherein the test sequence specifies;

a) new data for insertion into the production database during the test sequence, and

b) one or more queries limited only to the new data for execution against the production database during the test sequence, wherein the one or more queries cause a lock on data elements of the new data as specified by the one or more queries within the production database which in turn will trigger a data access failure for other production transactions requesting access to the locked data elements of the new data;

a Transaction Entity Object Observer to associate the new data with a test context by recording one or more row IDs corresponding to the inserted new data;

a query interface to insert the new data into the production database;

a query optimizer to inject a test context filter into the one or more queries based on the one or more row IDs; and

the query interface to execute the one or more queries having the test context filter injected therein against the production database, wherein the scope of the one or more queries is restricted to only the new data inserted into the production database during the test sequence via the test context filter preventing the new data introduced into to the production database during the test sequence from being discoverable to the production transactions due to the new data being associated with the test context and not a production context.

21. The system of claim 20, further comprising:

a web-server to implement the request interface to receive the test sequence from the client device;

a multi-tenant database system to implement the production database; and

wherein test sequence is to be received at the system as one of a plurality of requests received from a plurality of customer organizations, each customer organization being an entity selected from the group consisting of;

a separate and distinct remote organization, an organizational group within the host organization, a business partner of the host organization, or a customer organization that subscribes to cloud computing services provided by the host organization.