Cloning catalog objects

• US 10,534,793 B2
• Filed: 06/02/2016
• Issued: 01/14/2020
• Est. Priority Date: 02/19/2014
• Status: Active Grant

• Current Assignee: Snowflake, Inc.
• Original Assignee: Snowflake, Inc.
• Inventors: Motivala, Ashish, Dageville, Benoit
• Primary Examiner(s): Khong, Alexander
• Application Number:
  US15/171,859
• Publication Number:
  US 20160275160A1
• Time in Patent Office: 1,321 Days
• CPC Class Codes: A61F 5/566 Intra-oral devices
• CPC Class Codes: G06F 16/128 Details of file system snapshots on the file-level, e.g. snapshot creation, administration, deletion use of snapshots for error detection or correction G06F11/14, G06F11/16
• CPC Class Codes: G06F 16/148 File search processing
• CPC Class Codes: G06F 16/1827 Management specifically adapted to NAS management of storage area networks [SAN] G06F3/067
• CPC Class Codes: G06F 16/211 Schema design and management
• CPC Class Codes: G06F 16/221 Column-oriented storage; Management thereof
• CPC Class Codes: G06F 16/2365 Ensuring data consistency and integrity
• CPC Class Codes: G06F 16/24532 of parallel queries
• CPC Class Codes: G06F 16/24545 Selectivity estimation or determination
• CPC Class Codes: G06F 16/24552 Database cache management
• CPC Class Codes: G06F 16/2456 Join operations
• CPC Class Codes: G06F 16/2471 Distributed queries
• CPC Class Codes: G06F 16/254 Extract, transform and load [ETL] procedures, e.g. ETL data flows in data warehouses
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
• CPC Class Codes: G06F 16/273 Asynchronous replication or reconciliation
• CPC Class Codes: G06F 16/283 Multi-dimensional databases or data warehouses, e.g. MOLAP or ROLAP
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation
• CPC Class Codes: G06F 16/9538 Presentation of query results
• CPC Class Codes: G06F 9/4881 Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
• CPC Class Codes: G06F 9/5016 : the resource being the memory
• CPC Class Codes: G06F 9/5044 : considering hardware capabilities
• CPC Class Codes: G06F 9/5083 : Techniques for rebalancing the load in a distributed system
• CPC Class Codes: G06F 9/5088 : involving task migration
• CPC Class Codes: H04L 67/1095 : Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
• CPC Class Codes: H04L 67/1097 : for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
• CPC Class Codes: H04L 67/568 : Storing data temporarily at an intermediate stage, e.g. caching

Abstract

Example systems and methods for cloning catalog objects are described. In one implementation, a method identifies an original catalog object associated with a set of data and creates a duplicate copy of the original catalog object without copying the data itself. The method allows access to the set of data using the duplicate catalog object and supports modifying the data associated with the original catalog object independently of the duplicate catalog object. The duplicate catalog object can be deleted upon completion of modifying the data associated with the original catalog object.

Claims

1. A system comprising:

means for identifying a catalog object in a database comprising data stored in files, wherein the files cannot be updated or modified;

means for creating a duplicate copy of the catalog object without copying the data itself;

means for modifying at least one of the files of the catalog object by recreating the at least one of the files;

means for executing data access requests directed to the data stored in the files by reading the duplicate copy of the catalog object when the at least one of the files is being recreated;

means for adding additional files to either of the catalog object or the duplicate copy of the catalog object independently of another; and

means for deleting the duplicate copy of the catalog object upon completion of modifying the at least one of the files of the catalog object.

2. The system of claim 1, wherein the means for creating a duplicate copy of the catalog object further comprises means for copying metadata associated with the data stored in the files.

3. The system of claim 2, wherein the means for copying metadata further comprises means for copying an inventory of the data stored in the files.

4. The system of claim 2, wherein the means for copying metadata further comprises means for copying information regarding the data stored in the files that enables identification of the data stored in the files without requiring access to the data.

5. The system of claim 1, wherein the means for identifying a catalog object in a database comprises means for identifying a logical grouping of data in the database.

6. The system of claim 1, wherein the means for identifying a catalog object comprises means for identifying one of a table, a database, a schema, an account, a constraint, a file format, a function, a role, a sequence, a stage, a column, a user, or a volume.

7. The system of claim 1, wherein the means for modifying at least one of the files of the catalog object further comprises means for modifying such that modified data is not visible to the duplicate copy of the catalog object.

8. The system of claim 1, wherein the means for modifying at least one of the files of the catalog object further comprises means for modifying such that modified data is not visible to the catalog object in the database.

9. A system for cloning objects within a database, the system comprising:

a storage platform configured to store original objects associated with a set of data; and

a resource manager configured to;

identify a catalog object in the database, the catalog object comprising data stored in files, wherein the files cannot be updated or modified;

create a duplicate copy of the catalog object without copying the data itself;

allow modification of at least one of the files of the catalog object by recreating the at least one of the files;

assign tasks for executing data access requests directed to the data stored in the files, the tasks comprising reading the duplicate copy of the catalog object when the at least one of the files is being recreated;

add additional files to either of the catalog object or the duplicate copy of the catalog object independently of another; and

delete the duplicate copy of the catalog object upon completion of modification of the at least one of the files of the catalog object.

10. The system of claim 9, wherein the resource manager is further configured to copy metadata associated with the data stored in the files.

11. The system of claim 10, wherein the metadata includes an inventory of the data stored in the files.

12. The system of claim 10, wherein the metadata identifies information regarding the data stored in the files such that the data stored in the files can be identified without reading the data itself.

13. The system of claim 9, wherein the catalog object comprises a logical grouping of data in the database.

14. The system of claim 9, wherein the catalog object comprises one or more of a table, a database, a schema, an account, a constraint, a file format, a function, a role, a sequence, a stage, a column, a user, or a volume.

15. The system of claim 9, wherein modification of the at least one of the files comprises modifying such that modified data is not visible to the duplicate copy of the catalog object.

16. The system of claim 9, wherein modification of the at least one of the files comprises modifying such that modified data associated with the duplicate objects is not visible to the catalog object in the database.

17. A method comprising:

identifying a catalog object in a database, the catalog object comprising data stored in files, wherein the files cannot be updated or modified;

creating a duplicate copy of the catalog object without copying the data itself;

modifying at least one of the files of the catalog object by recreating the at least one of the files;

executing data access requests directed to the data stored in the files by reading the duplicate copy of the catalog object when the at least one of the files is being recreated;

adding additional files to either of the catalog object or the duplicate copy of the catalog object independently of another; and

deleting the duplicate copy of the catalog object upon completion of modifying the at least one of the files of the catalog object.

18. The method of claim 17, wherein creating a duplicate copy of the catalog object further comprises copying metadata associated with the data stored in the files.

19. The method of claim 18, wherein creating a duplicate copy of the catalog object further comprises copying an inventor of the data stored in the files.

20. The method of claim 18, wherein the metadata comprises information about the data stored in the files such that the data stored in the files can be identified without reading the data itself.

21. The method of claim 17, wherein identifying the catalog object comprises identifying a logical grouping of data in a data in the database.

22. The method of claim 17, wherein identifying the catalog object comprises identifying one or more of a table, a database, a schema, an account, a constraint, a file format, a function, a role, a sequence, a stage, a column, a user, or a volume.

23. The method of claim 17, wherein creating the duplicate copy of the catalog object comprises creating the duplicate copy such that the duplicate copy is read only.

24. The method of claim 17, wherein modifying the at least one of the files of the catalog object comprises modifying such that modified data is not visible to the duplicate copy of the catalog object.

25. The method of claim 17, wherein modifying the at least one of the files of the catalog object comprises modifying such that modified data is not visible to the catalog object in the database.

26. The method of claim 17, wherein creating the duplicate copy of the catalog object comprises creating the duplicate copy such that the data deleted from the data stored in the files remains visible to the duplicate copy of the catalog object.

27. The method of claim 17, wherein creating the duplicate copy of the catalog object comprises creating such that data deleted from the duplicate copy of the catalog object remains visible to the data stored in the files of the database.

28. The method of claim 17, wherein modifying the at least one of the files of the catalog object comprises modifying such that data added to the data stored in the files is not visible to the duplicate copy of the catalog object.

29. The method of claim 17, further comprising modifying the duplicate copy of the catalog object such that data inserted into the duplicate copy of the catalog object is not visible to the data stored in the files of the database.

30. The method of claim 17, further comprising deleting data originally stored in the files upon completion of modifying the at least one of the files of the catalog object.

31. The method of claim 17, wherein creating the duplicate copy of the catalog object further comprises updating a duplicate hierarchy of all generations of children.