SYSTEM AND METHOD FOR PERFORMING SHARD MIGRATION TO SUPPORT FUNCTIONS OF A CLOUD-BASED SERVICE

• US 20140012814A1
• Filed: 07/08/2013
• Published: 01/09/2014
• Est. Priority Date: 07/06/2012
• Status: Active Grant

• Current Assignee: Box Incorporated
• Original Assignee: Box Incorporated
• Inventors: Jourda, Florian, Trombley-Shapiro, Benjamin, Bercovici, Tamar
• Granted Patent: US 9,792,320 B2
• Time in Patent Office: Days
• US Class Current: 707/636
• CPC Class Codes: G06F 16/2379 Updates performed during online database operations; commit processing
• CPC Class Codes: G06F 16/273 Asynchronous replication or reconciliation

Abstract

Systems and methods of maintaining a horizontally scaled database based on data ownership for a cloud-based platform (e.g., cloud-based collaboration and/or storage platform/service) are disclosed. The system database comprises multiple shard databases, and all files and folders owned by a user are stored on a single shard database. When a user transfers ownership of a file and/or a folder to a second user, the transferred file and/or folder is stored on the shard database that stores all of the data for the second user.

Claims

1. A method of database partitioning comprising:

partitioning a database into multiple shard databases based upon ownership of objects stored in the database, wherein objects owned by a given user are stored in a single shard database, and

further wherein a user-initiated action that transfers ownership of an object from a first user to a second user results in movement of the object from a first shard database associated with the first user to a second shard database associated with the second user;

maintaining a mapping database that maps database object identifications to shard database identifications indicating in which one of the multiple shard databases respective objects are stored.

2. The method of claim 1, wherein at least a subset of users of the database are associated with an enterprise account provided by a cloud-based platform, and further wherein objects owned by the subset of users are stored in a single shard database.

3. The method of claim 2, further comprising:

permitting an administrator of the enterprise account to query database objects owned by the subset of users associated with the enterprise account.

4. The method of claim 1, further comprising:

receiving a database-related request for a first object;

upon determining that the request involves transfer of ownership of the first object from a given first user to a given second user, identifying a first shard database that stores objects owned by the first given user and a second shard database that stores objects owned by the given second user;

moving the first database object from the first shard database to the second shard database, if the first shard database is different from the second shard database;

updating the mapping database to reflect the movement of the first object to the second shard database.

5. The method of claim 4, wherein determining that the request involves transfer of ownership comprises identifying a field of the request as a mapping key used by the mapping database to identify which one of the multiple shard databases stores an object.

6. The method of claim 4, wherein the first object is a folder, the method further comprising:

determining one or more objects stored within the first object;

moving the one or more objects with the first object to the second shard database, if the first shard database is different from the second shard database.

7. The method of claim 6, wherein determining one or more objects stored within the first object comprises using a prefix query with the first object.

8. The method of claim 6, wherein a parent for each folder and a path for each folder are stored for use in responding to the prefix query.

9. The method of claim 1, further comprising:

receiving a database-related request for a second object;

upon determining that the request does not involve transfer of ownership of the second object, executing the database-related request.

10. The method of claim 9, wherein determining that the request does not involve transfer of ownership comprises determining that fields of the request do not include a mapping key, wherein the mapping key is used with the mapping database to determine in which one of the multiple shard databases a particular object is stored.

11. The method of claim 1, wherein objects stored in the database have a single owner.

12. The method of claim 10, wherein the multiple shard databases are used to store data for a cloud-based platform.

13. A method of performing shard migration for a cloud-based platform comprising:

receiving a user-initiated request that involves change of ownership of one or more items stored in one of multiple shard databases in the cloud-based platform, where items owned by a particular user are stored in a single shard database;

moving the one or more items from a source shard database associated with a first user to a destination shard database associated with a second user.

14. The method of claim 13, wherein moving the one or more items comprises:

selecting for updating the one or more items in the source shard database;

updating the one or more items in a mapping database to point to the destination shard database;

inserting the one or more items from the source shard database into the destination shard database;

deleting the one or more selected items from the source shard database.

15. The method of claim 14, wherein access by other users to the one or more items in the source shard database, the mapping database, and the destination shard database are locked while moving the one or more items.

16. The method of claim 14, further comprising upon determining that there is a problem with updating the one or more items in the mapping database, inserting the one or more items into the destination shard database, or deleting the one or more selected items from the source shard database, reinstating a state of the mapping database, the source shard database, and the destination shard database prior to moving the one or more items.

17. The method of claim 13, wherein the multiple shard databases are used to store the one or more items for in the cloud-based platform which enables the first and the second user to collaborate on the one or more items.

18. The method of claim 13, wherein each data object stored in the multiple shard databases has a single owner.

19. A system for performing shard migration to support functions of a cloud-based service, the system comprising:

a processor;

a memory having stored thereon instructions which, when executed by the processor, causes the system to;

receive a user-initiated request that involves change of ownership of one or more objects stored in one of multiple shard databases, where objects owned by a particular user are stored in a single shard database; and

move the one or more objects from a source shard database associated with a first user to a destination shard database associated with a second user;

wherein the multiple shard databases are used to store the one or more objects which enables the first and the second user to collaborate on via the cloud-based service.

20. The system of claim 19, wherein moving the one or more objects comprises:

updating the one or more objects in a mapping database to point to the destination shard database;

inserting the one or more objects from the source shard database into the destination shard database;

deleting the one or more selected items from the source shard database.

21. The system of claim 20, wherein the system is further caused to:

lock access to the source shard database, the mapping database, and the destination shard database while moving the one or more objects.

22. The system of claim 20, wherein the system is further caused to:

upon encountering a problem with moving the one or more objects, return the mapping database, the source shard database, and the destination shard database to a state prior to moving the one or more objects.

23. A machine-readable storage medium having stored thereon instructions which, when executed by a processor, cause the processor to:

partition a database into multiple shard databases based upon ownership of objects stored in the database, wherein objects owned by a given user are stored in a single shard database, and further wherein a user-initiated action that transfers ownership of an object from a first user to a second user results in movement of the object from a first shard database associated with the first user to a second shard database associated with the second user.

24. The machine-readable storage medium of claim 23, wherein the instructions further cause the processor to:

maintain a mapping database that maps database object identifications to shard database identifications indicating in which one of the multiple shard databases respective objects are stored.

25. The machine-readable storage medium of claim 23, wherein the instructions further cause the processor to:

receive a request pertaining to a first database object;

upon determining that the request involves transfer of ownership of the first object from a given first user to a given second user, identifying a first shard database that stores objects owned by the first given user and a second shard database that stores objects owned by the given second user;

moving the first object from the first shard database to the second shard database, if the first shard database is different from the second shard database;

updating the mapping database to reflect the movement of the first object to the second shard database.

26. The machine-readable storage medium of claim 25, wherein the first database object is a folder, and wherein the instructions further cause the processor to:

determine one or more objects stored within the first object;

move the one or more database objects with the first object to the second shard database, if the first shard database is different from the second shard database.

27. The machine-readable storage medium of claim 23, wherein the instructions further cause the processor to:

receive a request pertaining to a second object;

upon determining that the request does not involve transfer of ownership of the second object, execute the request.

28. A system for performing shard migration for a cloud-based collaboration service, the system comprising:

means for receiving a request and determining whether the request involves a change of ownership of one or more items in the cloud-based collaboration service;

means for moving the one or more items from a first shard to a second shard responsive to the request if the one or more items are currently stored in the first shard associated with a first owner and the second shard associated with a new owner is different from the first shard.

29. The system of claim 28, further comprising:

means for executing the request if the request does not involve a change of ownership in the cloud-based collaboration service.

30. The system of claim 28, further comprising:

means for updating a mapping metadata that stores mappings between each stored data object and an identification of which of the multiple shard stores the respective data object.