Flexible data storage system

• US 9,405,627 B2
• Filed: 08/21/2014
• Issued: 08/02/2016
• Est. Priority Date: 04/27/2007
• Status: Active Grant

• Current Assignee: Stripe, Inc.
• Original Assignee: IP3, Series 100 of Allied Security Trust I (Allied Security Trust)
• Inventors: Shuster, Gary Stephen
• Primary Examiner(s): Ruiz, Aracelis
• Application Number:
  US14/465,366
• Publication Number:
  US 20150149719A1
• Time in Patent Office: 712 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 11/1096 Parity calculation or recalculation after configuration or reconfiguration of the system
• CPC Class Codes: G06F 2211/1004 Adaptive RAID, i.e. RAID system adapts to changing circumstances, e.g. RAID1 becomes RAID5 as disks fill up
• CPC Class Codes: G06F 3/0607 by facilitating the process of upgrading existing storage systems, e.g. for improving compatibility between host and storage device
• CPC Class Codes: G06F 3/0608 Saving storage space on storage systems
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/064 Management of blocks
• CPC Class Codes: G06F 3/0644 Management of space entities, e.g. partitions, extents, pools
• CPC Class Codes: G06F 3/065 Replication mechanisms
• CPC Class Codes: G06F 3/0689 Disk arrays, e.g. RAID, JBOD

Abstract

Methods and systems for managing and locating available storage space in a system comprising data files stored in a plurality of storage devices and configured in accordance with various data storage schemes (mirroring, striping and parity-striping). A mapping table associated with each of the plurality of storage devices is used to determine the available locations and amount of available space in the storage devices. The data storage schemes for one or more of the stored data files are changed to a basic storage mode when the size of a new data file configured in accordance with an assigned data storage scheme exceeds the amount of available space. The configured new data file is stored in accordance with the assigned data storage scheme in one or more of the available locations and the locations of the new data file are recorded.

Claims

1. A system comprising:

a plurality of storage devices comprising stored data files, the stored data files stored in accordance with a plurality of data storage schemes;

a mapping table associated with each of the plurality of storage devices, the mapping table storing data representing available locations and amounts of available space in the plurality of storage devices,

wherein, the system is configured to change at least one data storage scheme of a corresponding stored data file when additional storage space is needed for a new data file, and the at least one data storage scheme to change is automatically selected by the system based on a user having identified the corresponding stored data file as non-critical.

2. The system of claim 1, wherein the plurality of data storage schemes comprises a plurality of RAID levels.

3. The system of claim 1, wherein the user identifies the corresponding stored data file as non-critical at a time when the system detects the available remaining space is insufficient for new data files.

4. The system of claim 1, wherein the mapping table is stored in a mirrored manner across each of the plurality of storage devices.

5. The system of claim 1, wherein the plurality of data storage schemes comprises at least two of mirroring, striping, and parity-striping.

6. The system of claim 1, further comprising:

assigning one of the plurality of data storage schemes to the new data file; and

storing the new data file on at least one of the plurality of storage devices in accordance with the assigned data storage scheme.

7. The system of claim 6, wherein the one of the plurality of data storage schemes is automatically assigned based on at least one of desired performance, fault tolerance, and redundancy for the new data file.

8. The system of claim 6, wherein a user is prompted to select and assign one of the plurality of data storage schemes to the new data file.

9. The system of claim 1, wherein the assigning one of the plurality of data storage schemes to the new data file comprises:

assigning a first one of the plurality of data storage schemes to a first part of the new data file; and

assigning a second one of the plurality of data storage schemes to a second part of the new data file; and

wherein storing the new data file comprises;

storing the first part of the new data file in accordance with the first one of the plurality of data storage schemes; and

storing the second part of the new data file in accordance with the second one of the plurality of data storage schemes.

10. The system of claim 9, wherein the first one of the plurality of data storage schemes comprises a mirrored data storage scheme, and wherein the second one of the plurality of data storage schemes comprises a non-mirrored data storage scheme.

11. The system of claim 9, wherein the first one of the plurality of data storage schemes comprises a striped data storage scheme, and wherein the second one of the plurality of data storage schemes comprises a non-striped data storage scheme.

12. The system of claim 9, wherein the first one of the plurality of data storage schemes comprises a parity-striped data storage scheme, and wherein the second one of the plurality of data storage schemes comprises a non-parity-striped data storage scheme.

13. The system of claim 1, wherein the at least one data storage scheme changed comprises a mirrored data file subject to frequent back-ups.

14. The system of claim 1:

wherein a first copy of one of the stored data files is stored in first disk sectors on a first one of the plurality of storage devices;

wherein a second copy of the one of the stored data files is stored in second disk sectors on a second one of the plurality of storage devices; and

wherein a physical location of the first disk sectors on the first one of the plurality of storage devices do not correspond to a physical location of the second disk sectors on the second one of the plurality of storage devices.

15. The system of claim 1, wherein the plurality of storage devices do not have identical available space.

16. The system of claim 1, wherein the plurality of storage devices have different size capacities.

17. The system of claim 1, wherein the at least one data storage scheme changed comprises a mirrored data file subject to frequent synchronization with an external device.

18. The system of claim 1, wherein the user identifies the corresponding stored data file as non-critical at a time when the user saves the data file.

19. A method comprising:

storing data files in a plurality of storage devices, the data files stored in accordance with a plurality of data storage schemes, wherein a portion of one of the data files is stored according to a first one of the plurality of data schemes on a first storage device, and a remaining portion of the one of the data files is stored according to a second one of the plurality of data schemes on a second storage device; and

storing, in a mapping table associated with at least one of the plurality of storage devices, data representing the locations and storage schemes of the data files, wherein the portion of the one of the data files is stored in first disk sectors on the first storage device, and the remaining portion is stored in second disk sectors on the second storage device; and

wherein a physical location of the first disk sectors on the first storage device does not correspond to a physical location of the second disk sectors on the second storage device.

20. The method of claim 19, wherein the mapping table further contains available locations and amounts of available space on at least one of the plurality of storage devices.

21. The method of claim 19, wherein the portion of the one of the data files is stored according to a mirroring data scheme and the remaining portion is stored according to a non-mirroring data scheme.

22. The method of claim 19, wherein the portion of the one of the data files is stored according to a striping data scheme and the remaining portion is stored according to a non-striping data scheme.

23. The method of claim 19, wherein the mapping table is stored in a mirrored manner across each of the plurality of storage devices.

24. The method of claim 19, wherein the mapping table is stored in accordance with one of RAID-0, RAID-1, RAID-5, and RAID-6.

25. The method of claim 19, further comprising:

assigning one of the plurality of data storage schemes to a new data file; and

storing the new data file on at least one of the plurality of storage devices in accordance with the assigned data storage scheme.

26. The method of claim 25, wherein assigning comprises assigning the one of the plurality of data storage schemes automatically based on at least one of desired performance, fault tolerance, and redundancy for the new data file.

27. The method of claim 25, wherein assigning comprises prompting a user to select and assign the assigned one of the plurality of data storage schemes to the new data file.

28. The method of claim 19, wherein the portion of the one of the data files is stored according to a mirrored data storage scheme, and wherein the remaining portion is stored according to a non-mirrored data storage scheme.

29. The method of claim 19, wherein the portion of the one of the data files is stored according to a striped data storage scheme, and wherein the remaining portion is stored according to a non-striped data storage scheme.

30. The method of claim 19, wherein the portion of the one of the data files is stored according to a parity-striped data storage scheme, and wherein the remaining portion is stored according to a non-parity-striped data storage scheme.

31. The method of claim 19, wherein the portion of the one of the data files is stored according to a parity-striping data scheme and the remaining portion is stored according to a non-parity-striping data scheme.

32. The method of claim 19, wherein the plurality of storage devices do not have identical available space.

33. The method of claim 19, wherein the plurality of storage devices have different size capacities.

34. The method of claim 19, wherein the mapping table is mirrored across the plurality of storage devices.

35. A system comprising:

a plurality of storage devices comprising stored data files, the stored data files stored in accordance with a plurality of data storage schemes, wherein a portion of one of the stored data files is stored in first disk sectors on a first storage device according to a first one of the plurality of data schemes, and a remaining portion of the one of the stored data files is stored in second disk sectors on a second storage device according to a second one of the plurality of data schemes;

a mapping table associated with each of the plurality of storage devices, the mapping table storing data representing available locations and amounts of available space in the plurality of storage devices; and

wherein a storage sequence of the first disk sectors on the first storage device does not correspond to a storage sequence of the second disk sectors on the second storage device.

36. The system of claim 35, wherein the first stored data file is stored in first non-contiguous disk sectors on the first storage device.

37. The system of claim 36, wherein the second stored data file is stored in second non-contiguous disk sectors on the second storage device.

38. The system of claim 35, wherein a mirrored data storage scheme is assigned to the first stored data file;

and

the first stored data file is stored on the first storage device in accordance with the mirrored data storage scheme.

39. The system of claim 38, wherein the mirrored data storage scheme is automatically assigned based on at least one of desired performance, fault tolerance, and redundancy for the first stored data file.

40. The system of claim 38, wherein a user is prompted to select and assign the mirrored data storage scheme to the first stored data file.

41. The system of claim 35, wherein the first and second storage devices do not have identical available space.

42. The system of claim 35, wherein the first and second storage devices have different size capacities.

43. The system of claim 35, wherein the portion of the one of the data files is stored according to a mirrored data storage scheme, and wherein the remaining portion is stored according to a non-mirrored data storage scheme.

44. The system of claim 35, wherein the portion of the one of the data files is stored according to first one of the plurality of data storage schemes comprises a striped data storage scheme, and wherein the remaining portion is stored according to a non-striped data storage scheme.

45. The system of claim 35, wherein the first one of the plurality of data storage schemes comprises a parity-striped data storage scheme, and wherein the second one of the plurality of data storage schemes comprises a non-parity-striped data storage scheme.

46. A method comprising:

reading the speed of a network and a respective drive speed of each of a plurality of networked storage devices and local storage devices;

determining whether to store a plurality of data files on one or more of the networked storage devices and/or one or more of the local storage devices based at least in part on the respective drive speed; and

storing the data files on one or more of the plurality of the networked storage devices and/or one or more of the local storage devices in accordance with a plurality of storage schemes.

47. The method of claim 46, wherein at least one of the plurality of data files is stored in accordance with a mirrored data scheme, wherein the one data file is stored on a first local storage device and a mirrored copy of the one data file is stored on a first networked storage device.

48. The method of claim 47, further comprising reading data from the first local storage device when the reading of data from the first networked storage devices is slow.

49. The method of claim 47, wherein the mirrored data storage scheme comprises a RAID-1 storage scheme.

50. The method of claim 47, wherein the first local storage device and the first networked storage device do not have identical available space.

51. The method of claim 47, wherein the first local storage device and the first networked storage device have different size capacities.

52. The method of claim 46, further comprising:

identifying one or more networked storage devices as storage targets based at least in part upon the capability of the storage devices to provide one or more data files to users of the networked storage devices who share the one or more data files.

53. The method of claim 52, further comprising placing a mirrored copy of the one or more data files on at least one user'"'"'s networked storage device based on the user'"'"'s networked storage device being identified as one of the storage targets.

54. The method of claim 46, further comprising storing, in a mapping table associated with each of the networked and local storage devices, data representing available locations and amounts of available space in each of the plurality of networked and local storage devices.