Optimizing access to production data

• US 10,558,480 B2
• Filed: 09/10/2015
• Issued: 02/11/2020
• Est. Priority Date: 09/10/2015
• Status: Active Grant

• Current Assignee: Veritas Technologies, LLC (Whitehouse Group Ltd.)
• Original Assignee: Veritas Technologies, LLC (Whitehouse Group Ltd.)
• Inventors: Dalal, Chirag, Bharadwaj, Vaijayanti Rakshit
• Primary Examiner(s): Alrobaye, Idriss N
• Assistant Examiner(s): Franklin, Richard B
• Application Number:
  US14/850,072
• Publication Number:
  US 20170075708A1
• Time in Patent Office: 1,615 Days
• CPC Class Codes: G06F 16/182 Distributed file systems
• CPC Class Codes: G06F 16/188 Virtual file systems
• CPC Class Codes: G06F 2009/45595 Network integration; Enabling network access in virtual machine instances
• CPC Class Codes: G06F 3/0604 Improving or facilitating administration, e.g. storage management
• CPC Class Codes: G06F 3/065 Replication mechanisms
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 9/45533 Hypervisors; Virtual machine monitors
• CPC Class Codes: G06F 9/45558 Hypervisor-specific management and integration aspects

Abstract

Various systems, methods, and processes for optimizing access to production data in application development and testing environments are disclosed. If an input/output (I/O) operation is a read operation, a storage location on a virtual storage unit at which the read operation is to be performed is determined. Also determined is whether an earlier write operation was performed at the storage location. If an earlier write operation was performed at the storage location, the read operation is performed on one or more virtual data files. However, if the earlier write operation was not performed at the storage location, the read operation is performed on allocated storage space.

Claims

1. A method comprising:

receiving an input/output operation from a first application, at a storage system, wherein

the input/output operation is directed to a virtual data file stored in an allocated storage space,

the allocated storage space has been allocated within a virtual disk in a virtual storage unit in the storage system,

the storage system comprises one or more storage devices, in which is stored

production data comprising one or more production data files, and

the virtual storage unit,

the one or more production data files are configured for present use by a second application, and

the one or more production data files and virtual storage unit are stored separately from one another in the storage system;

determining whether the input/output operation is a read operation or a write operation;

in response to a determination that the input/output operation is the write operation, performing the write operation on the virtual data file; and

in response to a determination that the input/output operation is the read operation,

determining a storage location on the virtual storage unit at which the read operation is to be performed,

determining whether an earlier write operation was performed at the storage location,

in response to determining that the earlier write operation was performed at the storage location, performing the read operation at a location in the virtual data file corresponding to the storage location, and

in response to determining that the earlier write operation was not performed at the storage location, performing the read operation on the allocated storage space at the storage location, wherein

the performing the read operation on the allocated storage space comprises

serving data from a location in the one or more production data files corresponding to the storage location,

the data is responsive to the read operation of the allocated storage space, and

the serving is facilitated by the allocated storage space being mapped to the production data, at least with regard to the storage location.

2. The method of claim 1, further comprising:

allocating the allocated storage space on the virtual disk, wherein

the one or more production data files are accessible from a physical computing device executing the second application,

the first application is an application being executed in an application development and testing environment,

the second application is an application being executed in a production environment,

the input/output operation is generated by a virtual machine executed by a virtualization server,

the physical computing device and the virtualization server are separate from one another, and

the virtualization server and the storage system are communicatively coupled to one another.

3. The method of claim 2, further comprising:

receiving an allocation map from the virtual machine, wherein

the allocation map comprises information regarding the allocated storage space,

the allocated storage space is allocated by the virtual machine, and

the allocated storage space corresponds, at least in part, to at least one of the one or more production data files;

converting information in the allocation map to one or more offsets associated with the one or more production data files, wherein

the one or more offsets are based on a storage structure of the one or more production data files; and

overlaying the storage structure over the allocation map, wherein

the overlaying facilitates the serving by permitting the storage system to re-map the input/output operation from the allocated storage space to the one or more production data files.

4. The method of claim 3, wherein

the overlaying further facilitates the serving by permitting the virtual machine to access the one or more production data files by directing the read operation to the allocated storage space instead of the one or more production data files.

5. The method of claim 2, further comprising:

receiving a request from the virtualization server to create the virtual storage unit; and

based on the receiving, creating the virtual storage unit in the storage system, wherein

the virtual storage unit is used by the virtual machine to allocate the allocated storage space, and

the virtual disk on the virtual storage unit is created as a sparse file.

6. The method of claim 2, further comprising:

importing the virtual disk from the storage system to the virtualization server, wherein

the importing is performed by a client,

the client is a Network File System (NFS) client, and

the virtual disk is imported to a hypervisor executed by the virtualization server; and

exporting the virtual disk from the hypervisor to the virtual machine, wherein

the exporting is performed by the hypervisor.

7. The method of claim 6, wherein

the storage system is part of the production environment, and

the virtual machine and the client are part of the application development and testing environment.

8. The method of claim 2, wherein

the production data is part of one or more live production datasets.

9. The method of claim 2, wherein

the virtual disk in the storage system is a sparse file,

the virtual machine employs a thin-provisioned virtual disk, and

the thin-provisioned virtual disk stores changes to the one or more production data files made by the virtual machine.

10. The method of claim 1, wherein

the data is served from the one of the one or more production data files using a storage structure overlaid over an allocation map associated with the allocated storage space, and

the storage structure being overlaid over the allocation map facilitates serving the data by permitting the storage system to re-map the read operation from the allocated storage space to the one or more production data files.

11. A non-transitory computer readable storage medium comprising program instructions executable to:

receive an input/output operation from a first application, at a storage system, wherein

the input/output operation is directed to a virtual data file stored in an allocated storage space,

the allocated storage space has been allocated within a virtual disk in a virtual storage unit in the storage system,

the storage system comprises one or more storage devices, in which is stored

production data comprising one or more production data files, and

the virtual storage unit,

the one or more production data files are configured for present use by a second application, and

the one or more production data files and virtual storage unit are stored separately from one another in the storage system;

determine whether the input/output operation is a read operation or a write operation;

in response to a determination that the input/output operation is the write operation, perform the write operation on the virtual data file; and

in response to a determination that the input/output operation is the read operation,

determine a storage location on the virtual storage unit at which the read operation is to be performed,

determine whether an earlier write operation was performed at the storage location,

in response to determining that the earlier write operation was performed at the storage location, perform the read operation at a location in the virtual data file corresponding to the storage location, and

in response to determining that the earlier write operation was not performed at the storage location, perform the read operation on the allocated storage space at the storage location, wherein

the program instructions executable to perform the read operation on the allocated storage space comprise further program instructions executable to

serve data from a location in the one or more production data files corresponding to the storage location,

the data is responsive to the read operation of the allocated storage space, and

serving of the data is facilitated by the allocated storage space being mapped to the production data, at least with regard to the storage location.

12. The non-transitory computer readable storage medium of claim 11, further comprising program instructions executable to:

allocate the allocated storage space on the virtual disk, wherein

the one or more production data files are accessible from a physical computing device executing the second application,

the first application is an application being executed in an application development and testing environment,

the second application is an application being executed in a production environment,

the input/output operation is generated by a virtual machine executed by a virtualization server,

the physical computing device and the virtualization server are separate from one another, and

the virtualization server and the storage system are communicatively coupled to one another.

13. The non-transitory computer readable storage medium of claim 12, further comprising program instructions executable to:

receive an allocation map from the virtual machine, wherein

the allocation map comprises information regarding the allocated storage space,

the allocated storage space is allocated by the virtual machine, and

the allocated storage space corresponds, at least in part, to at least one of the one or more production data files;

convert information in the allocation map to one or more offsets associated with the one or more production data files, wherein

the one or more offsets are based on a storage structure of the one or more production data files; and

overlay the storage structure over the allocation map, wherein

the program instructions executable to facilitate the serving of the data by permitting the storage system to re-map the input/output operation from the allocated storage space to the one or more production data files.

14. The non-transitory computer readable storage medium of claim 13, wherein

the production data is part of one or more live production datasets,

the virtual disk in the storage system is a sparse file,

the virtual machine employs a thin-provisioned virtual disk,

the thin-provisioned virtual disk stores changes to the one or more production data files made by the virtual machine, and

the program instructions executable to overlay further facilitate the serving by permitting the virtual machine to access the one or more production data files by directing the read operation to the allocated storage space instead of the one or more production data files.

15. The non-transitory computer readable storage medium of claim 12, further comprising program instructions executable to:

receive a request from the virtualization server to create the virtual storage unit;

based on receipt of the request,

create the virtual storage unit in the storage system, wherein

the virtual storage unit is used by the virtual machine to allocate the allocated storage space, and

the virtual disk on the virtual storage unit is created as a sparse file;

import the virtual disk from the storage system to the virtualization server, wherein

the program instructions executable to import are executed by a client,

the client is a Network File System (NFS) client, and

the virtual disk is imported to a hypervisor executed by the virtualization server; and

export the virtual disk from the hypervisor to the virtual machine, wherein

the exporting is performed by the hypervisor.

16. A system comprising:

one or more processors; and

a non-transitory computer readable storage medium coupled to the one or more processors, wherein the non-transitory computer readable storage medium stores program instructions executable by the one or more processors to

receive an input/output operation from a first application, at a storage system, wherein

the input/output operation is directed to a virtual data file stored in an allocated storage space,

the allocated storage space has been allocated within a virtual disk in a virtual storage unit in the storage system,

the storage system comprises one or more storage devices, in which is stored

production data comprising one or more production data files, and

the virtual storage unit,

the one or more production data files are configured for present use by a second application, and

the one or more production data files and virtual storage unit are stored separately from one another in the storage system,

determine whether the input/output operation is a read operation or a write operation,

in response to a determination that the input/output operation is the write operation, perform the write operation on the virtual data file,

in response to a determination that the input/output operation is the read operation,

determine a storage location on the virtual storage unit at which the read operation is to be performed,

determine whether an earlier write operation was performed at the storage location,

in response to determining that the earlier write operation was performed at the storage location, perform the read operation at a location in the virtual data file corresponding to the storage location, and

in response to determining that the earlier write operation was not performed at the storage location, perform the read operation on the allocated storage space at the storage location, wherein

the program instructions executable by the one or more processors to perform the read operation on the allocated storage space comprise program instructions executable by the one or more processors to serve data from a location in the one or more production data files corresponding to the storage location,

the data is responsive to the read operation of the allocated storage space, and

serving of the data is facilitated by the allocated storage space being mapped to the production data, at least with regard to the storage location.

17. The system of claim 16, further comprising program instructions executable by the one or more processors to:

allocate the allocated storage space on the virtual disk, wherein

the one or more production data files are accessible from a physical computing device executing the second application,

the first application is an application being executed in an application development and testing environment,

the second application is an application being executed in a production environment,

the input/output operation is generated by a virtual machine executed by a virtualization server,

the physical computing device and the virtualization server are separate from one another, and

the virtualization server and the storage system are communicatively coupled to one another.

18. The system of claim 17, further comprising program instructions executable by the one or more processors to:

receive an allocation map from the virtual machine, wherein

the allocation map comprises information regarding the allocated storage space,

the allocated storage space is allocated by the virtual machine, and

the allocated storage space corresponds, at least in part, to at least one of the one or more production data files;

convert information in the allocation map to one or more offsets associated with the one or more production data files, wherein

the one or more offsets are based on a storage structure of the one or more production data files; and

overlay the storage structure over the allocation map, wherein

the program instructions executable to facilitate serving of the data by permitting the storage system to re-map the input/output operation from the allocated storage space to the one or more production data files.

19. The system of claim 18, wherein

the program instructions executable by the one or more processors to overlay further facilitate the serving by permitting the virtual machine to access the one or more production data files by directing the read operation to the allocated storage space instead of the production data files.

20. The system of claim 17, further comprising program instructions executable by the one or more processors to:

receive a request from the virtualization server to create the virtual storage unit; and

based on receipt of the request,

create the virtual storage unit in the storage system, wherein

the virtual storage unit is used by the virtual machine to allocate the allocated storage space, and

the virtual disk on the virtual storage unit is created as a sparse file.

21. The system of claim 17, further comprising program instructions executable by the one or more processors to:

import the virtual disk from the storage system to the virtualization server, wherein

the program instructions executable by the one or more processors to import are executed by a client,

the client is a Network File System (NFS) client, and

the virtual disk is imported to a hypervisor executed by the virtualization server; and

export the virtual disk from the hypervisor to the virtual machine, wherein

the exporting is performed by the hypervisor.