METHODS, DEVICES, AND COMPUTER PROGRAM PRODUCTS FOR PROCESSING DATA

• US 20200133533A1
• Filed: 05/17/2019
• Published: 04/30/2020
• Est. Priority Date: 10/31/2018
• Status: Active Grant

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Inventors: Zhao, Junping, Hou, Andrew Anzhou
• Granted Patent: US 11,029,866 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 3/0604 Improving or facilitating administration, e.g. storage management
• CPC Class Codes: G06F 3/0611 in relation to response time
• CPC Class Codes: G06F 3/0631 by allocating resources to storage systems
• CPC Class Codes: G06F 3/0653 Monitoring storage devices or systems
• CPC Class Codes: G06F 3/0659 Command handling arrangements, e.g. command buffers, queues, command scheduling
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 3/0673 Single storage device
• CPC Class Codes: G06N 20/00 Machine learning

Abstract

A method comprises: determining whether an event triggering processing of data at a storage device occurs, the data being predetermined to be processed at a computing device associated with the storage device; in response to determining that the event occurs, determining available resources of the storage device; and in response to an amount of the available resources exceeding a first predetermined threshold, causing the storage device to process the data and provide the processed data to the computing device. As such, operations to be performed to data may be adaptively allocated, deployed and adjusted on the storage device and the computing device, so that computing capabilities of both the storage device and the computing device may be fully leveraged, and further the computing efficiency of the entire data processing system may be improved.

Claims

1. A method of processing data, comprising:

determining whether an event triggering processing of data at a storage device occurs, the data being predetermined to be processed at a computing device associated with the storage device;

in response to determining that the event occurs, determining available resources of the storage device; and

in response to an amount of the available resources exceeding a first predetermined threshold, causing the storage device to process the data and provide the processed data to the computing device.

2. The method according to claim 1, wherein determining whether the event triggering processing of data at the storage device occurs comprises at least one of:

in response to determining that the data is detected to be to-be-processed data stored in the storage device, determining that the event occurs; and

in response to determining that a request for processing the data is received from the computing device, determining that the event occurs.

3. The method according to claim 1, wherein determining the available resources of the storage device comprises at least one of:

determining available computing resources of the storage device; and

determining available storage space of the storage device.

4. The method according to claim 1, wherein processing the data comprises:

performing, on the data, at least one of;

shuffling, decoding, re-sizing, and data augmentation.

5. The method according to claim 1, further comprising:

notifying the computing device that the storage device processes the data.

6. The method according to claim 1, wherein processing the data further comprises:

determining the updated available resources of the storage device, the available resources being updated due to processing the data; and

in response to an amount of the updated available resources being below a second predetermined threshold, performing at least one of;

pausing processing of the data; and

reducing operations of processing the data.

7. The method according to claim 6, wherein reducing the operations of processing the data comprises:

notifying the computing device of the reduced operations.

8. The method according to claim 1, wherein processing the data comprises:

determining a negotiation result between the storage device and the computing device regarding processing the data, the negotiation result indicating which operations among operations of processing the data will be performed by the storage device; and

processing the data based on the negotiation result.

9. A computer program product being tangibly stored on a non-transient computer readable medium and comprising machine executable instructions which, when being executed, causing a machine to perform steps of the method of claim 1.

10. A device for processing data, comprising at least one processing unit;

at least one memory coupled to the at least one processing unit and storing instructions to be executed by the at least one processing unit, the instructions, when being executed by the at least one processing unit, causing the device to perform steps of the method of claim 1.

11. A method of processing data, the method comprising:

sending, from a computing device to a storage device associated with the computing device, a request for processing data at the storage device, the data being predetermined to be processed at the computing device; and

receiving the processed data from the storage device, the data being processed by the storage device when available resources of the storage device exceeds a first predetermined threshold.

12. The method according to claim 11, wherein sending the request comprises:

determining available resources of the computing device; and

in response to an amount of the available resources of the computing device being below a third predetermined threshold, sending the request to the storage device.

13. The method according to claim 12, wherein determining the available resources of the computing device comprises at least one of:

determining available computing resources of the computing device; and

determining available storage space of the computing device.

14. The method according to claim 11, further comprising:

determining a negotiation result between the computing device and the storage device regarding processing the data, the negotiation result indicating which operations among operations of processing the data will be performed by the storage device.

15. The method according to claim 11, wherein processing the data comprises:

performing, on the data, at least one of;

shuffling, decoding, re-sizing, and data augmentation.

16. A computer program product being tangibly stored on a non-transient computer readable medium and comprising machine executable instructions which, when being executed, causing a machine to perform steps of the method of claim 11.

17. A device for processing data, comprising:

at least one processing unit;

at least one memory coupled to the at least one processing unit and storing instructions to be executed by the at least one processing unit, the instructions, when being executed by the at least one processing unit, causing the device to perform acts comprising;

sending, from a computing device to a storage device associated with the computing device, a request for processing data at the storage device, the data being predetermined to be processed at the computing device; and

receiving the processed data from the storage device, the data being processed by the storage device when available resources of the storage device exceeds a first predetermined threshold.

18. The device according to claim 17, wherein sending the request comprises:

determining available resources of the computing device; and

in response to an amount of the available resources of the computing device being below a third predetermined threshold, sending the request to the storage device.

19. The device according to claim 18, wherein determining the available resources of the computing device comprises at least one of:

determining available computing resources of the computing device; and

determining available storage space of the computing device.

20. The device according to claim 17, the acts further comprising:

determining a negotiation result between the computing device and the storage device regarding processing the data, the negotiation result indicating which operations among operations of processing the data will be performed by the storage device.