Offloading probabilistic computations in data analytics applications

• US 9,697,034 B2
• Filed: 08/07/2015
• Issued: 07/04/2017
• Est. Priority Date: 08/07/2015
• Status: Active Grant

• Current Assignee: Futurewei Technologies Incorporated (Huawei Investment & Holding Co., Ltd.)
• Original Assignee: Futurewei Technologies Incorporated (Huawei Investment & Holding Co., Ltd.)
• Inventors: Shi, Guangyu, Palani, Gopinath, Chadha, Vineet
• Primary Examiner(s): KESSLER, GREGORY AARON
• Application Number:
  US14/821,320
• Publication Number:
  US 20170039080A1
• Time in Patent Office: 697 Days
• Field of Search: None
• CPC Class Codes: G06F 16/183 Provision of network file services by network file servers, e.g. by using NFS, CIFS network file access protocols H04L67/1097
• CPC Class Codes: G06F 16/188 Virtual file systems
• CPC Class Codes: G06F 2009/45583 Memory management, e.g. access or allocation
• CPC Class Codes: G06F 2009/45595 Network integration; Enabling network access in virtual machine instances
• CPC Class Codes: G06F 2209/509 Offload
• CPC Class Codes: G06F 9/45558 Hypervisor-specific management and integration aspects
• CPC Class Codes: G06F 9/5027 the resource being a machine, e.g. CPUs, Servers, Terminals
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

An approach to offloading probabilistic computations is described. An application server comprising a memory and a processor and coupled to a network-attached storage device configured to create a dedicated process in response to a procedural call to a virtual machine container based on a data request is disclosed. The processor forwards the data request to the network-attached storage device, programs one or more virtual machines to perform a probabilistic computation based on the procedural call, and directs the probabilistic computation to a first virtual machine of the one or more virtual machines. The request for data is transformed into a modified call using a virtualized lookup call.

Claims

1. An apparatus for offloading a probabilistic computation to a storage device, the apparatus comprising:

an application server comprising a memory and a processor and coupled to a network-attached storage device configured to create a dedicated process in response to a procedural call configured to invoke a probabilistic operation to a virtual machine container based on a data request,

wherein the processor forwards the data request to the network-attached storage device, programs one or more virtual machines to perform a probabilistic computation based on the procedural call, and directs the probabilistic computation to a first virtual machine of the one or more virtual machines, and the procedural call is generated using a virtualized lookup call.

2. The apparatus of claim 1, wherein the processor registers a process identifier of a user program to establish a channel of communication between the user program and the network-attached storage device.

3. The apparatus of claim 1, wherein the processor changes a behavior of one or more storage containers of the one or more virtual machines using a template.

4. The apparatus of claim 1, wherein at least one of the virtual machines is configured to execute a probabilistic algorithm.

5. The apparatus of claim 1, wherein at least one of the virtual machines performs a Log Log counting operation.

6. The apparatus of claim 1, wherein at least one of the virtual machines performs a membership test.

7. The apparatus of claim 1, wherein at least one of the virtual machines performs a bloom filter operation.

8. In a computer storage system, a method for offloading a probabilistic computation to a storage device, the method comprising:

registering a process identifier with a processor using a user-level application to establish a channel of communication with the processor;

transforming a request for data into a modified call configured to invoke a probabilistic operation using a virtualized lookup call;

forwarding the modified call to the user-level application using a virtualized NFS procedural call and the channel of communication;

creating an inter-process communication channel between a kernel address space of the processor and a user address space; and

using the inter-process communication channel to forward an NFS call to a virtual machine to perform the probabilistic computation.

9. The method of claim 8, further comprising passing the probabilistic computation to a secured container of the virtual machine.

10. The method of claim 9, wherein the secured container provides a sandbox environment for executing the probabilistic computation.

11. The method of claim 8, further comprising forking an output of the virtual machine to create a new process.

12. The method of claim 11, wherein the new process is executed by a second virtual machine.

13. The method of claim 8, wherein the virtual machine is configured to perform a Log Log counting operation.

14. The method of claim 8, wherein the virtual machine is configured to perform a membership test.

15. The method of claim 8, wherein the virtual machine is configured to perform a bloom filter operation.

16. A computer program product tangibly embodied in a computer-readable storage device and comprising instructions that when executed by a processor perform a method for offloading a probabilistic computation to a storage device, the method comprising:

registering a process identifier with a first processor using a user-level application to establish a channel of communication with the first processor and fork an output of a virtual machine to create a new process;

transforming a request for data into a modified call configured to invoke a probabilistic operation using a virtualized lookup call;

forwarding the modified call to the user-level application using a virtualized NFS procedural call and the channel of communication;

creating an inter-process communication channel between a kernel address space of the first processor and a user address space; and

using the inter-process communication channel to forward an NFS call to the virtual machine to perform the probabilistic computation.

17. The method of claim 16, further comprising passing the probabilistic computation to a secured container of the virtual machine.

18. The method of claim 17, wherein the probabilistic computation is passed to the secured container as a remote procedural call.

19. The method of claim 17, wherein the secured container provides a sandbox environment for executing the probabilistic computation.

20. The method of claim 17, wherein the virtual machine is configured to perform a membership test.