Virtualised interface functions

• US 8,447,904 B2
• Filed: 12/14/2009
• Issued: 05/21/2013
• Est. Priority Date: 12/18/2008
• Status: Active Grant

• Current Assignee: Xilinx, Inc. (Advanced Micro Devices, Inc.)
• Original Assignee: SolarFlare Communications Incorporated (Advanced Micro Devices, Inc.)
• Inventors: Riddoch, David
• Primary Examiner(s): Tsai, Henry
• Assistant Examiner(s): Sun, Michael
• Application Number:
  US12/637,554
• Publication Number:
  US 20100161847A1
• Time in Patent Office: 1,254 Days
• Field of Search: None
• US Class Current: 710/74
• CPC Class Codes: G06F 12/1027 using associative or pseudo-associative address translation means, e.g. translation look-aside buffer [TLB]
• CPC Class Codes: G06F 12/1081 for peripheral access to main memory, e.g. direct memory access [DMA]
• CPC Class Codes: G06F 12/145 the protection being virtual, e.g. for virtual blocks or segments before a translation mechanism
• CPC Class Codes: G06F 12/1491 in a hierarchical protection system, e.g. privilege levels, memory rings
• CPC Class Codes: G06F 13/102 where the programme performs an interfacing function, e.g. device driver G06F13/105 takes precedence; scheduling within device drivers G06F9/52; contention policies within device drivers G06F9/4881
• CPC Class Codes: G06F 2213/0026 PCI express
• CPC Class Codes: G06F 2213/0058 Bus-related hardware virtualisation

Abstract

Roughly described, a data processing system comprises a memory addressable by a range of physical memory addresses; a plurality of non-privileged software domains each having a virtual memory address space; a privileged software domain; a memory management unit operable to perform virtual address translation of a virtual memory address into a physical memory address; and an I/O device supporting virtualised interfaces each associated with a respective non-privileged software domain, the I/O device comprising an operation management unit operable to perform virtual address translation in one or more of the virtual memory address spaces; wherein, for I/O operations requested by a virtualised interface, the I/O device invokes the operation management unit to perform virtual address translation for those I/O operations meeting first criteria and to invoke the memory management unit to perform virtual address translation for those I/O operations which do not meet the first criteria.

Claims

1. A data processing system comprising:

a memory addressable by a range of physical memory addresses;

at least one first software domain having a first privilege level and supporting a trusted I/O management entity;

a plurality of second software domains each having a privilege level which is below the first privilege level and each having a virtual memory address space;

a memory management unit configured to perform virtual address translation of a virtual memory address into a physical memory address; and

an input and/or output device supporting a plurality of virtualised interfaces, each virtualised interface being associated with a respective one of the non-privileged software domains, the input and/or output device further comprising an address translation data store maintained up-to-date by the trusted I/O management entity and at least one operation management unit having access to the address translation data store and being configured to perform virtual address translation in one or more of the virtual memory address spaces;

wherein, for I/O operations requested by a virtualised interface of the input and/or output device, the input and/or output device is configured to invoke the operation management unit to perform virtual address translation for those I/O operations meeting one or more predefined first criteria and to invoke the memory management unit to perform virtual address translation for those I/O operations which do not meet the one or more predefined first criteria.

2. The data processing system as claimed in claim 1, wherein each virtualised interface of the input and/or output device is assigned a unique identifier, and an I/O operation requested by a particular virtualised interface uses the identifier of that interface.

3. The data processing system as claimed in claim 2, wherein the input and/or output device provides a physical interface having a unique identifier.

4. The data processing system as claimed in claim 1, wherein each virtualised interface of the input and/or output device is assigned a unique identifier,

wherein the input and/or output device provides a physical interface having a unique identifier,

and wherein the input and/or output device is configured to use the identifier of its physical interface for those I/O operations which meet the one or more predefined first criteria.

5. The data processing system as claimed in claim 1, wherein the operation management unit is further configured to perform memory protection by preventing I/O operations requested by a particular virtualised interface of the Input and/or output device from accessing areas of the memory which the software domain associated with that virtualised interface is not permitted to access, and the input and/or output device is further configured to invoke the operation management unit to perform memory protection for the said input and/or output operations meeting one or more predefined second criteria.

6. The data processing system as claimed in claim 5, wherein the memory management unit is further configured to perform memory protection by preventing I/O operations requested by a particular virtualised interface of the input and/or output device from accessing areas of the memory which the software domain associated with that virtualised interface is not permitted to access, and the input and/or output device is further configured to invoke the memory management unit to perform memory protection for the said input and/or output operations which do not meet the one or more predefined second criteria.

7. The data processing system as claimed in claim 5, wherein the predefined first and second criteria are the same criteria.

8. The data processing system as claimed in claim 1, wherein the memory management unit is further configured to perform memory protection by preventing I/O operations requested by a particular virtualised interface of the input and/or output device from accessing areas of the memory which the software domain associated with that virtualised interface is not permitted to access,

and wherein the predefined first criteria are chosen such that the operation management unit is configured to perform virtual address translation for all I/O operations requested by at least one virtualised interface of the input and/or output device and the memory management unit is configured to perform memory protection for all those I/O operations.

9. The data processing system as claimed in claim 1, wherein the operation management unit is further configured to perform memory protection by preventing I/O operations requested by a particular virtualised interface of the Input and/or output device from accessing areas of the memory which the software domain associated with that virtualised interface is not permitted to access,

and wherein the predefined first and second criteria are chosen such that the operation management unit is configured to perform memory protection for all I/O operations requested by at least one virtualised interface of the input and/or output device and the memory management unit is configured to perform virtual address translation for all those I/O operations.

10. The data processing system as claimed in claim 5, wherein each virtualised interface of the input and/or output device is assigned a unique identifier, and wherein the input and/or output device further provides a physical interface having a unique identifier, and wherein the input and/or output device is configured to use the identifier of its physical interface for those I/O operations which meet the one or more predefined first criteria and the one or more predefined second criteria.

11. The data processing system as claimed in claim 10, wherein the input and/or output device is configured to use the identifier of the virtualised interface requesting the I/O operation for those I/O operations which do not meet either the one or more predefined first criteria or the one or more predefined second criteria.

12. The data processing system as claimed in claim 5, wherein each virtualised interface of the input and/or output device is assigned a unique identifier,

and wherein the one or more predefined second criteria include the identifier of the virtualised interface such that memory protection for a particular I/O operation is selectively performed at the memory management unit or the operation management unit in dependence on which virtualised interface is requesting the I/O operation.

13. The data processing system as claimed in claim 1, wherein the one or more predefined first criteria include the identifier of the virtualised interface such that virtual address translation for a particular I/O operation is selectively performed at the memory management unit or the operation management unit in dependence on which virtualised interface is requesting the I/O operation.

14. The data processing system as claimed in claim 1, wherein the one or more predefined first criteria includes the type of I/O operation.

15. The data processing system as claimed in claim 5, wherein the one or more predefined second criteria includes the type of I/O operation.

16. The data processing system as claimed claim 1, wherein the input and/or output device is a network interface device.

17. The data processing system as claimed in claim 15, wherein the input and/or output device is a network interface device,

and wherein the I/O operation types include DMA reads, DMA writes, control path operations, data path operations and an interrupts.

18. The data processing system as claimed in claim 16, wherein the one or more predefined first criteria are chosen such that virtual address translation for I/O operations belonging to a control path are performed at the memory management unit and virtual address translation for I/O operations belonging to the data path are performed at the operation management unit.

19. The data processing system as claimed in claim 18, wherein I/O operations belonging to the control path include descriptor fetches, interrupts, configuration operations and notification operations.

20. The data processing system as claimed in claim 16, wherein the one or more predefined first criteria are chosen such that virtual address translation for data transmission operations and interrupts are performed at the memory management unit and virtual address translation for data reception operations, descriptor operations and completion operations are performed at the operation management unit.

21. The data processing system as claimed in claim 1, wherein at least one of the second software domains supports at least one application having a lower privilege level than its supporting domain, each application being granted access to the virtualised interface associated with its supporting domain and to an application memory address space within the address space of the supporting domain.

22. The data processing system as claimed in claim 21, wherein, for I/O operations requested on behalf of an application, the input and/or output device is configured to:

(i) at the operation management unit, perform translation of an application memory address into a virtual memory address of the supporting domain; and

(ii) subsequently, invoke the memory management unit to perform virtual address translation of the virtual memory address into a physical memory address.

23. The data processing system as claimed in claim 21, wherein each application is granted direct access to a sub-interface of the virtualised interface of the supporting domain and each sub-interface is assigned a unique identifier.

24. The data processing system as claimed in claim 22, wherein the one or more predefined first criteria are chosen such that the memory management unit performs virtual address translation of the virtual memory address into a physical memory address.

25. The data processing system as claimed in claim 1, wherein the predefined criteria are stored at the input and/or output device.

26. The data processing system as claimed in claim 25, wherein a device driver supported at the first software domain is configured to maintain the predefined criteria stored at the input and/or output device.

27. The data processing system as claimed in claim 1, wherein the input and/or output device comprises a plurality of the said operation management units, each operation management unit being supported at a respective one of the virtualised interfaces, and, on invoking the operation management unit, the input and/or output device is configured to invoke the instance of the operation management unit supported at the virtualised interface requesting the I/O operation.

28. The data processing system as claimed in claim 1, wherein the first software domain maintains a memory address translation table.

29. The data processing system as claimed in claim 28, wherein the input and/or output device maintains a copy of the translation table.

30. The data processing system as claimed in claim 1, wherein the first software domain is a hypervisor and the second software domains are guest operating systems supported by the hypervisor.

31. The data processing system as claimed in claim 1, wherein the first software domain is an operating system kernel and the second software domains are user level entities supported by the kernel.

32. The data processing system as claimed in claim 1, wherein the input and/or output device is a PCI Express device and the virtualised interfaces of the input and/or output device are provided in accordance with the PCI-IOV specification.

33. The data processing system as claimed in claim 2, wherein each identifier is a PCI Requester ID.

34. The data processing system as claimed in claim 6, wherein the predefined first and second criteria are the same criteria.

35. The data processing system as claimed in claim 5, wherein the predefined criteria are stored at the input and/or output device.

36. The data processing system as claimed in claim 5, wherein the input and/or output device comprises a plurality of the said operation management units, each operation management unit being supported at a respective one of the virtualised interfaces, and, on invoking the operation management unit, the input and/or output device is configured to invoke the instance of the operation management unit supported at the virtualised interface requesting the I/O operation.

37. A method for handling virtual memory addresses at a data processing system comprising a memory and an input and/or output device, the data processing system supporting (a) at least one privileged software domain and (b) a plurality of non-privileged software domains each having a virtual memory address space and each being associated with a respective virtualised interface of the input and/or output device, the method comprising:

receiving at a virtualised interface of the input and/or output device at least one virtual memory address from the respective one of the non-privileged software domains;

the virtualised interface requesting an I/O operation using the at least one virtual memory address;

determining at the input and/or output device whether the I/O operation request meets one or more predefined first criteria;

and;

if the one or more first criteria are met, performing translation of the virtual memory address into a physical memory address at an operation management unit of the input and/or output device, the operation management unit having access to an address translation data store of the input and/or output device which is maintained up-to-date by a trusted I/O management entity of the privileged software domain;

otherwise, invoking a memory management unit of the data processing system to perform translation of the virtual memory address into a physical memory address.

38. The method as claimed in claim 37, further comprising:

determining at the input and/or output device whether the I/O operation request meets one or more predefined second criteria;

and;

if the one or more predefined second criteria are met, the input and/or output device preventing the I/O operation requested by the virtualised interface from accessing areas of the memory which the non-privileged software domain associated with the virtualised interface is not permitted to access;

otherwise, invoking a memory management unit of the data processing system to perform memory protection for the I/O operation.