Hardware virtualized input output memory management unit

• US 10,642,501 B1
• Filed: 01/05/2015
• Issued: 05/05/2020
• Est. Priority Date: 01/10/2014
• Status: Active Grant

• Current Assignee: MIPS Tech, LLC
• Original Assignee: MIPS Tech, LLC
• Inventors: Patel, Sanjay, Rozario, Ranjit J
• Primary Examiner(s): Cygiel, Gary W.
• Application Number:
  US14/589,693
• Time in Patent Office: 1,947 Days
• Field of Search: None
• CPC Class Codes: G06F 12/0811 with multilevel cache hierarchies
• CPC Class Codes: G06F 12/0833 in combination with broadcast means (e.g. for invalidation or updating)
• CPC Class Codes: G06F 12/1009 using page tables, e.g. page table structures
• CPC Class Codes: G06F 12/1027 using associative or pseudo-associative address translation means, e.g. translation look-aside buffer [TLB]
• CPC Class Codes: G06F 12/1081 for peripheral access to main memory, e.g. direct memory access [DMA]
• CPC Class Codes: G06F 2009/45579 I/O management, e.g. providing access to device drivers or storage
• CPC Class Codes: G06F 2009/45583 Memory management, e.g. access or allocation
• CPC Class Codes: G06F 2212/1016 Performance improvement
• CPC Class Codes: G06F 2212/152 Virtualized environment, e.g. logically partitioned system
• CPC Class Codes: G06F 2212/283 Plural cache memories
• CPC Class Codes: G06F 2212/62 Details of cache specific to multiprocessor cache arrangements
• CPC Class Codes: G06F 2212/683 Invalidation
• CPC Class Codes: G06F 2212/684 TLB miss handling
• CPC Class Codes: G06F 3/061 Improving I/O performance
• CPC Class Codes: G06F 3/0613 in relation to throughput
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/064 Management of blocks
• CPC Class Codes: G06F 3/0644 Management of space entities, e.g. partitions, extents, pools
• CPC Class Codes: G06F 3/0659 Command handling arrangements, e.g. command buffers, queues, command scheduling
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 3/068 : Hybrid storage device
• CPC Class Codes: G06F 9/4411 : Configuring for operating with peripheral devices; Loading of device drivers
• CPC Class Codes: G06F 9/45558 : Hypervisor-specific management and integration aspects

Abstract

Aspects relate to Input/Output (IO) Memory Management Units (MMUs) that include hardware structures for implementing virtualization. Some implementations allow guests to setup and maintain device IO tables within memory regions to which those guests have been given permissions by a hypervisor. Some implementations provide hardware page table walking capability within the IOMMU, while other implementations provide static tables. Such static tables may be maintained by a hypervisor on behalf of guests. Some implementations reduce a frequency of interrupts or invocation of hypervisor by allowing transactions to be setup by guests without hypervisor involvement within their assigned device IO regions. Devices may communicate with IOMMU to setup the requested memory transaction, and completion thereof may be signaled to the guest without hypervisor involvement. Various other aspects will be evident from the disclosure.

Claims

1. An apparatus for performing computation, comprising:

a physical processor element capable of being configured to execute a hypervisor that hosts one or more guest Operating Systems (OS) by presenting a respective virtualized machine interface to each hosted guest OS;

a physical memory;

an Input/Output (I/O) device; and

an I/O Memory Management Unit (IOMMU) coupled to the physical processor element, the IOMMU configured to;

receive from the hypervisor a direct mapping between a guest address for a hosted guest OS, and an address in the physical memory,

store the mapping in a Translation Lookaside Buffer (TLB) maintained within the IOMMU,

store a mapping in a device table between a guest identifier for the guest OS and an identifier for the I/O device,

receive from the I/O device a request to access the physical memory, the request specifying an identifier for the I/O device, and responsive to receiving the request, to lookup the specified I/O device identifier in the device table, wherein the device table is modified by a corresponding entry in a device remap table,

if the device table, as modified by the corresponding entry in the device remap table, comprises a matching entry, then to obtain a guest identifier from the matching entry, and use that obtained guest identifier and a device address provided with the request to index the TLB to determine an address in the physical memory that corresponds to the device address, and

initiate fulfillment of the I/O device request.

2. The apparatus for performing computation of claim 1, wherein the IOMMU is configured to initiate fulfillment of the I/O device request by forwarding data relating to the I/O device request to a cache hierarchy.

3. The apparatus for performing computation of claim 1, wherein the IOMMU is configured to initiate fulfillment of the I/O device request by returning the physical address to a controller.

4. The apparatus for performing computation of claim 1, further comprising a Translation Lookaside Buffer (TLB) populated with entries from the IOMMU, the TLB coupled with a Graphics Processing Unit (GPU) configured to use the TLB to map virtual addresses used by the GPU to physical addresses in the physical memory.

5. The apparatus for performing computation of claim 4, wherein the TLB is read only by the GPU.

6. The apparatus for performing computation of claim 5, wherein the GPU is configured to request an update to entries in the TLB.

7. The apparatus for performing computation of claim 6, wherein the GPU is configured to send requests to the hypervisor, which, responsive to determining that the GPU may validly access portions of the physical memory identified in the requests, communicates with the IOMMU to service the requests.

8. The apparatus for performing computation of claim 4, wherein the GPU is configured to map a guest virtual address to a guest physical address, and provide that guest physical address for translation into a physical address in the physical memory.

9. The apparatus for performing computation of claim 1 wherein the IOMMU terminates device access in response to detection of a protocol error.

10. An Input/Output Memory Management Unit (IOMMU), comprising:

a device table storing entries mapping respective guest identifiers to respective I/O devices, each guest identifier identifying a respective GuestOS executing on a processor coupled with the IOMMU, and indicating read and write permissions to be accorded the I/O device;

a device remap table, wherein the device remap table is used to modify corresponding entries in the device table;

a Translation Lookaside Buffer (TLB) storing entries directly mapping device addresses supplied in I/O device requests to physical addresses within a system memory; and

circuitry configured to receive an I/O device request, verify that the I/O device request maps to a valid guest identifier using the device table, as modified by the device remap table, and use the TLB to identify a physical address corresponding to a device address supplied in the received I/O device request.

11. The Input/Output Memory Management Unit (IOMMU) of claim 10, wherein the device address supplied with the I/O device request is a guest physical address.

12. The Input/Output Memory Management Unit (IOMMU) of claim 10, further comprising a command queue for receiving, from a hypervisor, commands to program the TLB with entries.

13. The Input/Output Memory Management Unit (IOMMU) of claim 10, further comprising one or more registers coupled for receiving commands to program the TLB with entries.

14. The Input/Output Memory Management Unit (IOMMU) of claim 10, further comprising a memory and one or more pointers to locations in the memory at which are stored commands to program the TLB with entries.

15. The Input/Output Memory Management Unit (IOMMU) of claim 10, wherein the IOMMU is configured to determine if an I/O device request is associated with a privileged mode of execution, and if so, then to bypass translation of an address specified by that I/O device request and to use the address as a root physical address.

16. The Input/Output Memory Management Unit (IOMMU) of claim 15 wherein the privileged mode of execution is a root mode.

17. The Input/Output Memory Management Unit (IOMMU) of claim 10, wherein entries of the TLB comprise a respective group identifier, and any I/O device specifying a given group identifier can share TLB entries having that group identifier.

18. The Input/Output Memory Management Unit (IOMMU) of claim 17, wherein the IOMMU is configured to receive a command to invalidate all TLB entries corresponding to a specified group identifier and responsively to set all TLB entries matching to that group identifier to invalid.

19. The Input/Output Memory Management Unit (IOMMU) of claim 17, wherein the group identifier is obtained from a device table maintained by the IOMMU, which tracks device-specific permissions to be accorded to particular guest identifiers.

20. The Input/Output Memory Management Unit (IOMMU) of claim 10, wherein one or more entries of the TLB comprise both a device identifier and a group identifier.

21. The Input/Output Memory Management Unit (IOMMU) of claim 20 wherein the device identifier is distinct from the group identifier.

22. The Input/Output Memory Management Unit (IOMMU) of claim 10, further comprising a TLB configuration register comprising a field for setting a page size applicable to entries in the TLB.

23. The Input/Output Memory Management Unit (IOMMU) of claim 10, wherein the TLB supports entries having a fixed page size or a variable page size.

24. The Input/Output Memory Management Unit (IOMMU) of claim 10, wherein the IOMMU is further configured to receive a request from a GuestOS to program an I/O device, obtain a guest identifier from the request, and use the device table to determine whether that guest identifier has authorization to access that I/O device.

25. The Input/Output Memory Management Unit (IOMMU) of claim 10, further comprising an error queue for signaling when the I/O device has insufficient permission to complete the I/O device request, based on the read and write permissions in the device table entry.

26. The Input/Output Memory Management Unit (IOMMU) of claim 10, further comprising a register storing a pointer to a location in a memory at which an error queue is located, said error queue signaling, based on the read and write permissions in the device table entry, when the I/O device has insufficient permission to complete the I/O device request.

27. The Input/Output Memory Management Unit (IOMMU) of claim 10, wherein the IOMMU is configured to receive synchronization commands that bracket one or more other commands, and after completing the one or more other commands, to signal to a process that generated the one or more other commands that those commands are completed.

28. The Input/Output Memory Management Unit (IOMMU) of claim 10, wherein the IOMMU is configured to receive a prefetch command, which indicates that a page table entry for a particular device address to physical address is to be loaded into the TLB.