CACHE MAINTENANCE OPERATIONS IN A DATA PROCESSING SYSTEM

• US 20200133865A1
• Filed: 10/29/2018
• Published: 04/30/2020
• Est. Priority Date: 10/29/2018
• Status: Active Grant

• Current Assignee: Arm Limited (SoftBank Group Corp.)
• Original Assignee: Arm Limited (SoftBank Group Corp.)
• Inventors: MANNAVA, Phanindra Kumar, MATHEWSON, Bruce James, JALAL, Jamshed, MEYER, Paul Gilbert
• Granted Patent: US 10,783,080 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 12/0246 in block erasable memory, e.g. flash memory
• CPC Class Codes: G06F 12/0868 Data transfer between cache memory and other subsystems, e.g. storage devices or host systems
• CPC Class Codes: G06F 12/0871 Allocation or management of cache space
• CPC Class Codes: G06F 12/1009 using page tables, e.g. page table structures
• CPC Class Codes: G06F 2212/7202 Allocation control and policies
• CPC Class Codes: G06F 3/061 Improving I/O performance
• CPC Class Codes: G06F 3/0646 Horizontal data movement in storage systems, i.e. moving data in between storage devices or systems
• CPC Class Codes: G06F 3/0649 Lifecycle management
• CPC Class Codes: G06F 3/0679 Non-volatile semiconductor memory device, e.g. flash memory, one time programmable memory [OTP]
• CPC Class Codes: G06F 3/0685 Hybrid storage combining heterogeneous device types, e.g. hierarchical storage, hybrid arrays
• CPC Class Codes: G06F 9/52 Program synchronisation; Mutual exclusion, e.g. by means of semaphores
• CPC Class Codes: G06F 9/522 Barrier synchronisation
• CPC Class Codes: G06F 9/54 Interprogram communication

Abstract

An interconnect system and method of operating the system are disclosed. A master device has access to a cache and a slave device has an associated data storage device for long-term storage of data items. The master device can initiate a cache maintenance operation in the interconnect system with respect to a data item temporarily stored in the cache causing action to be taken by the slave device with respect to storage of the data item in the data storage device. For long latency operations the master device can issue a separated cache maintenance request specifying the data item and the slave device. In response an intermediate device signals an acknowledgment response indicating that it has taken on responsibility for completion of the cache maintenance operation and issues the separated cache maintenance request to the slave device. The slave device signals the acknowledgement response to the intermediate device and on completion of the cache maintenance operation with respect to the data item stored in the data storage device signals a completion response to the master device.

Claims

1. An interconnect system comprising:

a master device;

a cache for temporary storage of data items for low-latency access by the master device;

an intermediate device;

a slave device, wherein the slave device has an associated data storage device for long-term storage of the data items; and

interconnect circuitry connecting the master device, the intermediate device, and the slave device,

wherein the master device is capable of initiating a cache maintenance operation in the interconnect system with respect to a data item temporarily stored in the cache, wherein the cache maintenance operation comprises action taken by the slave device with respect to storage of the data item in the data storage device,

wherein, when the cache maintenance operation is predetermined as a long-latency operation the master device is capable of issuing to the intermediate device a separated cache maintenance request specifying the data item and the slave device;

the intermediate device is responsive to the separated cache maintenance request to signal an acknowledgment response to the master device indicative that the intermediate device has taken on responsibility for completion of the cache maintenance operation with respect to the data item and to issue the separated cache maintenance request to the slave device; and

the slave device is responsive to receipt of the separated cache maintenance request to signal the acknowledgement response to the intermediate device and is responsive to completion of the cache maintenance operation with respect to the data item stored in the data storage device to signal a completion response to the master device.

2. The interconnect system of claim 1, wherein the data storage device for long-term storage of the data items comprises non-volatile storage and the cache maintenance operation is a persistent cache maintenance operation to cause the data item to be written to the non-volatile storage and the persistent cache maintenance operation is predetermined as a long-latency operation.

3. The interconnect system of claim 1, wherein the master device comprises a request buffer to store request details of issued requests, wherein the master device is responsive to receipt of the acknowledgment response from the intermediate device to cease storage of request details of the separated cache maintenance request.

4. The interconnect system of claim 3, wherein the master device comprises a request counter, wherein the request counter is responsive to issuance to the intermediate device of the separated cache maintenance request to increment and wherein the request counter is responsive to receipt of the completion response from the slave device to decrement.

5. The interconnect system of claim 1, wherein the master device is capable of issuing the separated cache maintenance request comprising a group identifier, the intermediate device is capable of signalling the acknowledgment response to the master device further comprising the group identifier, and the slave device is capable of signalling the acknowledgment response comprising the group identifier and of signalling the completion response comprising the group identifier.

6. The interconnect system of claim 5, wherein the master device is capable of specifying the group identifier from a set of group identifiers and the master device comprises a set of request counters corresponding to the set of group identifiers, wherein each request counter of the set of request counters is responsive to issuance to the intermediate device of the separated cache maintenance request specifying a corresponding group identifier from the set of group identifiers to increment and wherein the each request counter of the set of request counters is responsive to receipt of the completion response from the slave device specifying the corresponding group identifier to decrement.

7. The interconnect system of claim 5, wherein the master device is capable of associating the group identifier with a staged data processing operation comprising at least one first stage operation and at least one second stage operation, and wherein the master device is capable of issuing the separated cache maintenance request comprising the group identifier in association with the at least one first stage operation, and the master device is capable of withholding initiation of the at least one second stage operation until receipt of the completion response comprising the group identifier from the slave device.

8. The interconnect system of claim 7, wherein the master device comprises a request counter having an initial value, wherein the request counter is responsive to issuance to the intermediate device of the separated cache maintenance request comprising the group identifier to increment and wherein the request counter is responsive to receipt of the completion response comprising the group identifier from the slave device to decrement, and the master device is capable of withholding initiation of the at least one second stage operation until the request counter returns to the initial value.

9. The interconnect system of claim 7, wherein the staged data processing operation comprises at least one of:

a privilege level transition;

a translation table update;

execution of a barrier instruction; and

execution of a specified set of instructions.

10. The interconnect system of claim 9, wherein the master device is arranged to execute sets of instructions corresponding to distinct physical or virtual threads and the specified set of instructions is associated with a specified thread.

11. The interconnect system of claim 1, wherein the slave device is responsive to completion of the cache maintenance operation with respect to the data item stored in the data storage device to signal the completion response to the master device via the intermediate device.

12. The interconnect system of claim 1, wherein the intermediate device is capable of performing cache coherency operations with respect to the data items, and wherein the intermediate device is responsive to the separated cache maintenance request to perform a cache coherency operation with respect to the data item specified in the separated cache maintenance request before signalling the acknowledgment response to the master device and before issuing the separated cache maintenance request to the slave device.

13. The interconnect system of claim 5, wherein the master device is capable of issuing the separated cache maintenance request comprising an identifier for the data item, an identifier for the slave device, an identifier for the master device and the group identifier,

wherein the slave device is responsive to receipt of the separated cache maintenance request to initiate the cache maintenance operation with respect to the data item stored in the data storage device, to store the identifier for the master device and the group identifier, and to discard the identifier for the data item and the identifier for the slave device,

and the completion response generated by the slave device comprises the identifier for the master device and the group identifier.

14. The interconnect system of claim 1, wherein the cache is a local cache associated with the master device.

15. The interconnect system of claim 1, wherein the cache is associated with at least one of the intermediate device and the interconnect circuitry.

16. The interconnect system of claim 1, wherein the slave device is responsive to receipt of the separated cache maintenance request to evaluate a response condition in dependence on the cache maintenance operation and the slave device is responsive to evaluation of the response condition as true to combine the acknowledgement response and the completion response as a combined response signalled to the intermediate device.

17. The interconnect system of claim 16, wherein the slave device is arranged to evaluate the response condition to be true when an estimated completion latency of the cache maintenance operation is below a latency threshold.

18. The interconnect system of claim 1, further comprising volatile storage associated with the slave device and the persistent cache maintenance operation is arranged to cause the data item to copied from the volatile storage to the non-volatile storage.

19. A method of operating an interconnect system comprising a master device, a cache for temporary storage of data items for low-latency access by the master device, an intermediate device, a slave device, wherein the slave device has an associated data storage device for long-term storage of the data items, and interconnect circuitry connecting the master device, the intermediate device, and the slave device, the method comprising:

initiating from the master device a cache maintenance operation in the interconnect system with respect to a data item temporarily stored in the cache, wherein the cache maintenance operation comprises action taken by the slave device with respect to storage of the data item in the data storage device,

wherein, when the cache maintenance operation is predetermined as a long-latency operation, the initiating comprises issuing from the master device to the intermediate device a separated cache maintenance request specifying the data item and the slave device, and the method further comprises the steps of;

signalling an acknowledgment response to the master device from the intermediate device in response to the separated cache maintenance request indicative that the intermediate device has taken on responsibility for completion of the cache maintenance operation with respect to the data item;

issuing the separated cache maintenance request to the slave device;

signalling the acknowledgement response to the intermediate device from the slave device in response to receipt of the separated cache maintenance request; and

signalling a completion response to the master device from the slave device in response to completion of the cache maintenance operation with respect to the data item stored in the data storage device.

20. An interconnect system comprising:

master device means;

caching means for temporary storage of data items for low-latency access by the master device means;

intermediate device means;

slave device means, wherein the slave device means has associated data storage means for long-term storage of the data items;

interconnect means for connecting the master device means, the intermediate device means, and the slave device means;

means for initiating from the master device means a cache maintenance operation in the interconnect system with respect to a data item temporarily stored in the caching means, wherein the cache maintenance operation comprises action taken by the slave device means with respect to storage of the data item in the data storage means,

wherein, when the cache maintenance operation is predetermined as a long-latency operation, the means for initiating comprises means for issuing from the master device to the intermediate device means a separated cache maintenance request specifying the data item and the slave device means, and the interconnect system further comprises;

means for signalling an acknowledgment response to the master device means from the intermediate device means in response to the separated cache maintenance request indicative that the intermediate device means has taken on responsibility for completion of the cache maintenance operation with respect to the data item;

means for issuing the separated cache maintenance request to the slave device means;

means for signalling the acknowledgement response to the intermediate device means from the slave device means in response to receipt of the separated cache maintenance request; and

means for signalling a completion response to the master device means from the slave device means in response to completion of the cache maintenance operation with respect to the data item stored in the data storage means.