linux.git/arch/arm64/include/asm/mpam.h, branch master

arm_mpam: resctrl: Add CDP emulation

2026-03-27T15:30:10+00:00

Intel RDT's CDP feature allows the cache to use a different control value
depending on whether the accesses was for instruction fetch or a data
access. MPAM's equivalent feature is the other way up: the CPU assigns a
different partid label to traffic depending on whether it was instruction
fetch or a data access, which causes the cache to use a different control
value based solely on the partid.

MPAM can emulate CDP, with the side effect that the alternative partid is
seen by all MSC, it can't be enabled per-MSC.

Add the resctrl hooks to turn this on or off. Add the helpers that match a
closid against a task, which need to be aware that the value written to
hardware is not the same as the one resctrl is using.

Update the 'arm64_mpam_global_default' variable the arch code uses during
context switch to know when the per-cpu value should be used instead. Also,
update these per-cpu values and sync the resulting mpam partid/pmg
configuration to hardware.

resctrl can enable CDP for L2 caches, L3 caches or both. When it is enabled
by one and not the other MPAM globally enabled CDP but hides the effect
on the other cache resource. This hiding is possible as CPOR is the only
supported cache control and that uses a resource bitmap; two partids with
the same bitmap act as one.

Awkwardly, the MB controls don't implement CDP and CDP can't be hidden as
the memory bandwidth control is a maximum per partid which can't be
modelled with more partids. If the total maximum is used for both the data
and instruction partids then then the maximum may be exceeded and if it is
split in two then the one using more bandwidth will hit a lower
limit. Hence, hide the MB controls completely if CDP is enabled for any
resource.

Tested-by: Gavin Shan 
Tested-by: Shaopeng Tan 
Tested-by: Peter Newman 
Tested-by: Zeng Heng 
Tested-by: Punit Agrawal 
Tested-by: Jesse Chick 
Cc: Dave Martin 
Cc: Amit Singh Tomar 
Reviewed-by: Zeng Heng 
Reviewed-by: Shaopeng Tan 
Reviewed-by: Jonathan Cameron 
Reviewed-by: Gavin Shan 
Co-developed-by: Ben Horgan 
Signed-off-by: Ben Horgan 
Signed-off-by: James Morse

arm64: mpam: Add helpers to change a task or cpu's MPAM PARTID/PMG values

2026-03-27T15:29:09+00:00

Care must be taken when modifying the PARTID and PMG of a task in any
per-task structure as writing these values may race with the task being
scheduled in, and reading the modified values.

Add helpers to set the task properties, and the CPU default value.  These
use WRITE_ONCE() that pairs with the READ_ONCE() in mpam_get_regval() to
avoid causing torn values.

Tested-by: Gavin Shan 
Tested-by: Shaopeng Tan 
Tested-by: Peter Newman 
Tested-by: Zeng Heng 
Tested-by: Punit Agrawal 
Tested-by: Jesse Chick 
Cc: Dave Martin 
Reviewed-by: Zeng Heng 
Reviewed-by: Shaopeng Tan 
Reviewed-by: Jonathan Cameron 
Reviewed-by: Catalin Marinas 
Reviewed-by: Gavin Shan 
Co-developed-by: Ben Horgan 
Signed-off-by: Ben Horgan 
Signed-off-by: James Morse

arm64: mpam: Initialise and context switch the MPAMSM_EL1 register

2026-03-27T15:29:02+00:00

The MPAMSM_EL1 sets the MPAM labels, PMG and PARTID, for loads and stores
generated by a shared SMCU. Disable the traps so the kernel can use it and
set it to the same configuration as the per-EL cpu MPAM configuration.

If an SMCU is not shared with other cpus then it is implementation
defined whether the configuration from MPAMSM_EL1 is used or that from
the appropriate MPAMy_ELx. As we set the same, PMG_D and PARTID_D,
configuration for MPAM0_EL1, MPAM1_EL1 and MPAMSM_EL1 the resulting
configuration is the same regardless.

The range of valid configurations for the PARTID and PMG in MPAMSM_EL1 is
not currently specified in Arm Architectural Reference Manual but the
architect has confirmed that it is intended to be the same as that for the
cpu configuration in the MPAMy_ELx registers.

Tested-by: Gavin Shan 
Tested-by: Shaopeng Tan 
Tested-by: Peter Newman 
Tested-by: Zeng Heng 
Tested-by: Punit Agrawal 
Tested-by: Jesse Chick 
Reviewed-by: Zeng Heng 
Reviewed-by: Shaopeng Tan 
Reviewed-by: Jonathan Cameron 
Reviewed-by: Gavin Shan 
Reviewed-by: Catalin Marinas 
Reviewed-by: James Morse 
Signed-off-by: Ben Horgan 
Signed-off-by: James Morse

arm64: mpam: Context switch the MPAM registers

2026-03-27T15:27:59+00:00

MPAM allows traffic in the SoC to be labeled by the OS, these labels are
used to apply policy in caches and bandwidth regulators, and to monitor
traffic in the SoC. The label is made up of a PARTID and PMG value. The x86
equivalent calls these CLOSID and RMID, but they don't map precisely.

MPAM has two CPU system registers that is used to hold the PARTID and PMG
values that traffic generated at each exception level will use. These can
be set per-task by the resctrl file system. (resctrl is the defacto
interface for controlling this stuff).

Add a helper to switch this.

struct task_struct's separate CLOSID and RMID fields are insufficient to
implement resctrl using MPAM, as resctrl can change the PARTID (CLOSID) and
PMG (sort of like the RMID) separately. On x86, the rmid is an independent
number, so a race that writes a mismatched closid and rmid into hardware is
benign. On arm64, the pmg bits extend the partid.
(i.e. partid-5 has a pmg-0 that is not the same as partid-6's pmg-0).  In
this case, mismatching the values will 'dirty' a pmg value that resctrl
believes is clean, and is not tracking with its 'limbo' code.

To avoid this, the partid and pmg are always read and written as a
pair. This requires a new u64 field. In struct task_struct there are two
u32, rmid and closid for the x86 case, but as we can't use them here do
something else. Add this new field, mpam_partid_pmg, to struct thread_info
to avoid adding more architecture specific code to struct task_struct.
Always use READ_ONCE()/WRITE_ONCE() when accessing this field.

Resctrl allows a per-cpu 'default' value to be set, this overrides the
values when scheduling a task in the default control-group, which has
PARTID 0. The way 'code data prioritisation' gets emulated means the
register value for the default group needs to be a variable.

The current system register value is kept in a per-cpu variable to avoid
writing to the system register if the value isn't going to change.  Writes
to this register may reset the hardware state for regulating bandwidth.

Finally, there is no reason to context switch these registers unless there
is a driver changing the values in struct task_struct. Hide the whole thing
behind a static key. This also allows the driver to disable MPAM in
response to errors reported by hardware. Move the existing static key to
belong to the arch code, as in the future the MPAM driver may become a
loadable module.

All this should depend on whether there is an MPAM driver, hide it behind
CONFIG_ARM64_MPAM.

Tested-by: Gavin Shan 
Tested-by: Shaopeng Tan 
Tested-by: Peter Newman 
Tested-by: Zeng Heng 
Tested-by: Punit Agrawal 
Tested-by: Jesse Chick 
CC: Amit Singh Tomar 
Reviewed-by: Zeng Heng 
Reviewed-by: Shaopeng Tan 
Reviewed-by: Jonathan Cameron 
Reviewed-by: Gavin Shan 
Reviewed-by: Catalin Marinas 
Co-developed-by: Ben Horgan 
Signed-off-by: Ben Horgan 
Signed-off-by: James Morse