linux-stable.git/include/linux/memory-tiers.h, branch v6.9.2

base/node / acpi: Change 'node_hmem_attrs' to 'access_coordinates'

2023-12-22T22:23:13+00:00

Dan Williams suggested changing the struct 'node_hmem_attrs' to
'access_coordinates' [1]. The struct is a container of r/w-latency and
r/w-bandwidth numbers. Moving forward, this container will also be used by
CXL to store the performance characteristics of each link hop in
the PCIE/CXL topology. So, where node_hmem_attrs is just the access
parameters of a memory-node, access_coordinates applies more broadly
to hardware topology characteristics. The observation is that seemed like
an exercise in having the application identify "where" it falls on a
spectrum of bandwidth and latency needs. For the tuple of
read/write-latency and read/write-bandwidth, "coordinates" is not a perfect
fit. Sometimes it is just conveying values in isolation and not a
"location" relative to other performance points, but in the end this data
is used to identify the performance operation point of a given memory-node.
[2]

Link: http://lore.kernel.org/r/64471313421f7_1b66294d5@dwillia2-xfh.jf.intel.com.notmuch/
Link: https://lore.kernel.org/linux-cxl/645e6215ee0de_1e6f2945e@dwillia2-xfh.jf.intel.com.notmuch/
Suggested-by: Dan Williams 
Reviewed-by: Dan Williams 
Reviewed-by: Jonathan Cameron 
Signed-off-by: Dave Jiang 
Acked-by: Greg Kroah-Hartman 
Link: https://lore.kernel.org/r/170319615734.2212653.15319394025985499185.stgit@djiang5-mobl3
Signed-off-by: Dan Williams

dax, kmem: calculate abstract distance with general interface

2023-10-16T22:44:39+00:00

Previously, a fixed abstract distance MEMTIER_DEFAULT_DAX_ADISTANCE is
used for slow memory type in kmem driver.  This limits the usage of kmem
driver, for example, it cannot be used for HBM (high bandwidth memory).

So, we use the general abstract distance calculation mechanism in kmem
drivers to get more accurate abstract distance on systems with proper
support.  The original MEMTIER_DEFAULT_DAX_ADISTANCE is used as fallback
only.

Now, multiple memory types may be managed by kmem.  These memory types are
put into the "kmem_memory_types" list and protected by
kmem_memory_type_lock.

Link: https://lkml.kernel.org/r/20230926060628.265989-5-ying.huang@intel.com
Signed-off-by: "Huang, Ying" 
Tested-by: Bharata B Rao 
Reviewed-by: Dave Jiang 
Reviewed-by: Alistair Popple 
Cc: Aneesh Kumar K.V 
Cc: Wei Xu 
Cc: Dan Williams 
Cc: Dave Hansen 
Cc: Davidlohr Bueso 
Cc: Johannes Weiner 
Cc: Jonathan Cameron 
Cc: Michal Hocko 
Cc: Yang Shi 
Cc: Rafael J Wysocki 
Signed-off-by: Andrew Morton

acpi, hmat: calculate abstract distance with HMAT

2023-10-16T22:44:39+00:00

A memory tiering abstract distance calculation algorithm based on ACPI
HMAT is implemented.  The basic idea is as follows.

The performance attributes of system default DRAM nodes are recorded as
the base line.  Whose abstract distance is MEMTIER_ADISTANCE_DRAM.  Then,
the ratio of the abstract distance of a memory node (target) to
MEMTIER_ADISTANCE_DRAM is scaled based on the ratio of the performance
attributes of the node to that of the default DRAM nodes.

The functions to record the read/write latency/bandwidth of the default
DRAM nodes and calculate abstract distance according to read/write
latency/bandwidth ratio will be used by CXL CDAT (Coherent Device
Attribute Table) and other memory device drivers.  So, they are put in
memory-tiers.c.

Link: https://lkml.kernel.org/r/20230926060628.265989-4-ying.huang@intel.com
Signed-off-by: "Huang, Ying" 
Tested-by: Bharata B Rao 
Reviewed-by: Dave Jiang 
Reviewed-by: Alistair Popple 
Cc: Aneesh Kumar K.V 
Cc: Wei Xu 
Cc: Dan Williams 
Cc: Dave Hansen 
Cc: Davidlohr Bueso 
Cc: Johannes Weiner 
Cc: Jonathan Cameron 
Cc: Michal Hocko 
Cc: Yang Shi 
Cc: Rafael J Wysocki 
Signed-off-by: Andrew Morton

memory tiering: add abstract distance calculation algorithms management

2023-10-16T22:44:38+00:00

Patch series "memory tiering: calculate abstract distance based on ACPI
HMAT", v4.

We have the explicit memory tiers framework to manage systems with
multiple types of memory, e.g., DRAM in DIMM slots and CXL memory devices.
Where, same kind of memory devices will be grouped into memory types,
then put into memory tiers.  To describe the performance of a memory type,
abstract distance is defined.  Which is in direct proportion to the memory
latency and inversely proportional to the memory bandwidth.  To keep the
code as simple as possible, fixed abstract distance is used in dax/kmem to
describe slow memory such as Optane DCPMM.

To support more memory types, in this series, we added the abstract
distance calculation algorithm management mechanism, provided a algorithm
implementation based on ACPI HMAT, and used the general abstract distance
calculation interface in dax/kmem driver.  So, dax/kmem can support HBM
(high bandwidth memory) in addition to the original Optane DCPMM.


This patch (of 4):

The abstract distance may be calculated by various drivers, such as ACPI
HMAT, CXL CDAT, etc.  While it may be used by various code which hot-add
memory node, such as dax/kmem etc.  To decouple the algorithm users and
the providers, the abstract distance calculation algorithms management
mechanism is implemented in this patch.  It provides interface for the
providers to register the implementation, and interface for the users.

Multiple algorithm implementations can cooperate via calculating abstract
distance for different memory nodes.  The preference of algorithm
implementations can be specified via priority (notifier_block.priority).

Link: https://lkml.kernel.org/r/20230926060628.265989-1-ying.huang@intel.com
Link: https://lkml.kernel.org/r/20230926060628.265989-2-ying.huang@intel.com
Signed-off-by: "Huang, Ying" 
Tested-by: Bharata B Rao 
Reviewed-by: Alistair Popple 
Reviewed-by: Dave Jiang 
Cc: Aneesh Kumar K.V 
Cc: Wei Xu 
Cc: Dan Williams 
Cc: Dave Hansen 
Cc: Davidlohr Bueso 
Cc: Johannes Weiner 
Cc: Jonathan Cameron 
Cc: Michal Hocko 
Cc: Yang Shi 
Cc: Rafael J Wysocki 
Signed-off-by: Andrew Morton

acpi,mm: fix typo sibiling -> sibling

2023-10-06T21:44:10+00:00

First found this typo as reviewing memory tier code. Fix it by sed like:
$ sed -i 's/sibiling/sibling/g' $(git grep -l sibiling)

so the acpi one will be corrected as well.

Link: https://lkml.kernel.org/r/20230802092856.819328-1-lizhijian@cn.fujitsu.com
Signed-off-by: Li Zhijian 
Cc: Aneesh Kumar K.V 
Cc: Huang, Ying 
Cc: Len Brown 
Cc: "Rafael J. Wysocki" 
Signed-off-by: Andrew Morton

memory tier: rename destroy_memory_type() to put_memory_type()

2023-08-18T17:12:11+00:00

It appears that destroy_memory_type() isn't a very good name because we
usually will not free the memory_type here.  So rename it to a more
appropriate name i.e.  put_memory_type().

Link: https://lkml.kernel.org/r/20230706063905.543800-1-linmiaohe@huawei.com
Signed-off-by: Miaohe Lin 
Suggested-by: Huang, Ying 
Reviewed-by: "Huang, Ying" 
Reviewed-by: Xiao Yang 
Cc: Dan Williams 
Cc: Dave Jiang 
Cc: Vishal Verma 
Signed-off-by: Andrew Morton

memory: move hotplug memory notifier priority to same file for easy sorting

2022-11-09T01:37:17+00:00

The priority of hotplug memory callback is defined in a different file. 
And there are some callers using numbers directly.  Collect them together
into include/linux/memory.h for easy reading.  This allows us to sort
their priorities more intuitively without additional comments.

Link: https://lkml.kernel.org/r/20220923033347.3935160-9-liushixin2@huawei.com
Signed-off-by: Liu Shixin 
Cc: Christoph Lameter 
Cc: David Hildenbrand 
Cc: Kefeng Wang 
Cc: Waiman Long 
Cc: zefan li 
Signed-off-by: Andrew Morton

mm/demotion: update node_is_toptier to work with memory tiers

2022-09-27T02:46:12+00:00

With memory tier support we can have memory only NUMA nodes in the top
tier from which we want to avoid promotion tracking NUMA faults.  Update
node_is_toptier to work with memory tiers.  All NUMA nodes are by default
top tier nodes.  With lower(slower) memory tiers added we consider all
memory tiers above a memory tier having CPU NUMA nodes as a top memory
tier

[sj@kernel.org: include missed header file, memory-tiers.h]
  Link: https://lkml.kernel.org/r/20220820190720.248704-1-sj@kernel.org
[akpm@linux-foundation.org: mm/memory.c needs linux/memory-tiers.h]
[aneesh.kumar@linux.ibm.com: make toptier_distance inclusive upper bound of toptiers]
  Link: https://lkml.kernel.org/r/20220830081457.118960-1-aneesh.kumar@linux.ibm.com
Link: https://lkml.kernel.org/r/20220818131042.113280-10-aneesh.kumar@linux.ibm.com
Signed-off-by: Aneesh Kumar K.V 
Reviewed-by: "Huang, Ying" 
Acked-by: Wei Xu 
Cc: Alistair Popple 
Cc: Bharata B Rao 
Cc: Dan Williams 
Cc: Dave Hansen 
Cc: Davidlohr Bueso 
Cc: Hesham Almatary 
Cc: Jagdish Gediya 
Cc: Johannes Weiner 
Cc: Jonathan Cameron 
Cc: Michal Hocko 
Cc: Tim Chen 
Cc: Yang Shi 
Cc: SeongJae Park 
Signed-off-by: Andrew Morton

mm/demotion: demote pages according to allocation fallback order

2022-09-27T02:46:12+00:00

Currently, a higher tier node can only be demoted to selected nodes on the
next lower tier as defined by the demotion path.  This strict demotion
order does not work in all use cases (e.g.  some use cases may want to
allow cross-socket demotion to another node in the same demotion tier as a
fallback when the preferred demotion node is out of space).  This demotion
order is also inconsistent with the page allocation fallback order when
all the nodes in a higher tier are out of space: The page allocation can
fall back to any node from any lower tier, whereas the demotion order
doesn't allow that currently.

This patch adds support to get all the allowed demotion targets for a
memory tier.  demote_page_list() function is now modified to utilize this
allowed node mask as the fallback allocation mask.

Link: https://lkml.kernel.org/r/20220818131042.113280-9-aneesh.kumar@linux.ibm.com
Signed-off-by: Jagdish Gediya 
Signed-off-by: Aneesh Kumar K.V 
Reviewed-by: "Huang, Ying" 
Acked-by: Wei Xu 
Cc: Alistair Popple 
Cc: Bharata B Rao 
Cc: Dan Williams 
Cc: Dave Hansen 
Cc: Davidlohr Bueso 
Cc: Hesham Almatary 
Cc: Johannes Weiner 
Cc: Jonathan Cameron 
Cc: Michal Hocko 
Cc: Tim Chen 
Cc: Yang Shi 
Cc: SeongJae Park 
Signed-off-by: Andrew Morton

mm/demotion: drop memtier from memtype

2022-09-27T02:46:12+00:00

Now that we track node-specific memtier in pg_data_t, we can drop memtier
from memtype.

Link: https://lkml.kernel.org/r/20220818131042.113280-8-aneesh.kumar@linux.ibm.com
Signed-off-by: Aneesh Kumar K.V 
Reviewed-by: "Huang, Ying" 
Acked-by: Wei Xu 
Cc: Alistair Popple 
Cc: Bharata B Rao 
Cc: Dan Williams 
Cc: Dave Hansen 
Cc: Davidlohr Bueso 
Cc: Hesham Almatary 
Cc: Jagdish Gediya 
Cc: Johannes Weiner 
Cc: Jonathan Cameron 
Cc: Michal Hocko 
Cc: Tim Chen 
Cc: Yang Shi 
Cc: SeongJae Park 
Signed-off-by: Andrew Morton