linux.git/include/net/page_pool.h, branch v6.1

net: page_pool: introduce ethtool stats

2022-04-15T09:43:47+00:00

Introduce page_pool APIs to report stats through ethtool and reduce
duplicated code in each driver.

Signed-off-by: Lorenzo Bianconi 
Reviewed-by: Jakub Kicinski 
Reviewed-by: Ilias Apalodimas 
Signed-off-by: David S. Miller

page_pool: Add function to batch and return stats

2022-03-03T09:55:28+00:00

Adds a function page_pool_get_stats which can be used by drivers to obtain
stats for a specified page_pool.

Signed-off-by: Joe Damato 
Acked-by: Jesper Dangaard Brouer 
Reviewed-by: Ilias Apalodimas 
Signed-off-by: David S. Miller

page_pool: Add recycle stats

2022-03-03T09:55:28+00:00

Add per-cpu stats tracking page pool recycling events:
	- cached: recycling placed page in the page pool cache
	- cache_full: page pool cache was full
	- ring: page placed into the ptr ring
	- ring_full: page released from page pool because the ptr ring was full
	- released_refcnt: page released (and not recycled) because refcnt > 1

Signed-off-by: Joe Damato 
Acked-by: Jesper Dangaard Brouer 
Reviewed-by: Ilias Apalodimas 
Signed-off-by: David S. Miller

page_pool: Add allocation stats

2022-03-03T09:55:28+00:00

Add per-pool statistics counters for the allocation path of a page pool.
These stats are incremented in softirq context, so no locking or per-cpu
variables are needed.

This code is disabled by default and a kernel config option is provided for
users who wish to enable them.

The statistics added are:
	- fast: successful fast path allocations
	- slow: slow path order-0 allocations
	- slow_high_order: slow path high order allocations
	- empty: ptr ring is empty, so a slow path allocation was forced.
	- refill: an allocation which triggered a refill of the cache
	- waive: pages obtained from the ptr ring that cannot be added to
	  the cache due to a NUMA mismatch.

Signed-off-by: Joe Damato 
Acked-by: Jesper Dangaard Brouer 
Reviewed-by: Ilias Apalodimas 
Signed-off-by: David S. Miller

page_pool: Refactor page_pool to enable fragmenting after allocation

2022-02-03T11:49:03+00:00

This change is meant to permit a driver to perform "fragmenting" of the
page from within the driver instead of the current model which requires
pre-partitioning the page. The main motivation behind this is to support
use cases where the page will be split up by the driver after DMA instead
of before.

With this change it becomes possible to start using page pool to replace
some of the existing use cases where multiple references were being used
for a single page, but the number needed was unknown as the size could be
dynamic.

For example, with this code it would be possible to do something like
the following to handle allocation:
  page = page_pool_alloc_pages();
  if (!page)
    return NULL;
  page_pool_fragment_page(page, DRIVER_PAGECNT_BIAS_MAX);
  rx_buf->page = page;
  rx_buf->pagecnt_bias = DRIVER_PAGECNT_BIAS_MAX;

Then we would process a received buffer by handling it with:
  rx_buf->pagecnt_bias--;

Once the page has been fully consumed we could then flush the remaining
instances with:
  if (page_pool_defrag_page(page, rx_buf->pagecnt_bias))
    continue;
  page_pool_put_defragged_page(pool, page -1, !!budget);

The general idea is that we want to have the ability to allocate a page
with excess fragment count and then trim off the unneeded fragments.

Signed-off-by: Alexander Duyck 
Reviewed-by: Ilias Apalodimas 
Signed-off-by: David S. Miller

page_pool: Store the XDP mem id

2022-01-06T03:46:32+00:00

Store the XDP mem ID inside the page_pool struct so it can be retrieved
later for use in bpf_prog_run().

Signed-off-by: Toke Høiland-Jørgensen 
Signed-off-by: Alexei Starovoitov 
Acked-by: Jesper Dangaard Brouer 
Link: https://lore.kernel.org/bpf/20220103150812.87914-4-toke@redhat.com

page_pool: Add callback to init pages when they are allocated

2022-01-06T03:46:32+00:00

Add a new callback function to page_pool that, if set, will be called every
time a new page is allocated. This will be used from bpf_test_run() to
initialise the page data with the data provided by userspace when running
XDP programs with redirect turned on.

Signed-off-by: Toke Høiland-Jørgensen 
Signed-off-by: Alexei Starovoitov 
Acked-by: John Fastabend 
Acked-by: Jesper Dangaard Brouer 
Link: https://lore.kernel.org/bpf/20220103150812.87914-3-toke@redhat.com

page_pool: Revert "page_pool: disable dma mapping support..."

2021-11-18T11:29:40+00:00

This reverts commit d00e60ee54b12de945b8493cf18c1ada9e422514.

As reported by Guillaume in [1]:
Enabling LPAE always enables CONFIG_ARCH_DMA_ADDR_T_64BIT
in 32-bit systems, which breaks the bootup proceess when a
ethernet driver is using page pool with PP_FLAG_DMA_MAP flag.
As we were hoping we had no active consumers for such system
when we removed the dma mapping support, and LPAE seems like
a common feature for 32 bits system, so revert it.

1. https://www.spinics.net/lists/netdev/msg779890.html

Fixes: d00e60ee54b1 ("page_pool: disable dma mapping support for 32-bit arch with 64-bit DMA")
Signed-off-by: Yunsheng Lin 
Reported-by: "kernelci.org bot" 
Tested-by: "kernelci.org bot" 
Acked-by: Jesper Dangaard Brouer 
Acked-by: Ilias Apalodimas 
Signed-off-by: David S. Miller

page_pool: disable dma mapping support for 32-bit arch with 64-bit DMA

2021-10-15T09:54:20+00:00

As the 32-bit arch with 64-bit DMA seems to rare those days,
and page pool might carry a lot of code and complexity for
systems that possibly.

So disable dma mapping support for such systems, if drivers
really want to work on such systems, they have to implement
their own DMA-mapping fallback tracking outside page_pool.

Reviewed-by: Ilias Apalodimas 
Signed-off-by: Yunsheng Lin 
Acked-by: Jesper Dangaard Brouer 
Signed-off-by: David S. Miller

page_pool: add frag page recycling support in page pool

2021-08-09T22:49:00+00:00

Currently page pool only support page recycling when there
is only one user of the page, and the split page reusing
implemented in the most driver can not use the page pool as
bing-pong way of reusing requires the multi user support in
page pool.

Those reusing or recycling has below limitations:
1. page from page pool can only be used be one user in order
   for the page recycling to happen.
2. Bing-pong way of reusing in most driver does not support
   multi desc using different part of the same page in order
   to save memory.

So add multi-users support and frag page recycling in page
pool to overcome the above limitation.

Signed-off-by: Yunsheng Lin 
Signed-off-by: Jakub Kicinski