linux.git/Documentation/core-api, branch v5.7

bpf: Restrict bpf_trace_printk()'s %s usage and add %pks, %pus specifier

2020-05-15T15:10:36+00:00

Usage of plain %s conversion specifier in bpf_trace_printk() suffers from the
very same issue as bpf_probe_read{,str}() helpers, that is, it is broken on
archs with overlapping address ranges.

While the helpers have been addressed through work in 6ae08ae3dea2 ("bpf: Add
probe_read_{user, kernel} and probe_read_{user, kernel}_str helpers"), we need
an option for bpf_trace_printk() as well to fix it.

Similarly as with the helpers, force users to make an explicit choice by adding
%pks and %pus specifier to bpf_trace_printk() which will then pick the corresponding
strncpy_from_unsafe*() variant to perform the access under KERNEL_DS or USER_DS.
The %pk* (kernel specifier) and %pu* (user specifier) can later also be extended
for other objects aside strings that are probed and printed under tracing, and
reused out of other facilities like bpf_seq_printf() or BTF based type printing.

Existing behavior of %s for current users is still kept working for archs where it
is not broken and therefore gated through CONFIG_ARCH_HAS_NON_OVERLAPPING_ADDRESS_SPACE.
For archs not having this property we fall-back to pick probing under KERNEL_DS as
a sensible default.

Fixes: 8d3b7dce8622 ("bpf: add support for %s specifier to bpf_trace_printk()")
Reported-by: Linus Torvalds 
Reported-by: Christoph Hellwig 
Signed-off-by: Daniel Borkmann 
Signed-off-by: Alexei Starovoitov 
Cc: Masami Hiramatsu 
Cc: Brendan Gregg 
Link: https://lore.kernel.org/bpf/20200515101118.6508-4-daniel@iogearbox.net

docs: timekeeping: Use correct prototype for deprecated functions

2020-04-15T20:48:26+00:00

Use the correct prototypes for do_gettimeofday(), getnstimeofday() and
getnstimeofday64(). All of these returned void and passed the return
value by reference. This should make the documentation of their
deprecation and replacements easier to search for.

Signed-off-by: Chris Packham 
Acked-by: Arnd Bergmann 
Link: https://lore.kernel.org/r/20200414221222.23996-1-chris.packham@alliedtelesis.co.nz
Signed-off-by: Jonathan Corbet

mm: add pagemap.h to the fine documentation

2020-04-02T16:35:29+00:00

The documentation currently does not include the deathless prose written
to describe functions in pagemap.h because it's not included in any rst
file.  Fix up the mismatches between parameter names and the documentation
and add the file to mm-api.

Signed-off-by: Matthew Wilcox (Oracle) 
Signed-off-by: Andrew Morton 
Reviewed-by: Zi Yan 
Reviewed-by: John Hubbard 
Cc: Jonathan Corbet 
Link: http://lkml.kernel.org/r/20200221220045.24989-1-willy@infradead.org
Signed-off-by: Linus Torvalds

mm: dump_page(): additional diagnostics for huge pinned pages

2020-04-02T16:35:27+00:00

As part of pin_user_pages() and related API calls, pages are "dma-pinned".
For the case of compound pages of order > 1, the per-page accounting of
dma pins is accomplished via the 3rd struct page in the compound page.  In
order to support debugging of any pin_user_pages()- related problems,
enhance dump_page() so as to report the pin count in that case.

Documentation/core-api/pin_user_pages.rst is also updated accordingly.

Signed-off-by: John Hubbard 
Signed-off-by: Andrew Morton 
Acked-by: Kirill A. Shutemov 
Cc: Jan Kara 
Cc: Matthew Wilcox 
Cc: Ira Weiny 
Cc: Jérôme Glisse 
Cc: Al Viro 
Cc: Christoph Hellwig 
Cc: Dan Williams 
Cc: Dave Chinner 
Cc: Jason Gunthorpe 
Cc: Jonathan Corbet 
Cc: Michal Hocko 
Cc: Mike Kravetz 
Cc: Shuah Khan 
Cc: Vlastimil Babka 
Link: http://lkml.kernel.org/r/20200211001536.1027652-13-jhubbard@nvidia.com
Signed-off-by: Linus Torvalds

mm/gup: /proc/vmstat: pin_user_pages (FOLL_PIN) reporting

2020-04-02T16:35:27+00:00

Now that pages are "DMA-pinned" via pin_user_page*(), and unpinned via
unpin_user_pages*(), we need some visibility into whether all of this is
working correctly.

Add two new fields to /proc/vmstat:

    nr_foll_pin_acquired
    nr_foll_pin_released

These are documented in Documentation/core-api/pin_user_pages.rst.  They
represent the number of pages (since boot time) that have been pinned
("nr_foll_pin_acquired") and unpinned ("nr_foll_pin_released"), via
pin_user_pages*() and unpin_user_pages*().

In the absence of long-running DMA or RDMA operations that hold pages
pinned, the above two fields will normally be equal to each other.

Also: update Documentation/core-api/pin_user_pages.rst, to remove an
earlier (now confirmed untrue) claim about a performance problem with
/proc/vmstat.

Also: update Documentation/core-api/pin_user_pages.rst to rename the new
/proc/vmstat entries, to the names listed here.

Signed-off-by: John Hubbard 
Signed-off-by: Andrew Morton 
Reviewed-by: Jan Kara 
Acked-by: Kirill A. Shutemov 
Cc: Ira Weiny 
Cc: Jérôme Glisse 
Cc: "Matthew Wilcox (Oracle)" 
Cc: Al Viro 
Cc: Christoph Hellwig 
Cc: Dan Williams 
Cc: Dave Chinner 
Cc: Jason Gunthorpe 
Cc: Jonathan Corbet 
Cc: Michal Hocko 
Cc: Mike Kravetz 
Cc: Shuah Khan 
Cc: Vlastimil Babka 
Link: http://lkml.kernel.org/r/20200211001536.1027652-9-jhubbard@nvidia.com
Signed-off-by: Linus Torvalds

mm/gup: page->hpage_pinned_refcount: exact pin counts for huge pages

2020-04-02T16:35:27+00:00

For huge pages (and in fact, any compound page), the GUP_PIN_COUNTING_BIAS
scheme tends to overflow too easily, each tail page increments the head
page->_refcount by GUP_PIN_COUNTING_BIAS (1024).  That limits the number
of huge pages that can be pinned.

This patch removes that limitation, by using an exact form of pin counting
for compound pages of order > 1.  The "order > 1" is required because this
approach uses the 3rd struct page in the compound page, and order 1
compound pages only have two pages, so that won't work there.

A new struct page field, hpage_pinned_refcount, has been added, replacing
a padding field in the union (so no new space is used).

This enhancement also has a useful side effect: huge pages and compound
pages (of order > 1) do not suffer from the "potential false positives"
problem that is discussed in the page_dma_pinned() comment block.  That is
because these compound pages have extra space for tracking things, so they
get exact pin counts instead of overloading page->_refcount.

Documentation/core-api/pin_user_pages.rst is updated accordingly.

Suggested-by: Jan Kara 
Signed-off-by: John Hubbard 
Signed-off-by: Andrew Morton 
Reviewed-by: Jan Kara 
Acked-by: Kirill A. Shutemov 
Cc: Ira Weiny 
Cc: Jérôme Glisse 
Cc: "Matthew Wilcox (Oracle)" 
Cc: Al Viro 
Cc: Christoph Hellwig 
Cc: Dan Williams 
Cc: Dave Chinner 
Cc: Jason Gunthorpe 
Cc: Jonathan Corbet 
Cc: Michal Hocko 
Cc: Mike Kravetz 
Cc: Shuah Khan 
Cc: Vlastimil Babka 
Link: http://lkml.kernel.org/r/20200211001536.1027652-8-jhubbard@nvidia.com
Signed-off-by: Linus Torvalds

mm/gup: track FOLL_PIN pages

2020-04-02T16:35:27+00:00

Add tracking of pages that were pinned via FOLL_PIN.  This tracking is
implemented via overloading of page->_refcount: pins are added by adding
GUP_PIN_COUNTING_BIAS (1024) to the refcount.  This provides a fuzzy
indication of pinning, and it can have false positives (and that's OK).
Please see the pre-existing Documentation/core-api/pin_user_pages.rst for
details.

As mentioned in pin_user_pages.rst, callers who effectively set FOLL_PIN
(typically via pin_user_pages*()) are required to ultimately free such
pages via unpin_user_page().

Please also note the limitation, discussed in pin_user_pages.rst under the
"TODO: for 1GB and larger huge pages" section.  (That limitation will be
removed in a following patch.)

The effect of a FOLL_PIN flag is similar to that of FOLL_GET, and may be
thought of as "FOLL_GET for DIO and/or RDMA use".

Pages that have been pinned via FOLL_PIN are identifiable via a new
function call:

   bool page_maybe_dma_pinned(struct page *page);

What to do in response to encountering such a page, is left to later
patchsets. There is discussion about this in [1], [2], [3], and [4].

This also changes a BUG_ON(), to a WARN_ON(), in follow_page_mask().

[1] Some slow progress on get_user_pages() (Apr 2, 2019):
    https://lwn.net/Articles/784574/
[2] DMA and get_user_pages() (LPC: Dec 12, 2018):
    https://lwn.net/Articles/774411/
[3] The trouble with get_user_pages() (Apr 30, 2018):
    https://lwn.net/Articles/753027/
[4] LWN kernel index: get_user_pages():
    https://lwn.net/Kernel/Index/#Memory_management-get_user_pages

[jhubbard@nvidia.com: add kerneldoc]
  Link: http://lkml.kernel.org/r/20200307021157.235726-1-jhubbard@nvidia.com
[imbrenda@linux.ibm.com: if pin fails, we need to unpin, a simple put_page will not be enough]
  Link: http://lkml.kernel.org/r/20200306132537.783769-2-imbrenda@linux.ibm.com
[akpm@linux-foundation.org: fix put_compound_head defined but not used]
Suggested-by: Jan Kara 
Suggested-by: Jérôme Glisse 
Signed-off-by: John Hubbard 
Signed-off-by: Claudio Imbrenda 
Signed-off-by: Andrew Morton 
Reviewed-by: Jan Kara 
Acked-by: Kirill A. Shutemov 
Cc: Ira Weiny 
Cc: "Matthew Wilcox (Oracle)" 
Cc: Al Viro 
Cc: Christoph Hellwig 
Cc: Dan Williams 
Cc: Dave Chinner 
Cc: Jason Gunthorpe 
Cc: Jonathan Corbet 
Cc: Michal Hocko 
Cc: Mike Kravetz 
Cc: Shuah Khan 
Cc: Vlastimil Babka 
Link: http://lkml.kernel.org/r/20200211001536.1027652-7-jhubbard@nvidia.com
Signed-off-by: Linus Torvalds

docs: move core-api/ioctl.rst to driver-api/

2020-03-10T17:21:08+00:00

The ioctl() documentation belongs with the rest of the driver-oriented
info, so move it there.

Signed-off-by: Jonathan Corbet

docs: move gcc-plugins to the kbuild manual

2020-03-10T17:20:46+00:00

Information about GCC plugins is relevant to kernel building, so move this
document to the kbuild manual.

Acked-by: Masahiro Yamada 
Signed-off-by: Jonathan Corbet

docs: Organize core-api/index.rst

2020-03-10T17:20:36+00:00

The core-api manual has become a big, disorganized mess.  Try to bring a
small amount of order to it by organizing the documents into
subcategories.

Signed-off-by: Jonathan Corbet