linux-stable.git/kernel/printk/printk_ringbuffer.c, branch v6.8.4

printk: Use prb_first_seq() as base for 32bit seq macros

2024-03-26T22:17:39+00:00

[ Upstream commit 90ad525c2d9a8a6591ab822234a94b82871ef8e0 ]

Note: This change only applies to 32bit architectures. On 64bit
      architectures the macros are NOPs.

Currently prb_next_seq() is used as the base for the 32bit seq
macros __u64seq_to_ulseq() and __ulseq_to_u64seq(). However, in
a follow-up commit, prb_next_seq() will need to make use of the
32bit seq macros.

Use prb_first_seq() as the base for the 32bit seq macros instead
because it is guaranteed to return 64bit sequence numbers without
relying on any 32bit seq macros.

Signed-off-by: John Ogness 
Reviewed-by: Petr Mladek 
Link: https://lore.kernel.org/r/20240207134103.1357162-4-john.ogness@linutronix.de
Signed-off-by: Petr Mladek 
Signed-off-by: Sasha Levin

printk: ringbuffer: Skip non-finalized records in panic

2024-03-26T22:16:40+00:00

[ Upstream commit b1c4c67a5e90db8fbdb5b5504fe16e17b564cca8 ]

Normally a reader will stop once reaching a non-finalized
record. However, when a panic happens, writers from other CPUs
(or an interrupted context on the panic CPU) may have been
writing a record and were unable to finalize it. The panic CPU
will reserve/commit/finalize its panic records, but these will
be located after the non-finalized records. This results in
panic() not flushing the panic messages.

Extend _prb_read_valid() to skip over non-finalized records if
on the panic CPU.

Fixes: 896fbe20b4e2 ("printk: use the lockless ringbuffer")
Signed-off-by: John Ogness 
Reviewed-by: Petr Mladek 
Link: https://lore.kernel.org/r/20240207134103.1357162-11-john.ogness@linutronix.de
Signed-off-by: Petr Mladek 
Signed-off-by: Sasha Levin

printk: ringbuffer: Cleanup reader terminology

2024-03-26T22:16:40+00:00

[ Upstream commit 584528d621459d1a5c31da7a591218ad3bb96d6c ]

With the lockless ringbuffer, it is allowed that multiple
CPUs/contexts write simultaneously into the buffer. This creates
an ambiguity as some writers will finalize sooner.

The documentation for the prb_read functions is not clear as it
refers to "not yet written" and "no data available". Clarify the
return values and language to be in terms of the reader: records
available for reading.

Signed-off-by: John Ogness 
Reviewed-by: Petr Mladek 
Link: https://lore.kernel.org/r/20240207134103.1357162-9-john.ogness@linutronix.de
Signed-off-by: Petr Mladek 
Stable-dep-of: b1c4c67a5e90 ("printk: ringbuffer: Skip non-finalized records in panic")
Signed-off-by: Sasha Levin

printk: Wait for all reserved records with pr_flush()

2024-03-26T22:16:40+00:00

[ Upstream commit ac7d7844c64d15603daa3e905a311ddcfbb4bc91 ]

Currently pr_flush() will only wait for records that were
available to readers at the time of the call (using
prb_next_seq()). But there may be more records (non-finalized)
that have following finalized records. pr_flush() should wait
for these to print as well. Particularly because any trailing
finalized records may be the messages that the calling context
wants to ensure are printed.

Add a new ringbuffer function prb_next_reserve_seq() to return
the sequence number following the most recently reserved record.
This guarantees that pr_flush() will wait until all current
printk() messages (completed or in progress) have been printed.

Fixes: 3b604ca81202 ("printk: add pr_flush()")
Signed-off-by: John Ogness 
Reviewed-by: Petr Mladek 
Link: https://lore.kernel.org/r/20240207134103.1357162-10-john.ogness@linutronix.de
Signed-off-by: Petr Mladek 
Signed-off-by: Sasha Levin

printk: ringbuffer: Do not skip non-finalized records with prb_next_seq()

2024-03-26T22:16:40+00:00

[ Upstream commit 5f72e52ba959e50680b8d83599da1368cd7a6ee2 ]

Commit f244b4dc53e5 ("printk: ringbuffer: Improve
prb_next_seq() performance") introduced an optimization for
prb_next_seq() by using best-effort to track recently finalized
records. However, the order of finalization does not
necessarily match the order of the records. The optimization
changed prb_next_seq() to return inconsistent results, possibly
yielding sequence numbers that are not available to readers
because they are preceded by non-finalized records or they are
not yet visible to the reader CPU.

Rather than simply best-effort tracking recently finalized
records, force the committing writer to read records and
increment the last "contiguous block" of finalized records. In
order to do this, the sequence number instead of ID must be
stored because ID's cannot be directly compared.

A new memory barrier pair is introduced to guarantee that a
reader can always read the records up until the sequence number
returned by prb_next_seq() (unless the records have since
been overwritten in the ringbuffer).

This restores the original functionality of prb_next_seq()
while also keeping the optimization.

For 32bit systems, only the lower 32 bits of the sequence
number are stored. When reading the value, it is expanded to
the full 64bit sequence number using the 32bit seq macros,
which fold in the value returned by prb_first_seq().

Fixes: f244b4dc53e5 ("printk: ringbuffer: Improve prb_next_seq() performance")
Signed-off-by: John Ogness 
Reviewed-by: Petr Mladek 
Link: https://lore.kernel.org/r/20240207134103.1357162-5-john.ogness@linutronix.de
Signed-off-by: Petr Mladek 
Signed-off-by: Sasha Levin

printk: ringbuffer: Fix truncating buffer size min_t cast

2023-08-14T11:05:22+00:00

If an output buffer size exceeded U16_MAX, the min_t(u16, ...) cast in
copy_data() was causing writes to truncate. This manifested as output
bytes being skipped, seen as %NUL bytes in pstore dumps when the available
record size was larger than 65536. Fix the cast to no longer truncate
the calculation.

Cc: Petr Mladek 
Cc: Sergey Senozhatsky 
Cc: Steven Rostedt 
Cc: John Ogness 
Reported-by: Vijay Balakrishna 
Link: https://lore.kernel.org/lkml/d8bb1ec7-a4c5-43a2-9de0-9643a70b899f@linux.microsoft.com/
Fixes: b6cf8b3f3312 ("printk: add lockless ringbuffer")
Cc: stable@vger.kernel.org
Signed-off-by: Kees Cook 
Tested-by: Vijay Balakrishna 
Tested-by: Guilherme G. Piccoli  # Steam Deck
Reviewed-by: Tyler Hicks (Microsoft) 
Tested-by: Tyler Hicks (Microsoft) 
Reviewed-by: John Ogness 
Reviewed-by: Sergey Senozhatsky 
Reviewed-by: Petr Mladek 
Signed-off-by: Petr Mladek 
Link: https://lore.kernel.org/r/20230811054528.never.165-kees@kernel.org

printk: fix a typo of comment

2022-11-22T11:10:15+00:00

Fix a typo of comment

Signed-off-by: Wang Honghui 
Reviewed-by: John Ogness 
Signed-off-by: Petr Mladek 
Link: https://lore.kernel.org/r/77522C532189E547+Y3yG91g6XALbtdJr@TP-P15V.lan
Link: https://lore.kernel.org/r/0C7C980DB815FAE1+Y3yNXJCqZ3Nzxa5V@TP-P15V.lan

printk: ringbuffer: Improve prb_next_seq() performance

2022-01-26T15:00:32+00:00

prb_next_seq() always iterates from the first known sequence number.
In the worst case, it might loop 8k times for 256kB buffer,
15k times for 512kB buffer, and 64k times for 2MB buffer.

It was reported that polling and reading using syslog interface
might occupy 50% of CPU.

Speedup the search by storing @id of the last finalized descriptor.

The loop is still needed because the @id is stored and read in the best
effort way. An atomic variable is used to keep the @id consistent.
But the stores and reads are not serialized against each other.
The descriptor could get reused in the meantime. The related sequence
number will be used only when it is still valid.

An invalid value should be read _only_ when there is a flood of messages
and the ringbuffer is rapidly reused. The performance is the least
problem in this case.

Reported-by: Chunlei Wang 
Signed-off-by: Mukesh Ojha 
Reviewed-by: John Ogness 
Signed-off-by: Petr Mladek 
Link: https://lore.kernel.org/r/1642770388-17327-1-git-send-email-quic_mojha@quicinc.com
Link: https://lore.kernel.org/lkml/YXlddJxLh77DKfIO@alley/T/#m43062e8b2a17f8dbc8c6ccdb8851fb0dbaabbb14

Merge branch 'printk-rework' into for-linus

2021-01-21T15:06:21+00:00

printk: ringbuffer: fix line counting

2021-01-15T10:30:03+00:00

Counting text lines in a record simply involves counting the number
of newline characters (+1). However, it is searching the full data
block for newline characters, even though the text data can be (and
often is) a subset of that area. Since the extra area in the data
block was never initialized, the result is that extra newlines may
be seen and counted.

Restrict newline searching to the text data length.

Fixes: b6cf8b3f3312 ("printk: add lockless ringbuffer")
Signed-off-by: John Ogness 
Reviewed-by: Petr Mladek 
Acked-by: Sergey Senozhatsky 
Signed-off-by: Petr Mladek 
Link: https://lore.kernel.org/r/20210113144234.6545-1-john.ogness@linutronix.de