linux-stable.git/fs/erofs/decompressor.c, branch linux-5.4.y

erofs: fix lz4 inplace decompression

2024-11-08T15:20:49+00:00

commit 3c12466b6b7bf1e56f9b32c366a3d83d87afb4de upstream.

Currently EROFS can map another compressed buffer for inplace
decompression, that was used to handle the cases that some pages of
compressed data are actually not in-place I/O.

However, like most simple LZ77 algorithms, LZ4 expects the compressed
data is arranged at the end of the decompressed buffer and it
explicitly uses memmove() to handle overlapping:
  __________________________________________________________
 |_ direction of decompression --> ____ |_ compressed data _|

Although EROFS arranges compressed data like this, it typically maps two
individual virtual buffers so the relative order is uncertain.
Previously, it was hardly observed since LZ4 only uses memmove() for
short overlapped literals and x86/arm64 memmove implementations seem to
completely cover it up and they don't have this issue.  Juhyung reported
that EROFS data corruption can be found on a new Intel x86 processor.
After some analysis, it seems that recent x86 processors with the new
FSRM feature expose this issue with "rep movsb".

Let's strictly use the decompressed buffer for lz4 inplace
decompression for now.  Later, as an useful improvement, we could try
to tie up these two buffers together in the correct order.

Reported-and-tested-by: Juhyung Park 
Closes: https://lore.kernel.org/r/CAD14+f2AVKf8Fa2OO1aAUdDNTDsVzzR6ctU_oJSmTyd6zSYR2Q@mail.gmail.com
Fixes: 0ffd71bcc3a0 ("staging: erofs: introduce LZ4 decompression inplace")
Fixes: 598162d05080 ("erofs: support decompress big pcluster for lz4 backend")
Cc: stable  # 5.4+
Tested-by: Yifan Zhao 
Signed-off-by: Gao Xiang 
Link: https://lore.kernel.org/r/20231206045534.3920847-1-hsiangkao@linux.alibaba.com
Signed-off-by: Greg Kroah-Hartman

erofs: avoid consecutive detection for Highmem memory

2022-08-25T09:17:36+00:00

[ Upstream commit 448b5a1548d87c246c3d0c3df8480d3c6eb6c11a ]

Currently, vmap()s are avoided if physical addresses are
consecutive for decompressed buffers.

I observed that is very common for 4KiB pclusters since the
numbers of decompressed pages are almost 2 or 3.

However, such detection doesn't work for Highmem pages on
32-bit machines, let's fix it now.

Reported-by: Liu Jinbao 
Fixes: 7fc45dbc938a ("staging: erofs: introduce generic decompression backend")
Link: https://lore.kernel.org/r/20220708101001.21242-1-hsiangkao@linux.alibaba.com
Signed-off-by: Gao Xiang 
Signed-off-by: Sasha Levin

erofs: fix out-of-bound read for shifted uncompressed block

2020-02-11T12:35:23+00:00

commit 4d2024370d877f9ac8b98694bcff666da6a5d333 upstream.

rq->out[1] should be valid before accessing. Otherwise,
in very rare cases, out-of-bound dirty onstack rq->out[1]
can equal to *in and lead to unintended memmove behavior.

Link: https://lore.kernel.org/r/20200107022546.19432-1-gaoxiang25@huawei.com
Fixes: 7fc45dbc938a ("staging: erofs: introduce generic decompression backend")
Cc:  # 5.3+
Reviewed-by: Chao Yu 
Signed-off-by: Gao Xiang 
Signed-off-by: Greg Kroah-Hartman

erofs: rename errln/infoln/debugln to erofs_{err, info, dbg}

2019-09-05T18:10:09+00:00

Add prefix "erofs_" to these functions and print
sb->s_id as a prefix to erofs_{err, info} so that
the user knows which file system is affected.

Reported-by: Christoph Hellwig 
Signed-off-by: Gao Xiang 
Link: https://lore.kernel.org/r/20190904020912.63925-23-gaoxiang25@huawei.com
Signed-off-by: Greg Kroah-Hartman

erofs: kill use_vmap module parameter

2019-09-05T18:10:09+00:00

As Christoph said [1],
"vm_map_ram is supposed to generally behave better.  So if
it doesn't please report that that to the arch maintainer
and linux-mm so that they can look into the issue.  Having
user make choices of deep down kernel internals is just
a horrible interface.

Please talk to maintainers of other bits of the kernel
if you see issues and / or need enhancements. "

Let's redo the previous conclusion and kill the vmap
approach.

[1] https://lore.kernel.org/r/20190830165533.GA10909@infradead.org/
Reported-by: Christoph Hellwig 
Signed-off-by: Gao Xiang 
Link: https://lore.kernel.org/r/20190904020912.63925-21-gaoxiang25@huawei.com
Signed-off-by: Greg Kroah-Hartman

erofs: add "erofs_" prefix for common and short functions

2019-09-05T18:10:08+00:00

Add erofs_ prefix to free_inode, alloc_inode, ...

Reported-by: Christoph Hellwig 
Signed-off-by: Gao Xiang 
Link: https://lore.kernel.org/r/20190904020912.63925-19-gaoxiang25@huawei.com
Signed-off-by: Greg Kroah-Hartman

erofs: use feature_incompat rather than requirements

2019-09-05T18:10:07+00:00

As Christoph said [1], "This is only cosmetic, why
not stick to feature_compat and feature_incompat?"

In my thought, requirements means "incompatible"
instead of "feature" though.

[1] https://lore.kernel.org/r/20190902125109.GA9826@infradead.org/
Reported-by: Christoph Hellwig 
Signed-off-by: Gao Xiang 
Link: https://lore.kernel.org/r/20190904020912.63925-7-gaoxiang25@huawei.com
Signed-off-by: Greg Kroah-Hartman

erofs: remove all likely/unlikely annotations

2019-08-30T07:02:02+00:00

As Dan Carpenter suggested [1], I have to remove
all erofs likely/unlikely annotations.

[1] https://lore.kernel.org/linux-fsdevel/20190829154346.GK23584@kadam/
Reported-by: Dan Carpenter 
Signed-off-by: Gao Xiang 
Link: https://lore.kernel.org/r/20190829163827.203274-1-gaoxiang25@huawei.com
Signed-off-by: Greg Kroah-Hartman

erofs: move erofs out of staging

2019-08-24T12:20:10+00:00

EROFS filesystem has been merged into linux-staging for a year.

EROFS is designed to be a better solution of saving extra storage
space with guaranteed end-to-end performance for read-only files
with the help of reduced metadata, fixed-sized output compression
and decompression inplace technologies.

In the past year, EROFS was greatly improved by many people as
a staging driver, self-tested, betaed by a large number of our
internal users, successfully applied to almost all in-service
HUAWEI smartphones as the part of EMUI 9.1 and proven to be stable
enough to be moved out of staging.

EROFS is a self-contained filesystem driver. Although there are
still some TODOs to be more generic, we have a dedicated team
actively keeping on working on EROFS in order to make it better
with the evolution of Linux kernel as the other in-kernel filesystems.

As Pavel suggested, it's better to do as one commit since git
can do moves and all histories will be saved in this way.

Let's promote it from staging and enhance it more actively as
a "real" part of kernel for more wider scenarios!

Cc: Greg Kroah-Hartman 
Cc: Alexander Viro 
Cc: Andrew Morton 
Cc: Stephen Rothwell 
Cc: Theodore Ts'o 
Cc: Pavel Machek 
Cc: David Sterba 
Cc: Amir Goldstein 
Cc: Christoph Hellwig 
Cc: Darrick J . Wong 
Cc: Dave Chinner 
Cc: Jaegeuk Kim 
Cc: Jan Kara 
Cc: Richard Weinberger 
Cc: Linus Torvalds 
Cc: Chao Yu 
Cc: Miao Xie 
Cc: Li Guifu 
Cc: Fang Wei 
Signed-off-by: Gao Xiang 
Link: https://lore.kernel.org/r/20190822213659.5501-1-hsiangkao@aol.com
Signed-off-by: Greg Kroah-Hartman