linux-stable.git/tools/lib/python/kdoc/c_lex.py, branch master

docs: c_lex.py: store logger on its data

2026-03-25T19:36:46+00:00

By having the logger stored there, any code using CTokenizer can
log messages there.

Signed-off-by: Mauro Carvalho Chehab 
Signed-off-by: Jonathan Corbet 
Message-ID: <467979dc18149e4b2a7113c178e0cb07919632f2.1774256269.git.mchehab+huawei@kernel.org>

docs: kdoc: add c_lex to generated documentation

2026-03-22T21:10:40+00:00

Do some fixes at groups() description for it to be parsed by
Sphinx and add it to the documentation.

Signed-off-by: Mauro Carvalho Chehab 
Signed-off-by: Jonathan Corbet 
Message-ID: <799178cf30dd4022fdb1d029ba998a458e037b52.1773823995.git.mchehab+huawei@kernel.org>

docs: c_lex: setup a logger to report tokenizer issues

2026-03-22T21:02:29+00:00

Report file that has issues detected via CMatch and CTokenizer.

This is done by setting up a logger that will be overriden by
kdoc_parser, when used on it.

Signed-off-by: Mauro Carvalho Chehab 
Reviewed-by: Aleksandr Loktionov 
Signed-off-by: Jonathan Corbet 
Message-ID: <903ad83ae176196a50444e66177a4f5bcdef5199.1773770483.git.mchehab+huawei@kernel.org>

docs: c_lex: properly implement a sub() method for CMatch

2026-03-22T21:02:29+00:00

Implement a sub() method to do what it is expected, parsing
backref arguments like \0, \1, \2, ...

Signed-off-by: Mauro Carvalho Chehab 
Signed-off-by: Jonathan Corbet 
Message-ID:

docs: kdoc: create a CMatch to match nested C blocks

2026-03-22T21:02:29+00:00

The NextMatch code is complex, and will become even more complex
if we add there support for arguments.

Now that we have a tokenizer, we can use a better solution,
easier to be understood.

Yet, to improve performance, it is better to make it use a
previously tokenized code, changing its ABI.

So, reimplement NextMatch using the CTokener class. Once it
is done, we can drop NestedMatch.

Signed-off-by: Mauro Carvalho Chehab 
Signed-off-by: Jonathan Corbet 
Message-ID:

docs: add a C tokenizer to be used by kernel-doc

2026-03-22T21:02:29+00:00

Handling C code purely using regular expressions doesn't work well.

Add a C tokenizer to help doing it the right way.

The tokenizer was written using as basis the Python re documentation
tokenizer example from:
    https://docs.python.org/3/library/re.html#writing-a-tokenizer

Signed-off-by: Mauro Carvalho Chehab 
Signed-off-by: Jonathan Corbet 
Message-ID: <39787bb8022e10c65df40c746077f7f66d07ffed.1773770483.git.mchehab+huawei@kernel.org>