We at Red Hat/Fedora have generally tried to have a per file breakdown of
every config option we set.  This makes it easy for us to add new options
when they are exposed and keep a changelog of why they were set.

A Fedora example is here:
  https://src.fedoraproject.org/cgit/rpms/kernel.git/tree/configs/fedora/generic

Using various merge scripts, we build up a config file and run it through
'make listnewconfig' and 'make oldnoconfig'.   The idea is to print out new
config options that haven't been manually set and use the default until
a patch is posted to set it properly.

To speed things up, it would be nice to make it easier to generate a
patch to post the default setting.  The output of 'make listnewconfig'
has two issues that limit us:

- it doesn't provide the default value
- it doesn't provide the new 'choice' options that get flagged in
  'oldconfig'

This patch extends 'listnewconfig' to address the above two issues.

This allows us to run a script

make listnewconfig | rhconfig-tool -o patches; git send-email patches/

The output of 'make listnewconfig':

CONFIG_NET_EMATCH_IPT
CONFIG_IPVLAN
CONFIG_ICE
CONFIG_NET_VENDOR_NI
CONFIG_IEEE802154_MCR20A
CONFIG_IR_IMON_DECODER
CONFIG_IR_IMON_RAW

The new output of 'make listnewconfig':

CONFIG_KERNEL_XZ=n
CONFIG_KERNEL_LZO=n
CONFIG_NET_EMATCH_IPT=n
CONFIG_IPVLAN=n
CONFIG_ICE=n
CONFIG_NET_VENDOR_NI=y
CONFIG_IEEE802154_MCR20A=n
CONFIG_IR_IMON_DECODER=n
CONFIG_IR_IMON_RAW=n

Signed-off-by: Don Zickus <dzickus@redhat.com>
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

Files generated by if_changed* must be added to 'targets' to include
*.cmd files.  Otherwise, they would be regenerated every time.

The build system automatically adds objects to 'targets' where
appropriate, such as obj-y, extra-y, etc. but does nothing for
intermediate files.  So, each Makefile needs to add them by itself.

There are some common cases where objects are generated by chained
rules.  Lexers and parsers are compiled like follows:

   %.lex.o <- %.lex.c <- %.l
   %.tab.o <- %.tab.c <- %.y

They are common patterns, so it is reasonable to take care of them
in the core Makefile instead of requiring each Makefile to do so.

At this moment, you cannot delete 'target += zconf.lex.c' in the
Kconfig Makefile because zconf.lex.c is included from zconf.tab.c
instead of being compiled separately.  It should be deleted after
Kconfig is more refactored.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Acked-by: Frank Rowand <frowand.list@gmail.com>

Files suffixed by .lex.c, .tab.[ch] are generated lexers, parsers,
respectively.  Clean them up globally from the top Makefile.

Some of the final host programs those lexer/parser are linked into
are necessary for building external modules, but the intermediates
are unneeded.  They can be cleaned away by 'make clean' instead of
'make mrproper'.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Acked-by: Frank Rowand <frowand.list@gmail.com>

These patterns are common to host programs that require lexer and parser.
Move them to the top .gitignore.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Acked-by: Frank Rowand <frowand.list@gmail.com>

Pull Kconfig updates from Masahiro Yamada:

 - improve checkpatch for more precise Kconfig code checking

 - clarify effective selects by grouping reverse dependencies in help

 - do not write out '# CONFIG_FOO is not set' from invisible symbols

 - make oldconfig as silent as it should be

 - rename 'silentoldconfig' to 'syncconfig'

 - add unit-test framework and several test cases

 - warn unmet dependency of tristate symbols

 - make unmet dependency warnings readable, removing false positives

 - improve recursive include detection

 - use yylineno to simplify the line number tracking

 - misc cleanups

* tag 'kconfig-v4.17' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild: (30 commits)
  kconfig: use yylineno option instead of manual lineno increments
  kconfig: detect recursive inclusion earlier
  kconfig: remove duplicated file name and lineno of recursive inclusion
  kconfig: do not include both curses.h and ncurses.h for nconfig
  kconfig: make unmet dependency warnings readable
  kconfig: warn unmet direct dependency of tristate symbols selected by y
  kconfig: tests: test if recursive inclusion is detected
  kconfig: tests: test if recursive dependencies are detected
  kconfig: tests: test randconfig for choice in choice
  kconfig: tests: test defconfig when two choices interact
  kconfig: tests: check visibility of tristate choice values in y choice
  kconfig: tests: check unneeded "is not set" with unmet dependency
  kconfig: tests: test if new symbols in choice are asked
  kconfig: tests: test automatic submenu creation
  kconfig: tests: add basic choice tests
  kconfig: tests: add framework for Kconfig unit testing
  kbuild: add PYTHON2 and PYTHON3 variables
  kconfig: remove redundant streamline_config.pl prerequisite
  kconfig: rename silentoldconfig to syncconfig
  kconfig: invoke oldconfig instead of silentoldconfig from local*config
  ...

Tracking the line number by hand is error-prone since you need to
increment it in every \n matching pattern.

If '%option yylineno' is set, flex defines 'yylineno' to contain the
current line number and automatically updates it each time it reads a
\n character.  This is much more convenient although the lexer does
not initializes yylineno, so you need to set it to 1 each time you
start reading a new file, and restore it you go back to the previous
file.

I tested this with DEBUG_PARSE, and confirmed the same dump message
was produced.

I removed the perf-report option.  Otherwise, I see the following
message:
  %option yylineno entails a performance penalty ONLY on rules that
  can match newline characters

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

Currently, the recursive inclusion is not detected when the offending
file is about to be included; it is detected the offending file is
about to include the *next* file.  This is because the detection loop
does not involve the file being included.

Do this check against the file that is about to be included so that
the recursive inclusion is detected before unneeded parsing happens.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

As in the unit test, the error message for the recursive inclusion
looks like this:

  Kconfig.inc1:4: recursive inclusion detected. Inclusion path:
    current file : 'Kconfig.inc1'
    included from: 'Kconfig.inc3:1'
    included from: 'Kconfig.inc2:3'
    included from: 'Kconfig.inc1:4'

The 'Kconfig.inc1:4' is duplicated in the first and last lines.
Also, the single quotes do not help readability.

Change the message like follows:

  Recursive inclusion detected.
  Inclusion path:
    current file : Kconfig.inc1
    included from: Kconfig.inc3:1
    included from: Kconfig.inc2:3
    included from: Kconfig.inc1:4

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

nconf.h includes <curses.h> and "ncurses.h", but it does not need to
include both.  Generally, it should fall back to curses.h only when
ncurses.h is not found.  But, looks like it has never happened;
these includes have been here for many years since commit 692d97c380c6
("kconfig: new configuration interface (nconfig)"), and nobody has
complained about hard-coding of ncurses.h .  Let's simply drop the
curses.h inclusion.

I replaced "ncurses.h" with <ncurses.h> since it is not a local file.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

Currently, the unmet dependency warnings end up with endlessly long
expressions, most of which are false positives.

Here is test code to demonstrate how it currently works.

[Test Case]

  config DEP1
          def_bool y

  config DEP2
          bool "DEP2"

  config A
          bool "A"
          select E

  config B
          bool "B"
          depends on DEP2
          select E

  config C
          bool "C"
          depends on DEP1 && DEP2
          select E

  config D
          def_bool n
          select E

  config E
          bool
          depends on DEP1 && DEP2

[Result]

  $ make config
  scripts/kconfig/conf  --oldaskconfig Kconfig
  *
  * Linux Kernel Configuration
  *
  DEP2 (DEP2) [N/y/?] (NEW) n
  A (A) [N/y/?] (NEW) y
  warning: (A && B && D) selects E which has unmet direct
  dependencies (DEP1 && DEP2)

Here, I see some points to be improved.

First, '(A || B || D)' would make more sense than '(A && B && D)'.
I am not sure if this is intentional, but expr_simplify_unmet_dep()
turns OR expressions into AND, like follows:

        case E_OR:
                return expr_alloc_and(

Second, we see false positives.  'A' is a real unmet dependency.
'B' is false positive because 'DEP1' is fixed to 'y', and 'B' depends
on 'DEP2'.  'C' was correctly dropped by expr_simplify_unmet_dep().
'D' is also false positive because it has no chance to be enabled.
Current expr_simplify_unmet_dep() cannot avoid those false positives.

After all, I decided to use the same helpers as used for printing
reverse dependencies in the help.

With this commit, unreadable warnings (most of the reported symbols are
false positives) in the real world:

$ make ARCH=score allyesconfig
scripts/kconfig/conf  --allyesconfig Kconfig
warning: (HWSPINLOCK_QCOM && AHCI_MTK && STMMAC_PLATFORM &&
 DWMAC_IPQ806X && DWMAC_LPC18XX && DWMAC_OXNAS && DWMAC_ROCKCHIP &&
 DWMAC_SOCFPGA && DWMAC_STI && TI_CPSW && PINCTRL_GEMINI &&
 PINCTRL_OXNAS && PINCTRL_ROCKCHIP && PINCTRL_DOVE &&
 PINCTRL_ARMADA_37XX && PINCTRL_STM32 && S3C2410_WATCHDOG &&
 VIDEO_OMAP3 && VIDEO_S5P_FIMC && USB_XHCI_MTK && RTC_DRV_AT91SAM9 &&
 LPC18XX_DMAMUX && VIDEO_OMAP4 && COMMON_CLK_GEMINI &&
 COMMON_CLK_ASPEED && COMMON_CLK_NXP && COMMON_CLK_OXNAS &&
 COMMON_CLK_BOSTON && QCOM_ADSP_PIL && QCOM_Q6V5_PIL && QCOM_GSBI &&
 ATMEL_EBI && ST_IRQCHIP && RESET_IMX7 && PHY_HI6220_USB &&
 PHY_RALINK_USB && PHY_ROCKCHIP_PCIE && PHY_DA8XX_USB) selects
 MFD_SYSCON which has unmet direct dependencies (HAS_IOMEM)
warning: (PINCTRL_AT91 && PINCTRL_AT91PIO4 && PINCTRL_OXNAS &&
 PINCTRL_PISTACHIO && PINCTRL_PIC32 && PINCTRL_MESON &&
 PINCTRL_NOMADIK && PINCTRL_MTK && PINCTRL_MT7622 && GPIO_TB10X)
 selects OF_GPIO which has unmet direct dependencies (GPIOLIB && OF &&
 HAS_IOMEM)
warning: (FAULT_INJECTION_STACKTRACE_FILTER && LATENCYTOP && LOCKDEP)
 selects FRAME_POINTER which has unmet direct dependencies
 (DEBUG_KERNEL && (CRIS || M68K || FRV || UML || SUPERH || BLACKFIN ||
 MN10300 || METAG) || ARCH_WANT_FRAME_POINTERS)

will be turned into:

$ make ARCH=score allyesconfig
scripts/kconfig/conf  --allyesconfig Kconfig

WARNING: unmet direct dependencies detected for MFD_SYSCON
  Depends on [n]: HAS_IOMEM [=n]
  Selected by [y]:
  - PINCTRL_STM32 [=y] && PINCTRL [=y] && (ARCH_STM32 ||
 COMPILE_TEST [=y]) && OF [=y]
  - RTC_DRV_AT91SAM9 [=y] && RTC_CLASS [=y] && (ARCH_AT91 ||
 COMPILE_TEST [=y])
  - RESET_IMX7 [=y] && RESET_CONTROLLER [=y]
  - PHY_HI6220_USB [=y] && (ARCH_HISI && ARM64 ||
 COMPILE_TEST [=y])
  - PHY_RALINK_USB [=y] && (RALINK || COMPILE_TEST [=y])
  - PHY_ROCKCHIP_PCIE [=y] && (ARCH_ROCKCHIP && OF [=y] ||
 COMPILE_TEST [=y])

WARNING: unmet direct dependencies detected for OF_GPIO
  Depends on [n]: GPIOLIB [=y] && OF [=y] && HAS_IOMEM [=n]
  Selected by [y]:
  - PINCTRL_MTK [=y] && PINCTRL [=y] && (ARCH_MEDIATEK ||
 COMPILE_TEST [=y]) && OF [=y]
  - PINCTRL_MT7622 [=y] && PINCTRL [=y] && (ARCH_MEDIATEK ||
 COMPILE_TEST [=y]) && OF [=y] && (ARM64 || COMPILE_TEST [=y])

WARNING: unmet direct dependencies detected for FRAME_POINTER
  Depends on [n]: DEBUG_KERNEL [=y] && (CRIS || M68K || FRV || UML ||
 SUPERH || BLACKFIN || MN10300 || METAG) || ARCH_WANT_FRAME_POINTERS [=n]
  Selected by [y]:
  - LATENCYTOP [=y] && DEBUG_KERNEL [=y] && STACKTRACE_SUPPORT [=y] &&
 PROC_FS [=y] && !MIPS && !PPC && !S390 && !MICROBLAZE && !ARM_UNWIND &&
 !ARC && !X86

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Petr Vorel <petr.vorel@gmail.com>