| Age | Commit message (Collapse) | Author |
|---|
|
Currently, nix-support/propagated-build-inputs is parsed by splitting on
a single space. This means that if this file contains multiple spaces
separating two paths, the build_inputs list will end up containing an
empty string.
Instead, call split() with no arguments, which splits on runs of
whitespace and also ignores whitespace at the beginning and end of the
string, eliminating the need for strip().
|
|
Now that we only visit each path once, a few things can be simplified.
We no longer have to keep a list of different dependency chains leading
to a package, since only one chain will ever be found. Also, the already
visited check also takes care of cycles, so the other cycle check can be
removed.
|
|
The hook performs a depth first search on the graph defined by
propagatedBuildInputs. This traverses all paths through the graph,
except for any cycles. In the worst case with a highly connected graph,
this search can take exponential time. In practice, this means that in
cases with long dependency chains and multiple packages depending on the
same package, the hook can take several hours to run.
Avoid this problem by keeping track of already visited paths and only
visiting each path once. This makes the search complete in linear time.
The visible effect of this change is that, if a conflict is found, only
one dependency chain that leads to the conflicting package is printed,
rather than all the possible dependency chains.
|
|
Albeit counter-intutive, the `propagatedBuildInputs`
mechanism and the corresponding package files in
`nix-support/propagated-build-inputs`
can form a dependency cycle.
This can happen if a package adds itself to this file,
or if multiple outputs of one derivation reference each other.
An example for this is the `patchPpdFilesHook`:
In its mission to collect dependency packages with binaries
that might be required by the dependent package to be created,
it sometimes picks up the dependent package itself.
This indicates that if a file of the dependent package
is used, the package itself should also be installed.
In the case of a multiple output package,
it is also possible that two outputs depend on each other,
creating a dependency cycle.
Since commit 2651ddc7b0788932df9da7416ccd1618d76c11fe,
the `find_packages` function in `catch_conflicts.py`
recursively collects all `propagated-build-inputs` files.
If it encounters a dependency cycle, it must not follow the
cycle to avoid infinite recursion (and a stack overflow).
The commit at hand adds a check so that the function skips over
a package that it already encountered and processed earlier.
This does not loosen the script's checks as the script
still recursively collects all propagated build inputs.
|
|
|
|
|
|
This changes the non-legacy version of pythonCatchConflictsHook
to recursively scan the output of the target derivation as well
as its propagatedBuildInputs for duplicate dependencies.
Previously, we did scan sys.path but did prove problematic as it
produced false positives i.e. when build-time dependencies of
hooks - such as setuptools in pythonCatchConflictsHook itself -
where mistakenly flagged as duplicates; even though the are
not included in the outputs of the target dervation.
As all python runtime-dependencies are currently passed via
propagatedBuildInputs in nixpkgs, scanning that plus
site-packages seems sufficient to catch all conflicts that
matter at runtime and less likely to produce false positives.
The legacyHook in catch_conflicts_py2.py needs to be migrated
as well, if it's still needed.
|
|
By restoring and diverting to the old version.
Previously the newer language features and use of more modern stdlib
imports broke the hook on Python<3.10.
|
|
To escape the pkg_resources API deprecation:
> catch-conflicts.py:1: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
Also remove exceptions for the previus bootstrap packages.
|
|
Python does add the script's directory into "sys.path". For the case of
"catch_conflicts.py" this means "/nix/store" is added to "sys.path". This can
result in very long delays if the store contains a lot of entries.
(moved from master commit 76213d102c1eaa91e733b326d9d312b60103d88a)
|
