Bug 7997

Btw, one more reason why I think this is important: whatever dependency is missing between 2 packages can have serious effects on the consistency of the cache, leaving as valid some packages which instead would be invalidated, if the dependency was correct.

FWIW, we do have scripts/test-dependencies.sh which does several builds to uncover undeclared dependencies - see the comments at the top of the file for details. This can take many hours to run, but is being run regularly by Martin Jansa on some of his autobuilders (not the Yocto Project Autobuilder).

From what I see, it seems that the script is too deterministic.

The key of my proposal is to introduce significant randomness in the execution of the build (number of CPUs, memory footprint, bandwidth, even randomly nuking certain recipes), so that over time they would expose a larger amount of missing dependency that normally are hidden, due to the typical build sequence.

This doesn't imply an increment in the number of test runs, it is an increment of the entropy of the build system, to add more variance to the typical build sequence.

To be honest I have never been a fan of random builds. Randomness means there is a chance that you will never find the issue you're looking for, and you will spend a load of time building things that will duplicate the results of the build testing that we already do.

Within the core metadata I think our dependencies are in a fairly good state thanks to the dependency QA checks we have and regular running of test-dependencies.sh. If we want to solve this in a better way however, I think we should be trying to do it as deterministically as possible. There are a number of potential approaches:

1) Instrument the build process (perhaps at a low level) so we know where it's looking and what it has found. This is effectively what swabber was supposed to do for accesses to the host system only, although the implementation made the tool impractical to use.

2) Use recipe-specific sysroots - then there can never be anything more than the declared dependencies in the sysroot for a recipe's build process to find and use. Covered under bug 1560.

3) Look for configure script options reported by ./configure --help that are different to previous builds so we know when new optional dependency checks are introduced. I prototyped this and it might be interesting for knowing when we should add new PACKAGECONFIG options, but it won't solve the issue for optional dependencies that aren't exposed as configure script options (or custom configure processes, or dependencies that come in outside of the configure script).

(In reply to comment #4)
> To be honest I have never been a fan of random builds. Randomness means
> there is a chance that you will never find the issue you're looking for, and
> you will spend a load of time building things that will duplicate the
> results of the build testing that we already do.

That's not the type of randomness I was proposing.
But it's irrelevant. I described the problem and gave also a proposal.
If you have a better way to solve the problem is solved, I'm totally cool with it.

> Within the core metadata I think our dependencies are in a fairly good state
> thanks to the dependency QA checks we have and regular running of
> test-dependencies.sh. If we want to solve this in a better way however, I
> think we should be trying to do it as deterministically as possible.

The problem is that the build sequence is only partially affected by the explicit dependency.

You do get other pseudo random factors affecting it: number of cores, memory, bandwidth, etc.

So, currently, there is already lots of randomness built in the build process.

> There
> are a number of potential approaches:
> 
> 1) Instrument the build process (perhaps at a low level) so we know where
> it's looking and what it has found. This is effectively what swabber was
> supposed to do for accesses to the host system only, although the
> implementation made the tool impractical to use.
> 
> 2) Use recipe-specific sysroots - then there can never be anything more than
> the declared dependencies in the sysroot for a recipe's build process to
> find and use. Covered under bug 1560.
> 
> 3) Look for configure script options reported by ./configure --help that are
> different to previous builds so we know when new optional dependency checks
> are introduced. I prototyped this and it might be interesting for knowing
> when we should add new PACKAGECONFIG options, but it won't solve the issue
> for optional dependencies that aren't exposed as configure script options
> (or custom configure processes, or dependencies that come in outside of the
> configure script).

The recipe-specific sysroot seems the more bullet-proof approach, as it will not provide what is not explicitly required.
Even if it was a special mode with additional overhead, it would be ok to arrange for periodical special builds.

But, as I said, as long as it works, everything is ok.

test-dependencies is exactly what you're after, because it attempts to build every recipe with a totally clean sysroot, so any undeclared dependencies are definitely not present.

Also, at a personal level, when developing recipes the script "wipe-sysroot" can be used to efficiently clear the sysroot so you can then do a build to verify the build dependencies.

Proposing to close this, between the existing tools (test-dependencies, wipe-sysroot, and developers actually using them for their own layers) and bug 1560 I think we've got this covered.

Closing, as I said earlier we have some tooling to help with most of this.