Bug 8158

Summary: INCOMPATIBLE_LICENSES has unexpected results with GNU libraries
Product: [Build System, Metadata & Runtime] OE-Core Reporter: Jussi Kukkonen <jku>
Component: deploymentAssignee: Joe Slater <joe.slater>
Status: RESOLVED WORKSFORME QA Contact:
Severity: normal    
Priority: Medium+ CC: bogdanx.a.voiculescu, elizabeth.flanagan, randy.macleod, sgw
Version: unspecified   
Target Milestone: 4.99   
Hardware: x86   
OS: Multiple   
See Also: https://bugzilla.yoctoproject.org/show_bug.cgi?id=8197
Whiteboard:
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: Don't know

Description Jussi Kukkonen 2015-08-13 16:48:49 UTC
This sort of license blacklist
    INCOMPATIBLE_LICENSES = "*GPLv3"
seems to lead to very unexpected results when combined with the current standard licensing scheme for GNU software ("LGPLv3+ | GPLv2+").

As far as I can tell e.g. the "non-gpl3" autobuilder* seems to happily build gmp-6.0.0 which is "LGPLv3 | GPLv2".

I couldn't figure out how the checker works yet but could it be that the license checker goes "oh, it's available as GPLv2 and that's not blacklisted"? Technically that is correct but it also means that now anything that links to gmp, and (by dependency) to nettle or gnutls has to be GPLv2 compatible... I'm going to guess that most users of INCOMPATIBLE_LICENSES were not planning that.

If my logic above is correct then maybe the autobuilder should just explicitly blacklist the combo "GPLany | LGPLv3" if that's possible (and docs should have the same suggestion) since this is the common dangerous combination?

CCing beth: ross ratted you out as a domain expert.


*) https://autobuilder.yoctoproject.org/main/builders/nightly-non-gpl3/builds/437/steps/CheckForGPLv3/logs/stdio
Comment 1 Jussi Kukkonen 2015-09-04 07:34:36 UTC
My test case gmp now has a non-lgpl3 version available in master but the non-gpl3 autobuilder still picks the "LGPLv3+ | GPLv2+" version.
Comment 2 Beth Flanagan 2015-09-16 15:55:30 UTC
Fixed in autobuilder:

http://git.yoctoproject.org/cgit/cgit.cgi/yocto-autobuilder/commit/?id=05e209132bd9ab7fe91cfce6d77eebf4c54ea978

and

http://git.yoctoproject.org/cgit/cgit.cgi/yocto-autobuilder/commit/?id=e7299636b173e1a5930f46d4566abe094dbfa7f2

Verified on yoctodev autobuilder cluster:

[pokybuild@yct51 ~]$ cd yocto-autobuilder/yocto-worker/nightly-non-gpl3/
[pokybuild@yct51 nightly-non-gpl3]$ cd build/
[pokybuild@yct51 build]$ for x in `find ./build/tmp/deploy/licenses -name "license.manifest"`; do cat $x|grep -E "GPLv3|GPL-3"; done
Comment 3 Mihail Stanciu 2015-09-21 09:46:29 UTC
Verified.
Comment 4 Jussi Kukkonen 2015-09-21 10:21:41 UTC
(In reply to comment #3)
> Verified.

Has it been verified that the build really does not include gmp-6.0.0? I haven't seen a "non-gpl3" build yet that didn't... The last one (from 16th) seems to not have the license warning anymore but gmp-6.0.0 is still getting built, which seems worse than before.
Comment 5 Mihail Stanciu 2015-09-21 11:47:06 UTC
(In reply to comment #4)
> (In reply to comment #3)
> > Verified.
> 
> Has it been verified that the build really does not include gmp-6.0.0? I
> haven't seen a "non-gpl3" build yet that didn't... The last one (from 16th)
> seems to not have the license warning anymore but gmp-6.0.0 is still getting
> built, which seems worse than before.

Sorry, should have been more specific. Verified that the patch is in and it does check for GPLv3.
We can't really check for more, as we don't have access to the AB infrastructure.
Comment 6 Mihail Stanciu 2015-09-21 14:17:38 UTC
Reopening.

Tried a local build using a freshly downloaded AB(made sure the patches were included) and gmp is still being built and still being included in the resulting image.

On the image i found libgmp.so.10.2.0 under /usr/lib, which I THINK is the library that should be skipped.

Oddly enough it's not showing up in the package manifest or the license manifest, that's why the "check for gmp" step isn't signaling anything.
Comment 7 Beth Flanagan 2015-09-23 12:15:31 UTC
pidge@buile ~/yocto-autobuilder/yocto-worker/nightly-non-gpl3/build/meta $ more recipes-support/gmp/gmp_6.0.0.bb 
require gmp.inc

LICENSE="GPLv2+ | LGPLv3+"

REVISION="a"

LIC_FILES_CHKSUM = "file://COPYING;md5=d32239bcb673463ab874e80d47fae504 \
                   file://COPYING.LESSERv3;md5=6a6a8e020838b23406c81b19c1d46df6 \
                   file://COPYINGv2;md5=b234ee4d69f5fce4486a80fdaf4a4263 \
"


As gmp is either GPLv2+ or LGPLv3+ this is actually the correct behaviour:

From the package.manifest we see that:

PACKAGE NAME: gmp
PACKAGE VERSION: 6.0.0
RECIPE NAME: gmp
LICENSE: GPLv2+

So, it is picking the correct license here so it should be on the image.
Comment 8 Jussi Kukkonen 2015-09-23 12:38:48 UTC
> As gmp is either GPLv2+ or LGPLv3+ this is actually the correct behaviour:

Like I said in my original report:

| Technically that is correct but it also means that now anything that
| links to gmp, and (by dependency) to nettle or gnutls has to be GPLv2
| compatible... I'm going to guess that most users of
| INCOMPATIBLE_LICENSES were not planning that.

The people who are allergic to *GPL3 in general are not going to be fine with their libraries suddenly going GPL2 (not LGPL, GPL). Technically the build is 100% correct but I think it's currently testing a set of software that no-one is interested in.
Comment 9 Beth Flanagan 2015-09-23 13:09:51 UTC
The problem here is that we're going to have to do quite a bit of interspection during the build to figure out who links what to what. We can't blacklist GPLv2+|LGPLv3. So we'll need to do things that really impact the build.

I'm not sure if there is something here that we can actually do during the build. I'm going to bump this to 2.1 to give us time to consider it.
Comment 10 Beth Flanagan 2016-02-10 17:18:57 UTC
I'm dropping this to a medium and moving this out to 2.2. Part of the issue here is that we're going to have to go through all of the RDEPENDS and check if those are compatible with the license on the other side of the |. I can see this getting a bit messy.
Comment 11 Beth Flanagan 2016-05-11 18:31:46 UTC
I'm wondering if part of this isn't a use case for spdx.bbclass. Tracy, can we look at if this.
Comment 12 Joshua Lock 2017-06-02 11:52:25 UTC
The Intel RefKit team have developed some code to do license checking on generated images[1]. This is used to perform refkit specific license checks[2] but could easily be used to provide a selftest which sets INCOMPATIBLE_LICENSE and checks to see whether the generated image has any (L)GPLv3 components.

1. http://git.yoctoproject.org/clean/cgit.cgi/intel-iot-refkit/tree/meta-refkit-core/lib/licensecheck.py
2. http://git.yoctoproject.org/clean/cgit.cgi/intel-iot-refkit/tree/meta-iotqa/lib/oeqa/selftest/refkit-license-check.py
3. http://git.yoctoproject.org/clean/cgit.cgi/intel-iot-refkit/commit/?id=5620309e2b22835bc43329b380127dcf20ace535
Comment 13 Jussi Kukkonen 2017-07-05 14:11:00 UTC
(In reply to comment #12)
> The Intel RefKit team have developed some code to do license checking on
> generated images[1]. This is used to perform refkit specific license
> checks[2] but could easily be used to provide a selftest which sets
> INCOMPATIBLE_LICENSE and checks to see whether the generated image has any
> (L)GPLv3 components.

I had a quick look here and I'm not sure I follow... I mean, yes it's probably possible to write a test like that but would it help? We already know the non-gpl3 build ends up with libraries with effective "LGPLv3|GPLv2" licenses.

I still think the results of INCOPATIBLE_LICENSES is unexpected but I'm not sure if there's anything we can reasonably do that would really help,  especially now that we've removed the obsolete lgplv2 versions (moved to meta-gplv2).

I'll build a non-gpl3 image and do some tests.
Comment 14 Joshua Lock 2017-07-13 15:55:15 UTC
(In reply to comment #13)
> (In reply to comment #12)
> > The Intel RefKit team have developed some code to do license checking on
> > generated images[1]. This is used to perform refkit specific license
> > checks[2] but could easily be used to provide a selftest which sets
> > INCOMPATIBLE_LICENSE and checks to see whether the generated image has any
> > (L)GPLv3 components.
> 
> I had a quick look here and I'm not sure I follow... I mean, yes it's
> probably possible to write a test like that but would it help? We already
> know the non-gpl3 build ends up with libraries with effective "LGPLv3|GPLv2"
> licenses.

Would it help? I'm not sure, I certainly defer to you here. Especially as you filed this bug.

> I still think the results of INCOPATIBLE_LICENSES is unexpected but I'm not
> sure if there's anything we can reasonably do that would really help, 
> especially now that we've removed the obsolete lgplv2 versions (moved to
> meta-gplv2).

Perhaps some warning laden documentation to point people at? Maybe we could even generate a warning when the option is set that would warn about the generated content requiring further review?

> I'll build a non-gpl3 image and do some tests.

Thanks
Comment 15 Randy MacLeod 2020-10-08 08:31:11 UTC
Joe can you see if this is still a problem?
Comment 16 Randy MacLeod 2022-10-14 18:25:39 UTC
I didn't check it myself but I asked Joe and he said that:
  Nothing with an incompatible license is put on the target.

Also we have have several test for the license rules:

$ git log --oneline -11 meta/lib/oeqa/selftest/cases/incompatible_lic.py
ce08cf4825 lib: Add copyright statements to files without one
a74792b15d selftest/runtime_test/incompatible_lic: Use IMAGE_CLASSES for testimage
d01b1919dc selftest/incompatible_lic: Remove references to AVAILABLE_LICENSES
82f24d2197 license: Rework INCOMPATIBLE_LICENSE wildcard handling
d6449581c9 base/license: Rework INCOMPATIBLE_LICENSE variable handling
ebee9854d7 selftest: add core-image-weston to no-gpl3-no-meta-gpl2 image test

from 2021, 2022.

Re-open with details if there's still a problem.