The following is a new proposed test case to further validate the behavior of the sstate-cache. Step 1: (as root) add a new user: test1 (as root) add a new user: test2 Step 2: (as user test1) * make sure that the test1 directory is only accessible by the test1 user: chmod 0700 ~test1 * clone the Poky repository * setup conf/local.conf * perform core-image-minimal build bitbake core-image-minimal Step 3: (as root) copy the sstate-cache and downloads to a location accessible by multiple users (i.e. a new /test directory) Set the directories and files as only readable, not writeable. chown -R root:root /test/sstate-cache /test/downloads chmod -R ugo+rx /test/sstate-cache /test/downloads chmod -R go-w /test/sstate-cache /test/downloads Step 4: (as user test1) * remove the poky directory, and build directory (this ensures that none of the previous paths exist on the disk...) Step 5: (as user test2) * make sure that the test1 directory is only accessible by the test1 user: chmod 0700 ~test1 * clone the Poky repository * setup conf/local.conf -- add the following (adjust directories as necessary) SSTATE_MIRRORS ?= "\ file://.* file:///test/sstate-cache/" PREMIRRORS ?= "\ bzr://.*/.* file:///test/downloads/ \n \ cvs://.*/.* file:///test/downloads/ \n \ git://.*/.* file:///test/downloads/ \n \ hg://.*/.* file:///test/downloads/ \n \ osc://.*/.* file:///test/downloads/ \n \ p4://.*/.* file:///test/downloads/ \n \ svk://.*/.* file:///test/downloads/ \n \ svn://.*/.* file:///test/downloads/ \n \ ftp://.*/.* file:///test/downloads/ \n \ http://.*/.* file:///test/downloads/ \n \ https://.*/.* file:///test/downloads/ \n" * perform the core-image-minimal build [expected results is this will pass, and be considerably quicker then the original build. It should re-use the sstate-cache for everything.] bitbake core-image-minimal * perform additional builds to verify behavior of the sysroot, i.e. build psplash bitbake psplash (psplash should build and package properly... in existing tests this is where the failure has occurred, indicating a problem with the compiler or sysroot setup.)
Assigned to Darren to verify the test case...
Following Mark's instructions: Host OS: Fedora 15 64bit [test2@f16-64 build]$ bitbake psplash Loading cache: 100% |##############################################| ETA: 00:00:00 Loaded 1105 entries from dependency cache. OE Build Configuration: BB_VERSION = "1.15.1" TARGET_ARCH = "i586" TARGET_OS = "linux" MACHINE = "qemux86" DISTRO = "poky" DISTRO_VERSION = "1.1+snapshot-20120314" TUNE_FEATURES = "m32 i586" TARGET_FPU = "" meta meta-yocto = "master:cb771ef55afc9acb0babdc4237bf5320cdc8ff6b" ... ERROR: Function failed: do_configure (see /home/test2/poky/build/tmp/work/i586-poky-linux/psplash-0.1+git1+e05374aae945bcfc6d962ed0d7b2774b77987e1d-r2/temp/log.do_configure.23906 for further information) ERROR: Logfile of failure stored in: /home/test2/poky/build/tmp/work/i586-poky-linux/psplash-0.1+git1+e05374aae945bcfc6d962ed0d7b2774b77987e1d-r2/temp/log.do_configure.23906 Log data follows: | DEBUG: SITE files ['endian-little', 'bit-32', 'ix86-common', 'common-linux', 'common-glibc', 'i586-linux', 'common'] | automake (GNU automake) 1.11.2 | Copyright (C) 2011 Free Software Foundation, Inc. | License GPLv2+: GNU GPL version 2 or later <http://gnu.org/licenses/gpl-2.0.html> | This is free software: you are free to change and redistribute it. | There is NO WARRANTY, to the extent permitted by law. | | Written by Tom Tromey <tromey@redhat.com> | and Alexandre Duret-Lutz <adl@gnu.org>. | AUTOV is 1.11 | NOTE: Executing autoreconf --verbose --install --force --exclude=autopoint -I/home/test2/poky/build/tmp/sysroots/x86_64-linux/usr/share/aclocal-1.11 -I /home/test2/poky/build/tmp/work/i586-poky-linux/psplash-0.1+git1+e05374aae945bcfc6d962ed0d7b2774b77987e1d-r2/git/aclocal-copy/ | autoreconf: Entering directory `.' | autoreconf: configure.ac: not using Gettext | autoreconf: running: aclocal -I /home/test2/poky/build/tmp/sysroots/x86_64-linux/usr/share/aclocal-1.11 -I /home/test2/poky/build/tmp/work/i586-poky-linux/psplash-0.1+git1+e05374aae945bcfc6d962ed0d7b2774b77987e1d-r2/git/aclocal-copy/ -I /home/test2/poky/build/tmp/sysroots/x86_64-linux/usr/share/aclocal-1.11 -I /home/test2/poky/build/tmp/work/i586-poky-linux/psplash-0.1+git1+e05374aae945bcfc6d962ed0d7b2774b77987e1d-r2/git/aclocal-copy/ --force --warnings=cross | aclocal: unknown warning category `cross' | autoreconf: configure.ac: tracing | autoreconf: configure.ac: not using Libtool | autoreconf: running: /home/test2/poky/build/tmp/sysroots/x86_64-linux/usr/bin/autoconf --include=/home/test2/poky/build/tmp/sysroots/x86_64-linux/usr/share/aclocal-1.11 --include=/home/test2/poky/build/tmp/work/i586-poky-linux/psplash-0.1+git1+e05374aae945bcfc6d962ed0d7b2774b77987e1d-r2/git/aclocal-copy/ --force --warnings=cross | autoreconf: running: /home/test2/poky/build/tmp/sysroots/x86_64-linux/usr/bin/autoheader --include=/home/test2/poky/build/tmp/sysroots/x86_64-linux/usr/share/aclocal-1.11 --include=/home/test2/poky/build/tmp/work/i586-poky-linux/psplash-0.1+git1+e05374aae945bcfc6d962ed0d7b2774b77987e1d-r2/git/aclocal-copy/ --force --warnings=cross | autoreconf: running: automake --foreign --add-missing --copy --force-missing --warnings=cross | automake: unknown warning category `cross' | configure.ac:3: installing `./install-sh' | configure.ac:3: installing `./missing' | Makefile.am: installing `./depcomp' | autoreconf: running: gnu-configize | autoreconf: Leaving directory `.' | NOTE: Running /home/test2/poky/build/tmp/work/i586-poky-linux/psplash-0.1+git1+e05374aae945bcfc6d962ed0d7b2774b77987e1d-r2/git/configure --build=x86_64-linux --host=i586-poky-linux --target=i586-poky-linux --prefix=/usr --exec_prefix=/usr --bindir=/usr/bin --sbindir=/usr/sbin --libexecdir=/usr/libexec --datadir=/usr/share --sysconfdir=/etc --sharedstatedir=/com --localstatedir=/var --libdir=/usr/lib --includedir=/usr/include --oldincludedir=/usr/include --infodir=/usr/share/info --mandir=/usr/share/man --disable-silent-rules --disable-dependency-tracking --with-libtool-sysroot=/home/test2/poky/build/tmp/sysroots/qemux86 | configure: WARNING: unrecognized options: --disable-silent-rules, --with-libtool-sysroot | configure: loading site script /home/test2/poky/meta/site/endian-little | configure: loading site script /home/test2/poky/meta/site/ix86-common | configure: loading site script /home/test2/poky/meta/site/common-linux | configure: loading site script /home/test2/poky/meta/site/common-glibc | configure: loading site script /home/test2/poky/meta/site/common | configure: loading site script /home/test2/poky/build/tmp/sysroots/qemux86/usr/share/i586-poky-linux_config_site.d/eglibc_config | configure: loading site script /home/test2/poky/build/tmp/sysroots/qemux86/usr/share/i586-poky-linux_config_site.d/ncurses_config | checking for a BSD-compatible install... /usr/bin/install -c | checking whether build environment is sane... yes | checking for i586-poky-linux-strip... i586-poky-linux-strip | checking for a thread-safe mkdir -p... /bin/mkdir -p | checking for gawk... gawk | checking whether make sets $(MAKE)... yes | checking whether to enable maintainer-specific portions of Makefiles... no | checkinERROR: Function failed: do_configure (see /home/test2/poky/build/tmp/work/i586-poky-linux/psplash-0.1+git1+e05374aae945bcfc6d962ed0d7b2774b77987e1d-r2/temp/log.do_configure.23906 for further information) | g for style of include used by make... GNU | checking for i586-poky-linux-gcc... ccache i586-poky-linux-gcc -m32 -march=i586 --sysroot=/home/test2/poky/build/tmp/sysroots/qemux86 | checking whether the C compiler works... yes | checking for C compiler default output file name... a.out | checking for suffix of executables... | checking whether we are cross compiling... yes | checking for suffix of object files... configure: error: in `/home/test2/poky/build/tmp/work/i586-poky-linux/psplash-0.1+git1+e05374aae945bcfc6d962ed0d7b2774b77987e1d-r2/git': | configure: error: cannot compute suffix of object files: cannot compile | See `config.log' for more details | ERROR: oe_runconf failed NOTE: package psplash-0.1+git1+e05374aae945bcfc6d962ed0d7b2774b77987e1d-r2: task do_configure: Failed ERROR: Task 6 (/home/test2/poky/meta/recipes-core/psplash/psplash_git.bb, do_configure) failed with exit code '1' NOTE: Tasks Summary: Attempted 790 tasks of which 785 didn't need to be rerun and 1 failed. Summary: 1 task failed: /home/test2/poky/meta/recipes-core/psplash/psplash_git.bb, do_configure Summary: There was 1 ERROR message shown, returning a non-zero exit code. Taking a look at the config.log in the ${WORKDIR}/git I grepped for test1 and found: [test2@f16-64 git]$ grep test1 * config.log:Configured with: /home/test1/poky/build/tmp/work-shared/gcc-4.6.3+svnr184847-r23/gcc-4_6-branch/configure --build=x86_64-linux --host=x86_64-linux --target=i586-poky-linux --prefix=/home/test1/poky/build/tmp/sysroots/x86_64-linux/usr --exec_prefix=/home/test1/poky/build/tmp/sysroots/x86_64-linux/usr --bindir=/home/test /poky/build/tmp/sysroots/x86_64-linux/usr/bin/i586-poky-linux --sbindir=/home/test /poky/build/tmp/sysroots/x86_64-linux/usr/bin/i586-poky-linux --libexecdir=/home/test1/poky/build/tmp/sysroots/x86_64-linux/usr/libexec/i586-poky-linux --datadir=/home/test1/poky/build/tmp/sysroots/x86_64-linux/usr/share --sysconfdir=/home/test1/poky/build/tmp/sysroots/x86_64-linux/etc --sharedstatedir=/home/test1/poky/build/tmp/sysroots/x86_64-linux/com --localstatedir=/home/test1/poky/build/tmp/sysroots/x86_64-linux/var --libdir=/home/test1/poky/build/tmp/sysroots/x86_64-linux/usr/lib/i586-poky-linux --includedir=/home/test1/poky/build/tmp/sysroots/x86_64-linux/usr/include --oldincludedir=/home/test1/poky/build/tmp/sysroots/x86_64-linux/usr/include --infodir=/home/test1/poky/build/tmp/sysroots/x86_64-linux/usr/share/info --mandir=/home/test1/poky/build/tmp/sysroots/x86_64-linux/usr/share/man --disable-silent-rules --disable-dependency-tracking --with-libtool-sysroot=/home/test1/poky/build/tmp/sysroots/x86_64-linux --with-gnu-ld --enable-shared --enable-languages=c,c++ --enable-threads=posix --disable-multilib --enable-c99 --enable-long-long --enable-symvers=gnu --enable-libstdcxx-pch --program-prefix=i586-poky-linux- --enable-target-optspace --enable-lto --enable-libssp --disable-bootstrap --disable-libgomp --disable-libmudflap --with-system-zlib --with-linker-hash-style=gnu --enable-cheaders=c_global --with-local-prefix=/home/test1/poky/build/tmp/sysroots/qemux86/usr --with-gxx-include-dir=/home/test1/poky/build/tmp/sysroots/qemux86/usr/include/c++ --with-sysroot=/home/test1/poky/build/tmp/sysroots/qemux86 --with-build-sysroot=/home/test1/poky/build/tmp/sysroots/qemux86 --enable-poison-system-directories --with-headers=/home/test1/poky/build/tmp/sysroots/qemux86/usr/include --disable-libunwind-exceptions --with-mpfr=/home/test1/poky/build/tmp/sysroots/x86_64-linux/usr --with-system-zlib --disable-nls --enable-__cxa_atexit config.log:cc1: error: /home/test1/poky/build/tmp/sysroots/qemux86/home/test1/poky/build/tmp/sysroots/qemux86/usr/include: Permission denied config.log:cc1: error: /home/test1/poky/build/tmp/sysroots/qemux86/usr/include: Permission denied Hopefully this provides some additional detail validating Mark's initial report. I'm handing this off to Jiajun now for further review.
Darren was able to reproduce the failure using the proposed test-case. Note: To be clear, this default is not about the failure (there is already another defect covering that), but instead about adding this additional test to ensure we don't get failures like this in the future.
I will add the testing into QA 1.2 M4 cycle. And we are setting up the regression test server for some build feature testing automation. According to the description for the failure, we could add a case on regression test server to cover the case in regular testing.
To be automated by the RO QA team.
Moving this to future. This was tested manually, but the intention is to have it automated. The initial idea was to test this with autotest but the idea of using autotest with yocto has been dropped. We will investigate more ways into the future.
Corneliu will look into it, to see if this worth being automated.
Looking into a way to automate this with oe-selftest. Initial thoughts: The test should check for the existance of 2 pre-existing test users (ex: selftest_user_1 and selftest_user_2) and will run commands on their behalf. Specifics about the shell environment and permissions of these users will be determined along the way. Any thoughts on this are welcome!
Hello Mark, We have been working on a similar test that has been merged in master: https://bugzilla.yoctoproject.org/show_bug.cgi?id=2081 We are thinking at molding this test to be similar to the others. Here is the proposed process: We will use only 1 user 1) create poky/build1 2) in poky/build1 create core-image-minimal 3) move poky/build1/sstate-cache and poky/build1/downloads to a world-readable location, say /test/sstate-cache/ respectively /test/downloads/ and also make them world-readable *NOTE: Because we will use these as MIRRORS, is there any point in making them read-only? Would bitbake attempt to modify mirror sources? 4) remove poky/build1 5) create poky/build2 and make test/sstate-cache and test/downloads mirrors for it 6) create core-image-minimal in poky/build2 Will the above test scenario detect the issues you have in mind?
The general test looks fine, but it needs to be done over two users. We have found (in the past) some cases where being the same user worked, being a different use did not. Being the same user allows things to be written to the "old" compile paths it didn't detect the issue. Once we switched to two different users, then it would fail as it couldn't read/write the other users directory due to permissions protections.
Moving to 1.7
I think this test is still needed. As commented on last time, the only piece missing to the test case proposal is running it with two different user ids.
Bulk move from 4.99 or 0.00 to 5.99