14790 2022-04-19 12:37:38 +0000 dbus: fatal error: X11/Xlib.h: No such file or directory 2022-06-09 09:43:59 +0000 1 1 7 Build System, Metadata & Runtime OE-Core oe-core other unspecified x86 x86_64 RESOLVED FIXED AB-INT Medium+ normal 4.1 M2 1 luca.ceresoli richard.purdie alex.kanavin alexandre.belloni luca.ceresoli randy.macleod richard.purdie ross.burton --- --- No (bug/feature does not impact docs) oldest_to_newest 93124 0 luca.ceresoli 2022-04-19 12:37:38 +0000 dbus_1.14.0 fails do_compile: | ../../dbus-1.14.0/tools/dbus-launch.c:45:10: fatal error: X11/Xlib.h: No such file or directory | 45 | #include <X11/Xlib.h> | | ^~~~~~~~~~~~ Exactly the same build has been started 6 times and failed once. oe-selftest-centos on centos8-ty-1 Failed: https://autobuilder.yoctoproject.org/typhoon/#/builders/79/builds/3463/steps/15/logs/stdio One of the succeeded runs: https://autobuilder.yoctoproject.org/typhoon/#/builders/79/builds/3466/steps/15/logs/stdio 93127 1 luca.ceresoli 2022-04-19 20:17:54 +0000 https://autobuilder.yoctoproject.org/typhoon/#/builders/80/builds/3419/steps/14/logs/stdio oe-selftest-debian on debian11-ty-3 93150 2 randy.macleod 2022-04-21 14:43:36 +0000 This shouldn't be possible even in a self-test... 93173 3 alex.kanavin 2022-04-22 08:43:59 +0000 This is indeed very odd. If it happens, inspecting the build directory of dbus would be very helpful. 93269 4 richard.purdie 2022-05-14 09:15:39 +0000 This happened again: https://autobuilder.yoctoproject.org/typhoon/#/builders/80/builds/3522/steps/14/logs/stdio dbus/1.14.0-r0/temp$ cat log.task_order do_populate_lic_setscene (3707212): log.do_populate_lic_setscene.3707212 do_deploy_source_date_epoch_setscene (3713069): log.do_deploy_source_date_epoch_setscene.3713069 do_fetch (3723141): log.do_fetch.3723141 do_unpack (3727703): log.do_unpack.3727703 do_patch (3732761): log.do_patch.3732761 do_populate_sysroot_setscene (315304): log.do_populate_sysroot_setscene.315304 do_prepare_recipe_sysroot (316001): log.do_prepare_recipe_sysroot.316001 do_configure (318403): log.do_configure.318403 do_package_write_rpm_setscene (370306): log.do_package_write_rpm_setscene.370306 do_package_qa_setscene (370301): log.do_package_qa_setscene.370301 do_packagedata_setscene (370314): log.do_packagedata_setscene.370314 do_packagedata_setscene (1813411): log.do_packagedata_setscene.1813411 do_populate_sysroot_setscene (997805): log.do_populate_sysroot_setscene.997805 do_configure_ptest_base (1007456): log.do_configure_ptest_base.1007456 do_compile (1007878): log.do_compile.1007878 do_package_write_rpm_setscene (1102663): log.do_package_write_rpm_setscene.1102663 do_package_qa_setscene (1102659): log.do_package_qa_setscene.1102659 do_packagedata_setscene (1117273): log.do_packagedata_setscene.1117273 do_package_write_rpm_setscene (1216055): log.do_package_write_rpm_setscene.1216055 do_packagedata_setscene (1217019): log.do_packagedata_setscene.1217019 do_package_write_rpm_setscene (2128997): log.do_package_write_rpm_setscene.2128997 do_packagedata_setscene (2129656): log.do_packagedata_setscene.2129656 $ grep Xlib.h * log.do_compile:../../dbus-1.14.0/tools/dbus-launch-x11.c:37:10: fatal error: X11/Xlib.h: No such file or directory log.do_compile: 37 | #include <X11/Xlib.h> log.do_compile:../../dbus-1.14.0/tools/dbus-launch.c:45:10: fatal error: X11/Xlib.h: No such file or directory log.do_compile: 45 | #include <X11/Xlib.h> log.do_compile.1007878:../../dbus-1.14.0/tools/dbus-launch-x11.c:37:10: fatal error: X11/Xlib.h: No such file or directory log.do_compile.1007878: 37 | #include <X11/Xlib.h> log.do_compile.1007878:../../dbus-1.14.0/tools/dbus-launch.c:45:10: fatal error: X11/Xlib.h: No such file or directory log.do_compile.1007878: 45 | #include <X11/Xlib.h> log.do_populate_sysroot_setscene:DEBUG: Removing manifest: /home/pokybuild/yocto-worker/oe-selftest-debian/build/build-st-1848430/tmp/work/core2-64-poky-linux/dbus/1.14.0-r0/recipe-sysroot/usr/include/X11/Xlib.h log.do_populate_sysroot_setscene.997805:DEBUG: Removing manifest: /home/pokybuild/yocto-worker/oe-selftest-debian/build/build-st-1848430/tmp/work/core2-64-poky-linux/dbus/1.14.0-r0/recipe-sysroot/usr/include/X11/Xlib.h which is preceded by: NOTE: /home/pokybuild/yocto-worker/oe-selftest-debian/build/build-st-1848430/tmp/work/core2-64-poky-linux/dbus/1.14.0-r0/recipe-sysroot-native/installeddeps/libx11.co mplete no longer exists, removing from sysroot so populate_sysroot was run from sstate which noticed some invalid entries in the sysroot and removed them. do_compile then ran, the files were missing, things broke. The question is why prepare_recipe_sysroot didn't rerun before do_compile. Looking like some kind of sstate issue. 93270 5 richard.purdie 2022-05-14 09:19:08 +0000 There are many builds from the oe-selftest in this builddir. I confirmed the second do_populate_sysroot and the failing do_compile were in the same bitbake invocation. 93271 6 richard.purdie 2022-05-14 09:47:25 +0000 I don't know how to reproduce this. libx11 do_populate_sysroot doesn't have a stamp in that build and isn't present. dbus do_prepare_recipe_sysroot does depend upon it as far as I can tell locally. A local build with libx11 do_(package|packagedata|package_qa|package_write_rpm|populate_lic) present but not do_populate_sysroot triggers an install of do_populate_sysroot if you run dbus -c compile. Tests run in this build were: 2022-05-13 15:29:12,275 - oe-selftest - INFO - 0: 1/45 9/483 (82.39s) (archiver.Archiver.test_archiver_allows_to_filter_on_recipe_name) 2022-05-13 15:29:50,950 - oe-selftest - INFO - 0: 2/45 22/483 (38.67s) (archiver.Archiver.test_archiver_filters_by_type) 2022-05-13 15:32:35,615 - oe-selftest - INFO - 0: 3/45 27/483 (164.67s) (archiver.Archiver.test_archiver_filters_by_type_and_name) 2022-05-13 15:33:07,683 - oe-selftest - INFO - 0: 4/45 32/483 (32.06s) (archiver.Archiver.test_archiver_mode_configured) 2022-05-13 15:33:34,337 - oe-selftest - INFO - 0: 5/45 36/483 (26.65s) (archiver.Archiver.test_archiver_mode_diff) 2022-05-13 15:34:01,885 - oe-selftest - INFO - 0: 6/45 43/483 (27.55s) (archiver.Archiver.test_archiver_mode_dumpdata) 2022-05-13 15:34:32,558 - oe-selftest - INFO - 0: 7/45 48/483 (30.67s) (archiver.Archiver.test_archiver_mode_mirror) 2022-05-13 15:35:25,979 - oe-selftest - INFO - 0: 8/45 53/483 (53.42s) (archiver.Archiver.test_archiver_mode_mirror_combined) 2022-05-13 15:35:54,942 - oe-selftest - INFO - 0: 9/45 59/483 (28.96s) (archiver.Archiver.test_archiver_mode_mirror_excludes) 2022-05-13 15:36:27,772 - oe-selftest - INFO - 0: 10/45 60/483 (32.83s) (archiver.Archiver.test_archiver_mode_mirror_gitsm) 2022-05-13 15:38:28,636 - oe-selftest - INFO - 0: 11/45 64/483 (120.86s) (archiver.Archiver.test_archiver_mode_mirror_gitsm_shallow) 2022-05-13 15:38:56,903 - oe-selftest - INFO - 0: 12/45 66/483 (28.27s) (archiver.Archiver.test_archiver_mode_original) 2022-05-13 15:39:34,915 - oe-selftest - INFO - 0: 13/45 67/483 (38.01s) (archiver.Archiver.test_archiver_mode_patched) 2022-05-13 15:40:02,638 - oe-selftest - INFO - 0: 14/45 68/483 (27.72s) (archiver.Archiver.test_archiver_mode_recipe) 2022-05-13 15:40:33,220 - oe-selftest - INFO - 0: 15/45 69/483 (30.58s) (archiver.Archiver.test_archiver_srpm_mode) 2022-05-13 17:44:05,816 - oe-selftest - INFO - 0: 16/45 141/483 (7406.99s) (devtool.DevtoolExtractTests.test_devtool_build_image) 2022-05-13 17:47:54,596 - oe-selftest - INFO - 0: 17/45 142/483 (228.78s) (devtool.DevtoolExtractTests.test_devtool_deploy_target) 2022-05-13 17:48:12,048 - oe-selftest - INFO - 0: 18/45 143/483 (17.45s) (devtool.DevtoolExtractTests.test_devtool_extract) 2022-05-13 17:48:31,335 - oe-selftest - INFO - 0: 19/45 144/483 (19.30s) (devtool.DevtoolExtractTests.test_devtool_extract_virtual) 2022-05-13 17:51:20,990 - oe-selftest - INFO - 0: 20/45 145/483 (169.66s) (devtool.DevtoolExtractTests.test_devtool_reset_all) 2022-05-13 17:56:11,921 - oe-selftest - INFO - 0: 21/45 147/483 (289.60s) (incompatible_lic.IncompatibleLicensePerImageTests.test_bash_and_license) 2022-05-13 17:56:58,176 - oe-selftest - INFO - 0: 22/45 151/483 (46.25s) (incompatible_lic.IncompatibleLicensePerImageTests.test_bash_default) 2022-05-13 17:57:44,009 - oe-selftest - INFO - 0: 23/45 154/483 (45.83s) (incompatible_lic.IncompatibleLicensePerImageTests.test_bash_license_exceptions) 2022-05-13 18:02:09,667 - oe-selftest - INFO - 0: 24/45 176/483 (265.67s) (incompatible_lic.IncompatibleLicensePerImageTests.test_bash_or_license) 2022-05-13 18:02:09,670 - oe-selftest - INFO - 0: 25/45 177/483 (0.00s) (oelib.license.TestIsIncluded.test_tests) 2022-05-13 18:57:29,918 - oe-selftest - INFO - 0: 26/45 301/483 (3320.25s) (overlayfs.OverlayFSTests.test_correct_image_fstab) 2022-05-13 18:58:32,103 - oe-selftest - INFO - 0: 27/45 302/483 (62.19s) (overlayfs.OverlayFSTests.test_correct_image_unit) 2022-05-13 18:58:42,302 - oe-selftest - INFO - 0: 28/45 303/483 (10.20s) (overlayfs.OverlayFSTests.test_distro_features_missing) 2022-05-13 18:58:45,660 - oe-selftest - INFO - 0: 29/45 304/483 (3.36s) (overlayfs.OverlayFSTests.test_mount_unit_not_set) 2022-05-13 19:11:19,498 - oe-selftest - INFO - 0: 30/45 326/483 (753.84s) (overlayfs.OverlayFSTests.test_not_all_units_installed) 2022-05-13 19:12:02,698 - oe-selftest - INFO - 0: 31/45 331/483 (43.20s) (overlayfs.OverlayFSTests.test_not_all_units_installed_but_qa_skipped) 2022-05-13 19:12:07,301 - oe-selftest - INFO - 0: 32/45 333/483 (4.60s) (overlayfs.OverlayFSTests.test_wrong_mount_unit_set) 2022-05-13 19:13:39,266 - oe-selftest - INFO - 0: 33/45 343/483 (91.97s) (runqemu.RunqemuTests.test_boot_deploy) 2022-05-13 19:13:55,365 - oe-selftest - INFO - 0: 34/45 346/483 (16.10s) (runqemu.RunqemuTests.test_boot_deploy_hddimg) 2022-05-13 19:14:11,192 - oe-selftest - INFO - 0: 35/45 350/483 (15.83s) (runqemu.RunqemuTests.test_boot_machine) 2022-05-13 19:14:25,574 - oe-selftest - INFO - 0: 36/45 351/483 (14.39s) (runqemu.RunqemuTests.test_boot_machine_ext4) 2022-05-13 19:14:42,391 - oe-selftest - INFO - 0: 37/45 353/483 (16.82s) (runqemu.RunqemuTests.test_boot_machine_iso) 2022-05-13 19:14:57,974 - oe-selftest - INFO - 0: 38/45 355/483 (15.59s) (runqemu.RunqemuTests.test_boot_machine_slirp) 2022-05-13 19:15:11,924 - oe-selftest - INFO - 0: 39/45 358/483 (13.95s) (runqemu.RunqemuTests.test_boot_machine_slirp_qcow2) 2022-05-13 19:15:23,504 - oe-selftest - INFO - 0: 40/45 361/483 (11.59s) (runqemu.RunqemuTests.test_boot_qemu_boot) 2022-05-13 19:15:38,308 - oe-selftest - INFO - 0: 41/45 363/483 (14.81s) (runqemu.RunqemuTests.test_boot_recipe_image) 2022-05-13 19:15:53,784 - oe-selftest - INFO - 0: 42/45 364/483 (15.48s) (runqemu.RunqemuTests.test_boot_recipe_image_vdi) 2022-05-13 19:15:56,570 - oe-selftest - INFO - imagefeatures.ImageFeatures.test_useradd_static (subunit.RemotedTestCase) 2022-05-13 19:15:56,572 - oe-selftest - INFO - ... FAIL 93272 7 richard.purdie 2022-05-14 10:02:20 +0000 The big clue is probably that: cases/overlayfs.py:DISTRO_FEATURES += "systemd overlayfs" cases/overlayfs.py:DISTRO_FEATURES += "systemd overlayfs" cases/overlayfs.py:DISTRO_FEATURES += "systemd overlayfs" cases/overlayfs.py:DISTRO_FEATURES += "systemd overlayfs" cases/overlayfs.py:DISTRO_FEATURES += "systemd overlayfs" cases/overlayfs.py:DISTRO_FEATURES += "systemd" cases/overlayfs.py:DISTRO_FEATURES += "systemd" cases/overlayfs.py:DISTRO_FEATURES += "systemd" cases/overlayfs.py:DISTRO_FEATURES += "systemd" which removes x11 from DISTRO_FEATURES since += clears it. This is in the list above, ran before this failure and will disable x11 from dbus but it probably doesn't get rebuilt properly. 93273 8 richard.purdie 2022-05-14 13:19:21 +0000 bitbake dbus then setting DISTRO_FEATURES += "systemd overlayfs" then running bitbake quilt-native results in libx11's complete link in installeddeps in recipe-sysroot-native in dbus being "broken". What I can't seem to reproduce is a scenario where "bitbake dbus -c compile" then doesn't restore them. Also I'd note the autobuilder did run a MACHINE=genericx86-64 build which triggers dbus do_packgedata_setscene to run. It is possible that changing between two machines with the same package arch is somehow involved. 93274 9 richard.purdie 2022-05-14 13:20:16 +0000 I'd also note that rerunning the failed autobuilder build on the same worker didn't reproduce a second time so sstate and hashequiv are somehow involved too and allow us to cache around the issue. 93275 10 richard.purdie 2022-05-14 20:07:36 +0000 I extracted the wrong list of tests, the correct one is: 2022-05-13 16:14:46,146 - oe-selftest - INFO - 14: 1/66 96/483 (2774.46s) (devtool.DevtoolAddTests.test_devtool_add) 2022-05-13 16:16:20,423 - oe-selftest - INFO - 14: 2/66 97/483 (94.28s) (devtool.DevtoolAddTests.test_devtool_add_fetch) 2022-05-13 16:17:54,793 - oe-selftest - INFO - 14: 3/66 98/483 (94.38s) (devtool.DevtoolAddTests.test_devtool_add_fetch_git) 2022-05-13 16:18:18,993 - oe-selftest - INFO - 14: 4/66 99/483 (24.20s) (devtool.DevtoolAddTests.test_devtool_add_fetch_simple) 2022-05-13 16:53:03,988 - oe-selftest - INFO - 14: 5/66 131/483 (2084.99s) (devtool.DevtoolAddTests.test_devtool_add_git_local) 2022-05-13 17:14:21,311 - oe-selftest - INFO - 14: 6/66 132/483 (1277.32s) (devtool.DevtoolAddTests.test_devtool_add_library) 2022-05-13 17:14:24,757 - oe-selftest - INFO - 14: 7/66 133/483 (3.45s) (devtool.DevtoolAddTests.test_devtool_add_npm) 2022-05-13 18:32:54,063 - oe-selftest - INFO - 14: 8/66 257/483 (4708.50s) (imagefeatures.ImageFeatures.test_all_users_can_connect_via_ssh_without_password) 2022-05-13 18:33:39,480 - oe-selftest - INFO - 14: 9/66 261/483 (45.42s) (imagefeatures.ImageFeatures.test_bmap) 2022-05-13 18:34:00,913 - oe-selftest - INFO - 14: 10/66 264/483 (21.43s) (imagefeatures.ImageFeatures.test_empty_image) 2022-05-13 18:35:30,569 - oe-selftest - INFO - 14: 11/66 271/483 (89.66s) (imagefeatures.ImageFeatures.test_hypervisor_fmts) 2022-05-13 18:36:58,732 - oe-selftest - INFO - 14: 12/66 275/483 (88.16s) (imagefeatures.ImageFeatures.test_image_fstypes) 2022-05-13 19:12:43,722 - oe-selftest - INFO - 14: 13/66 337/483 (2144.99s) (imagefeatures.ImageFeatures.test_image_gen_debugfs) 2022-05-13 19:13:40,653 - oe-selftest - INFO - 14: 14/66 344/483 (56.93s) (imagefeatures.ImageFeatures.test_long_chain_conversion) 2022-05-13 19:14:07,854 - oe-selftest - INFO - 14: 15/66 349/483 (27.21s) (imagefeatures.ImageFeatures.test_no_busybox_base_utils) 2022-05-13 19:15:04,620 - oe-selftest - INFO - 14: 16/66 357/483 (56.77s) (imagefeatures.ImageFeatures.test_non_root_user_can_connect_via_ssh_without_password) 2022-05-13 19:15:56,572 - oe-selftest - INFO - 14: 17/66 365/483 (51.95s) (imagefeatures.ImageFeatures.test_useradd_static) 93276 11 richard.purdie 2022-05-15 11:37:58 +0000 This bug is proving a pain to reproduce. It doesn't give this error but this set of steps does end up with a broken build directory in what is probably a related issue: Add DISTRO_FEATURES += "systemd" bitbake dbus Remove DISTRO_FEATURES += "systemd" bitbake dbus bitbake dbus -c clean bitbake dbus -c configure Add DISTRO_FEATURES += "systemd" bitbake quilt-native bitbake dbus -c populate_sysroot Remove DISTRO_FEATURES += "systemd" bitbake dbus -c compile 93429 12 richard.purdie 2022-06-09 09:43:59 +0000 https://git.yoctoproject.org/poky/commit/?id=2962cd877b7629b0be2962505796f5e60c31578e