The Yocto documentation describes a procedure to "Replicate a Build Offline": https://docs.yoctoproject.org/4.2.2/dev-manual/building.html?highlight=bb_no_network#replicating-a-build-offline The documentation says that it is sufficient to keep the tar files creates by setting 'BB_GENERATE_MIRROR_TARBALLS = "1"' and that it is possible to delete any Git or other SCM repositories from the downloads directory, but this is not working. Regarding Git the documentation specifically says that it is possible to delete all ${DL_DIR}/git2/* directories. When the removal of the ${DL_DIR}/git2/* directories is skipped in the steps below, the offline build succeeds. How to reproduce: ~/src$ git clone -b mickledore-4.2.2 https://git.yoctoproject.org/poky Cloning into 'poky'... ... ~/src$ cd poky/ ~/src/poky$ . oe-init-build-env # Now edited conf/local.conf and added: BB_GENERATE_MIRROR_TARBALLS = "1". # Tarballs are created as expected. Git directories are removed from the download directory. ~/src/poky/build$ bitbake core-image-minimal --runonly=fetch ... ~/src/poky/build$ rm -rf downloads/git2/* # Now I edit conf/local.conf again and add the following according to the documentation # SOURCE_MIRROR_URL ?= "file://${TOPDIR}/downloads/" # INHERIT += "own-mirrors" # BB_NO_NETWORK = "1" ~/src/poky/build$ tail -10 conf/local.conf # CONF_VERSION is increased each time build/conf/ changes incompatibly and is used to # track the version of this file when it was generated. This can safely be ignored if # this doesn't mean anything to you. CONF_VERSION = "2" BB_GENERATE_MIRROR_TARBALLS = "1" SOURCE_MIRROR_URL ?= "file://${TOPDIR}/downloads/" INHERIT += "own-mirrors" BB_NO_NETWORK = "1" # An attempt to run the build fails early with these error messages: ~/src/poky/build$ bitbake core-image-minimal Loading cache: 100% | | ETA: --:--:-- Loaded 0 entries from dependency cache. Parsing recipes: 100% |##################################################################################| Time: 0:00:07 Parsing of 899 .bb files complete (0 cached, 899 parsed). 1799 targets, 49 skipped, 0 masked, 0 errors. NOTE: Resolving any missing task queue dependencies Build Configuration: BB_VERSION = "2.4.0" BUILD_SYS = "x86_64-linux" NATIVELSBSTRING = "universal" TARGET_SYS = "x86_64-poky-linux" MACHINE = "qemux86-64" DISTRO = "poky" DISTRO_VERSION = "4.2.2" TUNE_FEATURES = "m64 core2" TARGET_FPU = "" meta meta-poky meta-yocto-bsp = "HEAD:6e17b3e644ca15b8b4afd071ccaa6f172a0e681a" Initialising tasks: 100% |###############################################################################| Time: 0:00:01 Sstate summary: Wanted 1074 Local 0 Mirrors 0 Missed 1074 Current 0 (0% match, 0% complete) NOTE: Executing Tasks ERROR: gnu-config-native-20230216+gitAUTOINC+63acb96f92-r0 do_unpack: Bitbake Fetcher Error: UnpackError('No up to date source found: clone directory not available or not up to date: /home/stm/src/poky/build/downloads/git2/git.savannah.gnu.org.git.config.git; shallow clone not enabled', 'git://git.savannah.gnu.org/git/config.git;protocol=https;branch=master') ERROR: Logfile of failure stored in: /home/stm/src/poky/build/tmp/work/x86_64-linux/gnu-config-native/20230216+gitAUTOINC+63acb96f92-r0/temp/log.do_unpack.19635 ERROR: Task (virtual:native:/home/stm/src/poky/meta/recipes-devtools/gnu-config/gnu-config_git.bb:do_unpack) failed with exit code '1' ERROR: pkgconfig-native-0.29.2+gitAUTOINC+d97db4fae4-r0 do_unpack: Bitbake Fetcher Error: UnpackError('No up to date source found: clone directory not available or not up to date: /home/stm/src/poky/build/downloads/git2/gitlab.freedesktop.org.pkg-config.pkg-config.git; shallow clone not enabled', 'git://gitlab.freedesktop.org/pkg-config/pkg-config.git;branch=master;protocol=https') ERROR: Logfile of failure stored in: /home/stm/src/poky/build/tmp/work/x86_64-linux/pkgconfig-native/0.29.2+gitAUTOINC+d97db4fae4-r0/temp/log.do_unpack.19647 ERROR: Task (virtual:native:/home/stm/src/poky/meta/recipes-devtools/pkgconfig/pkgconfig_git.bb:do_unpack) failed with exit code '1' NOTE: Tasks Summary: Attempted 44 tasks of which 22 didn't need to be rerun and 2 failed. Summary: 2 tasks failed: virtual:native:/home/stm/src/poky/meta/recipes-devtools/gnu-config/gnu-config_git.bb:do_unpack virtual:native:/home/stm/src/poky/meta/recipes-devtools/pkgconfig/pkgconfig_git.bb:do_unpack Summary: There were 2 ERROR messages, returning a non-zero exit code.
Stephan, Thanks for the bug report. Are you able to test this on master? We don't have anyone available to work on the issue right not but the Robert and/or Hongxu may have some insight into the issue so I've CCed them.
(In reply to Randy MacLeod from comment #1) > Stephan, > Thanks for the bug report. Thanks for looking into this. > Are you able to test this on master? I repeated the steps from the description, with the only difference that I checked out master: ~/src$ git clone -b master https://git.yoctoproject.org/poky This is the commit that I got: commit bdc74e5ff3c16fed24829fd494e5860754c08778 (HEAD -> master, origin/master, origin/HEAD) Author: Etienne Cordonnier <ecordonnier@snap.com> Date: Tue Aug 29 13:45:47 2023 +0200 migration-guides: system-conf -> systemd-conf The error is reproducible in the same manner as from the mickledore branch.
Bulk move to 5.0 M1. -- Randy
I was able to reproduce the symptoms using the steps indicated. However, this repro omits a step from the cited Yocto documentation, specifically: "2. Start With a Clean Build: You can start with a clean build by removing the ${TMPDIR} directory or using a new Build Directory." After I removed the ${TMPDIR} directory and re-ran bitbake, the problem did not occur. However bitbake later failed in do_package_write_rpm for openssl.
(In reply to Alan Perry from comment #4) > However, this repro omits a step from the cited Yocto documentation, > specifically: > > "2. Start With a Clean Build: You can start with a clean build by removing > the ${TMPDIR} directory or using a new Build Directory." You are right, thanks for pointing that out. I had somehow assumed that I am already in the "Clean Build" state as the packages were only fetched and no actual build of the packages has been performed. When I remove the ${TMPDIR} directory before invoking "bitbake core-image-minimal" the build works as expected without any network access. I did not see the problem with do_package_write_rpm for openssl. I'm building the Mickledore release. I'm sorry for the confusion, this bug report is invalid.