1465 2011-09-09 01:21:49 +0000 bitbake cannot fetch local files from SRC_URI when -b is used 2012-06-22 08:34:39 +0000 1 1 7 Build System, Metadata & Runtime BitBake bitbake unspecified x86 Multiple RESOLVED FIXED (P1) Patches sent for some issues, adding a sanity test remains. Medium normal 1.2.1 1 Martin.Jansa laurentiu.palcu andrei jessica.zhang poky.bs.watcher poky.watcher richard.purdie scott.a.garman sgw song.liu --- --- --- oldest_to_newest 16112 0 Martin.Jansa 2011-09-09 01:21:49 +0000 $ bitbake -c unpack -b openembedded-core/meta/recipes-core/base-files/base-files_3.0.14.bb WARNING: Buildfile specified, dependencies will not be handled. If this is not what you want, do not use -b / --buildfile. OE Build Configuration: BB_VERSION = "1.13.3" TARGET_ARCH = "arm" TARGET_OS = "linux-gnueabi" MACHINE = "om-gta02" DISTRO = "shr" DISTRO_VERSION = "20110811" TUNE_FEATURES = "armv4 thumb arm920t" TARGET_FPU = "soft" meta-jama = "master:c9300f537bab3cf7a6400e9a0f799f8a54559784" meta-shr meta-fso meta-nokia meta-htc meta-palm meta-openmoko meta-zaurus = "master:b287197790ec4ac9ddbed185d4df1205a55531ac" meta-oe meta-efl meta-gnome = "shr:c5ba433b740fdb88f1a7631fa884ea39c03df35a" meta = "shr:fe056d9f1ebdf1b4ec5d423e2a6ea8cb7737d66c" NOTE: Preparing runqueue NOTE: Executing RunQueue Tasks NOTE: Running task 1 of 2 (ID: 1, openembedded-core/meta/recipes-core/base-files/base-files_3.0.14.bb, do_fetch) NOTE: package base-files-3.0.14-r69: task do_fetch: Started ERROR: Function 'Fetcher failure for URL: 'file://rotation'. Unable to fetch URL file://rotation from any source.' failed ERROR: Logfile of failure stored in: /OE/shr-core/tmp/work/om_gta02-oe-linux-gnueabi/base-files-3.0.14-r69/temp/log.do_fetch.28071 Log data follows: | ERROR: Function 'Fetcher failure for URL: 'file://rotation'. Unable to fetch URL file://rotation from any source.' failed NOTE: package base-files-3.0.14-r69: task do_fetch: Failed ERROR: Task 1 (openembedded-core/meta/recipes-core/base-files/base-files_3.0.14.bb, do_fetch) failed with exit code '1' ERROR: 'openembedded-core/meta/recipes-core/base-files/base-files_3.0.14.bb' failed but the rotation file is where it should be: openembedded-core/meta/recipes-core/base-files/base-files/rotation and without -b it fetches rotation file fine. More details were on oe-core ML http://lists.linuxtogo.org/pipermail/openembedded-core/2011-August/008097.html and http://lists.linuxtogo.org/pipermail/openembedded-core/2011-May/003431.html where is also reported which bitbake commit broke this behavior http://git.openembedded.org/cgit.cgi/bitbake/commit/?id=6e9119c42d3f4cb3dab3d25856eee8b4bf4ad94f 17242 1 richard.purdie 2011-11-17 06:17:18 +0000 This bug doesn't seem to be as simple as states here as bitbake -b base-files -c unpack works fine for me :( It is likely therefore some kind of complication in the layering but its making it harder to reproduce. 17259 2 richard.purdie 2011-11-18 06:40:43 +0000 http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=9d4f95587424c9e2d3ef7891c8ac7007f8db1f00 is a valid fix and seems to resolve the problem for Martin even if I can't reproduce it. 20839 3 Martin.Jansa 2012-04-24 16:04:15 +0000 Part of original fix was reverted in http://git.yoctoproject.org/cgit.cgi/poky/commit/bitbake/lib/bb/parse/parse_py/BBHandler.py?id=9d4f95587424c9e2d3ef7891c8ac7007f8db1f00 so it's broken again 20840 4 Martin.Jansa 2012-04-24 16:07:36 +0000 Ah :/ not this commit but something else.. but still it's broken (and only broken when relative path is used in -b) 21500 5 richard.purdie 2012-05-21 16:24:14 +0000 I think the working directories are not getting consistently set for some reason. 21506 6 richard.purdie 2012-05-21 22:51:42 +0000 Martin, can you please provide a test case to reproduce this? I was hunting for a different problem today and came back to looking at this for various reasons but I can't replicate the bug :(. I suspect it depends on some particular directory layout, some particular environment script or combination of things which I don't have... 21511 7 Martin.Jansa 2012-05-22 07:27:23 +0000 I wasn't able to reproduce it today with oe-core only (but IIRC it was failing with oe-core only too), so I've tried to narrow it with default SHR setup NOTE: Preparing runqueue NOTE: Executing RunQueue Tasks NOTE: Running task 1 of 3 (ID: 2, openembedded-core/meta/recipes-core/base-files/base-files_3.0.14.bb, do_rm_old_works) NOTE: Running task 2 of 3 (ID: 1, openembedded-core/meta/recipes-core/base-files/base-files_3.0.14.bb, do_fetch) NOTE: package base-files-3.0.14-r77: task do_rm_old_works: Started NOTE: package base-files-3.0.14-r77: task do_rm_old_works: Succeeded NOTE: package base-files-3.0.14-r77: task do_fetch: Started ERROR: Function failed: Fetcher failure for URL: 'file://rotation'. Unable to fetch URL from any source. ERROR: Logfile of failure stored in: /OE/shr-core/tmp-eglibc/work/om_gta02-oe-linux-gnueabi/base-files/base-files-3.0.14-r77/temp/log.do_fetch.5922 Log data follows: | DEBUG: Trying PREMIRRORS | DEBUG: For url ['file', '', 'rotation', '', '', {}] comparing ['cvs', '.*', '/.*', '', '', {}] to ['http', 'build.shr-project.org', '/sources/', '', '', {}] | DEBUG: For url ['file', '', 'rotation', '', '', {}] comparing ['svn', '.*', '/.*', '', '', {}] to ['http', 'build.shr-project.org', '/sources/', '', '', {}] | DEBUG: For url ['file', '', 'rotation', '', '', {}] comparing ['git', '.*', '/.*', '', '', {}] to ['http', 'build.shr-project.org', '/sources/', '', '', {}] | DEBUG: For url ['file', '', 'rotation', '', '', {}] comparing ['hg', '.*', '/.*', '', '', {}] to ['http', 'build.shr-project.org', '/sources/', '', '', {}] | DEBUG: For url ['file', '', 'rotation', '', '', {}] comparing ['bzr', '.*', '/.*', '', '', {}] to ['http', 'build.shr-project.org', '/sources/', '', '', {}] | DEBUG: Trying Upstream | ERROR: Function failed: Fetcher failure for URL: 'file://rotation'. Unable to fetch URL from any source. NOTE: package base-files-3.0.14-r77: task do_fetch: Failed ERROR: Task 1 (openembedded-core/meta/recipes-core/base-files/base-files_3.0.14.bb, do_fetch) failed with exit code '1' NOTE: Tasks Summary: Attempted 2 tasks of which 0 didn't need to be rerun and 1 failed. Tried to disable distro PREMIRRORS: NOTE: Preparing runqueue NOTE: Executing RunQueue Tasks NOTE: Running task 2 of 3 (ID: 1, openembedded-core/meta/recipes-core/base-files/base-files_3.0.14.bb, do_fetch) NOTE: package base-files-3.0.14-r77: task do_fetch: Started ERROR: Function failed: Fetcher failure for URL: 'file://rotation'. Unable to fetch URL from any source. ERROR: Logfile of failure stored in: /OE/shr-core/tmp-eglibc/work/om_gta02-oe-linux-gnueabi/base-files/base-files-3.0.14-r77/temp/log.do_fetch.6078 Log data follows: | DEBUG: Trying PREMIRRORS | DEBUG: Trying Upstream | ERROR: Function failed: Fetcher failure for URL: 'file://rotation'. Unable to fetch URL from any source. NOTE: package base-files-3.0.14-r77: task do_fetch: Failed ERROR: Task 1 (openembedded-core/meta/recipes-core/base-files/base-files_3.0.14.bb, do_fetch) failed with exit code '1' NOTE: Tasks Summary: Attempted 2 tasks of which 1 didn't need to be rerun and 1 failed. Removed all .bbappends for base-files: OE om-gta02@shr ~/shr-core $ rm -f meta-smartphone/meta-nokia/recipes-core/base-files/base-files_3.0.14.bbappend meta-smartphone/meta-shr/recipes-core/base-files/base-files_3.0.14.bbappend meta-smartphone/meta-openmoko/recipes-core/base-files/base-files_3.0.14.bbappend meta-smartphone/meta-samsung/recipes-core/base-files/base-files_3.0.14.bbappend NOTE: Preparing runqueue NOTE: Executing RunQueue Tasks NOTE: Running task 1 of 3 (ID: 2, openembedded-core/meta/recipes-core/base-files/base-files_3.0.14.bb, do_rm_old_works) NOTE: Running task 2 of 3 (ID: 1, openembedded-core/meta/recipes-core/base-files/base-files_3.0.14.bb, do_fetch) NOTE: package base-files-3.0.14-r71: task do_rm_old_works: Started NOTE: package base-files-3.0.14-r71: task do_rm_old_works: Succeeded NOTE: package base-files-3.0.14-r71: task do_fetch: Started ERROR: Function failed: Fetcher failure for URL: 'file://rotation'. Unable to fetch URL from any source. ERROR: Logfile of failure stored in: /OE/shr-core/tmp-eglibc/work/om_gta02-oe-linux-gnueabi/base-files/base-files-3.0.14-r71/temp/log.do_fetch.6241 Log data follows: | DEBUG: Trying PREMIRRORS | DEBUG: Trying Upstream | ERROR: Function failed: Fetcher failure for URL: 'file://rotation'. Unable to fetch URL from any source. NOTE: package base-files-3.0.14-r71: task do_fetch: Failed ERROR: Task 1 (openembedded-core/meta/recipes-core/base-files/base-files_3.0.14.bb, do_fetch) failed with exit code '1' NOTE: Tasks Summary: Attempted 2 tasks of which 0 didn't need to be rerun and 1 failed. Still the same, tried without -b to reparse (interestingly nothing was used from cache) OE om-gta02@shr ~/shr-core $ bitbake -c unpack -f base-files WARNING: /OE/shr-core/meta-openembedded/meta-oe/recipes-connectivity/networkmanager/networkmanager_0.9.2.0.bb: it is recommended to set SYSTEMD_PACKAGES as <existing-package>-systemd | ETA: 0:00:16 NOTE: shr DOES NOT support libiconv because libiconv is provided by LIBC######################################################################################################## | ETA: 0:00:09 NOTE: shr DOES NOT support libiconv because libiconv is provided by LIBC########################################################################################################################## | ETA: 0:00:05 WARNING: /OE/shr-core/openembedded-core/meta/recipes-extended/lighttpd/lighttpd_1.4.30.bb: it is recommended to set SYSTEMD_PACKAGES as <existing-package>-systemd NOTE: shr DOES NOT support initscripts because use service file for systemd now NOTE: shr DOES NOT support external-csl-toolchain because use internal toolchain NOTE: shr DOES NOT support sysvinit because use systemd now############################################################################################################################################### | ETA: 0:00:04 Parsing recipes: 100% |#######################################################################################################################################################################################################| Time: 0:00:42 Parsing of 1718 .bb files complete (0 cached, 1718 parsed). 2157 targets, 70 skipped, 0 masked, 0 errors. WARNING: No bb files matched BBFILE_PATTERN_systemd-layer '^/OE/shr-core/meta-openembedded/meta-systemd/' Build Configuration: BB_VERSION = "1.15.2" TARGET_ARCH = "arm" TARGET_OS = "linux-gnueabi" MACHINE = "om-gta02" DISTRO = "shr" DISTRO_VERSION = "20120522" TUNE_FEATURES = "armv4 thumb arm920t" TARGET_FPU = "soft" meta-jama = "master:aa61d65a0a21be6eab0619c1ad3a5bef6c912f92" meta-shr meta-aurora meta-fso = "jansa/test:a4b5b9c2e987dc87cf0a4f890428443a9cea3733" meta-oe toolchain-layer meta-efl meta-gnome meta-gpe meta-initramfs meta-systemd = "jansa/test:7b64162e153c3538ddc98e804cc0208d5f50212a" meta-osmocombb meta-nokia meta-htc meta-palm meta-openmoko meta-samsung = "jansa/test:a4b5b9c2e987dc87cf0a4f890428443a9cea3733" meta-handheld = "jansa/spitz:f2169618f96b14491911a2361bf04f1b8c5f65a2" meta-browser = "master:c47f59df2e723495679c751cbdf6a8c6adec4b6a" meta = "jansa/test:89f4092b93452f77d0d30b394629a928341a01d0" NOTE: Resolving any missing task queue dependencies NOTE: Preparing runqueue NOTE: Executing RunQueue Tasks NOTE: Running task 2 of 3 (ID: 1, /OE/shr-core/openembedded-core/meta/recipes-core/base-files/base-files_3.0.14.bb, do_fetch) NOTE: package base-files-3.0.14-r71: task do_fetch: Started NOTE: package base-files-3.0.14-r71: task do_fetch: Succeeded NOTE: Running task 3 of 3 (ID: 0, /OE/shr-core/openembedded-core/meta/recipes-core/base-files/base-files_3.0.14.bb, do_unpack) NOTE: package base-files-3.0.14-r71: task do_unpack: Started NOTE: package base-files-3.0.14-r71: task do_unpack: Succeeded NOTE: Tasks Summary: Attempted 3 tasks of which 1 didn't need to be rerun and all succeeded. Summary: There were 3 WARNING messages shown. And with -b just after it: OE om-gta02@shr ~/shr-core $ bitbake -f -c unpack -b openembedded-core/meta/recipes-core/base-files/base-files_3.0.14.bb WARNING: Buildfile specified, dependencies will not be handled. If this is not what you want, do not use -b / --buildfile. Build Configuration: BB_VERSION = "1.15.2" TARGET_ARCH = "arm" TARGET_OS = "linux-gnueabi" MACHINE = "om-gta02" DISTRO = "shr" DISTRO_VERSION = "20120522" TUNE_FEATURES = "armv4 thumb arm920t" TARGET_FPU = "soft" meta-jama = "master:aa61d65a0a21be6eab0619c1ad3a5bef6c912f92" meta-shr meta-aurora meta-fso = "jansa/test:a4b5b9c2e987dc87cf0a4f890428443a9cea3733" meta-oe toolchain-layer meta-efl meta-gnome meta-gpe meta-initramfs meta-systemd = "jansa/test:7b64162e153c3538ddc98e804cc0208d5f50212a" meta-osmocombb meta-nokia meta-htc meta-palm meta-openmoko meta-samsung = "jansa/test:a4b5b9c2e987dc87cf0a4f890428443a9cea3733" meta-handheld = "jansa/spitz:f2169618f96b14491911a2361bf04f1b8c5f65a2" meta-browser = "master:c47f59df2e723495679c751cbdf6a8c6adec4b6a" meta = "jansa/test:89f4092b93452f77d0d30b394629a928341a01d0" NOTE: Preparing runqueue NOTE: Executing RunQueue Tasks NOTE: Running task 3 of 3 (ID: 0, openembedded-core/meta/recipes-core/base-files/base-files_3.0.14.bb, do_unpack) NOTE: package base-files-3.0.14-r71: task do_unpack: Started ERROR: Error executing a python function in openembedded-core/meta/recipes-core/base-files/base-files_3.0.14.bb: OSError: [Errno 2] No such file or directory: '/OE/shr-core/downloads/rotation' ERROR: The stack trace of python calls that resulted in this exception/failure was: ERROR: File "base_do_unpack", line 19, in <module> ERROR: ERROR: File "base_do_unpack", line 14, in base_do_unpack ERROR: ERROR: File "/usr/lib64/python2.7/site-packages/bb/fetch2/__init__.py", line 1075, in unpack ERROR: ud.method.unpack(ud, root, self.d) ERROR: ERROR: File "/usr/lib64/python2.7/site-packages/bb/fetch2/__init__.py", line 775, in unpack ERROR: if (file != dest) and not (os.path.exists(dest) and os.path.samefile(file, dest)): ERROR: ERROR: File "/usr/lib64/python2.7/posixpath.py", line 153, in samefile ERROR: s1 = os.stat(f1) ERROR: ERROR: The code that was being executed was: ERROR: 0015: except bb.fetch2.BBFetchException, e: ERROR: 0016: raise bb.build.FuncFailed(e) ERROR: 0017: ERROR: 0018: ERROR: *** 0019:base_do_unpack(d) ERROR: 0020: ERROR: (file: 'base_do_unpack', lineno: 19, function: <module>) ERROR: 0010: rootdir = localdata.getVar('WORKDIR', True) ERROR: 0011: ERROR: 0012: try: ERROR: 0013: fetcher = bb.fetch2.Fetch(src_uri, localdata) ERROR: *** 0014: fetcher.unpack(rootdir) ERROR: 0015: except bb.fetch2.BBFetchException, e: ERROR: 0016: raise bb.build.FuncFailed(e) ERROR: 0017: ERROR: 0018: ERROR: (file: 'base_do_unpack', lineno: 14, function: base_do_unpack) ERROR: Function failed: base_do_unpack ERROR: Logfile of failure stored in: /OE/shr-core/tmp-eglibc/work/om_gta02-oe-linux-gnueabi/base-files/base-files-3.0.14-r71/temp/log.do_unpack.10111 NOTE: package base-files-3.0.14-r71: task do_unpack: Failed ERROR: Task 0 (openembedded-core/meta/recipes-core/base-files/base-files_3.0.14.bb, do_unpack) failed with exit code '1' NOTE: Tasks Summary: Attempted 3 tasks of which 2 didn't need to be rerun and 1 failed. Summary: 1 task failed: openembedded-core/meta/recipes-core/base-files/base-files_3.0.14.bb, do_unpack Summary: There was 1 WARNING message shown. Summary: There were 34 ERROR messages shown, returning a non-zero exit code. OE om-gta02@shr ~/shr-core $ ls /OE/shr-core/downloads/rotation* /OE/shr-core/downloads/rotation.done /OE/shr-core/downloads/rotation.lock OE om-gta02@shr ~/shr-core $ rm -f /OE/shr-core/downloads/rotation* OE om-gta02@shr ~/shr-core $ bitbake -f -c unpack -b openembedded-core/meta/recipes-core/base-files/base-files_3.0.14.bb fails the same, but this time only .lock is created OE om-gta02@shr ~/shr-core $ ls /OE/shr-core/downloads/rotation* /OE/shr-core/downloads/rotation.lock And now I'm sort of stuck here, -f -c unpack with -b always fails like this (even after another reparse and cleansstate, without -b still works) and if I do -c cleansstate and then -c unpack _without_ -f then I'm back to first error. Layout is like this: OE om-gta02@shr ~/shr-core $ which bitbake /OE/shr-core/openembedded-core/scripts/bitbake OE om-gta02@shr ~/shr-core $ ls -ld meta* open* lrwxrwxrwx 1 bitbake bitbake 16 Nov 29 12:41 meta-angstrom -> ../meta-angstrom lrwxrwxrwx 1 bitbake bitbake 16 May 17 13:15 meta-browser -> ../meta-browser/ lrwxrwxrwx 1 bitbake bitbake 17 Apr 20 10:50 meta-chromium -> ../meta-chromium/ lrwxrwxrwx 1 bitbake bitbake 16 Nov 29 12:41 meta-handheld -> ../meta-handheld lrwxrwxrwx 1 bitbake bitbake 12 Nov 29 12:41 meta-jama -> ../meta-jama lrwxrwxrwx 1 bitbake bitbake 16 Jan 16 16:15 meta-mozilla -> ../meta-mozilla/ lrwxrwxrwx 1 bitbake bitbake 20 Nov 29 12:41 meta-openembedded -> ../meta-openembedded lrwxrwxrwx 1 bitbake bitbake 18 Nov 29 12:41 meta-smartphone -> ../meta-smartphone lrwxrwxrwx 1 bitbake bitbake 21 Nov 29 12:41 openembedded-core -> ../openembedded-core/ OE om-gta02@shr ~/shr-core $ pwd /OE/shr-core OE om-gta02@shr ~/shr-core $ env | grep BB BBFETCH2=True BB_ENV_EXTRAWHITE=MACHINE DISTRO http_proxy ftp_proxy https_proxy all_proxy ALL_PROXY no_proxy SSH_AGENT_PID SSH_AUTH_SOCK BB_SRCREV_POLICY SDKMACHINE BB_NUMBER_THREADS GIT_PROXY_COMMAND PSEUDO_DISABLED PSEUDO_BUILD OE om-gta02@shr ~/shr-core $ cat conf/bblayers.conf # LAYER_CONF_VERSION is increased each time build/conf/bblayers.conf # changes incompatibly LCONF_VERSION = "4" BBFILES ?= "" # Add your overlay location to BBLAYERS # Make sure to have a conf/layers.conf in there BBLAYERS = " \ ${TOPDIR}/meta-jama \ ${TOPDIR}/meta-smartphone/meta-shr \ ${TOPDIR}/meta-smartphone/meta-aurora \ ${TOPDIR}/meta-smartphone/meta-fso \ ${TOPDIR}/meta-openembedded/meta-oe \ ${TOPDIR}/meta-openembedded/toolchain-layer \ ${TOPDIR}/meta-openembedded/meta-efl \ ${TOPDIR}/meta-openembedded/meta-gnome \ ${TOPDIR}/meta-openembedded/meta-gpe \ ${TOPDIR}/meta-openembedded/meta-initramfs \ ${TOPDIR}/meta-openembedded/meta-systemd \ ${TOPDIR}/meta-smartphone/meta-osmocombb \ ${TOPDIR}/meta-smartphone/meta-nokia \ ${TOPDIR}/meta-smartphone/meta-htc \ ${TOPDIR}/meta-smartphone/meta-palm \ ${TOPDIR}/meta-smartphone/meta-openmoko \ ${TOPDIR}/meta-smartphone/meta-samsung \ ${TOPDIR}/meta-handheld \ ${TOPDIR}/meta-browser \ ${TOPDIR}/openembedded-core/meta \ " OE om-gta02@shr ~/shr-core $ cat setup-env # setup up the environment for a bitbake build. This is used # by the makefile and may also be directly sourced from an # interactive shell. The makefile uses 'env -i' to ensure # no variables are inherited apart from CCACHE_DISABLE and # CCACHE_DIR # # topdir must exist in the directory and define TOPDIR to # the full path name of the working directory . ./conf/topdir.conf test -e ./setup-local || echo "setup-local does not exist, please update your conf/auto.conf and copy setup-local from common directory" . ./setup-local test -n "$TOPDIR" -a -d "$TOPDIR" || { echo "environment: TOPDIR not defined" >&2 return } # # the following must match the definitions in common/conf/site.conf test ! -d "${TOPDIR}/bitbake/bin" || export PATH="${TOPDIR}/bitbake/bin:${PATH}" test ! -d "${TOPDIR}/openembedded-core/scripts" || export PATH="${TOPDIR}/openembedded-core/scripts:${PATH}" # used in bitbake wrapper for pseudodone location export BUILDDIR="${TOPDIR}" #export BBPATH="${TOPDIR}:${TOPDIR}/openembedded-core/meta" export BB_ENV_EXTRAWHITE="MACHINE DISTRO http_proxy ftp_proxy https_proxy all_proxy ALL_PROXY no_proxy SSH_AGENT_PID SSH_AUTH_SOCK BB_SRCREV_POLICY SDKMACHINE BB_NUMBER_THREADS GIT_PROXY_COMMAND PSEUDO_DISABLED PSEUDO_BUILD" #export BB_ENV_EXTRAWHITE="MACHINE DISTRO http_proxy ftp_proxy https_proxy all_proxy ALL_PROXY no_proxy SSH_AGENT_PID SSH_AUTH_SOCK BB_SRCREV_POLICY SDKMACHINE BB_NUMBER_THREADS GIT_PROXY_COMMAND PSEUDO_DISABLED PSEUDO_BUILD LD_LIBRARY_PATH" #export LD_LIBRARY_PATH="/usr/lib/gcc/x86_64-pc-linux-gnu/4.6.2/" export LANG=C export MACHINE export DISTRO #NOTE: if you add export definitions here add them below too! # # unset the following (unnecessary for the makefile, but safe) unset LC_CTYPE unset CC unset CXX unset MFLAGS unset MAKEFLAGS unset MAKE_TARGET # # make bb into a 'safe' bitbake alias bb-safe="env -i \ HOME='${HOME}' \ PYTHONPATH='${PYTHONPATH}' \ BBPATH='${BBPATH}' \ PATH='${PATH}' \ LD_LIBRARY_PATH='${LD_LIBRARY_PATH}' \ LANG='${LANG}' \ '${TOPDIR}/bitbake/bin/bitbake'" alias bb="bitbake" # # remove TOPDIR - not required unset TOPDIR if [ ${TERM:-dumb} = dumb ]; then export PS1='OE $MACHINE@$DISTRO \w \$ ' else export PS1='\[\033[01;32m\]OE $MACHINE@$DISTRO\[\033[01;34m\] \w \$\[\033[00m\] ' fi echo "Altered environment for ${MACHINE}@${DISTRO} development" What else could be usefull? 21543 8 richard.purdie 2012-05-23 10:49:05 +0000 I think I have a handle on this now. Could you run: bitbake -e | grep BBPATH and share the value of BBPATH that is being used? I have a suspicion that something in your layer config is injecting a "::" or other empty entry in there, before the meta layer is included. For example, if locally I have: BBPATH="/media/data1/build1/poky/build/meta-yocto::/media/data1/build1/poky/meta-qt3:/media/data1/build1/poky/build/meta" I can reproduce this error. I effecitvely stumbled upon this by accident trying to recreate your error. If you run "bitbake base-files -e" and look at the value of FILE, it will also likely not have an asbolute path which will corrupt FILE_DIRNAME and FILESPATH in turn. Simplistically, we can do something like: diff --git a/bitbake/lib/bb/utils.py b/bitbake/lib/bb/utils.py index 7a73419..fc389a3 100644 --- a/bitbake/lib/bb/utils.py +++ b/bitbake/lib/bb/utils.py @@ -721,6 +721,8 @@ def which(path, item, direction = 0): for p in paths: next = os.path.join(p, item) if os.path.exists(next): + if not os.path.isabs(next): + next = os.path.abspath(next) return next return "" to work around this. I'm wondering if we should error about empty directories in that variable though. If you could see if the above guesses are correct for your environment I'd appreciate it. It looks like it comes down to the exact layer config you're using. 21544 9 Martin.Jansa 2012-05-23 12:22:44 +0000 I don't have :: in BBPATH but there is non-existent dir "/OE/shr-core/openembedded" from initial BBPATH defined setup-env export BBPATH="${TOPDIR}:${TOPDIR}/openembedded" BBPATH="/OE/shr-core:/OE/shr-core/openembedded:/OE/shr-core/meta-smartphone/meta-shr:/OE/shr-core/meta-smartphone/meta-aurora:/OE/shr-core/meta-smartphone/meta-fso:/OE/shr-core/meta-openembedded/meta-oe:/OE/shr-core/meta-openembedded/toolchain-layer:/OE/shr-core/meta-openembedded/meta-efl:/OE/shr-core/meta-openembedded/meta-gnome:/OE/shr-core/meta-openembedded/meta-gpe:/OE/shr-core/meta-openembedded/meta-systemd:/OE/shr-core/meta-smartphone/meta-osmocombb:/OE/shr-core/meta-smartphone/meta-nokia:/OE/shr-core/meta-smartphone/meta-htc:/OE/shr-core/meta-smartphone/meta-palm:/OE/shr-core/meta-smartphone/meta-openmoko:/OE/shr-core/meta-smartphone/meta-samsung:/OE/shr-core/meta-browser:/OE/shr-core/openembedded-core/meta" Later today or tomorrow I'll try to remove it and then your patch. 21547 10 richard.purdie 2012-05-23 12:51:09 +0000 Do you have the value of the FILE variable handy in the failing cases (bitbake -e vase-files | grep ^FILE). I'm pretty sure the value of FILE is wrong, the question is how/why... 21548 11 Martin.Jansa 2012-05-23 13:19:05 +0000 OE om-gta02@shr ~/shr-core $ bitbake -e -b openembedded-core/meta/recipes-core/base-files/base-files_3.0.14.bb | grep ^FILE= FILE="openembedded-core/meta/recipes-core/base-files/base-files_3.0.14.bb" OE om-gta02@shr ~/shr-core $ bitbake -e base-files | grep ^FILE= FILE="/OE/shr-core/openembedded-core/meta/recipes-core/base-files/base-files_3.0.14.bb" Now I've found what could cause '::' in BBPATH meta-openembedded/meta-initramfs/conf/layer.conf and meta-handheld/conf/layer.conf are using BBPATH := "${LAYERDIR}:${BBPATH}" while all other layers are I'm using have BBPATH .= ":${LAYERDIR}" And with meta-initramfs/meta-handheld layers included I get '::' base-files.b.env:BBPATH="/OE/shr-core/meta-handheld:/OE/shr-core/meta-openembedded/meta-initramfs::/OE/shr-core/meta-jama:/OE/shr-core/meta-smartphone/meta-shr:/OE/shr-core/meta-smartphone/meta-aurora:/OE/shr-core/meta-smartphone/meta-fso:/OE/shr-core/meta-openembedded/meta-oe:/OE/shr-core/meta-openembedded/toolchain-layer:/OE/shr-core/meta-openembedded/meta-efl:/OE/shr-core/meta-openembedded/meta-gnome:/OE/shr-core/meta-openembedded/meta-gpe:/OE/shr-core/meta-openembedded/meta-systemd:/OE/shr-core/meta-smartphone/meta-osmocombb:/OE/shr-core/meta-smartphone/meta-nokia:/OE/shr-core/meta-smartphone/meta-htc:/OE/shr-core/meta-smartphone/meta-palm:/OE/shr-core/meta-smartphone/meta-openmoko:/OE/shr-core/meta-smartphone/meta-samsung:/OE/shr-core/meta-browser:/OE/shr-core/openembedded-core/meta" 21549 12 Martin.Jansa 2012-05-23 13:26:24 +0000 I can also reproduce it with BBPATH starting with ':'. 21550 13 richard.purdie 2012-05-23 13:42:59 +0000 Thanks Martin, that helps a lot. So to summarise, we have several issues: a) bb.utils.which() can sometimes not return absolute paths, My patch fixes that. b) :: can make it into BBPATH accidentally for whatever reason. We should detect and warn about that as it doesn't make sense. c) buildFile() in cooker should always be resolving paths into absolute paths anyway. The code in matchFiles therefore also needs fixing so non-absolute paths don't get passed any further within bitbake. 21551 14 Martin.Jansa 2012-05-23 13:57:59 +0000 b) Can we somehow not only warn about leading/trailing ':' and '::', but also ignore /remove them in runtime? I'm thinking about scenario when someone is using layers maintained by someone else or from some older release and cannot update/fix their layer.conf immediately. Well if this "-c unpack -b" is only case when ':' matter then it's not important, but if there are some other hidden issues with such BBPATH.. 21555 15 Martin.Jansa 2012-05-23 17:13:17 +0000 I've sent patches for layer.conf in meta-handheld: http://patchwork.openembedded.org/patch/28485/ meta-initramfs: http://patchwork.openembedded.org/patch/28487/ but there are at least 2 more in oe-core repository with the same BBPATH append: meta-hob/conf/layer.conf:BBPATH := "${BBPATH}:${LAYERDIR}" meta-skeleton/conf/layer.conf:BBPATH := "${BBPATH}:${LAYERDIR}" 21556 16 richard.purdie 2012-05-23 17:18:59 +0000 (In reply to comment #15) > I've sent patches for layer.conf in > meta-handheld: > http://patchwork.openembedded.org/patch/28485/ > meta-initramfs: > http://patchwork.openembedded.org/patch/28487/ > > but there are at least 2 more in oe-core repository with the same BBPATH > append: > meta-hob/conf/layer.conf:BBPATH := "${BBPATH}:${LAYERDIR}" > meta-skeleton/conf/layer.conf:BBPATH := "${BBPATH}:${LAYERDIR}" These aren't wrong as such, they just assume they're being included after the oe-core layer... 21558 17 Martin.Jansa 2012-05-23 18:27:15 +0000 Well, yes, they work, but order of layers in BBLAYERS together with BBFILE_PRIORITY in each layer.conf is confusing enough already for many people.. so IMHO it's better when layers respect BBLAYERS order not introduce another "variable" to priority mix. 21619 18 richard.purdie 2012-05-25 13:00:36 +0000 Patches for issues a) and c) have merged: http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=0b14db452413df414fbadf14d6dae287bc5ae9ca http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=2d3ff5e6e17c3c8e0ba000c696da54e51167f15d I'm going to leave b) to Laurentiu. I don't think fixing this up dynamically makes sense, we need to tell the user there is a problem and let them fix it. We need to detect any empty path variable, so "." and "" at any point of BBPATH are an issue. There is already a PATH check in sanity.bbclass which could easily be used as a reference for this. 22122 19 laurentiu.palcu 2012-06-13 13:24:06 +0000 Patch for b) has been also merged. For a while, we will only show a warning message and ask the user to fix the BBPATH issue, but the build process will continue. After all BBPATH issues in the tree will be fixed(let's give it a month), the warning will become a fatal error and the build will not continue unless the problem is fixed. http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=d8d95d10bd6945047b4bab9ff9a4e948c73db205