Created attachment 2958 [details] build-appliance error log Build appliance image location: http://autobuilder.yoctoproject.org/pub/releases/yocto-2.1_M2.rc1/build-appliance/ How to receate this issue: On test host(Ubuntu 14), when using build appliance to build core-image-minimal, no matter what arch is chosen, the build fails. Piece of Errorlog message(the log has been attached): Build Configuration: BB_VERSION = "1.28.0" BUILD_SYS = "x86_64-linux" NATIVELSBSTRING = "poky-2.0+snapshot-20160127" TARGET_SYS = "mips-poky-linux" MACHINE = "qemumips" DISTRO = "poky" DISTRO_VERSION = "2.0+snapshot-20160201" TUNE_FEATURES = "o32 bigendian fpu-hard mips32r2" TARGET_FPU = "" meta meta-yocto meta-yocto-bsp = "<unknown>:<unknown>" NOTE: Preparing RunQueue NOTE: Executing SetScene Tasks NOTE: Executing RunQueue Tasks NOTE: Running task 1 of 1903 (ID: 1273, /home/builder/poky/meta/recipes-devtools/quilt/quilt-native_0.64.bb, do_fetch) NOTE: Running task 2 of 1903 (ID: 1587, /home/builder/poky/meta/recipes-extended/texinfo-dummy-native/texinfo-dummy-native.bb, do_fetch) ERROR: Execution of event handler 'run_buildstats' failed Traceback (most recent call last): File "/home/builder/poky/meta/classes/buildstats.bbclass", line 154, in run_buildstats(e=<bb.build.TaskSucceeded object at 0x7f43a769f210>): elif isinstance(e, bb.build.TaskSucceeded): > write_task_data("passed", os.path.join(taskdir, e.task), e, d) if e.task == "do_rootfs": File "/home/builder/poky/meta/classes/buildstats.bbclass", line 85, in write_task_data(status='passed', logfile='/home/builder/poky/build/tmp/buildstats/20160201020056/quilt-native-0.64-r0/do_fetch', e=<bb.build.TaskSucceeded object at 0x7f43a769f210>, d=<bb.data_smart.DataSmart object at 0x7f43a7606e50>): (e.task, elapsedtime))) > cpu, iostats, resources, childres = get_process_cputime(os.getpid()) if cpu: File "/home/builder/poky/meta/classes/buildstats.bbclass", line 29, in get_process_cputime(pid=1386): iostats = {} > with open("/proc/%d/io" % pid, "r") as f: while True: IOError: [Errno 2] No such file or directory: '/proc/1386/io' ERROR: Build of do_fetch failed ERROR: Traceback (most recent call last): File "/home/builder/poky/bitbake/lib/bb/build.py", line 560, in exec_task return _exec_task(fn, task, d, quieterr) File "/home/builder/poky/bitbake/lib/bb/build.py", line 534, in _exec_task event.fire(TaskSucceeded(task, logfn, localdata), localdata) File "/home/builder/poky/bitbake/lib/bb/event.py", line 171, in fire fire_class_handlers(event, d) File "/home/builder/poky/bitbake/lib/bb/event.py", line 110, in fire_class_handlers execute_handler(name, handler, event, d) File "/home/builder/poky/bitbake/lib/bb/event.py", line 82, in execute_handler ret = handler(event) File "/home/builder/poky/meta/classes/buildstats.bbclass", line 154, in run_buildstats write_task_data("passed", os.path.join(taskdir, e.task), e, d) File "/home/builder/poky/meta/classes/buildstats.bbclass", line 85, in write_task_data cpu, iostats, resources, childres = get_process_cputime(os.getpid()) File "/home/builder/poky/meta/classes/buildstats.bbclass", line 29, in get_process_cputime with open("/proc/%d/io" % pid, "r") as f: IOError: [Errno 2] No such file or directory: '/proc/1386/io' ERROR: Execution of event handler 'run_buildstats' failed Traceback (most recent call last): File "/home/builder/poky/meta/classes/buildstats.bbclass", line 154, in run_buildstats(e=<bb.build.TaskSucceeded object at 0x7f43a764dfd0>): elif isinstance(e, bb.build.TaskSucceeded): > write_task_data("passed", os.path.join(taskdir, e.task), e, d) if e.task == "do_rootfs": File "/home/builder/poky/meta/classes/buildstats.bbclass", line 85, in write_task_data(status='passed', logfile='/home/builder/poky/build/tmp/buildstats/20160201020056/texinfo-dummy-native-1.0-r0/do_fetch', e=<bb.build.TaskSucceeded object at 0x7f43a764dfd0>, d=<bb.data_smart.DataSmart object at 0x7f43aeaaf490>): (e.task, elapsedtime))) > cpu, iostats, resources, childres = get_process_cputime(os.getpid()) if cpu: File "/home/builder/poky/meta/classes/buildstats.bbclass", line 29, in get_process_cputime(pid=1387): iostats = {} > with open("/proc/%d/io" % pid, "r") as f: while True: IOError: [Errno 2] No such file or directory: '/proc/1387/io'
seems a process does not exist any more by the time we try to obtain some info about the process...
on second thought, it seems to be a permission error, as the preceding read executed without errors: with open("/proc/%d/stat" % pid, "r") as f: but the following one failed: with open("/proc/%d/io" % pid, "r") as f: Looking at the permissions we see they have different permission: -r-------- io -r--r--r-- stat
But then again, the error explicitly says: IOError: [Errno 2] No such file or directory
Something does not make sense: The Build Appliance recipe clones poky based on commit: SRCREV ?= "992e5774714ef44bc91d45e2aaf2acfc23c72d63" //12-01-2015 Precedes 2.1_M1 but the error message come from File "/home/builder/poky/meta/classes/buildstats.bbclass", line 29, in get_process_cputime(pid=1386): iostats = {} > with open("/proc/%d/io" % pid, "r") as f: while True: IOError: [Errno 2] No such file or directory: '/proc/1386/io' adfdca4df18f4b9d261dd865243accc868eeacc7 //12-18-2015 follows 2.1_M1 How can this happen? The BA image should not contain the new buidstats.bbclass! Also, the yocto master does not seem to have 2.1M2 tag, only 2.1M1.
So there is one logical explanation: I assumed BA was built using this GIT commit: SRCREV ?= "992e5774714ef44bc91d45e2aaf2acfc23c72d63" Apparently not, as that commit does not contain the code printing the error message. So somewhere during the build SRCREV was set to some other value. The proper way is to modify the recipe as: SRCREV = "992e5774714ef44bc91d45e2aaf2acfc23c72d63" to guarantee a known BA content. Once I rebuild BA with this, there were no errors. Speaking of a known BA content, it also populates the folder "downloads" using the envar: cp -RpL ${DL_DIR}/* ${IMAGE_ROOTFS}/home/builder/poky/build/downloads/ I don't think {DL_DIR} contents are unique, and can be quite huge. So best leaving this out of the recipe. This will make the BA binaries significantly smaller as well. I am not sure what was the the reason behind the decision to copy downloaded data from host machine to BA image during the BA build as opposed to download the data on demand when it is actually needed. (The user in essence downloads the same data anyway: Either as a part of a giant BA, or as required by recipe at the time the BA is used for building a recipe) The BA should somehow reflect its contents, as there are many BAs around, for various Yocto releases. One way is to tag the actual commit number to the name. Lastly, each build of BA leaves behind almost 100G of used disk space. The files that are no longer needed should be deleted automatically by the recipe. I am going to post a patch to fix various BA issues.
Created attachment 2982 [details] Several fixes for Build Appliance recipe
I built my own build-appliance image, "bitbake build-appiance-image" then ran it, "runqemu qemux86". Once it booted, I looked at /proc and it looks like the io files are missing from this image. To address this bug, we likely need to: a) Check for the io files during the sanity check and print an warning to user that statistics collection will be impacted if they are missing b) Check for the io files before opening them c) Figure out which kernel option needs to be set and set it for our qemux86* builds (and mention the option name in a).
(In reply to comment #7) > I built my own build-appliance image, "bitbake build-appiance-image" then > ran it, "runqemu qemux86". Once it booted, I looked at /proc and it looks > like the io files are missing from this image. > > To address this bug, we likely need to: > > a) Check for the io files during the sanity check and print an warning to > user that statistics collection will be impacted if they are missing > > b) Check for the io files before opening them > > c) Figure out which kernel option needs to be set and set it for our > qemux86* builds (and mention the option name in a). The problem may be in .config: CONFIG_TASKSTATS is not set
Created attachment 2995 [details] patch to prevent from gathering I/O stats if kernel does not allow it.
I am currently testing the patch patch to prevent from gathering I/O stats if kernel does not allow it. The patch fixes a, b, and 50% of c.
This issue still exists on 2.1_M2 rc3 build. git rev: master/152914f2983c5d69001de1d46ce99547fa1e75fe
(In reply to comment #11) > This issue still exists on 2.1_M2 rc3 build. > git rev: master/152914f2983c5d69001de1d46ce99547fa1e75fe Not surprising. It needs a patch. Sent one to openembedded-core@lists.openembedded.org moments ago.
Patch merged: https://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=4b32351f2def081580fcf5fbe723d005938743ad I am closing this as resolved, but see https://bugzilla.yoctoproject.org/show_bug.cgi?id=9210
Verified it with 2.1.1 RC2 git rev: krogoth/f5da2a5913319ad6ac2141438ba1aa17576326ab