Bug 8239

Summary: strange build failure with no output
Product: [QA/Testing] Build Testing Reporter: Jussi Kukkonen <jku>
Component: generalAssignee: Mariano Lopez <mariano.lopez>
Status: VERIFIED FIXED QA Contact: Cristina Agurida <cristina-danielax.agurida>
Severity: normal    
Priority: High CC: benjamin.esquivel, bogdanx.a.voiculescu, mariano.lopez, richard.purdie
Version: unspecified   
Target Milestone: 1.9 M3   
Hardware: x86   
OS: Multiple   
Whiteboard:
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: No (bug/feature does not impact docs)
Bug Depends on:    
Bug Blocks: 8249    

Description Jussi Kukkonen 2015-09-02 07:58:37 UTC 
"bitbake core-image-lsb core-image-lsb-sdk" fails after 12 minutes with no output at all. I don't know what to make of it.


https://autobuilder.yoctoproject.org/main/builders/nightly-mips-lsb/builds/468/




. ./oe-init-build-env; bitbake -k -w 'bitbake_eventlog.json' core-image-lsb core-image-lsb-sdk
 in dir /home/pokybuild/yocto-autobuilder/yocto-worker/nightly-mips-lsb/build (timeout 100000 secs)
 watching logfiles {}
 argv: . ./oe-init-build-env; bitbake -k -w 'bitbake_eventlog.json' core-image-lsb core-image-lsb-sdk
 environment:
  ADTQA_INST_PUBLISH_DIR=adt-installer-QA
  ADTREPO_DEV_PATH=/srv/www/vhosts/adtrepo-dev/
  ADTREPO_DEV_POPULATE=True
  ADTREPO_DEV_URL=http://adtrepo-dev.yoctoproject.org/
  ADTREPO_GENERATE_DEV_INSTALLER=True
  ADTREPO_GENERATE_INSTALLER=False
  ADTREPO_PATH=/srv/www/vhosts/adtrepo.yoctoproject.org/
  ADTREPO_POPULATE=False
  ADTREPO_URL=http://adtrepo.yoctoproject.org/
  ADT_INST_PUBLISH_DIR=adt-installer
  BA_PUBLISH_DIR=build-appliance
  BB_NUMBER_THREADS=16
  BUILD_HISTORY_COLLECT=True
  BUILD_HISTORY_DIR=/srv/www/vhosts/autobuilder.yoctoproject.org/buildhistory
  BUILD_HISTORY_REPO=ssh://git@git.yoctoproject.org/poky-buildhistory
  BUILD_HISTORY_WEB=True
  BUILD_HISTORY_WEB_DIR=/home/pokybuild/buildhistory-web
  BUILD_HISTORY_WHITELIST={'git://git.yoctoproject.org/poky':['master']}
  BUILD_PUBLISH_DIR=/srv/www/vhosts/autobuilder.yoctoproject.org/pub
  CLEAN_SOURCE_DIR=False
  DEB_PUBLISH_DIR=deb
  DEVKERNEL=linux-yocto
  DEVKERNEL_MUT_REPO={'git://git.yoctoproject.org/poky-contrib':['stage/master_under_test', 'sgw/mut', 'ross/mut'], 'git://git.yoctoproject.org/poky':['ross/mut']}
  DL_DIR=/srv/www/vhosts/autobuilder.yoctoproject.org/current_sources
  EMGD_DRIVER_DIR=/tmp/yocto-autobuilder/emgd-driver
  ERROR_REPORT_COLLECT=True
  GEN_IMG_MD5=True
  HOME=/home/pokybuild
  IMAGE_FSTYPES=tar.gz
  IPK_PUBLISH_DIR=ipk
  LANG=en_US.UTF-8
  LOGNAME=pokybuild
  LSB_SSTATE_DIR=/tmp/yocto-autobuilder/lsb-sstate
  LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arj=01;31:*.taz=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lz=01;31:*.xz=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.jpg=01;35:*.jpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.axv=01;35:*.anx=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.axa=00;36:*.oga=00;36:*.spx=00;36:*.xspf=00;36:
  MACHINE_PUBLISH_DIR=machines
  MAIL=/var/mail/pokybuild
  MAINTAIN_PERSISTDB=True
  MAKEFLAGS=
  MAKELEVEL=1
  MARKED_RELEASE_DIR=releases
  MFLAGS=
  OGIT_MIRROR_DIR=/home/pokybuild/git/mirror
  OGIT_TRASH_CRON_TIME=0 0 * * *
  OGIT_TRASH_DIR=/home/pokybuild/git/trash
  OGIT_TRASH_NICE_LEVEL=19
  OLDPWD=/home/pokybuild
  OPTIMIZED_GIT_CLONE=True
  PARALLEL_MAKE=16
  PATH=/home/pokybuild/yocto-autobuilder/bin:/home/pokybuild/yocto-autobuilder/yocto-autobuilder/scripts:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
  PERSISTDB_DIR=/srv/www/vhosts/autobuilder.yoctoproject.org/persistdb
  PUBLISH_BUILDS=True
  PUBLISH_SOURCE_MIRROR=True
  PUBLISH_SSTATE=False
  PWD=/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-mips-lsb/build
  PYTHONPATH=/home/pokybuild/yocto-autobuilder/lib/python2.7/site-packages/autobuilder/:/home/pokybuild/yocto-autobuilder/lib/python2.7/site-packages/:/home/pokybuild/yocto-autobuilder/lib/python2.7/site-packages/autobuilder/buildsteps:
  QA_MAIL_BCC=
  QA_MAIL_CC=elizabeth.flanagan@intel.com, alexandru.c.georgescu@intel.com, otavio@ossystems.com.br, yi.zhao@windriver.com
  QA_MAIL_SIG=\nThis is an automated message from\nThe Yocto Project Autobuilder\nGit: git://git.yoctoproject.org/yocto-autobuilder\nEmail: elizabeth.flanagan@intel.com
  QA_MAIL_TO=yocto@yoctoproject.org
  QEMU_PUBLISH_DIR=/machines/qemu
  RELEASE_PUBLISH_DIR=/srv/www/vhosts/autobuilder.yoctoproject.org/pub/releases
  RESOLVED_TRIGGERED_HEAD=True
  RESOLVE_TRIGGERED_HEAD=True
  RPM_PUBLISH_DIR=rpm
  SHELL=/bin/bash
  SHLVL=1
  SOURCE_DL_DIR=/srv/www/vhosts/autobuilder.yoctoproject.org/current_sources
  SOURCE_MIRROR_URL=/srv/www/vhosts/autobuilder.yoctoproject.org/pub/sources
  SOURCE_PUBLISH_DIR=/srv/www/vhosts/autobuilder.yoctoproject.org/current_sources
  SOURCE_SSTATE_DIR=/srv/www/vhosts/autobuilder.yoctoproject.org/pub/sstate
  SSTATE_DIR=/srv/www/vhosts/autobuilder.yoctoproject.org/pub/sstate
  SSTATE_PUBLISH_DIR=/srv/www/vhosts/autobuilder.yoctoproject.org/pub/sstate
  TERM=xterm
  TOASTER_UPLOAD_URL=http://toaster-ab-dev.yoctoproject.org/orm/eventfile
  USER=pokybuild
  WEB_ROOT=/srv/www/vhosts/autobuilder.yoctoproject.org/
  WEB_URL=http://autobuilder.yoctoproject.org/
  WORKERBASEDIR=/home/pokybuild/yocto-autobuilder/yocto-worker
  X8664TC_PUBLISH_DIR=toolchain/x86_64
  X86TC_PUBLISH_DIR=toolchain/i686
  XDG_SESSION_COOKIE=cce5631d6cf011d310ac1e7d532dd80e-1438592224.258724-563411987
  YOCTO_AB_CONFIG=/home/pokybuild/yocto-autobuilder/buildset-config/yoctoAB.conf
  _=./yocto-start-autobuilder
 using PTY: False

### Shell environment set up for builds. ###

You can now run 'bitbake <target>'

Common targets are:
    core-image-minimal
    core-image-sato
    meta-toolchain
    adt-installer
    meta-ide-support

You can also run generated qemu images with a command like 'runqemu qemux86'
program finished with exit code 1
elapsedTime=702.556848
Comment 1 Richard Purdie 2015-09-03 14:26:38 UTC 
There are two parts to this. It took 12 minutes since it was running chmod -st -R in the environment setup script. Bitbake was actually exiting immediately and should have said bitbake.lock couldn't be acquired. The -w option to bitbake has a bug and was preventing this from being shown.

The chmod -st -R issue is fixed with http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=d8caef4f6e38efb739866a1de38f3bae8f1754ed

There is a bitbake patch for the missing error message.

The reason bitbake.lock is being held is that the autobuilder tried to kill a previous task yet the runqemu process remained. We likely need to improve the SIGTERM handling there to ensure that does get killed. The bug should likely remain open until this last issue is resolved.
Comment 2 Richard Purdie 2015-09-03 14:28:25 UTC 
A likely good reproducer for this is "bitabke -c testimage core-image-sato", then Ctrl+C it after qemu starts and notice that qemu remains running. We need to fix it so qemu exits.
Comment 3 Mariano Lopez 2015-09-07 19:43:40 UTC 
Sent patch to the list to get the SIGTERM in testimage class
Comment 5 Cristina Agurida 2015-10-19 11:36:59 UTC 
Verified on master: a9053ac0829eb01f6f002587854d01e47a78e6f0