7954 2015-07-03 14:45:07 +0000 pigz has resource race issues 2020-01-23 16:10:47 +0000 1 1 7 Build System, Metadata & Runtime OE-Core devtools / tool chain 1.8 x86 Multiple RESOLVED WORKSFORME Medium normal Future 1 richard.purdie unassigned meta.mr.watcher meta.watcher mhalstead randy.macleod ross.burton --- --- No (bug/feature does not impact docs) oldest_to_newest 52131 0 richard.purdie 2015-07-03 14:45:07 +0000 A series of builds hung on centos6, a machine which has our 1.8 SDK installed in /opt/poky and has pigz as gzip. e.g. https://autobuilder.yoctoproject.org/main/builders/nightly-qa-skeleton/builds/369/steps/BuildImages/logs/stdio it would appear the machine could no longer fork (builds were running on the kernel AB and the main AB) and this resulted in various hung gzip processes: 300 ? S 0:00 [crypto/12] 301 ? S 0:00 [crypto/13] 302 ? S 0:00 [crypto/14] 303 ? S 0:00 [crypto/15] 304 ? S 0:00 [crypto/16] 305 ? S 0:00 [crypto/17] 306 ? S 0:00 [crypto/18] 307 ? S 0:00 [crypto/19] 308 ? S 0:00 [crypto/20] 309 ? S 0:00 [crypto/21] 310 ? S 0:00 [crypto/22] 311 ? S 0:00 [crypto/23] 319 ? S 0:00 [kthrotld/0] 320 ? S 0:00 [kthrotld/1] 321 ? S 0:00 [kthrotld/2] 322 ? S 0:00 [kthrotld/3] 323 ? S 0:00 [kthrotld/4] 324 ? S 0:00 [kthrotld/5] 325 ? S 0:00 [kthrotld/6] 326 ? S 0:00 [kthrotld/7] 327 ? S 0:00 [kthrotld/8] 328 ? S 0:00 [kthrotld/9] 329 ? S 0:00 [kthrotld/10] 330 ? S 0:00 [kthrotld/11] 331 ? S 0:00 [kthrotld/12] 332 ? S 0:00 [kthrotld/13] 333 ? S 0:00 [kthrotld/14] 334 ? S 0:00 [kthrotld/15] 335 ? S 0:00 [kthrotld/16] 336 ? S 0:00 [kthrotld/17] 337 ? S 0:00 [kthrotld/18] 338 ? S 0:00 [kthrotld/19] 339 ? S 0:00 [kthrotld/20] 340 ? S 0:00 [kthrotld/21] 341 ? S 0:00 [kthrotld/22] 342 ? S 0:00 [kthrotld/23] 344 ? S 0:00 [kpsmoused] 345 ? S 0:00 [usbhid_resumer] 346 ? S 0:00 [deferwq] 378 ? S 0:00 [kdmremove] 379 ? S 0:00 [kstriped] 527 ? S 0:00 [scsi_eh_0] 563 ? S 0:00 [scsi_eh_1] 564 ? S 0:00 [scsi_eh_2] 648 ? S 0:31 [kdmflush] 649 ? S 0:00 [kdmflush] 667 ? S 1:09 [jbd2/dm-0-8] 668 ? S 0:00 [ext4-dio-unwrit] 754 ? S<s 0:00 /sbin/udevd -d 793 ? SN 0:00 /bin/sh /home/pokybuild/yocto-autobuilder/yocto-controller/nightly-qa-logrotate/build/build/tmp/work/core2-64-poky-linux/rpm/5.4.14-r0/temp/run.sstate_create_package.12951 843 ? SNs 0:00 /opt/poky/1.8+snapshot/sysroots/x86_64-pokysdk-linux/usr/bin/python2.7 /home/pokybuild/yocto-autobuilder-kernel/yocto-worker/kernel-x86-64/build/bitbake/bin/bitbake-worker decafbad 847 ? SN 0:00 /bin/sh /home/pokybuild/yocto-autobuilder-kernel/yocto-worker/kernel-x86-64/build/build/tmp/sysroots/x86_64-linux/usr/bin/opkg-build dbus-test-ptest /home/pokybuild/yocto-autobuilder-k 927 ? SN 0:00 tar -czf /srv/www/vhosts/autobuilder.yoctoproject.org/pub/sstate//21/sstate:rpm:core2-64-poky-linux:5.4.14:r0:core2-64:3:21ea08e26a0dbf93e371565aea1efeeb_package_write_rpm.tgz.PATbucE7 948 ? SNl 0:00 gzip 1035 ? S 0:00 [edac-poller] 1556 ? SN 0:02 [kipmi0] 1593 ? S 0:00 [kvm-irqfd-clean] 1608 ? S 0:41 [kdmflush] 1646 ? S 0:00 [jbd2/sda1-8] 1647 ? S 0:00 [ext4-dio-unwrit] 1648 ? S 22:42 [jbd2/dm-2-8] 1649 ? S 0:00 [ext4-dio-unwrit] 1678 ? SN 0:00 /bin/sh /home/pokybuild/yocto-autobuilder-kernel/yocto-worker/kernel-x86-64/build/build/tmp/sysroots/x86_64-linux/usr/bin/opkg-build atk-locale-bg /home/pokybuild/yocto-autobuilder-ker 1691 ? S 0:01 [kauditd] 1850 ? S 1:40 [flush-253:0] 1874 ? SNs 0:07 /opt/poky/1.8+snapshot/sysroots/x86_64-pokysdk-linux/usr/bin/python2.7 /home/pokybuild/yocto-autobuilder/yocto-controller/ptest-x86/build/bitbake/bin/bitbake-worker decafbad 1930 ? S<sl 0:06 auditd 1950 ? Sl 0:01 /sbin/rsyslogd -i /var/run/syslogd.pid -c 5 1980 ? Ss 1:48 irqbalance --pid=/var/run/irqbalance.pid 1996 ? Ss 0:07 rpcbind 2016 ? Ss 0:00 rpc.statd 2099 ? SN 0:00 /bin/sh /home/pokybuild/yocto-autobuilder/yocto-controller/nightly-qa-skeleton/build/build/tmp/sysroots/x86_64-linux/usr/bin/opkg-build perl-module-pod-man /home/pokybuild/yocto-autobu 2157 ? SN 0:00 tar -cz --format=gnu -f /home/pokybuild/yocto-autobuilder/yocto-controller/nightly-qa-skeleton/build/build/tmp/work/core2-64-poky-linux/perl/5.22.0-r0/deploy-ipks/core2-64/IPKG_BUILD.3 2201 ? SN 0:00 gzip 3966 ? SN 0:00 /bin/sh /home/pokybuild/yocto-autobuilder-kernel/yocto-worker/kernel-x86-64/build/build/tmp/sysroots/x86_64-linux/usr/bin/opkg-build dbus-test-ptest /home/pokybuild/yocto-autobuilder-k 3987 ? SN 0:00 /bin/sh /home/pokybuild/yocto-autobuilder/yocto-controller/ptest-x86/build/build/tmp/work/core2-32-poky-linux/libproxy/0.4.11-r0/temp/run.sstate_create_package.11794 4016 ? SN 0:00 tar -X /home/pokybuild/yocto-autobuilder-kernel/yocto-worker/kernel-x86-64/build/build/tmp/work/core2-64-poky-linux/dbus-test/1.8.18-r0/deploy-ipks/core2-64/IPKG_BUILD.847/tarX -cz --f 4049 ? SNl 0:00 gzip 4256 ? SN 0:00 tar -czf /srv/www/vhosts/autobuilder.yoctoproject.org/pub/sstate//b2/sstate:libproxy:core2-32-poky-linux:0.4.11:r0:core2-32:3:b2ba1cc894aa6a38e8deafe83d6963ee_package.tgz.iI6gwuxl pack 4298 ? SNl 0:00 gzip 4829 ? Ss 0:07 /home/pokybuild/yocto-autobuilder-kernel/yocto-worker/kernel-x86-64/build/build/tmp/sysroots/x86_64-linux/usr/bin/pseudo -d 4861 ? SNs 0:00 /opt/poky/1.8+snapshot/sysroots/x86_64-pokysdk-linux/usr/bin/python2.7 /home/pokybuild/yocto-autobuilder-kernel/yocto-worker/kernel-x86-64/build/bitbake/bin/bitbake-worker decafbad 4993 ? SN 0:00 /bin/sh /home/pokybuild/yocto-autobuilder-kernel/yocto-worker/kernel-x86-64/build/build/tmp/sysroots/x86_64-linux/usr/bin/opkg-build atk-locale-bg /home/pokybuild/yocto-autobuilder-ker 5098 ? SN 0:00 tar -cz --format=gnu -f /home/pokybuild/yocto-autobuilder-kernel/yocto-worker/kernel-x86-64/build/build/tmp/work/core2-64-poky-linux/atk/2.16.0-r0/deploy-ipks/core2-64/IPKG_BUILD.1678/ 5137 ? SNl 0:00 gzip 5921 ? S 0:00 /home/pokybuild/yocto-autobuilder/yocto-controller/nightly-qa-skeleton/build/build/tmp/sysroots/x86_64-linux/usr/bin/pseudo bitbake-worker decafbad 5926 ? S 1:04 /opt/poky/1.8+snapshot/sysroots/x86_64-pokysdk-linux/usr/bin/python2.7 /home/pokybuild/yocto-autobuilder/yocto-controller/nightly-qa-skeleton/build/bitbake/bin/bitbake-worker decafbad 5972 ? S 0:00 /home/pokybuild/yocto-autobuilder/yocto-controller/nightly-qa-logrotate/build/build/tmp/sysroots/x86_64-linux/usr/bin/pseudo bitbake-worker decafbad 5999 ? S 1:27 /opt/poky/1.8+snapshot/sysroots/x86_64-pokysdk-linux/usr/bin/python2.7 /home/pokybuild/yocto-autobuilder/yocto-controller/nightly-qa-logrotate/build/bitbake/bin/bitbake-worker decafbad 6652 ? Ss 0:00 dbus-daemon --system 6684 ? S 0:34 [rpciod/0] 6685 ? S 0:07 [rpciod/1] 6686 ? S 0:20 [rpciod/2] 6687 ? S 0:07 [rpciod/3] 6688 ? S 0:08 [rpciod/4] 6689 ? S 0:07 [rpciod/5] 6690 ? S 0:08 [rpciod/6] 6691 ? S 0:06 [rpciod/7] 6692 ? S 0:07 [rpciod/8] 6693 ? S 0:06 [rpciod/9] 6694 ? S 0:07 [rpciod/10] 6695 ? S 0:06 [rpciod/11] 6696 ? S 0:08 [rpciod/12] 6697 ? S 0:06 [rpciod/13] 6698 ? S 16:24 [rpciod/14] 6699 ? S 0:06 [rpciod/15] 6700 ? S 0:07 [rpciod/16] 6701 ? S 0:06 [rpciod/17] 6702 ? S 0:07 [rpciod/18] 6703 ? S 0:06 [rpciod/19] 6704 ? S 0:06 [rpciod/20] 6705 ? S 0:06 [rpciod/21] 6706 ? S 0:06 [rpciod/22] 6707 ? S 0:06 [rpciod/23] 6709 ? S< 0:00 [kslowd000] 6710 ? S< 0:00 [kslowd001] 6711 ? S 5:43 [nfsiod] 6712 ? S 0:45 [nfsv4.0-svc] 6722 ? Ss 0:00 /usr/sbin/acpid 6732 ? Ssl 0:11 hald 6733 ? S 0:00 hald-runner 6776 ? S 0:00 hald-addon-input: Listening on /dev/input/event2 /dev/input/event0 6777 ? S 0:00 hald-addon-acpi: listening on acpid socket /var/run/acpid.socket 6797 ? Ss 0:00 /usr/sbin/sshd 6806 ? Ss 0:00 xinetd -stayalive -pidfile /var/run/xinetd.pid 6815 ? Ss 0:31 ntpd -u ntp:ntp -p /var/run/ntpd.pid -g 6892 ? Ss 0:06 /usr/libexec/postfix/master 6902 ? S 0:01 qmgr -l -t fifo -u 6917 ? Ss 0:00 /usr/sbin/abrtd 6925 ? Ss 0:00 abrt-dump-oops -d /var/spool/abrt -rwx /var/log/messages 6940 ? S 0:22 /bin/bash /usr/sbin/ksmtuned 6951 ? Ss 0:04 crond 6964 ? Ss 0:00 /usr/sbin/atd 7777 ? S 0:24 /usr/bin/Xvnc :1 -desktop centos6.osl.yoctoproject.org:1 (pokybuild) -auth /home/pokybuild/.Xauthority -geometry 1024x768 -rfbwait 30000 -rfbauth /home/pokybuild/.vnc/passwd -rfbport 5 7790 tty1 Ss+ 0:00 /sbin/mingetty /dev/tty1 7792 tty2 Ss+ 0:00 /sbin/mingetty /dev/tty2 7794 tty3 Ss+ 0:00 /sbin/mingetty /dev/tty3 7797 tty4 Ss+ 0:00 /sbin/mingetty /dev/tty4 7799 tty5 Ss+ 0:00 /sbin/mingetty /dev/tty5 7801 tty6 Ss+ 0:00 /sbin/mingetty /dev/tty6 7805 ? S< 0:00 /sbin/udevd -d 7806 ? S< 0:00 /sbin/udevd -d 7812 ? Sl 0:00 /usr/sbin/console-kit-daemon --no-daemon 7889 ? S 0:00 /usr/bin/xterm -geometry 80x50-50+150 7892 pts/0 Ss+ 0:00 bash 8055 ? SN 0:00 /bin/sh /home/pokybuild/yocto-autobuilder-kernel/yocto-worker/kernel-x86-64/build/build/tmp/work/core2-64-poky-linux/valgrind/3.10.1-r1/temp/run.sstate_create_package.19387 8275 ? SN 0:00 tar -czf /srv/www/vhosts/autobuilder.yoctoproject.org/pub/sstate//72/sstate:valgrind:core2-64-poky-linux:3.10.1:r1:core2-64:3:726a9990e8698e747af1ab05912ddd13_package_write_ipk.tgz.I1y 8325 ? SNl 0:00 gzip 9393 ? SN 0:00 /bin/sh /home/pokybuild/yocto-autobuilder-kernel/yocto-worker/kernel-ppc/build/build/tmp/work/ppc7400-poky-linux/util-linux/2.26.2-r0/temp/run.sstate_create_package.18323 9613 ? Ss 0:00 sshd: halstead [priv] 9724 ? SN 0:00 tar -czf /srv/www/vhosts/autobuilder.yoctoproject.org/pub/sstate//4a/sstate:util-linux:ppc7400-poky-linux:2.26.2:r0:ppc7400:3:4ac91c5cc5af51776b833a7e8b5090cc_package_write_ipk.tgz.tOi 9765 ? SNl 0:00 gzip 9808 ? Ss 1:39 /home/pokybuild/yocto-autobuilder/yocto-controller/ptest-x86/build/build/tmp/sysroots/x86_64-linux/usr/bin/pseudo -d 9862 ? S 0:05 sshd: halstead@pts/2 9879 pts/2 Ss+ 0:00 -bash 10060 ? Ss 0:02 /home/pokybuild/yocto-autobuilder-kernel/yocto-worker/kernel-x86-64/build/build/tmp/sysroots/x86_64-linux/usr/bin/pseudo -d 10677 ? SNs 0:05 /opt/poky/1.8+snapshot/sysroots/x86_64-pokysdk-linux/usr/bin/python2.7 /home/pokybuild/yocto-autobuilder/yocto-controller/nightly-qa-skeleton/build/bitbake/bin/bitbake-worker decafbad 11085 ? SN 0:00 /bin/sh /home/pokybuild/yocto-autobuilder/yocto-controller/ptest-x86/build/build/tmp/sysroots/x86_64-linux/usr/bin/opkg-build perl-module-extutils-testlib /home/pokybuild/yocto-autobui 11134 ? SN 0:00 /bin/sh /home/pokybuild/yocto-autobuilder-kernel/yocto-worker/kernel-x86-64/build/build/tmp/sysroots/x86_64-linux/usr/bin/opkg-build lttng-tools-doc /home/pokybuild/yocto-autobuilder-k 11254 ? SN 0:00 /bin/sh /home/pokybuild/yocto-autobuilder/yocto-controller/nightly-qa-logrotate/build/build/tmp/work/core2-64-poky-linux/rpm/5.4.14-r0/temp/run.sstate_create_package.15136 11538 ? SN 0:00 tar -czf /srv/www/vhosts/autobuilder.yoctoproject.org/pub/sstate//e4/sstate:rpm:core2-64-poky-linux:5.4.14:r0:core2-64:3:e4d7e7418ee2470b94c899b8a279a51b_populate_sysroot.tgz.9gs1E9W9 11576 ? SNl 0:00 gzip 11794 ? SNs 0:05 /opt/poky/1.8+snapshot/sysroots/x86_64-pokysdk-linux/usr/bin/python2.7 /home/pokybuild/yocto-autobuilder/yocto-controller/ptest-x86/build/bitbake/bin/bitbake-worker decafbad 11849 ? Ss 0:02 /home/pokybuild/yocto-autobuilder/yocto-controller/ptest-x86/build/build/tmp/sysroots/x86_64-linux/usr/bin/pseudo -d 12951 ? SNs 0:05 /opt/poky/1.8+snapshot/sysroots/x86_64-pokysdk-linux/usr/bin/python2.7 /home/pokybuild/yocto-autobuilder/yocto-controller/nightly-qa-logrotate/build/bitbake/bin/bitbake-worker decafbad 12981 ? Ss 0:07 /home/pokybuild/yocto-autobuilder/yocto-controller/nightly-qa-logrotate/build/build/tmp/sysroots/x86_64-linux/usr/bin/pseudo -d 13054 ? SN 0:00 /bin/sh /home/pokybuild/yocto-autobuilder/yocto-controller/ptest-x86/build/build/tmp/sysroots/x86_64-linux/usr/bin/opkg-build perl-module-extutils-testlib /home/pokybuild/yocto-autobui 13066 ? SN 0:00 tar -cz --format=gnu -f /home/pokybuild/yocto-autobuilder/yocto-controller/ptest-x86/build/build/tmp/work/core2-32-poky-linux/perl/5.22.0-r0/deploy-ipks/core2-32/IPKG_BUILD.11085/contr 13067 ? SN 0:00 /bin/sh /home/pokybuild/yocto-autobuilder-kernel/yocto-worker/kernel-x86-64/build/build/tmp/sysroots/x86_64-linux/usr/bin/opkg-build lttng-tools-doc /home/pokybuild/yocto-autobuilder-k 13087 ? SN 0:00 gzip 13096 ? SN 0:00 tar -X /home/pokybuild/yocto-autobuilder-kernel/yocto-worker/kernel-x86-64/build/build/tmp/work/core2-64-poky-linux/lttng-tools/2.6.0-r0/deploy-ipks/core2-64/IPKG_BUILD.11134/tarX -cz 13127 ? SNl 0:00 gzip 14410 ? SN 0:00 /bin/sh /home/pokybuild/yocto-autobuilder/yocto-controller/nightly-qa-skeleton/build/build/tmp/work/core2-64-poky-linux/glib-2.0/1_2.44.1-r0/temp/run.sstate_create_package.10677 14502 ? SN 0:00 tar -czf /srv/www/vhosts/autobuilder.yoctoproject.org/pub/sstate//fe/sstate:glib-2.0:core2-64-poky-linux:2.44.1:r0:core2-64:3:feeda92e608450c9fe4e848cb53cdca6_packagedata.tgz.iIGnBGVh 14519 ? SNl 0:00 gzip 15136 ? SNs 0:05 /opt/poky/1.8+snapshot/sysroots/x86_64-pokysdk-linux/usr/bin/python2.7 /home/pokybuild/yocto-autobuilder/yocto-controller/nightly-qa-logrotate/build/bitbake/bin/bitbake-worker decafbad 15564 ? S 0:00 /home/pokybuild/yocto-autobuilder/yocto-controller/ptest-x86/build/build/tmp/sysroots/x86_64-linux/usr/bin/pseudo bitbake-worker decafbad 15569 ? S 1:39 /opt/poky/1.8+snapshot/sysroots/x86_64-pokysdk-linux/usr/bin/python2.7 /home/pokybuild/yocto-autobuilder/yocto-controller/ptest-x86/build/bitbake/bin/bitbake-worker decafbad 16771 ? SNs 0:34 /opt/poky/1.8+snapshot/sysroots/x86_64-pokysdk-linux/usr/bin/python2.7 /home/pokybuild/yocto-autobuilder-kernel/yocto-worker/kernel-ppc/build/bitbake/bin/bitbake-worker decafbad 17268 ? S 0:00 pickup -l -t fifo -u 17764 ? S 0:00 [flush-253:2] 17777 ? Ss 0:00 sshd: rpurdie [priv] 17779 ? R 0:00 sshd: rpurdie@pts/4 17780 pts/4 Ss 0:00 -bash 17939 ? S 0:00 sleep 60 17940 pts/4 R+ 0:00 ps ax 18296 ? Ss 0:07 /home/pokybuild/yocto-autobuilder-kernel/yocto-worker/kernel-ppc/build/build/tmp/sysroots/x86_64-linux/usr/bin/pseudo -d 18323 ? SNs 0:06 /opt/poky/1.8+snapshot/sysroots/x86_64-pokysdk-linux/usr/bin/python2.7 /home/pokybuild/yocto-autobuilder-kernel/yocto-worker/kernel-ppc/build/bitbake/bin/bitbake-worker decafbad 18605 ? Ss 0:03 /home/pokybuild/yocto-autobuilder-kernel/yocto-worker/kernel-x86-64/build/build/tmp/sysroots/x86_64-linux/usr/bin/pseudo -d 18830 ? SNs 0:00 /opt/poky/1.8+snapshot/sysroots/x86_64-pokysdk-linux/usr/bin/python2.7 /home/pokybuild/yocto-autobuilder-kernel/yocto-worker/kernel-x86-64/build/bitbake/bin/bitbake-worker decafbad 18923 ? SNs 0:00 /opt/poky/1.8+snapshot/sysroots/x86_64-pokysdk-linux/usr/bin/python2.7 /home/pokybuild/yocto-autobuilder-kernel/yocto-worker/kernel-x86-64/build/bitbake/bin/bitbake-worker decafbad 19387 ? SNs 0:04 /opt/poky/1.8+snapshot/sysroots/x86_64-pokysdk-linux/usr/bin/python2.7 /home/pokybuild/yocto-autobuilder-kernel/yocto-worker/kernel-x86-64/build/bitbake/bin/bitbake-worker decafbad 19530 ? SNs 0:04 /opt/poky/1.8+snapshot/sysroots/x86_64-pokysdk-linux/usr/bin/python2.7 /home/pokybuild/yocto-autobuilder/yocto-controller/ptest-x86/build/bitbake/bin/bitbake-worker decafbad 22455 ? Ss 0:02 /home/pokybuild/yocto-autobuilder-kernel/yocto-worker/kernel-x86-64/build/build/tmp/sysroots/x86_64-linux/usr/bin/pseudo -d 23432 ? S 0:00 /bin/sh -c . ./oe-init-build-env; bitbake -k core-image-kernel-dev 23506 ? S 0:57 /opt/poky/1.8+snapshot/sysroots/x86_64-pokysdk-linux/usr/bin/python2.7 /home/pokybuild/yocto-autobuilder-kernel/yocto-worker/kernel-x86-64/build/bitbake/bin/bitbake -k core-image-kerne 24237 ? Sl 8:13 /opt/poky/1.8+snapshot/sysroots/x86_64-pokysdk-linux/usr/bin/python2.7 /home/pokybuild/yocto-autobuilder-kernel/yocto-worker/kernel-x86-64/build/bitbake/bin/bitbake -k core-image-kerne 25307 ? S 0:00 /bin/sh -c . ./oe-init-build-env; bitbake -k core-image-kernel-dev 25357 ? S 1:02 /opt/poky/1.8+snapshot/sysroots/x86_64-pokysdk-linux/usr/bin/python2.7 /home/pokybuild/yocto-autobuilder-kernel/yocto-worker/kernel-ppc/build/bitbake/bin/bitbake -k core-image-kernel-d 25726 ? Sl 8:29 /opt/poky/1.8+snapshot/sysroots/x86_64-pokysdk-linux/usr/bin/python2.7 /home/pokybuild/yocto-autobuilder-kernel/yocto-worker/kernel-ppc/build/bitbake/bin/bitbake -k core-image-kernel-d 26442 ? SNs 0:00 /opt/poky/1.8+snapshot/sysroots/x86_64-pokysdk-linux/usr/bin/python2.7 /home/pokybuild/yocto-autobuilder-kernel/yocto-worker/kernel-x86-64/build/bitbake/bin/bitbake-worker decafbad 26671 ? SNs 0:07 /opt/poky/1.8+snapshot/sysroots/x86_64-pokysdk-linux/usr/bin/python2.7 /home/pokybuild/yocto-autobuilder/yocto-controller/ptest-x86/build/bitbake/bin/bitbake-worker decafbad 28288 ? S 0:52 /opt/poky/1.8+snapshot/sysroots/x86_64-pokysdk-linux/usr/bin/python2.7 /home/pokybuild/yocto-autobuilder-kernel/yocto-worker/kernel-x86-64/build/bitbake/bin/bitbake-worker decafbad 29082 ? Sl 0:31 /usr/bin/python /home/pokybuild/yocto-autobuilder/bin/twistd --no_save -y buildbot.tac 29088 ? Sl 0:00 /opt/poky/1.8+snapshot/sysroots/x86_64-pokysdk-linux/usr/bin/python2.7 bin/buildworker-janitor /home/pokybuild/yocto-autobuilder/config/autobuilder.conf 29099 ? SNs 0:11 /opt/poky/1.8+snapshot/sysroots/x86_64-pokysdk-linux/usr/bin/python2.7 /home/pokybuild/yocto-autobuilder/yocto-controller/nightly-qa-skeleton/build/bitbake/bin/bitbake-worker decafbad 29127 ? SNs 0:11 /opt/poky/1.8+snapshot/sysroots/x86_64-pokysdk-linux/usr/bin/python2.7 /home/pokybuild/yocto-autobuilder/yocto-controller/nightly-qa-skeleton/build/bitbake/bin/bitbake-worker decafbad 29253 ? S 0:59 /opt/poky/1.8+snapshot/sysroots/x86_64-pokysdk-linux/usr/bin/python2.7 /home/pokybuild/yocto-autobuilder-kernel/yocto-worker/kernel-ppc/build/bitbake/bin/bitbake-worker decafbad 29782 ? SN 0:00 /bin/sh /home/pokybuild/yocto-autobuilder-kernel/yocto-worker/kernel-ppc/build/build/tmp/sysroots/x86_64-linux/usr/bin/opkg-build glibc-binary-localedata-es-py.iso-8859-1 /home/pokybui 29783 ? S 0:00 /home/pokybuild/yocto-autobuilder-kernel/yocto-worker/kernel-x86-64/build/build/tmp/sysroots/x86_64-linux/usr/bin/pseudo bitbake-worker decafbad 29787 ? S 0:46 /opt/poky/1.8+snapshot/sysroots/x86_64-pokysdk-linux/usr/bin/python2.7 /home/pokybuild/yocto-autobuilder-kernel/yocto-worker/kernel-x86-64/build/bitbake/bin/bitbake-worker decafbad 29928 ? Ss 1:54 /home/pokybuild/yocto-autobuilder/yocto-controller/nightly-qa-skeleton/build/build/tmp/sysroots/x86_64-linux/usr/bin/pseudo -d 29946 ? Sl 1:03 /usr/bin/python /home/pokybuild/yocto-autobuilder-kernel/bin/twistd --no_save -y buildbot.tac 29951 ? Sl 0:12 /usr/bin/python /home/pokybuild/yocto-autobuilder-kernel/bin/twistd --no_save -y buildbot.tac 29952 ? Sl 0:00 /opt/poky/1.8+snapshot/sysroots/x86_64-pokysdk-linux/usr/bin/python2.7 bin/buildworker-janitor /home/pokybuild/yocto-autobuilder-kernel/config/autobuilder.conf 30530 ? S 0:00 /bin/sh -c . ./oe-init-build-env; bitbake -k core-image-sato 30552 ? S 0:00 /bin/sh -c . ./oe-init-build-env; bitbake -k core-image-sato 30602 ? S 1:09 /opt/poky/1.8+snapshot/sysroots/x86_64-pokysdk-linux/usr/bin/python2.7 /home/pokybuild/yocto-autobuilder/yocto-controller/nightly-qa-logrotate/build/bitbake/bin/bitbake -k core-image-s 30621 ? S 1:09 /opt/poky/1.8+snapshot/sysroots/x86_64-pokysdk-linux/usr/bin/python2.7 /home/pokybuild/yocto-autobuilder/yocto-controller/nightly-qa-skeleton/build/bitbake/bin/bitbake -k core-image-sa 30656 ? S 0:00 /bin/sh -c . ./oe-init-build-env; bitbake -k -w 'bitbake_eventlog.json' core-image-sato-sdk 30691 ? S 1:16 /opt/poky/1.8+snapshot/sysroots/x86_64-pokysdk-linux/usr/bin/python2.7 /home/pokybuild/yocto-autobuilder/yocto-controller/ptest-x86/build/bitbake/bin/bitbake -k -w bitbake_eventlog.jso 30716 ? Sl 10:52 /opt/poky/1.8+snapshot/sysroots/x86_64-pokysdk-linux/usr/bin/python2.7 /home/pokybuild/yocto-autobuilder/yocto-controller/nightly-qa-logrotate/build/bitbake/bin/bitbake -k core-image-s 30773 ? Sl 10:02 /opt/poky/1.8+snapshot/sysroots/x86_64-pokysdk-linux/usr/bin/python2.7 /home/pokybuild/yocto-autobuilder/yocto-controller/nightly-qa-skeleton/build/bitbake/bin/bitbake -k core-image-sa 30864 ? Sl 14:07 /opt/poky/1.8+snapshot/sysroots/x86_64-pokysdk-linux/usr/bin/python2.7 /home/pokybuild/yocto-autobuilder/yocto-controller/ptest-x86/build/bitbake/bin/bitbake -k -w bitbake_eventlog.jso 31045 ? Ss 2:27 /home/pokybuild/yocto-autobuilder-kernel/yocto-worker/kernel-ppc/build/build/tmp/sysroots/x86_64-linux/usr/bin/pseudo -d 31568 ? S 1:31 /opt/poky/1.8+snapshot/sysroots/x86_64-pokysdk-linux/usr/bin/python2.7 /home/pokybuild/yocto-autobuilder/yocto-controller/nightly-qa-logrotate/build/bitbake/bin/bitbake-worker decafbad 31582 ? SNs 0:00 /opt/poky/1.8+snapshot/sysroots/x86_64-pokysdk-linux/usr/bin/python2.7 /home/pokybuild/yocto-autobuilder-kernel/yocto-worker/kernel-x86-64/build/bitbake/bin/bitbake-worker decafbad 31593 ? S 1:22 /opt/poky/1.8+snapshot/sysroots/x86_64-pokysdk-linux/usr/bin/python2.7 /home/pokybuild/yocto-autobuilder/yocto-controller/nightly-qa-skeleton/build/bitbake/bin/bitbake-worker decafbad 31618 ? S 1:44 /opt/poky/1.8+snapshot/sysroots/x86_64-pokysdk-linux/usr/bin/python2.7 /home/pokybuild/yocto-autobuilder/yocto-controller/ptest-x86/build/bitbake/bin/bitbake-worker decafbad 32088 ? SN 0:00 /bin/sh /home/pokybuild/yocto-autobuilder/yocto-controller/ptest-x86/build/build/tmp/work/core2-32-poky-linux/pango/1.36.8-r0/temp/run.sstate_create_package.19530 32271 ? SN 0:00 /bin/sh /home/pokybuild/yocto-autobuilder-kernel/yocto-worker/kernel-ppc/build/build/tmp/sysroots/x86_64-linux/usr/bin/opkg-build glibc-binary-localedata-es-py.iso-8859-1 /home/pokybui 32293 ? SN 0:00 tar -czf /srv/www/vhosts/autobuilder.yoctoproject.org/pub/sstate//9f/sstate:pango:core2-32-poky-linux:1.36.8:r0:core2-32:3:9fdebd3ee5521db1ff4c6d4284ffc45f_populate_sysroot.tgz.V6xyayF 32305 ? SN 0:00 tar -cz --format=gnu -f /home/pokybuild/yocto-autobuilder-kernel/yocto-worker/kernel-ppc/build/build/tmp/work/ppc7400-poky-linux/glibc-locale/2.21-r0/deploy-ipks/ppc7400/IPKG_BUILD.297 32324 ? SNl 0:00 gzip 32337 ? SN 0:00 gzip 32613 ? S 0:00 /home/pokybuild/yocto-autobuilder-kernel/yocto-worker/kernel-ppc/build/build/tmp/sysroots/x86_64-linux/usr/bin/pseudo bitbake-worker decafbad 32623 ? S 0:48 /opt/poky/1.8+snapshot/sysroots/x86_64-pokysdk-linux/usr/bin/python2.7 /home/pokybuild/yocto-autobuilder-kernel/yocto-worker/kernel-ppc/build/bitbake/bin/bitbake-worker decafbad 32699 ? SN 0:00 /bin/sh /home/pokybuild/yocto-autobuilder/yocto-controller/nightly-qa-skeleton/build/build/tmp/sysroots/x86_64-linux/usr/bin/opkg-build perl-module-pod-man /home/pokybuild/yocto-autobu [rpurdie@centos6 ~] autom4te[13297]: segfault at 11863678 ip 00007ff5f0e9bba9 sp 00007fffe9960d90 error 4 in Fcntl.so[7ff5f0e9a000+3000] autom4te[28191]: segfault at 15dee778 ip 00007fa27a866ba9 sp 00007fff13eb4bc0 error 4 in Fcntl.so[7fa27a865000+3000] autom4te[15797]: segfault at 813bcf8 ip 00007f46b70e2ba9 sp 00007ffff3731520 error 4 in Fcntl.so[7f46b70e1000+3000] autom4te[12375]: segfault at 145d9678 ip 00007f0be1c24ba9 sp 00007fff41f5e370 error 4 in Fcntl.so[7f0be1c23000+3000] autom4te[4953]: segfault at ce86278 ip 00007f7d59407ba9 sp 00007fffea82f310 error 4 in Fcntl.so[7f7d59406000+3000] autom4te[29942]: segfault at 10398f78 ip 00007f5cdfd20ba9 sp 00007fffe21b0160 error 4 in Fcntl.so[7f5cdfd1f000+3000] autom4te[25783]: segfault at 7bd8a78 ip 00007f7017f57ba9 sp 00007fffd5fb6150 error 4 in Fcntl.so[7f7017f56000+3000] autom4te[10309]: segfault at 167f36f8 ip 00007f4f314dfba9 sp 00007fffebaf39f0 error 4 in Fcntl.so[7f4f314de000+3000] autom4te[20382]: segfault at 16459ff8 ip 00007fe4202b1ba9 sp 00007fff9d97beb0 error 4 in Fcntl.so[7fe4202b0000+3000] autom4te[3916]: segfault at 85b6478 ip 00007fa14a71eba9 sp 00007fffcd992600 error 4 in Fcntl.so[7fa14a71d000+3000] autom4te[3951]: segfault at 14992cf8 ip 00007f4e45190ba9 sp 00007ffff75f9630 error 4 in Fcntl.so[7f4e4518f000+3000] autom4te[32132]: segfault at 75c0378 ip 00007f6134eacba9 sp 00007fffbe49bc90 error 4 in Fcntl.so[7f6134eab000+3000] autom4te[1891]: segfault at fbd35f8 ip 00007f8096935ba9 sp 00007fff3b65a690 error 4 in Fcntl.so[7f8096934000+3000] autom4te[20845]: segfault at 94d6678 ip 00007f0ae9ee7ba9 sp 00007fff7cf06750 error 4 in Fcntl.so[7f0ae9ee6000+3000] autom4te[29955]: segfault at 14d9cef8 ip 00007f9e3cae2ba9 sp 00007fffb34427f0 error 4 in Fcntl.so[7f9e3cae1000+3000] autom4te[6229]: segfault at a1d3978 ip 00007feaea468ba9 sp 00007fffb7d7f580 error 4 in Fcntl.so[7feaea467000+3000] autom4te[10646]: segfault at 151d24f8 ip 00007f0dada85ba9 sp 00007fffdc397300 error 4 in Fcntl.so[7f0dada84000+3000] autom4te[30164]: segfault at dbcd578 ip 00007fdc85461ba9 sp 00007fffc11a0830 error 4 in Fcntl.so[7fdc85460000+3000] autom4te[718]: segfault at 14b2c578 ip 00007f9d4b44cba9 sp 00007fff9bb656d0 error 4 in Fcntl.so[7f9d4b44b000+3000] autom4te[5500]: segfault at 83437f8 ip 00007f85eeaa4ba9 sp 00007fff33bad8f0 error 4 in Fcntl.so[7f85eeaa3000+3000] autom4te[9684]: segfault at 12087a78 ip 00007ff175f58ba9 sp 00007fff41c61880 error 4 in Fcntl.so[7ff175f57000+3000] autom4te[476]: segfault at acdb7f8 ip 00007f23007f6ba9 sp 00007fff38ee4fa0 error 4 in Fcntl.so[7f23007f5000+3000] autom4te[17301]: segfault at 7626af8 ip 00007fb472576ba9 sp 00007fffbc3dc590 error 4 in Fcntl.so[7fb472575000+3000] autom4te[26688]: segfault at 10162f78 ip 00007f8e1ece6ba9 sp 00007fff752dbef0 error 4 in Fcntl.so[7f8e1ece5000+3000] autom4te[23457]: segfault at b6207f8 ip 00007fc3e31e9ba9 sp 00007fff8f5875f0 error 4 in Fcntl.so[7fc3e31e8000+3000] autom4te[3300]: segfault at 589ebf8 ip 00007f038d729ba9 sp 00007fff7b3f1310 error 4 in Fcntl.so[7f038d728000+3000] autom4te[3422]: segfault at d6a0e78 ip 00007f7f3147bba9 sp 00007fffac6bceb0 error 4 in Fcntl.so[7f7f3147a000+3000] autom4te[2616]: segfault at dcabef8 ip 00007f1394ea4ba9 sp 00007fffde229a90 error 4 in Fcntl.so[7f1394ea3000+3000] autom4te[21843]: segfault at 13f3e4f8 ip 00007f13dc10eba9 sp 00007fffdf68e100 error 4 in Fcntl.so[7f13dc10d000+3000] autom4te[20412]: segfault at df0f378 ip 00007fc25fa89ba9 sp 00007fff2e6735e0 error 4 in Fcntl.so[7fc25fa88000+3000] autom4te[26531]: segfault at 959f278 ip 00007fef3780dba9 sp 00007fff7b341a20 error 4 in Fcntl.so[7fef3780c000+3000] autom4te[24754]: segfault at 6575e78 ip 00007fe310ca1ba9 sp 00007fff7ab6f690 error 4 in Fcntl.so[7fe310ca0000+3000] autom4te[15776]: segfault at 16735e78 ip 00007f073e26dba9 sp 00007fff36d07f80 error 4 in Fcntl.so[7f073e26c000+3000] autom4te[12121]: segfault at 155b6778 ip 00007feadbb08ba9 sp 00007fff84772640 error 4 in Fcntl.so[7feadbb07000+3000] autom4te[13527]: segfault at 9a9ecf8 ip 00007fe573e4fba9 sp 00007fff1975a370 error 4 in Fcntl.so[7fe573e4e000+3000] autom4te[15045]: segfault at daab0f8 ip 00007f3491cacba9 sp 00007fffc84a4df0 error 4 in Fcntl.so[7f3491cab000+3000] autom4te[14069]: segfault at 158e1df8 ip 00007feabb2d7ba9 sp 00007fffd1eea0b0 error 4 in Fcntl.so[7feabb2d6000+3000] autom4te[31905]: segfault at e709d78 ip 00007f7aa311fba9 sp 00007fff0c16d660 error 4 in Fcntl.so[7f7aa311e000+3000] autom4te[24969]: segfault at 14894bf8 ip 00007fda72c14ba9 sp 00007fff33d7bad0 error 4 in Fcntl.so[7fda72c13000+3000] autom4te[26206]: segfault at cc01df8 ip 00007f4d37a88ba9 sp 00007fff7ef1fab0 error 4 in Fcntl.so[7f4d37a87000+3000] autom4te[27846]: segfault at 12924ef8 ip 00007ff96e8b2ba9 sp 00007fff94ff2460 error 4 in Fcntl.so[7ff96e8b1000+3000] autom4te[1072]: segfault at 48eac78 ip 00007f8cc5c6aba9 sp 00007fffdfc868c0 error 4 in Fcntl.so[7f8cc5c69000+3000] autom4te[20275]: segfault at e9c77f8 ip 00007f9d95850ba9 sp 00007fff981d4e20 error 4 in Fcntl.so[7f9d9584f000+3000] autom4te[21051]: segfault at d713af8 ip 00007fbf4df8dba9 sp 00007fffff8b6660 error 4 in Fcntl.so[7fbf4df8c000+3000] autom4te[20652]: segfault at d8b27f8 ip 00007f0a4c11aba9 sp 00007fff0611abb0 error 4 in Fcntl.so[7f0a4c119000+3000] autom4te[26810]: segfault at b34e378 ip 00007fcfa4d44ba9 sp 00007fffb8d13c40 error 4 in Fcntl.so[7fcfa4d43000+3000] autom4te[19607]: segfault at 97174f8 ip 00007f7081c51ba9 sp 00007fff21083c40 error 4 in Fcntl.so[7f7081c50000+3000] autom4te[23852]: segfault at 51437f8 ip 00007f20fa905ba9 sp 00007fff5e2e6770 error 4 in Fcntl.so[7f20fa904000+3000] autom4te[18147]: segfault at f7dadf8 ip 00007f7eba88aba9 sp 00007fff4db4bb50 error 4 in Fcntl.so[7f7eba889000+3000] autom4te[27569]: segfault at 90e7cf8 ip 00007f5db0e96ba9 sp 00007fffc750b890 error 4 in Fcntl.so[7f5db0e95000+3000] autom4te[6167]: segfault at cba6678 ip 00007fcb6dfe9ba9 sp 00007fff66dbcf90 error 4 in Fcntl.so[7fcb6dfe8000+3000] autom4te[24585]: segfault at e444bf8 ip 00007fea88186ba9 sp 00007fff47bde4a0 error 4 in Fcntl.so[7fea88185000+3000] autom4te[12990]: segfault at 10fa6478 ip 00007fcb7d411ba9 sp 00007fff916ca280 error 4 in Fcntl.so[7fcb7d410000+3000] autom4te[1523]: segfault at 1308af78 ip 00007f9fcb443ba9 sp 00007fff90bd8f30 error 4 in Fcntl.so[7f9fcb442000+3000] autom4te[21206]: segfault at e2d9378 ip 00007fe944583ba9 sp 00007fffbd666d40 error 4 in Fcntl.so[7fe944582000+3000] autom4te[21249]: segfault at 129eaf78 ip 00007fd6b6090ba9 sp 00007fff8a351fa0 error 4 in Fcntl.so[7fd6b608f000+3000] autom4te[8361]: segfault at 675e578 ip 00007f1476054ba9 sp 00007fff3aed0fc0 error 4 in Fcntl.so[7f1476053000+3000] autom4te[30555]: segfault at 91a8a78 ip 00007f6bb717cba9 sp 00007fffc9e2d9c0 error 4 in Fcntl.so[7f6bb717b000+3000] autom4te[3669]: segfault at d0068f8 ip 00007f92a1956ba9 sp 00007ffff69bf920 error 4 in Fcntl.so[7f92a1955000+3000] autom4te[10692]: segfault at 1016c778 ip 00007fc1e4f64ba9 sp 00007fff2a053aa0 error 4 in Fcntl.so[7fc1e4f63000+3000] autom4te[28648]: segfault at 1476b5f8 ip 00007fd2d2879ba9 sp 00007fff21283670 error 4 in Fcntl.so[7fd2d2878000+3000] autom4te[13585]: segfault at 14d0a378 ip 00007fea98983ba9 sp 00007fffea11b7a0 error 4 in Fcntl.so[7fea98982000+3000] autom4te[25632]: segfault at 161c0678 ip 00007fd3746f0ba9 sp 00007fff8493e010 error 4 in Fcntl.so[7fd3746ef000+3000] autom4te[22075]: segfault at ad4a178 ip 00007ffd3f9a0ba9 sp 00007fffcd19c540 error 4 in Fcntl.so[7ffd3f99f000+3000] autom4te[28603]: segfault at 155dd3f8 ip 00007f3087dcaba9 sp 00007fffec5fc370 error 4 in Fcntl.so[7f3087dc9000+3000] autom4te[13257]: segfault at 96f04f8 ip 00007f9ca6891ba9 sp 00007fff7c5f2430 error 4 in Fcntl.so[7f9ca6890000+3000] autom4te[15849]: segfault at 85d4078 ip 00007f8caa7c3ba9 sp 00007fff20a01850 error 4 in Fcntl.so[7f8caa7c2000+3000] autom4te[16489]: segfault at 6ad60f8 ip 00007f9a67d20ba9 sp 00007fffc8779340 error 4 in Fcntl.so[7f9a67d1f000+3000] autom4te[20179]: segfault at a099978 ip 00007fc24f123ba9 sp 00007fffc3cedaf0 error 4 in Fcntl.so[7fc24f122000+3000] autom4te[16578]: segfault at d731ef8 ip 00007fe6db29bba9 sp 00007fff99c50920 error 4 in Fcntl.so[7fe6db29a000+3000] autom4te[25810]: segfault at 74524f8 ip 00007f6a04749ba9 sp 00007fff86f63bf0 error 4 in Fcntl.so[7f6a04748000+3000] autom4te[13673]: segfault at f60c478 ip 00007f167a93aba9 sp 00007fff218bae00 error 4 in Fcntl.so[7f167a939000+3000] autom4te[12960]: segfault at 148624f8 ip 00007f7f9521eba9 sp 00007fffd0cd0010 error 4 in Fcntl.so[7f7f9521d000+3000] autom4te[25213]: segfault at c5b5df8 ip 00007f76cbd3fba9 sp 00007fff0303d030 error 4 in Fcntl.so[7f76cbd3e000+3000] autom4te[16940]: segfault at 8637a78 ip 00007f9804678ba9 sp 00007fff69c7efb0 error 4 in Fcntl.so[7f9804677000+3000] autom4te[1003]: segfault at fd9d378 ip 00007fd9117c1ba9 sp 00007fffc1fcd400 error 4 in Fcntl.so[7fd9117c0000+3000] autom4te[7303]: segfault at 1142a478 ip 00007f7d9c519ba9 sp 00007fff59e59670 error 4 in Fcntl.so[7f7d9c518000+3000] autom4te[21681]: segfault at 72df4f8 ip 00007f20b99efba9 sp 00007fff01d153c0 error 4 in Fcntl.so[7f20b99ee000+3000] autom4te[4442]: segfault at b382ef8 ip 00007f5880aa3ba9 sp 00007fff2ce0f630 error 4 in Fcntl.so[7f5880aa2000+3000] autom4te[5366]: segfault at ce49cf8 ip 00007f2b85233ba9 sp 00007fffab565750 error 4 in Fcntl.so[7f2b85232000+3000] autom4te[7434]: segfault at 970e178 ip 00007fd19c72fba9 sp 00007fff00fcd210 error 4 in Fcntl.so[7fd19c72e000+3000] autom4te[11649]: segfault at de3e8f8 ip 00007f46d8f6cba9 sp 00007fffe6c2a2a0 error 4 in Fcntl.so[7f46d8f6b000+3000] autom4te[8404]: segfault at 14c882f8 ip 00007f6b85ff3ba9 sp 00007fffbdd37370 error 4 in Fcntl.so[7f6b85ff2000+3000] autom4te[18104]: segfault at 6b6d1f8 ip 00007f7f6273bba9 sp 00007fff1ee8d200 error 4 in Fcntl.so[7f7f6273a000+3000] autom4te[30582]: segfault at ab71978 ip 00007fe34e75dba9 sp 00007fff10e7f390 error 4 in Fcntl.so[7fe34e75c000+3000] autom4te[17814]: segfault at e29a4f8 ip 00007f019f801ba9 sp 00007fff3f6f77e0 error 4 in Fcntl.so[7f019f800000+3000] autom4te[19119]: segfault at a778e78 ip 00007f86eaea6ba9 sp 00007fff1636c080 error 4 in Fcntl.so[7f86eaea5000+3000] autom4te[31558]: segfault at e0f4c78 ip 00007fcf2ce77ba9 sp 00007fff5c7336f0 error 4 in Fcntl.so[7fcf2ce76000+3000] autom4te[32001]: segfault at 10f653f8 ip 00007f0725f69ba9 sp 00007fff9fb451d0 error 4 in Fcntl.so[7f0725f68000+3000] autom4te[8577]: segfault at 15e3b5f8 ip 00007f941b80bba9 sp 00007fff4b8d1870 error 4 in Fcntl.so[7f941b80a000+3000] autom4te[18146]: segfault at 7575df8 ip 00007f9171acdba9 sp 00007fff3d3384e0 error 4 in Fcntl.so[7f9171acc000+3000] autom4te[32544]: segfault at dc3e078 ip 00007f1da98e5ba9 sp 00007fffa9d1ff20 error 4 in Fcntl.so[7f1da98e4000+3000] autom4te[10423]: segfault at 64bed78 ip 00007fcd8c453ba9 sp 00007fff7718c1e0 error 4 in Fcntl.so[7fcd8c452000+3000] autom4te[23543]: segfault at c85c0f8 ip 00007f3b2d30fba9 sp 00007fffbc0f7990 error 4 in Fcntl.so[7f3b2d30e000+3000] autom4te[17116]: segfault at 4cd7578 ip 00007faea57e8ba9 sp 00007fff852098d0 error 4 in Fcntl.so[7faea57e7000+3000] autom4te[25707]: segfault at d4084f8 ip 00007ff054c27ba9 sp 00007fffeae3f4d0 error 4 in Fcntl.so[7ff054c26000+3000] autom4te[12745]: segfault at 77c2678 ip 00007fea2bc6cba9 sp 00007fffb2afc420 error 4 in Fcntl.so[7fea2bc6b000+3000] autom4te[18948]: segfault at 5464f78 ip 00007fdac1d3fba9 sp 00007fff00ec21b0 error 4 in Fcntl.so[7fdac1d3e000+3000] autom4te[23631]: segfault at 65b1378 ip 00007f69d1f8dba9 sp 00007fff259cdea0 error 4 in Fcntl.so[7f69d1f8c000+3000] autom4te[11520]: segfault at a15af78 ip 00007f890c8ffba9 sp 00007fff134e1ea0 error 4 in Fcntl.so[7f890c8fe000+3000] autom4te[1032]: segfault at 7fec7f8 ip 00007f600a806ba9 sp 00007fffde403100 error 4 in Fcntl.so[7f600a805000+3000] autom4te[13615]: segfault at db985f8 ip 00007f08e051bba9 sp 00007ffff24cccd0 error 4 in Fcntl.so[7f08e051a000+3000] autom4te[3877]: segfault at 8563ef8 ip 00007f8e49049ba9 sp 00007fff81dd4810 error 4 in Fcntl.so[7f8e49048000+3000] Clock: inserting leap second 23:59:60 UTC autom4te[13290]: segfault at dc98d78 ip 00007f7366095ba9 sp 00007fff790dec10 error 4 in Fcntl.so[7f7366094000+3000] autom4te[32272]: segfault at cf61df8 ip 00007f5c076afba9 sp 00007fff13b88360 error 4 in Fcntl.so[7f5c076ae000+3000] autom4te[19243]: segfault at 136cb4f8 ip 00007f4bce6edba9 sp 00007fff1ca23a20 error 4 in Fcntl.so[7f4bce6ec000+3000] autom4te[31622]: segfault at 16617af8 ip 00007f86f7a0eba9 sp 00007fffdd7eb020 error 4 in Fcntl.so[7f86f7a0d000+3000] autom4te[11569]: segfault at 8cf7178 ip 00007f362f7adba9 sp 00007fff464fe400 error 4 in Fcntl.so[7f362f7ac000+3000] autom4te[27216]: segfault at 82651f8 ip 00007f84040a9ba9 sp 00007fff62d245a0 error 4 in Fcntl.so[7f84040a8000+3000] autom4te[12684]: segfault at 8e9e978 ip 00007fe422a21ba9 sp 00007fff87900930 error 4 in Fcntl.so[7fe422a20000+3000] autom4te[10820]: segfault at 85cc3f8 ip 00007f0c299b7ba9 sp 00007fffa620a960 error 4 in Fcntl.so[7f0c299b6000+3000] autom4te[26939]: segfault at 5d27a78 ip 00007fdb4ab44ba9 sp 00007fff40f31540 error 4 in Fcntl.so[7fdb4ab43000+3000] qemu-system-x86[18519]: segfault at 2d0 ip 00007fa132aeb880 sp 00007fa13123cae8 error 4 in libpthread-2.12.so[7fa132adf000+17000] autom4te[31682]: segfault at d61adf8 ip 00007f5f6fe79ba9 sp 00007fff08ffdc90 error 4 in Fcntl.so[7f5f6fe78000+3000] autom4te[745]: segfault at 12475478 ip 00007fbb2a334ba9 sp 00007fff98bfa420 error 4 in Fcntl.so[7fbb2a333000+3000] autom4te[19854]: segfault at d13e0f8 ip 00007fef6a6f6ba9 sp 00007fffca62d080 error 4 in Fcntl.so[7fef6a6f5000+3000] autom4te[23200]: segfault at 9ade1f8 ip 00007f16b1657ba9 sp 00007fff49f2ea20 error 4 in Fcntl.so[7f16b1656000+3000] autom4te[4332]: segfault at 845d3f8 ip 00007f97a38ebba9 sp 00007fff56134ab0 error 4 in Fcntl.so[7f97a38ea000+3000] INFO: task syslinux:6843 blocked for more than 120 seconds. Not tainted 2.6.32-504.23.4.el6.x86_64 #1 "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. syslinux D 0000000000000000 0 6843 6761 0x00000080 ffff880197507d38 0000000000000086 ffff88084c4d5958 ffff880197507da8 ffff880197507ca8 ffffffff8128c539 00000000000158c0 ffff88084c4ced00 ffff880197507cf8 ffffffff8105c5d5 ffff880102231ad8 ffff880197507fd8 Call Trace: [<ffffffff8128c539>] ? cpumask_next_and+0x29/0x50 [<ffffffff8105c5d5>] ? select_idle_sibling+0x95/0x150 [<ffffffff8152a925>] schedule_timeout+0x215/0x2e0 [<ffffffff8105def4>] ? check_preempt_wakeup+0x1a4/0x260 [<ffffffff8106d5eb>] ? enqueue_task_fair+0xfb/0x100 [<ffffffff8152a5a3>] wait_for_common+0x123/0x180 [<ffffffff81064be0>] ? default_wake_function+0x0/0x20 [<ffffffff8152a6bd>] wait_for_completion+0x1d/0x20 [<ffffffff811ba4b5>] writeback_inodes_sb_nr+0x75/0xa0 [<ffffffff811ba523>] writeback_inodes_sb+0x43/0x50 [<ffffffff811c0cde>] __sync_filesystem+0x3e/0x90 [<ffffffff811c0e28>] sync_filesystems+0xf8/0x130 [<ffffffff811c0eb7>] sys_sync+0x17/0x40 [<ffffffff8100b0f2>] system_call_fastpath+0x16/0x1b INFO: task syslinux:10200 blocked for more than 120 seconds. Not tainted 2.6.32-504.23.4.el6.x86_64 #1 "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. syslinux D 0000000000000004 0 10200 9474 0x00000080 ffff880864cc7ea8 0000000000000086 0000000000000000 0000000000000007 ffff880864cc7e28 ffffffff8106d554 00020c83d8d86563 ffff88102366a040 ffff8800282758c0 00000001225af4fe ffff88091803d068 ffff880864cc7fd8 Call Trace: [<ffffffff8106d554>] ? enqueue_task_fair+0x64/0x100 [<ffffffff8152b3a6>] __mutex_lock_slowpath+0x96/0x210 [<ffffffff8152aecb>] mutex_lock+0x2b/0x50 [<ffffffff811c0d56>] sync_filesystems+0x26/0x130 [<ffffffff811c0eb7>] sys_sync+0x17/0x40 [<ffffffff8100b0f2>] system_call_fastpath+0x16/0x1b INFO: task syslinux:9758 blocked for more than 120 seconds. Not tainted 2.6.32-504.23.4.el6.x86_64 #1 "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. syslinux D 0000000000000000 0 9758 9687 0x00000080 ffff88083005fea8 0000000000000086 0000000000000000 0000000000000015 ffff88083005fe28 ffffffff8106d554 00020c9e27376e25 ffff88102366a040 ffff8800283558c0 00000001225cae49 ffff88102366bad8 ffff88083005ffd8 Call Trace: [<ffffffff8106d554>] ? enqueue_task_fair+0x64/0x100 [<ffffffff8152b3a6>] __mutex_lock_slowpath+0x96/0x210 [<ffffffff81064c35>] ? wake_up_process+0x15/0x20 [<ffffffff8152aecb>] mutex_lock+0x2b/0x50 [<ffffffff811c0d56>] sync_filesystems+0x26/0x130 [<ffffffff811c0eb7>] sys_sync+0x17/0x40 [<ffffffff8100b0f2>] system_call_fastpath+0x16/0x1b INFO: task syslinux:6843 blocked for more than 120 seconds. Not tainted 2.6.32-504.23.4.el6.x86_64 #1 "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. syslinux D 000000000000000c 0 6843 6761 0x00000080 ffff880197507d38 0000000000000086 ffffffff811244d0 ffff880197507d38 ffff880197507ca8 ffffffff8128c539 00000000000158c0 ffff88084c40ed00 ffff880197507cf8 ffffffff8105c5d5 ffff880102231ad8 ffff880197507fd8 Call Trace: [<ffffffff811244d0>] ? sync_page+0x0/0x50 [<ffffffff8128c539>] ? cpumask_next_and+0x29/0x50 [<ffffffff8105c5d5>] ? select_idle_sibling+0x95/0x150 [<ffffffff8152a925>] schedule_timeout+0x215/0x2e0 [<ffffffff81058678>] ? resched_task+0x68/0x80 [<ffffffff8105df12>] ? check_preempt_wakeup+0x1c2/0x260 [<ffffffff8106d5eb>] ? enqueue_task_fair+0xfb/0x100 [<ffffffff8152a5a3>] wait_for_common+0x123/0x180 [<ffffffff81064be0>] ? default_wake_function+0x0/0x20 [<ffffffff8152a6bd>] wait_for_completion+0x1d/0x20 [<ffffffff811ba5b8>] sync_inodes_sb+0x88/0x190 [<ffffffff811c0d22>] __sync_filesystem+0x82/0x90 [<ffffffff811c0e28>] sync_filesystems+0xf8/0x130 [<ffffffff811c0ec1>] sys_sync+0x21/0x40 [<ffffffff8100b0f2>] system_call_fastpath+0x16/0x1b INFO: task syslinux:10200 blocked for more than 120 seconds. Not tainted 2.6.32-504.23.4.el6.x86_64 #1 "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. syslinux D 0000000000000006 0 10200 9474 0x00000080 ffff880864cc7ea8 0000000000000086 0000000000000000 0000000000000007 ffff880864cc7e28 ffffffff8106d554 00020ca9e5aada55 ffff88102366a040 ffff8800282758c0 00000001225d7359 ffff88091803d068 ffff880864cc7fd8 Call Trace: [<ffffffff8106d554>] ? enqueue_task_fair+0x64/0x100 [<ffffffff8152b3a6>] __mutex_lock_slowpath+0x96/0x210 [<ffffffff8152aecb>] mutex_lock+0x2b/0x50 [<ffffffff811c0d56>] sync_filesystems+0x26/0x130 [<ffffffff811c0eb7>] sys_sync+0x17/0x40 [<ffffffff8100b0f2>] system_call_fastpath+0x16/0x1b INFO: task syslinux:9758 blocked for more than 120 seconds. Not tainted 2.6.32-504.23.4.el6.x86_64 #1 "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. syslinux D 0000000000000000 0 9758 9687 0x00000080 ffff88083005fea8 0000000000000086 0000000000000000 0000000000000015 ffff88083005fe28 ffffffff8106d554 00020c9e27376e25 ffff88102366a040 ffff8800283558c0 00000001225cae49 ffff88102366bad8 ffff88083005ffd8 Call Trace: [<ffffffff8106d554>] ? enqueue_task_fair+0x64/0x100 [<ffffffff8152b3a6>] __mutex_lock_slowpath+0x96/0x210 [<ffffffff81064c35>] ? wake_up_process+0x15/0x20 [<ffffffff8152aecb>] mutex_lock+0x2b/0x50 [<ffffffff811c0d56>] sync_filesystems+0x26/0x130 [<ffffffff811c0eb7>] sys_sync+0x17/0x40 [<ffffffff8100b0f2>] system_call_fastpath+0x16/0x1b INFO: task syslinux:6843 blocked for more than 120 seconds. Not tainted 2.6.32-504.23.4.el6.x86_64 #1 "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. syslinux D 000000000000000c 0 6843 6761 0x00000080 ffff880197507d38 0000000000000086 ffffffff811244d0 ffff880197507d38 ffff880197507ca8 ffffffff8128c539 00000000000158c0 ffff88084c40ed00 ffff880197507cf8 ffffffff8105c5d5 ffff880102231ad8 ffff880197507fd8 Call Trace: [<ffffffff811244d0>] ? sync_page+0x0/0x50 [<ffffffff8128c539>] ? cpumask_next_and+0x29/0x50 [<ffffffff8105c5d5>] ? select_idle_sibling+0x95/0x150 [<ffffffff8152a925>] schedule_timeout+0x215/0x2e0 [<ffffffff81058678>] ? resched_task+0x68/0x80 [<ffffffff8105df12>] ? check_preempt_wakeup+0x1c2/0x260 [<ffffffff8106d5eb>] ? enqueue_task_fair+0xfb/0x100 [<ffffffff8152a5a3>] wait_for_common+0x123/0x180 [<ffffffff81064be0>] ? default_wake_function+0x0/0x20 [<ffffffff8152a6bd>] wait_for_completion+0x1d/0x20 [<ffffffff811ba5b8>] sync_inodes_sb+0x88/0x190 [<ffffffff811c0d22>] __sync_filesystem+0x82/0x90 [<ffffffff811c0e28>] sync_filesystems+0xf8/0x130 [<ffffffff811c0ec1>] sys_sync+0x21/0x40 [<ffffffff8100b0f2>] system_call_fastpath+0x16/0x1b INFO: task syslinux:10200 blocked for more than 120 seconds. Not tainted 2.6.32-504.23.4.el6.x86_64 #1 "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. syslinux D 0000000000000006 0 10200 9474 0x00000080 ffff880864cc7ea8 0000000000000086 0000000000000000 0000000000000007 ffff880864cc7e28 ffffffff8106d554 00020ca9e5aada55 ffff88102366a040 ffff8800282758c0 00000001225d7359 ffff88091803d068 ffff880864cc7fd8 Call Trace: [<ffffffff8106d554>] ? enqueue_task_fair+0x64/0x100 [<ffffffff8152b3a6>] __mutex_lock_slowpath+0x96/0x210 [<ffffffff8152aecb>] mutex_lock+0x2b/0x50 [<ffffffff811c0d56>] sync_filesystems+0x26/0x130 [<ffffffff811c0eb7>] sys_sync+0x17/0x40 [<ffffffff8100b0f2>] system_call_fastpath+0x16/0x1b INFO: task syslinux:9758 blocked for more than 120 seconds. Not tainted 2.6.32-504.23.4.el6.x86_64 #1 "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. syslinux D 0000000000000000 0 9758 9687 0x00000080 ffff88083005fea8 0000000000000086 0000000000000000 0000000000000015 ffff88083005fe28 ffffffff8106d554 00020c9e27376e25 ffff88102366a040 ffff8800283558c0 00000001225cae49 ffff88102366bad8 ffff88083005ffd8 Call Trace: [<ffffffff8106d554>] ? enqueue_task_fair+0x64/0x100 [<ffffffff8152b3a6>] __mutex_lock_slowpath+0x96/0x210 [<ffffffff81064c35>] ? wake_up_process+0x15/0x20 [<ffffffff8152aecb>] mutex_lock+0x2b/0x50 [<ffffffff811c0d56>] sync_filesystems+0x26/0x130 [<ffffffff811c0eb7>] sys_sync+0x17/0x40 [<ffffffff8100b0f2>] system_call_fastpath+0x16/0x1b INFO: task syslinux:6843 blocked for more than 120 seconds. Not tainted 2.6.32-504.23.4.el6.x86_64 #1 "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. syslinux D 000000000000000c 0 6843 6761 0x00000080 ffff880197507d38 0000000000000086 ffffffff811244d0 ffff880197507d38 ffff880197507ca8 ffffffff8128c539 00000000000158c0 ffff88084c40ed00 ffff880197507cf8 ffffffff8105c5d5 ffff880102231ad8 ffff880197507fd8 Call Trace: [<ffffffff811244d0>] ? sync_page+0x0/0x50 [<ffffffff8128c539>] ? cpumask_next_and+0x29/0x50 [<ffffffff8105c5d5>] ? select_idle_sibling+0x95/0x150 [<ffffffff8152a925>] schedule_timeout+0x215/0x2e0 [<ffffffff81058678>] ? resched_task+0x68/0x80 [<ffffffff8105df12>] ? check_preempt_wakeup+0x1c2/0x260 [<ffffffff8106d5eb>] ? enqueue_task_fair+0xfb/0x100 [<ffffffff8152a5a3>] wait_for_common+0x123/0x180 [<ffffffff81064be0>] ? default_wake_function+0x0/0x20 [<ffffffff8152a6bd>] wait_for_completion+0x1d/0x20 [<ffffffff811ba5b8>] sync_inodes_sb+0x88/0x190 [<ffffffff811c0d22>] __sync_filesystem+0x82/0x90 [<ffffffff811c0e28>] sync_filesystems+0xf8/0x130 [<ffffffff811c0ec1>] sys_sync+0x21/0x40 [<ffffffff8100b0f2>] system_call_fastpath+0x16/0x1b autom4te[14195]: segfault at d59f4f8 ip 00007fb0b8caaba9 sp 00007fff81177170 error 4 in Fcntl.so[7fb0b8ca9000+3000] autom4te[21463]: segfault at 1330e1f8 ip 00007f750c570ba9 sp 00007fff7ca83770 error 4 in Fcntl.so[7f750c56f000+3000] autom4te[24400]: segfault at ffc0d78 ip 00007f06bc18dba9 sp 00007fffe21f6c10 error 4 in Fcntl.so[7f06bc18c000+3000] autom4te[31112]: segfault at 8d83bf8 ip 00007f4aa9fd6ba9 sp 00007fff5f66b160 error 4 in Fcntl.so[7f4aa9fd5000+3000] autom4te[21950]: segfault at a246df8 ip 00007ff6e9869ba9 sp 00007fff59968f90 error 4 in Fcntl.so[7ff6e9868000+3000] [rpurdie@centos6 ~]$ pstree init─┬─Xvnc ├─abrt-dump-oops ├─abrtd ├─acpid ├─atd ├─auditd───{auditd} ├─console-kit-dae───63*[{console-kit-da}] ├─crond ├─dbus-daemon ├─hald─┬─hald-runner─┬─hald-addon-acpi │ │ └─hald-addon-inpu │ └─{hald} ├─irqbalance ├─ksmtuned───sleep ├─master─┬─pickup │ └─qmgr ├─6*[mingetty] ├─ntpd ├─10*[pseudo] ├─2*[python2.7.real───2*[{python2.7.real}]] ├─rpc.statd ├─rpcbind ├─rsyslogd───3*[{rsyslogd}] ├─sshd─┬─sshd───sshd───bash │ └─sshd───sshd───bash───pstree ├─twistd─┬─sh───python2.7.real───python2.7.real─┬─pseudo───python2.7.real───python2.7.real───run.sstate_crea───tar───gzip───16*[{gzip}] │ │ ├─python2.7.real───python2.7.real───run.sstate_crea───tar───gzip───14*[{gzip}] │ │ └─3*[{python2.7.real}] │ ├─sh───python2.7.real───python2.7.real─┬─pseudo───python2.7.real─┬─python2.7.real───opkg-build───opkg-build───tar───gzip │ │ │ └─python2.7.real │ │ ├─python2.7.real───python2.7.real───run.sstate_crea───tar───gzip───2*[{gzip}] │ │ └─3*[{python2.7.real}] │ ├─sh───python2.7.real───python2.7.real─┬─pseudo───python2.7.real─┬─python2.7.real───opkg-build───opkg-build───tar───gzip │ │ │ ├─python2.7.real───run.sstate_crea───tar───gzip───13*[{gzip}] │ │ │ └─python2.7.real │ │ ├─python2.7.real───python2.7.real───run.sstate_crea───tar───gzip───7*[{gzip}] │ │ └─3*[{python2.7.real}] │ └─{twistd} ├─twistd───17*[{twistd}] ├─twistd─┬─sh───python2.7.real───python2.7.real─┬─pseudo───python2.7.real─┬─3*[python2.7.real] │ │ │ ├─python2.7.real───opkg-build───opkg-build───tar───gzip───{gzip} │ │ │ ├─python2.7.real───run.sstate_crea───tar───gzip───23*[{gzip}] │ │ │ ├─python2.7.real───opkg-build───opkg-build───tar───gzip───2*[{gzip}] │ │ │ └─python2.7.real───opkg-build───opkg-build───tar───gzip───11*[{gzip}] │ │ ├─python2.7.real │ │ └─3*[{python2.7.real}] │ ├─sh───python2.7.real───python2.7.real─┬─pseudo───python2.7.real─┬─python2.7.real───opkg-build───opkg-build───tar───gzip │ │ │ └─python2.7.real───run.sstate_crea───tar───gzip───17*[{gzip}] │ │ ├─python2.7.real │ │ └─3*[{python2.7.real}] │ └─{twistd} ├─udevd───2*[udevd] ├─xinetd └─xterm───bash The gzip processes were in futexes: rpurdie]# gdb -p 32305 GNU gdb (GDB) Red Hat Enterprise Linux (7.2-75.el6) Copyright (C) 2010 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-redhat-linux-gnu". For bug reporting instructions, please see: <http://www.gnu.org/software/gdb/bugs/>. Attaching to process 32305 Reading symbols from /opt/poky/1.8+snapshot/sysroots/x86_64-pokysdk-linux/usr/bin/tar...Missing separate debuginfo for /opt/poky/1.8+snapshot/sysroots/x86_64-pokysdk-linux/usr/bin/tar Try: yum --enablerepo='*-debug*' install /usr/lib/debug/.build-id/3a/f0af0ee8fdc1c7785821093610e17d1faa3d56.debug (no debugging symbols found)...done. Reading symbols from /home/pokybuild/yocto-autobuilder-kernel/yocto-worker/kernel-ppc/build/build/tmp/sysroots/x86_64-linux/usr/bin/../lib/pseudo/lib64/libpseudo.so...(no debugging symbols found)...done. Loaded symbols for /home/pokybuild/yocto-autobuilder-kernel/yocto-worker/kernel-ppc/build/build/tmp/sysroots/x86_64-linux/usr/bin/../lib/pseudo/lib64/libpseudo.so Reading symbols from /opt/poky/1.8+snapshot/sysroots/x86_64-pokysdk-linux/usr/bin/../../lib/libc.so.6...Missing separate debuginfo for /opt/poky/1.8+snapshot/sysroots/x86_64-pokysdk-linux/usr/bin/../../lib/libc.so.6 (no debugging symbols found)...done. Loaded symbols for /opt/poky/1.8+snapshot/sysroots/x86_64-pokysdk-linux/usr/bin/../../lib/libc.so.6 Reading symbols from /opt/poky/1.8+snapshot/sysroots/x86_64-pokysdk-linux/usr/bin/../../lib/libdl.so.2...Missing separate debuginfo for /opt/poky/1.8+snapshot/sysroots/x86_64-pokysdk-linux/usr/bin/../../lib/libdl.so.2 (no debugging symbols found)...done. Loaded symbols for /opt/poky/1.8+snapshot/sysroots/x86_64-pokysdk-linux/usr/bin/../../lib/libdl.so.2 Reading symbols from /opt/poky/1.8+snapshot/sysroots/x86_64-pokysdk-linux/usr/bin/../../lib/libpthread.so.0...Missing separate debuginfo for /opt/poky/1.8+snapshot/sysroots/x86_64-pokysdk-linux/usr/bin/../../lib/libpthread.so.0 (no debugging symbols found)...done. warning: File "/opt/poky/1.8+snapshot/sysroots/x86_64-pokysdk-linux/usr/bin/../../lib/libthread_db.so.1" auto-loading has been declined by your `auto-load safe-path' set to "/usr/share/gdb/auto-load:/usr/lib/debug:/usr/bin/mono-gdb.py". To enable execution of this file add add-auto-load-safe-path /opt/poky/1.8+snapshot/sysroots/x86_64-pokysdk-linux/usr/bin/../../lib/libthread_db.so.1 line to your configuration file "/root/.gdbinit". To completely disable this security protection add set auto-load safe-path / line to your configuration file "/root/.gdbinit". For more information about this security protection see the "Auto-loading safe path" section in the GDB manual. E.g., run from the shell: info "(gdb)Auto-loading safe path" warning: Unable to find libthread_db matching inferior's thread library, thread debugging will not be available. Loaded symbols for /opt/poky/1.8+snapshot/sysroots/x86_64-pokysdk-linux/usr/bin/../../lib/libpthread.so.0 Reading symbols from /opt/poky/1.8+snapshot/sysroots/x86_64-pokysdk-linux/lib/ld-linux-x86-64.so.2...Missing separate debuginfo for /opt/poky/1.8+snapshot/sysroots/x86_64-pokysdk-linux/lib/ld-linux-x86-64.so.2 (no debugging symbols found)...done. warning: File "/opt/poky/1.8+snapshot/sysroots/x86_64-pokysdk-linux/usr/bin/../../lib/libthread_db.so.1" auto-loading has been declined by your `auto-load safe-path' set to "/usr/share/gdb/auto-load:/usr/lib/debug:/usr/bin/mono-gdb.py". warning: Unable to find libthread_db matching inferior's thread library, thread debugging will not be available. Loaded symbols for /opt/poky/1.8+snapshot/sysroots/x86_64-pokysdk-linux/lib/ld-linux-x86-64.so.2 warning: File "/opt/poky/1.8+snapshot/sysroots/x86_64-pokysdk-linux/usr/bin/../../lib/libthread_db.so.1" auto-loading has been declined by your `auto-load safe-path' set to "/usr/share/gdb/auto-load:/usr/lib/debug:/usr/bin/mono-gdb.py". warning: Unable to find libthread_db matching inferior's thread library, thread debugging will not be available. 0x00007f82f5444b8a in waitpid () from /opt/poky/1.8+snapshot/sysroots/x86_64-pokysdk-linux/usr/bin/../../lib/libc.so.6 (gdb) bt #0 0x00007f82f5444b8a in waitpid () from /opt/poky/1.8+snapshot/sysroots/x86_64-pokysdk-linux/usr/bin/../../lib/libc.so.6 #1 0x0000000000421388 in ?? () #2 0x0000000000407cc3 in ?? () #3 0x000000000040dc52 in ?? () #4 0x000000000040607c in ?? () #5 0x00007f82f53ad810 in __libc_start_main () from /opt/poky/1.8+snapshot/sysroots/x86_64-pokysdk-linux/usr/bin/../../lib/libc.so.6 #6 0x0000000000406429 in ?? () (gdb) q A debugging session is active. Inferior 1 [process 32305] will be detached. Quit anyway? (y or n) y Detaching from program: /opt/poky/1.8+snapshot/sysroots/x86_64-pokysdk-linux/usr/bin/tar, process 32305 [root@centos6 rpurdie]# ls /proc/32337/fd/ 0 1 10 11 12 13 14 15 16 17 18 19 2 20 21 22 23 24 25 26 27 28 29 3 30 31 32 33 34 35 36 37 38 39 4 40 41 43 5 6 7 8 9 [root@centos6 rpurdie]# ls /proc/32337/fd/ -la total 0 dr-x------ 2 pokybuild users 0 Jul 3 13:21 . dr-xr-xr-x 8 pokybuild users 0 Jul 3 09:15 .. lr-x------ 1 pokybuild users 64 Jul 3 13:21 0 -> pipe:[505258883] l-wx------ 1 pokybuild users 64 Jul 3 13:21 1 -> /home/pokybuild/yocto-autobuilder-kernel/yocto-worker/kernel-ppc/build/build/tmp/work/ppc7400-poky-linux/glibc-locale/2.21-r0/deploy-ipks/ppc7400/IPKG_BUILD.29782/control.tar.gz lr-x------ 1 pokybuild users 64 Jul 3 13:21 10 -> inotify lr-x------ 1 pokybuild users 64 Jul 3 13:21 11 -> pipe:[488437917] lrwx------ 1 pokybuild users 64 Jul 3 13:21 12 -> socket:[488436209] lrwx------ 1 pokybuild users 64 Jul 3 13:21 13 -> /home/pokybuild/yocto-autobuilder-kernel/yocto-worker/kernel-ppc/build/build/bitbake.lock lr-x------ 1 pokybuild users 64 Jul 3 13:21 14 -> /dev/null l-wx------ 1 pokybuild users 64 Jul 3 13:21 15 -> pipe:[488437917] lr-x------ 1 pokybuild users 64 Jul 3 13:21 16 -> pipe:[488437921] l-wx------ 1 pokybuild users 64 Jul 3 13:21 17 -> pipe:[488437921] lr-x------ 1 pokybuild users 64 Jul 3 13:21 18 -> pipe:[488437925] l-wx------ 1 pokybuild users 64 Jul 3 13:21 19 -> pipe:[488437925] l-wx------ 1 pokybuild users 64 Jul 3 13:21 2 -> /home/pokybuild/yocto-autobuilder-kernel/yocto-worker/kernel-ppc/build/build/tmp/work/ppc7400-poky-linux/glibc-locale/2.21-r0/temp/log.do_package_write_ipk.16771 lr-x------ 1 pokybuild users 64 Jul 3 13:21 20 -> pipe:[488437929] l-wx------ 1 pokybuild users 64 Jul 3 13:21 21 -> pipe:[488437929] lr-x------ 1 pokybuild users 64 Jul 3 13:21 22 -> pipe:[497248080] lr-x------ 1 pokybuild users 64 Jul 3 13:21 23 -> pipe:[497205519] lrwx------ 1 pokybuild users 64 Jul 3 13:21 24 -> /dev/null lr-x------ 1 pokybuild users 64 Jul 3 13:21 25 -> pipe:[497234732] lr-x------ 1 pokybuild users 64 Jul 3 13:21 26 -> pipe:[497300527] lrwx------ 1 pokybuild users 64 Jul 3 13:21 27 -> /dev/null lr-x------ 1 pokybuild users 64 Jul 3 13:21 28 -> /dev/urandom lr-x------ 1 pokybuild users 64 Jul 3 13:21 29 -> pipe:[497110097] lr-x------ 1 pokybuild users 64 Jul 3 13:21 3 -> pipe:[488436203] lr-x------ 1 pokybuild users 64 Jul 3 13:21 30 -> pipe:[497319633] lr-x------ 1 pokybuild users 64 Jul 3 13:21 31 -> pipe:[497278476] lr-x------ 1 pokybuild users 64 Jul 3 13:21 32 -> pipe:[497322874] l-wx------ 1 pokybuild users 64 Jul 3 13:21 33 -> pipe:[497327234] lr-x------ 1 pokybuild users 64 Jul 3 13:21 34 -> /home/pokybuild/yocto-autobuilder-kernel/yocto-worker/kernel-ppc/build/build/tmp/sysroots/x86_64-linux/usr lr-x------ 1 pokybuild users 64 Jul 3 13:21 35 -> /home/pokybuild/yocto-autobuilder-kernel/yocto-worker/kernel-ppc/build/build/tmp/work/ppc7400-poky-linux/glibc-locale/2.21-r0/pseudo lrwx------ 1 pokybuild users 64 Jul 3 13:21 36 -> socket:[505258971] lr-x------ 1 pokybuild users 64 Jul 3 13:21 37 -> /dev/null l-wx------ 1 pokybuild users 64 Jul 3 13:21 38 -> /home/pokybuild/yocto-autobuilder-kernel/yocto-worker/kernel-ppc/build/build/tmp/work/ppc7400-poky-linux/glibc-locale/2.21-r0/temp/log.do_package_write_ipk.16771 lrwx------ 1 pokybuild users 64 Jul 3 13:21 39 -> /dev/null l-wx------ 1 pokybuild users 64 Jul 3 13:21 4 -> pipe:[488436203] l-wx------ 1 pokybuild users 64 Jul 3 13:21 40 -> pipe:[488469385] l-wx------ 1 pokybuild users 64 Jul 3 13:21 41 -> pipe:[488435461] lrwx------ 1 pokybuild users 64 Jul 3 13:21 43 -> /home/pokybuild/yocto-autobuilder-kernel/yocto-worker/kernel-ppc/build/build/tmp/work/ppc7400-poky-linux/glibc-locale/2.21-r0/packages-split/glibc-binary-localedata-es-py.iso-8859-1.lock lrwx------ 1 pokybuild users 64 Jul 3 13:21 5 -> socket:[488436201] lrwx------ 1 pokybuild users 64 Jul 3 13:21 6 -> socket:[488436202] lr-x------ 1 pokybuild users 64 Jul 3 13:21 7 -> inotify lrwx------ 1 pokybuild users 64 Jul 3 13:21 8 -> /dev/null lr-x------ 1 pokybuild users 64 Jul 3 13:21 9 -> /dev/urandom [root@centos6 rpurdie]# ls /proc/9765/fd/ -la total 0 dr-x------ 2 pokybuild users 0 Jul 3 13:14 . dr-xr-xr-x 8 pokybuild users 0 Jul 3 09:17 .. lr-x------ 1 pokybuild users 64 Jul 3 13:14 0 -> pipe:[505273133] l-wx------ 1 pokybuild users 64 Jul 3 13:14 1 -> /srv/www/vhosts/sstate.yoctoproject.org/4a/sstate:util-linux:ppc7400-poky-linux:2.26.2:r0:ppc7400:3:4ac91c5cc5af51776b833a7e8b5090cc_package_write_ipk.tgz.tOiEAvel l-wx------ 1 pokybuild users 64 Jul 3 13:14 2 -> pipe:[505272517] lr-x------ 1 pokybuild users 64 Jul 3 13:14 20 -> /home/pokybuild/yocto-autobuilder-kernel/yocto-worker/kernel-ppc/build/build/tmp/sysroots/x86_64-linux/usr lr-x------ 1 pokybuild users 64 Jul 3 13:14 21 -> /home/pokybuild/yocto-autobuilder-kernel/yocto-worker/kernel-ppc/build/build/tmp/work/ppc7400-poky-linux/util-linux/2.26.2-r0/pseudo lrwx------ 1 pokybuild users 64 Jul 3 13:14 22 -> socket:[505273258] strace -p 9765 -f Process 9765 attached with 18 threads - interrupt to quit [pid 9856] futex(0x63b8c0, FUTEX_WAIT_PRIVATE, 2, NULL <unfinished ...> [pid 9859] futex(0x63b8c0, FUTEX_WAIT_PRIVATE, 2, NULL <unfinished ...> [pid 9853] futex(0x63b8c0, FUTEX_WAIT_PRIVATE, 2, NULL <unfinished ...> [pid 9858] futex(0x63b8c0, FUTEX_WAIT_PRIVATE, 2, NULL <unfinished ...> [pid 9851] futex(0x63b8c0, FUTEX_WAIT_PRIVATE, 2, NULL <unfinished ...> [pid 9855] futex(0x63b8c0, FUTEX_WAIT_PRIVATE, 2, NULL <unfinished ...> [pid 9849] futex(0x63b8c0, FUTEX_WAIT_PRIVATE, 2, NULL <unfinished ...> [pid 9847] futex(0x63b8c0, FUTEX_WAIT_PRIVATE, 2, NULL <unfinished ...> [pid 9844] futex(0x63b8c0, FUTEX_WAIT_PRIVATE, 2, NULL <unfinished ...> [pid 9843] futex(0x63b8c0, FUTEX_WAIT_PRIVATE, 2, NULL <unfinished ...> [pid 9850] futex(0x63b8c0, FUTEX_WAIT_PRIVATE, 2, NULL <unfinished ...> [pid 9842] futex(0x63b8c0, FUTEX_WAIT_PRIVATE, 2, NULL <unfinished ...> [pid 9857] futex(0x63b8c0, FUTEX_WAIT_PRIVATE, 2, NULL <unfinished ...> [pid 9838] futex(0x63b8c0, FUTEX_WAIT_PRIVATE, 2, NULL <unfinished ...> [pid 9836] futex(0x63b8c0, FUTEX_WAIT_PRIVATE, 2, NULL <unfinished ...> [pid 9833] futex(0x63b8c0, FUTEX_WAIT_PRIVATE, 2, NULL <unfinished ...> [pid 9765] futex(0x63b8c0, FUTEX_WAIT_PRIVATE, 2, NULL <unfinished ...> [pid 9829] futex(0x74a75c, FUTEX_WAIT_PRIVATE, 1, NULL^C <unfinished ...> Likely we need to look at the error paths in pigz and figure out whether failures are well handled. 52145 1 2576 mhalstead 2015-07-06 17:14:16 +0000 Created attachment 2576 Open file descriptor logging during the build 52146 2 mhalstead 2015-07-06 17:14:56 +0000 If it's relevant it looks like pokybuild was out of allowed open file handles. We were nowhere near the system limit: sysctl fs.file-nr fs.file-nr = 1248 0 6553322 But pokybuild is much more constrained. ulimit -Sn 1024 ulimit -Hn 4096 During the build there were some points that the open descriptors went over 8000. Linux 2.6.32-504.23.4.el6.x86_64 (centos6.osl.yoctoproject.org) 07/03/2015 _x86_64_ (24 CPU) 12:00:01 AM dentunusd file-nr inode-nr pty-nr 12:10:01 AM 1301816 8064 844409 4 12:20:01 AM 1254938 6336 742260 4 12:30:02 AM 1907781 5712 1061746 4 12:40:01 AM 1657203 6624 990812 4 12:50:01 AM 2052072 7728 1160877 4 01:00:01 AM 2152702 8736 1226978 4 01:10:01 AM 2204908 8064 1227499 4 01:20:02 AM 2125034 6528 1139800 4 01:30:01 AM 2223994 8688 1174059 4 01:40:02 AM 1808354 5856 966341 4 01:50:01 AM 1853701 7008 900623 4 Should pokybuild's limits be raised? 52150 3 ross.burton 2015-07-06 20:04:57 +0000 8000 does seem a little excessive but we are running multiple builds, and each bitbake will open inotify monitors for all of the layers, so that's a lot of FDs. I say increase it. 52152 4 mhalstead 2015-07-06 20:29:22 +0000 Every builder was set to 1024/4096 except for debian8 which is 65536/65536. Inotify watches are max 8192 on all hosts except opensuse131 where it's 65536. I'll raise it to 65536 in every case since we have plenty of memory. Do we need to document that for end users? 52241 5 ross.burton 2015-07-09 14:47:30 +0000 To test the hypothesis that this is resource limits you can use ulimit in a shell. -n will set the number of open file descriptors, for a start (and despite the documentation, works on Linux). 86100 6 randy.macleod 2020-01-23 16:10:47 +0000 This hasn't been seen on the YP autobuilder or by other users so closing. 2576 2015-07-06 17:14:16 +0000 2015-07-06 17:14:16 +0000 Open file descriptor logging during the build file-nr.txt text/plain 5136 mhalstead TGludXggMi42LjMyLTUwNC4yMy40LmVsNi54ODZfNjQgKGNlbnRvczYub3NsLnlvY3RvcHJvamVj dC5vcmcpIAkwNy8wMi8yMDE1IAlfeDg2XzY0XwkoMjQgQ1BVKQoKMTE6MDA6MDEgUE0gZGVudHVu dXNkICAgZmlsZS1uciAgaW5vZGUtbnIgICAgcHR5LW5yCjExOjEwOjAxIFBNICAgMTMzNDAyNiAg ICAgIDE4NzIgICAgNjQ2ODM4ICAgICAgICAgNAoxMToyMDowMSBQTSAgIDE1NjI4ODggICAgICAz NjAwICAgMTMzNDY5OCAgICAgICAgIDQKMTE6MzA6MDEgUE0gICAxOTY4NDg4ICAgICAgMjAxNiAg IDE0ODA0NDMgICAgICAgICA0CjExOjQwOjAyIFBNICAgMTkxMjgwOSAgICAgIDMzNjAgICAxMjYz NzA2ICAgICAgICAgNAoxMTo1MDowMSBQTSAgIDE3NTc3OTYgICAgICA0NTEyICAgMTE0NTQ4MyAg ICAgICAgIDQKCgpMaW51eCAyLjYuMzItNTA0LjIzLjQuZWw2Lng4Nl82NCAoY2VudG9zNi5vc2wu eW9jdG9wcm9qZWN0Lm9yZykgCTA3LzAzLzIwMTUgCV94ODZfNjRfCSgyNCBDUFUpCgoxMjowMDow MSBBTSBkZW50dW51c2QgICBmaWxlLW5yICBpbm9kZS1uciAgICBwdHktbnIKMTI6MTA6MDEgQU0g ICAxMzAxODE2ICAgICAgODA2NCAgICA4NDQ0MDkgICAgICAgICA0CjEyOjIwOjAxIEFNICAgMTI1 NDkzOCAgICAgIDYzMzYgICAgNzQyMjYwICAgICAgICAgNAoxMjozMDowMiBBTSAgIDE5MDc3ODEg ICAgICA1NzEyICAgMTA2MTc0NiAgICAgICAgIDQKMTI6NDA6MDEgQU0gICAxNjU3MjAzICAgICAg NjYyNCAgICA5OTA4MTIgICAgICAgICA0CjEyOjUwOjAxIEFNICAgMjA1MjA3MiAgICAgIDc3Mjgg ICAxMTYwODc3ICAgICAgICAgNAowMTowMDowMSBBTSAgIDIxNTI3MDIgICAgICA4NzM2ICAgMTIy Njk3OCAgICAgICAgIDQKMDE6MTA6MDEgQU0gICAyMjA0OTA4ICAgICAgODA2NCAgIDEyMjc0OTkg ICAgICAgICA0CjAxOjIwOjAyIEFNICAgMjEyNTAzNCAgICAgIDY1MjggICAxMTM5ODAwICAgICAg ICAgNAowMTozMDowMSBBTSAgIDIyMjM5OTQgICAgICA4Njg4ICAgMTE3NDA1OSAgICAgICAgIDQK MDE6NDA6MDIgQU0gICAxODA4MzU0ICAgICAgNTg1NiAgICA5NjYzNDEgICAgICAgICA0CjAxOjUw OjAxIEFNICAgMTg1MzcwMSAgICAgIDcwMDggICAgOTAwNjIzICAgICAgICAgNAowMjowMDowMiBB TSAgIDIyNDM2NjEgICAgICA0NjA4ICAgMTA2NDk5NSAgICAgICAgIDQKMDI6MTA6MDIgQU0gICAy MTc5NTUwICAgICAgMzc5MiAgIDEwMjE3MjIgICAgICAgICA0CjAyOjIwOjAxIEFNICAgMjAxMzU4 OSAgICAgIDMzMTIgICAgOTg1Mzg3ICAgICAgICAgNAowMjozMDowMSBBTSAgIDE3NjcwMTAgICAg ICAzMDI0ICAgIDg4ODEyMCAgICAgICAgIDQKMDI6NDA6MDIgQU0gICAxNjYyODA0ICAgICAgMzIx NiAgICA4NjM5MTggICAgICAgICA0CjAyOjUwOjAxIEFNICAgMTUyNzc4OCAgICAgIDMwMjQgICAg ODE5NTI0ICAgICAgICAgNAowMzowMDowMSBBTSAgIDE1MDY0MDAgICAgICAzMTIwICAgIDgxMTY1 OCAgICAgICAgIDQKMDM6MTA6MDEgQU0gICAxNDAxMzU0ICAgICAgMzIxNiAgICA3NzE3NjAgICAg ICAgICA0CjAzOjIwOjAxIEFNICAgMTM0MDM0OCAgICAgIDMxNjggICAgNzQ2MTIwICAgICAgICAg NAowMzozMDowMyBBTSAgIDEzMDU0NTAgICAgICAzMTIwICAgIDc1ODc3NSAgICAgICAgIDQKMDM6 NDA6MDEgQU0gICAxNDI5MDYxICAgICAgMzA3MiAgICA5MDUyMzEgICAgICAgICA0CjAzOjUwOjAx IEFNICAgMTU1NjczOCAgICAgIDMwNzIgICAxMDY2NjAyICAgICAgICAgNAowNDowMDowMSBBTSAg IDE2ODk0MzEgICAgICAzMDcyICAgMTMyMTMyMCAgICAgICAgIDQKMDQ6MTA6MDEgQU0gICAxNzYw NTM0ICAgICAgMzA3MiAgIDE0NzU5NTIgICAgICAgICA0CjA0OjIwOjAxIEFNICAgMTg3MjAxOCAg ICAgIDMwNzIgICAxNjI0NDk3ICAgICAgICAgNAowNDozMDowMSBBTSAgIDE5NjY4MjcgICAgICAz MDI0ICAgMTczMjIyNiAgICAgICAgIDQKMDQ6NDA6MDEgQU0gICAxOTM5Mjg2ICAgICAgMzIxNiAg IDE3NTE0MzkgICAgICAgICA0CjA0OjUwOjAxIEFNICAgMTk0NjQ0OCAgICAgIDMyMTYgICAxNzcy MzM3ICAgICAgICAgNAowNTowMDowMSBBTSAgIDIwNjE3MjEgICAgICAzMjE2ICAgMTg5OTE3OCAg ICAgICAgIDQKMDU6MTA6MDEgQU0gICAyMTUwOTY2ICAgICAgMzE2OCAgIDIwMzI5ODIgICAgICAg ICA0CjA1OjIwOjAxIEFNICAgMjIxMzgzOCAgICAgIDMxNjggICAyMTAxMDU1ICAgICAgICAgNAow NTozMDowMiBBTSAgIDIxMDMwNTYgICAgICAzMTY4ICAgMjAzMjUwNyAgICAgICAgIDQKMDU6NDA6 MDEgQU0gICAyMTM4MzYzICAgICAgMzAyNCAgIDIwNjkwNTYgICAgICAgICA0CjA1OjUwOjAxIEFN ICAgMjEzMTcwOCAgICAgIDI5NzYgICAyMTEwMTg2ICAgICAgICAgNAowNjowMDowMSBBTSAgIDE5 ODgwMjkgICAgICAyOTI4ICAgMTk5MDEwMiAgICAgICAgIDQKMDY6MTA6MDEgQU0gICAxODgzMDgy ICAgICAgMzAyNCAgIDE4ODU0ODQgICAgICAgICA0CjA2OjIwOjAxIEFNICAgMTc5NTAxOCAgICAg IDI5NzYgICAxNzk3OTEzICAgICAgICAgNAowNjozMDowMSBBTSAgIDE3Mjc0MTMgICAgICAzMDI0 ICAgMTczMTMxNCAgICAgICAgIDQKMDY6NDA6MDEgQU0gICAxNzIzODIzICAgICAgMzEyMCAgIDE3 MjkyNDQgICAgICAgICA0CjA2OjUwOjAxIEFNICAgMTg0OTQ4MyAgICAgIDMxMjAgICAxODU2MDM3 ICAgICAgICAgNAowNzowMDowMSBBTSAgIDE5NDY0MzkgICAgICAzMDI0ICAgMTk1MzgyNiAgICAg ICAgIDQKMDc6MTA6MDEgQU0gICAyMDI3OTk0ICAgICAgMzAyNCAgIDIwMzU1OTYgICAgICAgICA0 CjA3OjIwOjAxIEFNICAgMjA4ODI1MyAgICAgIDMwMjQgICAyMDk2MTUwICAgICAgICAgNAowNzoz MDowMSBBTSAgIDIxNDg4MjIgICAgICAzMDI0ICAgMjE1NzIzOCAgICAgICAgIDQKMDc6NDA6MDEg QU0gICAyMjg5NDMxICAgICAgMzA3MiAgIDIyOTgxMTkgICAgICAgICA0CjA3OjUwOjAyIEFNICAg MjYyNTE3NiAgICAgIDMwNzIgICAyNjMzMzkzICAgICAgICAgNAowODowMDowMSBBTSAgIDI3Nzk2 MTMgICAgICAzMTIwICAgMjc4ODQxNyAgICAgICAgIDQKMDg6MTA6MDEgQU0gICAzMDAyNTA4ICAg ICAgMzA3MiAgIDMwMTE2NDcgICAgICAgICA0CjA4OjIwOjAxIEFNICAgMzUxOTE1MSAgICAgIDMw NzIgICAzNTI4MTM3ICAgICAgICAgNAowODozMDowMSBBTSAgIDM5NTM2OTUgICAgICAzMDI0ICAg Mzk2MjcyOCAgICAgICAgIDQKMDg6NDA6MDEgQU0gICA0Mzk3NjQ2ICAgICAgMzA3MiAgIDQ0MDY5 OTAgICAgICAgICA0CjA4OjUwOjAxIEFNICAgNDI2NDczMyAgICAgIDMwMjQgICA0Mjc0MDA5ICAg ICAgICAgNAowOTowMDowMSBBTSAgIDQxMDA5MjMgICAgICAzMDI0ICAgNDExMDYwMCAgICAgICAg IDQKMDk6MTA6MDEgQU0gICAzOTYwMTg5ICAgICAgMzAyNCAgIDM5NjkwMzAgICAgICAgICA0CjA5 OjIwOjAxIEFNICAgMzg2NDg1NiAgICAgIDI5NzYgICAzODczMTQyICAgICAgICAgNAowOTozMDow MSBBTSAgIDM4NjQ4NjIgICAgICAyOTI4ICAgMzg3MzE0NCAgICAgICAgIDQKMDk6NDA6MDEgQU0g ICAzODY0OTA2ICAgICAgMzA3MiAgIDM4NzMxNjQgICAgICAgICA0CjA5OjUwOjAxIEFNICAgMzg2 NDkwNiAgICAgIDMwNzIgICAzODczMTY0ICAgICAgICAgNAoxMDowMDowMSBBTSAgIDM4NjQ5MDcg ICAgICAzMDcyICAgMzg3MzE2NSAgICAgICAgIDQKMTA6MTA6MDEgQU0gICAzODY0OTE2ICAgICAg MzAyNCAgIDM4NzMxNjggICAgICAgICA0CjEwOjIwOjAxIEFNICAgMzg2NDkxNiAgICAgIDMwMjQg ICAzODczMTY4ICAgICAgICAgNAoxMDozMDowMSBBTSAgIDM4NjQ5MTYgICAgICAyOTc2ICAgMzg3 MzE2OCAgICAgICAgIDQKCjEwOjMwOjAxIEFNIGRlbnR1bnVzZCAgIGZpbGUtbnIgIGlub2RlLW5y ICAgIHB0eS1ucgoxMDo0MDowMiBBTSAgIDM4NjQ5MTkgICAgICAyOTc2ICAgMzg3MzE2NCAgICAg ICAgIDQKMTA6NTA6MDEgQU0gICAzODY0OTQwICAgICAgMjk3NiAgIDM4NzMxODYgICAgICAgICA0 CjExOjAwOjAxIEFNICAgMzg2NDk0MCAgICAgIDI5MjggICAzODczMTg2ICAgICAgICAgNAoxMTox MDowMSBBTSAgIDM4NjQ5NDAgICAgICAyOTc2ICAgMzg3MzE4NiAgICAgICAgIDQKMTE6MjA6MDEg QU0gICAzODY0OTQwICAgICAgMjk3NiAgIDM4NzMxODYgICAgICAgICA0CjExOjMwOjAxIEFNICAg Mzg2NDk0MCAgICAgIDI5NzYgICAzODczMTg2ICAgICAgICAgNAoxMTo0MDowMSBBTSAgIDM4NjQ5 NDAgICAgICAyOTI4ICAgMzg3MzE4NiAgICAgICAgIDQKMTE6NTA6MDEgQU0gICAzODY0OTU4ICAg ICAgMjk3NiAgIDM4NzMyMDUgICAgICAgICA0CjEyOjAwOjAxIFBNICAgMzg2NDk1OCAgICAgIDMw MjQgICAzODczMjA1ICAgICAgICAgNAoxMjoxMDowMSBQTSAgIDM4NjQ5NTggICAgICAyOTc2ICAg Mzg3MzIwNSAgICAgICAgIDQKMTI6MjA6MDEgUE0gICAzODY0OTU1ICAgICAgMjkyOCAgIDM4NzMy MDEgICAgICAgICA0CjEyOjMwOjAxIFBNICAgMzg2NDk1OCAgICAgIDMwMjQgICAzODczMjA1ICAg ICAgICAgNAoxMjo0MDowMSBQTSAgIDM4NjQ5NTggICAgICAyOTc2ICAgMzg3MzIwNSAgICAgICAg IDQKMTI6NTA6MDEgUE0gICAzODY0OTU4ICAgICAgMzAyNCAgIDM4NzMyMDUgICAgICAgICA0CjAx OjAwOjAxIFBNICAgMzg2NDk1OCAgICAgIDMxMjAgICAzODczMjA1ICAgICAgICAgNAowMToxMDow MSBQTSAgIDM4NjY5MDQgICAgICAzMTY4ICAgMzg3NTY1NCAgICAgICAgIDYKMDE6MjA6MDEgUE0g ICAzODY3ODYzICAgICAgMzMxMiAgIDM4NzYzNDMgICAgICAgICA2CjAxOjMwOjAxIFBNICAgMzg2 ODAzNCAgICAgIDMyNjQgICAzODc2NDU2ICAgICAgICAgNgowMTo0MDowMSBQTSAgIDM4NjgzNTIg ICAgICAzMzEyICAgMzg3NjY4OCAgICAgICAgIDcKMDE6NTA6MDEgUE0gICAzODcwNjM4ICAgICAg MzI2NCAgIDM4Nzg5MzggICAgICAgICA3CjAyOjAwOjAxIFBNICAgMzg3MDY0MCAgICAgIDMyNjQg ICAzODc4OTQyICAgICAgICAgNwowMjoxMDowMSBQTSAgIDM4NzE1MDQgICAgICAzMDcyICAgMzg3 OTcwMyAgICAgICAgIDcKMDI6MjA6MDEgUE0gICAzODcxNDYyICAgICAgMjkyOCAgIDM4Nzk2MjAg ICAgICAgICA1CjAyOjMwOjAxIFBNICAgMzg3MTQ3NSAgICAgIDI5MjggICAzODc5NjI1ICAgICAg ICAgNQoK