| Summary: | pseudo: Different file attributes detected by buildhistory | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Product: | [Build System, Metadata & Runtime] OE-Core | Reporter: | Martin Jansa <Martin.Jansa> | ||||||||
| Component: | core | Assignee: | Seebs <seebs> | ||||||||
| Status: | RESOLVED FIXED | QA Contact: | |||||||||
| Severity: | normal | ||||||||||
| Priority: | Medium+ | CC: | meta.mr.watcher, meta.watcher, richard.purdie, sgw | ||||||||
| Version: | 1.4 | ||||||||||
| Target Milestone: | 1.5.1 | ||||||||||
| Hardware: | x86 | ||||||||||
| OS: | Multiple | ||||||||||
| Whiteboard: | |||||||||||
| OS type for building Yocto: | --- | Type of Regression: | --- | ||||||||
| Verified: | Documentation change: | Don't know | |||||||||
| Attachments: |
|
||||||||||
|
Description
Martin Jansa
2013-09-06 07:37:36 UTC
Created attachment 1466 [details]
procps-pseudo-4.tgz
Created attachment 1467 [details]
procps-pseudo-5.tgz
It looks like -rwxr-xr-x is used everywhere except files-in-image.txt: OE @ ~/oe-core/tmp-eglibc $ ll from-scratch*/work/x86_64-oe-linux/procps/3.2.8-r11/package/usr/bin/slabtop -rwxr-xr-x 2 bitbake bitbake 12360 Sep 5 19:50 from-scratch1/work/x86_64-oe-linux/procps/3.2.8-r11/package/usr/bin/slabtop* -rwxr-xr-x 2 bitbake bitbake 12360 Sep 5 20:16 from-scratch2/work/x86_64-oe-linux/procps/3.2.8-r11/package/usr/bin/slabtop* -rwxr-xr-x 2 bitbake bitbake 12360 Sep 5 20:33 from-scratch3/work/x86_64-oe-linux/procps/3.2.8-r11/package/usr/bin/slabtop* -rwxr-xr-x 2 bitbake bitbake 12360 Sep 6 09:30 from-scratch4/work/x86_64-oe-linux/procps/3.2.8-r11/package/usr/bin/slabtop* -rwxr-xr-x 2 bitbake bitbake 12360 Sep 6 01:34 from-scratch5/work/x86_64-oe-linux/procps/3.2.8-r11/package/usr/bin/slabtop* OE @ ~/oe-core/tmp-eglibc $ ll from-scratch*/work/x86_64-oe-linux/procps/3.2.8-r11/packages-split/procps/usr/bin/slabtop -rwxr-xr-x 2 bitbake bitbake 12360 Sep 5 19:50 from-scratch1/work/x86_64-oe-linux/procps/3.2.8-r11/packages-split/procps/usr/bin/slabtop* -rwxr-xr-x 2 bitbake bitbake 12360 Sep 5 20:16 from-scratch2/work/x86_64-oe-linux/procps/3.2.8-r11/packages-split/procps/usr/bin/slabtop* -rwxr-xr-x 2 bitbake bitbake 12360 Sep 5 20:33 from-scratch3/work/x86_64-oe-linux/procps/3.2.8-r11/packages-split/procps/usr/bin/slabtop* -rwxr-xr-x 2 bitbake bitbake 12360 Sep 6 09:30 from-scratch4/work/x86_64-oe-linux/procps/3.2.8-r11/packages-split/procps/usr/bin/slabtop* -rwxr-xr-x 2 bitbake bitbake 12360 Sep 6 01:34 from-scratch5/work/x86_64-oe-linux/procps/3.2.8-r11/packages-split/procps/usr/bin/slabtop* OE @ ~/oe-core/tmp-eglibc $ ll from-scratch*/work/qemux86_64-oe-linux/core-image-minimal/1.0-r0/rootfs/usr/bin/slabtop -rwxr-xr-x 1 bitbake bitbake 12360 Sep 5 19:50 from-scratch1/work/qemux86_64-oe-linux/core-image-minimal/1.0-r0/rootfs/usr/bin/slabtop* -rwxr-xr-x 1 bitbake bitbake 12360 Sep 5 20:16 from-scratch2/work/qemux86_64-oe-linux/core-image-minimal/1.0-r0/rootfs/usr/bin/slabtop* -rwxr-xr-x 1 bitbake bitbake 12360 Sep 5 20:33 from-scratch3/work/qemux86_64-oe-linux/core-image-minimal/1.0-r0/rootfs/usr/bin/slabtop* -rwxr-xr-x 1 bitbake bitbake 12360 Sep 6 09:30 from-scratch4/work/qemux86_64-oe-linux/core-image-minimal/1.0-r0/rootfs/usr/bin/slabtop* -rwxr-xr-x 1 bitbake bitbake 12360 Sep 6 01:34 from-scratch5/work/qemux86_64-oe-linux/core-image-minimal/1.0-r0/rootfs/usr/bin/slabtop* OE @ ~/oe-core/tmp-eglibc $ sqlite3 core-image-minimal-pseudo-4/files.db sqlite> select * from files where path like '%slabtop%'; 206|/OE/oe-core/tmp-eglibc/work/qemux86_64-oe-linux/core-image-minimal/1.0-r0/rootfs/usr/bin/slabtop|2081|19244656|0|0|33261|0|0 OE @ ~/oe-core/tmp-eglibc $ sqlite3 core-image-minimal-pseudo-5/files.db sqlite> select * from files where path like '%slabtop%'; 206|/OE/oe-core/tmp-eglibc/work/qemux86_64-oe-linux/core-image-minimal/1.0-r0/rootfs/usr/bin/slabtop|2081|18927165|0|0|33133|0|0 OE @ ~/oe-core/tmp-eglibc $ sqlite3 core-image-minimal-pseudo-4/logs.db sqlite> select * from logs where path like '%slabtop%'; 16093|1378452666|6|5415||2081|19244656|33188|/OE/oe-core/tmp-eglibc/work/qemux86_64-oe-linux/core-image-minimal/1.0-r0/rootfs/usr/bin/slabtop|1|2|||0|0|0|opkg-cl|6 16094|1378452666|14|5415||2081|19244656|33188|/OE/oe-core/tmp-eglibc/work/qemux86_64-oe-linux/core-image-minimal/1.0-r0/rootfs/usr/bin/slabtop|1|2|||1026|1026|2|opkg-cl|6 16095|1378452666|3|5415||2081|19244656|33188|/OE/oe-core/tmp-eglibc/work/qemux86_64-oe-linux/core-image-minimal/1.0-r0/rootfs/usr/bin/slabtop|1|2|||0|0|0|opkg-cl|6 16096|1378452666|2|5415||2081|19244656|33261|/OE/oe-core/tmp-eglibc/work/qemux86_64-oe-linux/core-image-minimal/1.0-r0/rootfs/usr/bin/slabtop|1|2|||0|0|0|opkg-cl|6 54059|1378452693|16|11512||2081|19244656|33261|/OE/oe-core/tmp-eglibc/work/qemux86_64-oe-linux/core-image-minimal/1.0-r0/rootfs/usr/bin/slabtop|1|2|||0|0|0|/usr/bin/python2.7|3 72852|1378452729|16|12518||2081|19244656|33261|/OE/oe-core/tmp-eglibc/work/qemux86_64-oe-linux/core-image-minimal/1.0-r0/rootfs/usr/bin/slabtop|1|2|||0|0|0|du|3 80923|1378452730|16|12520||2081|19244656|33261|/OE/oe-core/tmp-eglibc/work/qemux86_64-oe-linux/core-image-minimal/1.0-r0/rootfs/usr/bin/slabtop|1|2|||0|0|0|tar|3 80925|1378452730|14|12520||2081|19244656|32813|/OE/oe-core/tmp-eglibc/work/qemux86_64-oe-linux/core-image-minimal/1.0-r0/rootfs/usr/bin/slabtop|1|2|||1026|1026|4|tar|6 80927|1378452730|10|12520||2081|19244656|33261|/OE/oe-core/tmp-eglibc/work/qemux86_64-oe-linux/core-image-minimal/1.0-r0/rootfs/usr/bin/slabtop|1|2|||0|0|0|tar|3 80930|1378452730|10|12520||2081|19244656|33261|/OE/oe-core/tmp-eglibc/work/qemux86_64-oe-linux/core-image-minimal/1.0-r0/rootfs/usr/bin/slabtop|1|2|||0|0|0|tar|3 83111|1378452734|16|12547||2081|19244656|33261|/OE/oe-core/tmp-eglibc/work/qemux86_64-oe-linux/core-image-minimal/1.0-r0/rootfs/usr/bin/slabtop|1|2|||0|0|0|du|3 115893|1378452746|16|14083||2081|19244656|33261|/OE/oe-core/tmp-eglibc/work/qemux86_64-oe-linux/core-image-minimal/1.0-r0/rootfs/usr/bin/slabtop|1|2|||0|0|0|stat|3 115895|1378452746|14|12554||2081|19244656|32813|/OE/oe-core/tmp-eglibc/work/qemux86_64-oe-linux/core-image-minimal/1.0-r0/rootfs/usr/bin/slabtop|1|2|||1026|1026|4|debugfs|6 115897|1378452746|10|12554||2081|19244656|33261|/OE/oe-core/tmp-eglibc/work/qemux86_64-oe-linux/core-image-minimal/1.0-r0/rootfs/usr/bin/slabtop|1|2|||0|0|0|debugfs|3 120567|1378452747|16|14279||2081|19244656|33261|/OE/oe-core/tmp-eglibc/work/qemux86_64-oe-linux/core-image-minimal/1.0-r0/rootfs/usr/bin/slabtop|1|2|||0|0|0|find|3 122446|1378452748|16|14337||2081|19244656|33261|/OE/oe-core/tmp-eglibc/work/qemux86_64-oe-linux/core-image-minimal/1.0-r0/rootfs/usr/bin/slabtop|1|2|||0|0|0|find|3 124174|1378452748|16|14350||2081|19244656|33261|/OE/oe-core/tmp-eglibc/work/qemux86_64-oe-linux/core-image-minimal/1.0-r0/rootfs/usr/bin/slabtop|1|2|||0|0|0|du|3 OE @ ~/oe-core/tmp-eglibc $ sqlite3 core-image-minimal-pseudo-5/logs.db sqlite> select * from logs where path like '%slabtop%'; 16097|1378424324|6|10619||2081|18927165|33188|/OE/oe-core/tmp-eglibc/work/qemux86_64-oe-linux/core-image-minimal/1.0-r0/rootfs/usr/bin/slabtop|1|2|||0|0|0|opkg-cl|6 16098|1378424324|14|10619||2081|18927165|33188|/OE/oe-core/tmp-eglibc/work/qemux86_64-oe-linux/core-image-minimal/1.0-r0/rootfs/usr/bin/slabtop|1|2|||1026|1026|2|opkg-cl|6 16099|1378424324|3|10619||2081|18927165|33188|/OE/oe-core/tmp-eglibc/work/qemux86_64-oe-linux/core-image-minimal/1.0-r0/rootfs/usr/bin/slabtop|1|2|||0|0|0|opkg-cl|6 16100|1378424324|2|10619||2081|18927165|33133|/OE/oe-core/tmp-eglibc/work/qemux86_64-oe-linux/core-image-minimal/1.0-r0/rootfs/usr/bin/slabtop|1|2|||0|0|0|opkg-cl|6 54015|1378424351|16|19727||2081|18927165|33133|/OE/oe-core/tmp-eglibc/work/qemux86_64-oe-linux/core-image-minimal/1.0-r0/rootfs/usr/bin/slabtop|1|2|||0|0|0|/usr/bin/python2.7|3 72808|1378424361|16|21005||2081|18927165|33133|/OE/oe-core/tmp-eglibc/work/qemux86_64-oe-linux/core-image-minimal/1.0-r0/rootfs/usr/bin/slabtop|1|2|||0|0|0|du|3 80613|1378424361|16|21013||2081|18927165|33133|/OE/oe-core/tmp-eglibc/work/qemux86_64-oe-linux/core-image-minimal/1.0-r0/rootfs/usr/bin/slabtop|1|2|||0|0|0|tar|3 80615|1378424361|14|21013||2081|18927165|32813|/OE/oe-core/tmp-eglibc/work/qemux86_64-oe-linux/core-image-minimal/1.0-r0/rootfs/usr/bin/slabtop|1|2|||1026|1026|4|tar|6 80616|1378424361|10|21013||2081|18927165|33133|/OE/oe-core/tmp-eglibc/work/qemux86_64-oe-linux/core-image-minimal/1.0-r0/rootfs/usr/bin/slabtop|1|2|||0|0|0|tar|3 80618|1378424361|10|21013||2081|18927165|33133|/OE/oe-core/tmp-eglibc/work/qemux86_64-oe-linux/core-image-minimal/1.0-r0/rootfs/usr/bin/slabtop|1|2|||0|0|0|tar|3 82802|1378424366|16|21224||2081|18927165|33133|/OE/oe-core/tmp-eglibc/work/qemux86_64-oe-linux/core-image-minimal/1.0-r0/rootfs/usr/bin/slabtop|1|2|||0|0|0|du|3 115366|1378424376|16|23609||2081|18927165|33133|/OE/oe-core/tmp-eglibc/work/qemux86_64-oe-linux/core-image-minimal/1.0-r0/rootfs/usr/bin/slabtop|1|2|||0|0|0|stat|3 115368|1378424376|14|21303||2081|18927165|32813|/OE/oe-core/tmp-eglibc/work/qemux86_64-oe-linux/core-image-minimal/1.0-r0/rootfs/usr/bin/slabtop|1|2|||1026|1026|4|debugfs|6 115369|1378424376|10|21303||2081|18927165|33133|/OE/oe-core/tmp-eglibc/work/qemux86_64-oe-linux/core-image-minimal/1.0-r0/rootfs/usr/bin/slabtop|1|2|||0|0|0|debugfs|3 119930|1378424377|16|23883||2081|18927165|33133|/OE/oe-core/tmp-eglibc/work/qemux86_64-oe-linux/core-image-minimal/1.0-r0/rootfs/usr/bin/slabtop|1|2|||0|0|0|find|3 121838|1378424378|16|23962||2081|18927165|33133|/OE/oe-core/tmp-eglibc/work/qemux86_64-oe-linux/core-image-minimal/1.0-r0/rootfs/usr/bin/slabtop|1|2|||0|0|0|find|3 123566|1378424378|16|23978||2081|18927165|33133|/OE/oe-core/tmp-eglibc/work/qemux86_64-oe-linux/core-image-minimal/1.0-r0/rootfs/usr/bin/slabtop|1|2|||0|0|0|du|3 OK, lets go back from rootfs WORKDIR to procps pseudo, because it's wrong in .ipk already: bitbake@jama:/OE$ dpkg --contents procps-4.ipk | grep slab -rwxr-xr-x root/root 12360 2013-09-06 09:30 ./usr/bin/slabtop bitbake@jama:/OE$ dpkg --contents procps-5.ipk | grep slab -r-xr-xr-x root/root 12360 2013-09-06 01:34 ./usr/bin/slabtop I don't know how to properly read this log yet, but the issue seems to start on id 7801/8031 when scratch4 logs python2.7 before strip is completed. from-scratch4: 7799|1378452647|15|4650||2081|19244451|33225|WORK/package/usr/bin/slabtop|1|2|||1026|1026|0|x86_64-oe-linux-strip|6 7800|1378452647|2|4650||2081|19244451|33261|WORK/package/usr/bin/slabtop|1|2|||0|0|0|x86_64-oe-linux-strip|6 7801|1378452647|2|4641||2081|19244451|33133|WORK/package/usr/bin/slabtop|1|2|||0|0|0|/usr/bin/python2.7|6 7802|1378452647|3|4650||2081|19244451|33133|WORK/package/usr/bin/slabtop|1|2|||0|0|0|x86_64-oe-linux-strip|6 7804|1378452647|2|4650||2081|19244451|33261|WORK/package/usr/bin/slabtop|1|2|||0|0|0|x86_64-oe-linux-strip|6 9182|1378452647|16|4300||2081|19244451|33261|WORK/package/usr/bin/slabtop|1|2|||0|0|0|/usr/bin/python2.7|3 9183|1378452647|16|4300||2081|19244451|33261|WORK/package/usr/bin/slabtop|1|2|||0|0|0|/usr/bin/python2.7|3 9184|1378452647|16|4300||2081|19244451|33261|WORK/package/usr/bin/slabtop|1|2|||0|0|0|/usr/bin/python2.7|3 9185|1378452647|11|4300||2081|19244451|33261|WORK/packages-split/procps/usr/bin/slabtop|1|2|||0|0|0|/usr/bin/python2.7|6 9186|1378452647|2|4300||2081|19244451|33261|WORK/packages-split/procps/usr/bin/slabtop|1|2|||0|0|0|/usr/bin/python2.7|6 9187|1378452647|3|4300||2081|19244451|33261|WORK/packages-split/procps/usr/bin/slabtop|1|2|||0|0|0|/usr/bin/python2.7|6 9310|1378452647|16|4300||2081|19244451|33261|WORK/package/usr/bin/slabtop|1|2|||0|0|0|/usr/bin/python2.7|3 from-scratch5: 8021|1378424090|15|31890||2081|18822904|33225|WORK/package/usr/bin/slabtop|1|2|||1026|1026|0|x86_64-oe-linux-strip|6 8022|1378424090|2|31890||2081|18822904|33261|WORK/package/usr/bin/slabtop|1|2|||0|0|0|x86_64-oe-linux-strip|6 8024|1378424090|3|31890||2081|18822904|33261|WORK/package/usr/bin/slabtop|1|2|||0|0|0|x86_64-oe-linux-strip|6 8026|1378424090|2|31890||2081|18822904|33261|WORK/package/usr/bin/slabtop|1|2|||0|0|0|x86_64-oe-linux-strip|6 8031|1378424090|2|31857||2081|18822904|33133|WORK/package/usr/bin/slabtop|1|2|||0|0|0|/usr/bin/python2.7|6 9260|1378424090|16|30521||2081|18822904|33133|WORK/package/usr/bin/slabtop|1|2|||0|0|0|/usr/bin/python2.7|3 9261|1378424090|16|30521||2081|18822904|33133|WORK/package/usr/bin/slabtop|1|2|||0|0|0|/usr/bin/python2.7|3 9262|1378424090|16|30521||2081|18822904|33133|WORK/package/usr/bin/slabtop|1|2|||0|0|0|/usr/bin/python2.7|3 9263|1378424090|11|30521||2081|18822904|33133|WORK/packages-split/procps/usr/bin/slabtop|1|2|||0|0|0|/usr/bin/python2.7|6 9264|1378424090|2|30521||2081|18822904|33133|WORK/packages-split/procps/usr/bin/slabtop|1|2|||0|0|0|/usr/bin/python2.7|6 9265|1378424090|3|30521||2081|18822904|33133|WORK/packages-split/procps/usr/bin/slabtop|1|2|||0|0|0|/usr/bin/python2.7|6 9388|1378424090|16|30521||2081|18822904|33133|WORK/package/usr/bin/slabtop|1|2|||0|0|0|/usr/bin/python2.7|3 It's easier to see in pseudolog output: --- from-scratch4/pseudo/slabtop.log4 2013-09-06 11:23:58.443923636 +0200 +++ from-scratch5/pseudo/slabtop.log4 2013-09-06 11:24:01.023923698 +0200 @@ -46,89 +46,89 @@ x86_64-oe-li fastop did-unlink: [mode 0755, -] WORK/package/usr/bin/slabtop x86_64-oe-li fastop rename: [mode 0711, -] WORK/package/usr/bin/slabtop x86_64-oe-li fastop chmod: [mode 0755, -] WORK/package/usr/bin/slabtop -/usr/bin/pyt fastop chmod: [mode 0555, -] WORK/package/usr/bin/slabtop -x86_64-oe-li fastop chown: [mode 0555, -] WORK/package/usr/bin/slabtop +x86_64-oe-li fastop chown: [mode 0755, -] WORK/package/usr/bin/slabtop x86_64-oe-li fastop chmod: [mode 0755, -] WORK/package/usr/bin/slabtop -/usr/bin/pyt op stat: [mode 0755, -] WORK/package/usr/bin/slabtop -/usr/bin/pyt op stat: [mode 0755, -] WORK/package/usr/bin/slabtop -/usr/bin/pyt op stat: [mode 0755, -] WORK/package/usr/bin/slabtop -/usr/bin/pyt fastop link: [mode 0755, -] WORK/packages-split/procps/usr/bin/slabtop -/usr/bin/pyt fastop chmod: [mode 0755, -] WORK/packages-split/procps/usr/bin/slabtop -/usr/bin/pyt fastop chown: [mode 0755, -] WORK/packages-split/procps/usr/bin/slabtop -/usr/bin/pyt op stat: [mode 0755, -] WORK/package/usr/bin/slabtop -/usr/bin/pyt op stat: [mode 0755, -] WORK/packages-split/procps/usr/bin/slabtop +/usr/bin/pyt fastop chmod: [mode 0555, -] WORK/package/usr/bin/slabtop +/usr/bin/pyt op stat: [mode 0555, -] WORK/package/usr/bin/slabtop +/usr/bin/pyt op stat: [mode 0555, -] WORK/package/usr/bin/slabtop +/usr/bin/pyt op stat: [mode 0555, -] WORK/package/usr/bin/slabtop +/usr/bin/pyt fastop link: [mode 0555, -] WORK/packages-split/procps/usr/bin/slabtop +/usr/bin/pyt fastop chmod: [mode 0555, -] WORK/packages-split/procps/usr/bin/slabtop +/usr/bin/pyt fastop chown: [mode 0555, -] WORK/packages-split/procps/usr/bin/slabtop +/usr/bin/pyt op stat: [mode 0555, -] WORK/package/usr/bin/slabtop +/usr/bin/pyt op stat: [mode 0555, -] WORK/packages-split/procps/usr/bin/slabtop /usr/bin/pyt fastop open: [mode 0755, r] WORK/packages-split/procps/usr/bin/slabtop ... only difference on following lines is 0555/0755 mode. So the problem is chmod from python happening a bit sooner in some builds. There is also some reuse of the same inode in logs:
sqlite> select distinct path from logs where ino=5530781;
/OE/oe-core/tmp-eglibc/work/x86_64-oe-linux/procps/3.2.8-r11/package/usr/bin/stctlQEd
/OE/oe-core/tmp-eglibc/work/x86_64-oe-linux/procps/3.2.8-r11/package/usr/bin/slabtop
/OE/oe-core/tmp-eglibc/work/x86_64-oe-linux/procps/3.2.8-r11/package/usr/bin/stNWC016
/OE/oe-core/tmp-eglibc/work/x86_64-oe-linux/procps/3.2.8-r11/package/usr/bin/free.procps
/OE/oe-core/tmp-eglibc/work/x86_64-oe-linux/procps/3.2.8-r11/packages-split/procps/usr/bin/free.procps
/OE/oe-core/tmp-eglibc/work/x86_64-oe-linux/procps/3.2.8-r11/sstate-build-package/package/usr/bin/free.procps
/OE/oe-core/tmp-eglibc/work/x86_64-oe-linux/procps/3.2.8-r11/sstate-build-package/packages-split/procps/usr/bin/free.procps
I was suspecting race-condition openembedded-core/meta/lib/oe/package.py:runstrip()
which has that chmod adding +W+R and is executed in parallel from package.bbclass:
import multiprocessing
nproc = multiprocessing.cpu_count()
pool = bb.utils.multiprocessingpool(nproc)
processed = list(pool.imap(oe.package.runstrip, sfiles))
pool.close()
pool.join()
But with added warnings in runstrip code it looks to be executed in the same order (if we can trust order in log file).
from-scratch7/log.txt:WARNING: runstrip: /OE/oe-core/tmp-eglibc/work/x86_64-oe-linux/procps/3.2.8-r11/package/usr/bin/slabtop
from-scratch7/log.txt:WARNING: runstrip: /OE/oe-core/tmp-eglibc/work/x86_64-oe-linux/procps/3.2.8-r11/package/usr/bin/free.procps
from-scratch7/log.txt:WARNING: runstrip chmod: /OE/oe-core/tmp-eglibc/work/x86_64-oe-linux/procps/3.2.8-r11/package/usr/bin/free.procps - 33261
from-scratch7/log.txt:WARNING: runstrip chmod: /OE/oe-core/tmp-eglibc/work/x86_64-oe-linux/procps/3.2.8-r11/package/usr/bin/slabtop - 33261
from-scratch8/log.txt:WARNING: runstrip: /OE/oe-core/tmp-eglibc/work/x86_64-oe-linux/procps/3.2.8-r11/package/usr/bin/slabtop
from-scratch8/log.txt:WARNING: runstrip: /OE/oe-core/tmp-eglibc/work/x86_64-oe-linux/procps/3.2.8-r11/package/usr/bin/free.procps
from-scratch8/log.txt:WARNING: runstrip chmod: /OE/oe-core/tmp-eglibc/work/x86_64-oe-linux/procps/3.2.8-r11/package/usr/bin/slabtop - 33261
from-scratch8/log.txt:WARNING: runstrip chmod: /OE/oe-core/tmp-eglibc/work/x86_64-oe-linux/procps/3.2.8-r11/package/usr/bin/free.procps - 33261
Peter, I hate to say this but I think this is a bug in pseudo, specifically in http://git.yoctoproject.org/cgit.cgi/pseudo/commit/?id=01bc8866a4aa5af17350b36e2347dc1695dbceb0. In one of the builds where I could show this breaking, this is the output from pseudolog (btw, please put the default format string in the help text!): $ PSEUDO_BINDIR=/media/build1/poky/build/tmp/sysroots/x86_64-linux/usr/bin PSEUDO_LIBDIR=/media/build1/poky/build/tmp/sysroots/x86_64-linux/usr/lib/pseudo/lib PSEUDO_LOCALSTATEDIR=/media/build1/poky/build/tmp/work/i586-poky-linux/procps/3.2.8-r12/pseudo/ PSEUDO_NOSYMLINKEXP=1 PSEUDO_PASSWD=/media/build1/poky/build/tmp/sysroots/qemux86 PSEUDO_PREFIX=/media/build1/poky/build/tmp/sysroots/x86_64-linux/usr ../../../../../sysroots/x86_64-linux/usr/bin/pseudolog -F "%s %R %-4y %7o: [mode %04m, %2a] %p %T" | grep package/usr/bin/slabtop 13:56:08 rm op may-unlink: [mode 0755, -] /media/build1/poky/build/tmp/work/i586-poky-linux/procps/3.2.8-r12/package/usr/bin/slabtop 13:56:08 rm fastop did-unlink: [mode 0755, -] /media/build1/poky/build/tmp/work/i586-poky-linux/procps/3.2.8-r12/package/usr/bin/slabtop 13:56:08 tar fastop creat: [mode 0500, -] /media/build1/poky/build/tmp/work/i586-poky-linux/procps/3.2.8-r12/package/usr/bin/slabtop 13:56:08 tar fastop open: [mode 0500, w] /media/build1/poky/build/tmp/work/i586-poky-linux/procps/3.2.8-r12/package/usr/bin/slabtop 13:56:09 python op stat: [mode 0555, -] /media/build1/poky/build/tmp/work/i586-poky-linux/procps/3.2.8-r12/package/usr/bin/slabtop 13:56:09 /media/build1/poky/build/tmp/work/x86_64-linux/file-native/5.14-r0/image//media/build1/poky/build/tmp/sysroots/x86_64-linux/usr/bin/file op stat: [mode 0555, -] /media/build1/poky/build/tmp/work/i586-poky-linux/procps/3.2.8-r12/package/usr/bin/slabtop 13:56:09 /media/build1/poky/build/tmp/work/x86_64-linux/file-native/5.14-r0/image//media/build1/poky/build/tmp/sysroots/x86_64-linux/usr/bin/file op stat: [mode 0555, -] /media/build1/poky/build/tmp/work/i586-poky-linux/procps/3.2.8-r12/package/usr/bin/slabtop 13:56:09 python op stat: [mode 0555, -] /media/build1/poky/build/tmp/work/i586-poky-linux/procps/3.2.8-r12/package/usr/bin/slabtop 13:56:09 python fastop chmod: [mode 0755, -] /media/build1/poky/build/tmp/work/i586-poky-linux/procps/3.2.8-r12/package/usr/bin/slabtop 13:56:09 /media/build1/poky/build/tmp/sysroots/x86_64-linux/usr/lib/rpm/bin/debugedit op stat: [mode 0755, -] /media/build1/poky/build/tmp/work/i586-poky-linux/procps/3.2.8-r12/package/usr/bin/slabtop 13:56:09 /media/build1/poky/build/tmp/sysroots/x86_64-linux/usr/lib/rpm/bin/debugedit fastop chmod: [mode 0755, -] /media/build1/poky/build/tmp/work/i586-poky-linux/procps/3.2.8-r12/package/usr/bin/slabtop 13:56:09 /media/build1/poky/build/tmp/sysroots/x86_64-linux/usr/lib/rpm/bin/debugedit fastop open: [mode 0055, r+] /media/build1/poky/build/tmp/work/i586-poky-linux/procps/3.2.8-r12/package/usr/bin/slabtop 13:56:09 /media/build1/poky/build/tmp/sysroots/x86_64-linux/usr/lib/rpm/bin/debugedit op fstat: [mode 0755, -] /media/build1/poky/build/tmp/work/i586-poky-linux/procps/3.2.8-r12/package/usr/bin/slabtop 13:56:09 /media/build1/poky/build/tmp/sysroots/x86_64-linux/usr/lib/rpm/bin/debugedit op fstat: [mode 0755, -] /media/build1/poky/build/tmp/work/i586-poky-linux/procps/3.2.8-r12/package/usr/bin/slabtop 13:56:09 /media/build1/poky/build/tmp/sysroots/x86_64-linux/usr/lib/rpm/bin/debugedit fastop chmod: [mode 0755, -] /media/build1/poky/build/tmp/work/i586-poky-linux/procps/3.2.8-r12/package/usr/bin/slabtop 13:56:09 i586-poky-linux-objcopy op stat: [mode 0755, -] /media/build1/poky/build/tmp/work/i586-poky-linux/procps/3.2.8-r12/package/usr/bin/slabtop 13:56:09 i586-poky-linux-objcopy op stat: [mode 0755, -] /media/build1/poky/build/tmp/work/i586-poky-linux/procps/3.2.8-r12/package/usr/bin/slabtop 13:56:09 i586-poky-linux-objcopy fastop open: [mode 0755, r] /media/build1/poky/build/tmp/work/i586-poky-linux/procps/3.2.8-r12/package/usr/bin/slabtop 13:56:09 i586-poky-linux-objcopy op stat: [mode 0755, -] /media/build1/poky/build/tmp/work/i586-poky-linux/procps/3.2.8-r12/package/usr/bin/slabtop 13:56:09 i586-poky-linux-objcopy op stat: [mode 0755, -] /media/build1/poky/build/tmp/work/i586-poky-linux/procps/3.2.8-r12/package/usr/bin/slabtop 13:56:09 i586-poky-linux-objcopy fastop open: [mode 0755, r] /media/build1/poky/build/tmp/work/i586-poky-linux/procps/3.2.8-r12/package/usr/bin/slabtop 13:56:09 i586-poky-linux-objcopy op stat: [mode 0755, -] /media/build1/poky/build/tmp/work/i586-poky-linux/procps/3.2.8-r12/package/usr/bin/slabtop 13:56:09 i586-poky-linux-objcopy op may-unlink: [mode 0755, -] /media/build1/poky/build/tmp/work/i586-poky-linux/procps/3.2.8-r12/package/usr/bin/slabtop 13:56:09 i586-poky-linux-objcopy fastop did-unlink: [mode 0755, -] /media/build1/poky/build/tmp/work/i586-poky-linux/procps/3.2.8-r12/package/usr/bin/slabtop 13:56:09 i586-poky-linux-objcopy fastop rename: [mode 0711, -] /media/build1/poky/build/tmp/work/i586-poky-linux/procps/3.2.8-r12/package/usr/bin/slabtop 13:56:09 i586-poky-linux-objcopy fastop chmod: [mode 0755, -] /media/build1/poky/build/tmp/work/i586-poky-linux/procps/3.2.8-r12/package/usr/bin/slabtop 13:56:09 i586-poky-linux-objcopy fastop chown: [mode 0755, -] /media/build1/poky/build/tmp/work/i586-poky-linux/procps/3.2.8-r12/package/usr/bin/slabtop 13:56:09 i586-poky-linux-objcopy fastop chmod: [mode 0755, -] /media/build1/poky/build/tmp/work/i586-poky-linux/procps/3.2.8-r12/package/usr/bin/slabtop 13:56:09 python fastop chmod: [mode 0555, -] /media/build1/poky/build/tmp/work/i586-poky-linux/procps/3.2.8-r12/package/usr/bin/slabtop 13:56:10 python op stat: [mode 0555, -] /media/build1/poky/build/tmp/work/i586-poky-linux/procps/3.2.8-r12/package/usr/bin/slabtop 13:56:10 python fastop chmod: [mode 0755, -] /media/build1/poky/build/tmp/work/i586-poky-linux/procps/3.2.8-r12/package/usr/bin/slabtop 13:56:10 i586-poky-linux-strip op stat: [mode 0755, -] /media/build1/poky/build/tmp/work/i586-poky-linux/procps/3.2.8-r12/package/usr/bin/slabtop 13:56:10 i586-poky-linux-strip op stat: [mode 0755, -] /media/build1/poky/build/tmp/work/i586-poky-linux/procps/3.2.8-r12/package/usr/bin/slabtop 13:56:10 i586-poky-linux-strip op stat: [mode 0755, -] /media/build1/poky/build/tmp/work/i586-poky-linux/procps/3.2.8-r12/package/usr/bin/slabtop 13:56:10 i586-poky-linux-strip fastop open: [mode 0755, r] /media/build1/poky/build/tmp/work/i586-poky-linux/procps/3.2.8-r12/package/usr/bin/slabtop 13:56:10 i586-poky-linux-strip op stat: [mode 0755, -] /media/build1/poky/build/tmp/work/i586-poky-linux/procps/3.2.8-r12/package/usr/bin/slabtop 13:56:10 i586-poky-linux-strip op may-unlink: [mode 0755, -] /media/build1/poky/build/tmp/work/i586-poky-linux/procps/3.2.8-r12/package/usr/bin/slabtop 13:56:10 i586-poky-linux-strip fastop did-unlink: [mode 0755, -] /media/build1/poky/build/tmp/work/i586-poky-linux/procps/3.2.8-r12/package/usr/bin/slabtop 13:56:10 i586-poky-linux-strip fastop rename: [mode 0711, -] /media/build1/poky/build/tmp/work/i586-poky-linux/procps/3.2.8-r12/package/usr/bin/slabtop 13:56:10 i586-poky-linux-strip fastop chmod: [mode 0755, -] /media/build1/poky/build/tmp/work/i586-poky-linux/procps/3.2.8-r12/package/usr/bin/slabtop 13:56:10 python fastop chmod: [mode 0555, -] /media/build1/poky/build/tmp/work/i586-poky-linux/procps/3.2.8-r12/package/usr/bin/slabtop 13:56:10 i586-poky-linux-strip fastop chown: [mode 0555, -] /media/build1/poky/build/tmp/work/i586-poky-linux/procps/3.2.8-r12/package/usr/bin/slabtop 13:56:10 i586-poky-linux-strip fastop chmod: [mode 0755, -] /media/build1/poky/build/tmp/work/i586-poky-linux/procps/3.2.8-r12/package/usr/bin/slabtop 13:56:10 python op stat: [mode 0755, -] /media/build1/poky/build/tmp/work/i586-poky-linux/procps/3.2.8-r12/package/usr/bin/slabtop 13:56:10 python op stat: [mode 0755, -] /media/build1/poky/build/tmp/work/i586-poky-linux/procps/3.2.8-r12/package/usr/bin/slabtop 13:56:10 python op stat: [mode 0755, -] /media/build1/poky/build/tmp/work/i586-poky-linux/procps/3.2.8-r12/package/usr/bin/slabtop 13:56:10 python op stat: [mode 0755, -] /media/build1/poky/build/tmp/work/i586-poky-linux/procps/3.2.8-r12/package/usr/bin/slabtop 13:56:12 find op stat: [mode 0755, -] /media/build1/poky/build/tmp/work/i586-poky-linux/procps/3.2.8-r12/package/usr/bin/slabtop what scares me in this log specifically is: 13:56:10 i586-poky-linux-strip fastop rename: [mode 0711, -] /media/build1/poky/build/tmp/work/i586-poky-linux/procps/3.2.8-r12/package/usr/bin/slabtop 13:56:10 i586-poky-linux-strip fastop chmod: [mode 0755, -] /media/build1/poky/build/tmp/work/i586-poky-linux/procps/3.2.8-r12/package/usr/bin/slabtop 13:56:10 python fastop chmod: [mode 0555, -] /media/build1/poky/build/tmp/work/i586-poky-linux/procps/3.2.8-r12/package/usr/bin/slabtop 13:56:10 i586-poky-linux-strip fastop chown: [mode 0555, -] /media/build1/poky/build/tmp/work/i586-poky-linux/procps/3.2.8-r12/package/usr/bin/slabtop 13:56:10 i586-poky-linux-strip fastop chmod: [mode 0755, -] /media/build1/poky/build/tmp/work/i586-poky-linux/procps/3.2.8-r12/package/usr/bin/slabtop 13:56:10 python op stat: [mode 0755, -] /media/build1/poky/build/tmp/work/i586-poky-linux/procps/3.2.8-r12/package/usr/bin/slabtop i.e. python ran its chmod 0555 *before* the strip process finished?! Looking at the code in lib/oe/package.py, that should not happen. Think about it though, strip does chmod 755, sends the request to the pseudo server but them immediately exits. The system exits back to the parent python process which does a chmod 0555 and again, carries straight on. How do we guarantee the 755 reaches the pseudo server before the 555? So I think there is a good reason you have to wait for a reply from the server. Its not to say ok/not ok, its to say "hey this change was now queued". We may have a way to keep some of the speedup. If at process exit time, pseudo were to ping the server and wait for a reply, this particular problem won't happen. You could devise some pathological case where process A does something to the file, sends a message to process B and it tries to do something but those cases should be very rare. Created attachment 1532 [details]
proposed patch for the atexit() approach
Here's a proposed patch to send a ping on shutdown (using atexit). I have verified that it compiled and ran, I haven't verified that it, say, actually works. Because I can't easily reproduce the problem.
It might be fairly possible to simulate this, except we'd have to update the pseudo server to have smarter/fancier processing of incoming messages to allow it to delay a specific message for a bit.
Looks good with that patch, at least procps reproduced doesn't show it anymore, I'll try to build it once more and then let whole image to rebuild from scratch over night. I added the patch, I couldn't reproduce the problem so it would appear to solve the race issue we were seeing... Complete core-image-minimal also doesn't show any differences in permissions now, so really looks good.
BTW: Unrelated issue from last build from scratch:
diff from-scratch20/buildhistory/images/qemux86_64/eglibc/core-image-minimal/installed-packages.txt from-scratch19/buildhistory/images/qemux86_64/eglibc/core-image-minimal/installed-packages.txt
8,13c8,13
< kernel-3.10.11-yocto-standard_3.10.11+git0+4f689aa30a_e1aa804148-r0.4_qemux86_64.ipk
< kernel-module-binfmt-misc_3.10.11+git0+4f689aa30a_e1aa804148-r0.4_qemux86_64.ipk
< kernel-module-cfbcopyarea_3.10.11+git0+4f689aa30a_e1aa804148-r0.4_qemux86_64.ipk
< kernel-module-cfbfillrect_3.10.11+git0+4f689aa30a_e1aa804148-r0.4_qemux86_64.ipk
< kernel-module-cfbimgblt_3.10.11+git0+4f689aa30a_e1aa804148-r0.4_qemux86_64.ipk
< kernel-module-uvesafb_3.10.11+git0+4f689aa30a_e1aa804148-r0.4_qemux86_64.ipk
---
> kernel-3.10.11-yocto-standard_3.10.11+git0+4f689aa30a_e1aa804148-r0.1_qemux86_64.ipk
> kernel-module-binfmt-misc_3.10.11+git0+4f689aa30a_e1aa804148-r0.1_qemux86_64.ipk
> kernel-module-cfbcopyarea_3.10.11+git0+4f689aa30a_e1aa804148-r0.1_qemux86_64.ipk
> kernel-module-cfbfillrect_3.10.11+git0+4f689aa30a_e1aa804148-r0.1_qemux86_64.ipk
> kernel-module-cfbimgblt_3.10.11+git0+4f689aa30a_e1aa804148-r0.1_qemux86_64.ipk
> kernel-module-uvesafb_3.10.11+git0+4f689aa30a_e1aa804148-r0.1_qemux86_64.ipk
Both builds were using local PR service and build log doesn't show any errors from PR service, only that it was started:
from-scratch19/log.txt:NOTE: Started PRServer with DBfile: /OE/oe-core/cache/prserv.sqlite3, IP: 127.0.0.1, PORT: 47078, PID: 17065
from-scratch20/log.txt:NOTE: Started PRServer with DBfile: /OE/oe-core/cache/prserv.sqlite3, IP: 127.0.0.1, PORT: 52381, PID: 24743
sqlite> select * from PRMAIN_nohist where version like 'linux-yocto-3.10.11%';
linux-yocto-3.10.11+gitAUTOINC+4f689aa30a_e1aa804148-r0|qemux86_64|c2bf1442ea618b900226b1ac7ae84b69|3
linux-yocto-3.10.11+gitAUTOINC+4f689aa30a_e1aa804148-r0|qemux86_64|7035787fa02a96383b10bf0abc60f255|4
linux-yocto-3.10.11+gitAUTOINC+4f689aa30a_e1aa804148-r0|qemux86_64|50c0c802cd8cfbba14e970a31183d5c4|5
-rw-rw-r-- 1 bitbake bitbake 141282 Sep 25 23:54 tmp-eglibc/from-scratch20/stamps/qemux86_64-oe-linux/linux-yocto/3.10.11+gitAUTOINC+4f689aa30a_e1aa804148-r0.do_package.sigdata.7035787fa02a96383b10bf0abc60f255
in build "from-scratch19" it was from sstate, so there is only do_package_setscene stamp
Fixed merged into OE-Core master. http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=a1d9b6df86b0829d591cae57ae351f6d96e63557 Its assumed seebs will catch up in the pseudo repository when he has time. |