Bug 533

Summary: SSTATE_MIRRORS causes meta tree pollution
Product: [Build System, Metadata & Runtime] Meta-yocto Reporter: Gary Thomas <gary>
Component: meta-yoctoAssignee: Richard Purdie <richard.purdie>
Status: RESOLVED FIXED QA Contact:
Severity: normal    
Priority: Undecided CC: ke.yu, kevin.tian, poky.bs.watcher, poky.watcher, richard.purdie, sgw
Version: unspecified   
Target Milestone: ---   
Hardware: x86   
OS: Multiple   
Whiteboard:
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: ---

Description Gary Thomas 2010-11-10 15:26:54 UTC 
See bug #526 for the setup scenario.

It seems that when you have SSTATE_MIRRORS enabled, certain 
files are ending up in the wrong place.  In particular, there
are a bunch of files "*deploy-ipk*" being placed in ${OEROOT}/...!!

Here's what I just discovered after trying my little experiment
(bug #526) yesterday.

$ find /tmp/poky-master/ -name "*deploy-ipk*"
/tmp/poky-master/meta/recipes-kernel/linux/files/sstate-linux-wrs-qemuarm-poky-linux-gnueabi-2.6.34+git0+d1cd5c80ee97e81e130be8c3de3965b770f320d6_0+4f4177b4bea5b8858acc1eeb788d80b7af0df962-r12-armv5te-1-3559c0381a4081ae6e23a5ba8bc7d435_deploy-ipk.tgz
/tmp/poky-master/meta/recipes-core/images/sstate-poky-image-minimal-qemuarm-poky-linux-gnueabi-1.0-r0-armv5te-1-393bfe7f6b63c8fb85b7fbf1281a33a3_deploy-ipk.tgz
/tmp/poky-master/meta/recipes-core/tasks/sstate-task-poky-boot-qemuarm-poky-linux-gnueabi-1.0-r7-armv5te-1-6cceb5879edee4d66ae5644664ee4586_deploy-ipk.tgz
/tmp/poky-master/meta/recipes-extended/zypper/zypper/sstate-zypper-armv5te-poky-linux-gnueabi-1.4.7-git0+9eb0e248e06c8d20ad054be2439149d9ede37531-r1-armv5te-1-f6d39a32074b296543290cf29d982031_deploy-ipk.tgz
/tmp/poky-master/meta/recipes-extended/procps/procps-3.2.7/sstate-procps-armv5te-poky-linux-gnueabi-3.2.7-r9-armv5te-1-f65081285df4fe29a846cfebd18ca2ca_deploy-ipk.tgz

/tmp/poky-master is my original Poky (${OEROOT} in old terminology)
tree.  My build tree is somewhere totally different.

This was after building the same image.  If I build something new,
it gets much worse:

$ find /tmp/poky-master/ -name "*deploy-ipk*"
/tmp/poky-master/meta/recipes-support/libcap/sstate-libcap-armv5te-poky-linux-gnueabi-2.19-r1-armv5te-1-04f875e9f6b875a448302ae572f13c80_deploy-ipk.tgz
/tmp/poky-master/meta/recipes-support/libevent/sstate-libevent-armv5te-poky-linux-gnueabi-1.4.14b-r0-armv5te-1-d6dab558e9ae7defd7ba47bef5830330_deploy-ipk.tgz
/tmp/poky-master/meta/recipes-connectivity/nfs-utils/sstate-libnfsidmap-armv5te-poky-linux-gnueabi-0.23-r0-armv5te-1-1034a99ba6e20f1e4c21e2baa756d7bc_deploy-ipk.tgz
/tmp/poky-master/meta/recipes-connectivity/nfs-utils/nfs-utils/sstate-nfs-utils-armv5te-poky-linux-gnueabi-1.2.2-r1-armv5te-1-1857f64d548cfe271a6b6f43db03867e_deploy-ipk.tgz
/tmp/poky-master/meta/recipes-connectivity/portmap/portmap-6.0/sstate-portmap-armv5te-poky-linux-gnueabi-6.0-r7-armv5te-1-7aa6ab3c84b6fdb798f2e2865c8b0b85_deploy-ipk.tgz
/tmp/poky-master/meta/recipes-kernel/linux/files/sstate-linux-wrs-qemuarm-poky-linux-gnueabi-2.6.34+git0+d1cd5c80ee97e81e130be8c3de3965b770f320d6_0+4f4177b4bea5b8858acc1eeb788d80b7af0df962-r12-armv5te-1-3559c0381a4081ae6e23a5ba8bc7d435_deploy-ipk.tgz
/tmp/poky-master/meta/recipes-core/images/sstate-poky-image-minimal-qemuarm-poky-linux-gnueabi-1.0-r0-armv5te-1-393bfe7f6b63c8fb85b7fbf1281a33a3_deploy-ipk.tgz
/tmp/poky-master/meta/recipes-core/tasks/sstate-task-poky-boot-qemuarm-poky-linux-gnueabi-1.0-r7-armv5te-1-6cceb5879edee4d66ae5644664ee4586_deploy-ipk.tgz
/tmp/poky-master/meta/recipes-core/util-linux/util-linux-2.17.2/sstate-util-linux-armv5te-poky-linux-gnueabi-2.17.2-r1-armv5te-1-7150d2c180d3f6453a1555f2384e0ad6_deploy-ipk.tgz
/tmp/poky-master/meta/recipes-devtools/flex/sstate-flex-armv5te-poky-linux-gnueabi-2.5.35-r1-armv5te-1-9823f95f24a63065e045fab33ffc3b8a_deploy-ipk.tgz
/tmp/poky-master/meta/recipes-devtools/bison/bison/sstate-bison-armv5te-poky-linux-gnueabi-2.4.2-r0-armv5te-1-1617549c8529ac8e79ff62051cbf937e_deploy-ipk.tgz
/tmp/poky-master/meta/recipes-extended/pam/libpam-1.1.1/sstate-libpam-armv5te-poky-linux-gnueabi-1.1.1-r1-armv5te-1-19848d9fbff9cb63512c4f862321e2b9_deploy-ipk.tgz
/tmp/poky-master/meta/recipes-extended/tcp-wrappers/tcp-wrappers-7.6/sstate-tcp-wrappers-armv5te-poky-linux-gnueabi-7.6-r0-armv5te-1-9ea49de542653cd771ead8864ee1ec8e_deploy-ipk.tgz
/tmp/poky-master/meta/recipes-extended/zypper/zypper/sstate-zypper-armv5te-poky-linux-gnueabi-1.4.7-git0+9eb0e248e06c8d20ad054be2439149d9ede37531-r1-armv5te-1-f6d39a32074b296543290cf29d982031_deploy-ipk.tgz
/tmp/poky-master/meta/recipes-extended/procps/procps-3.2.7/sstate-procps-armv5te-poky-linux-gnueabi-3.2.7-r9-armv5te-1-f65081285df4fe29a846cfebd18ca2ca_deploy-ipk.tgz

Conclusion: when SSTATE_MIRRORS are enabled, some intermediate packages
are being written in the Poky (${OEROOT}) tree which should not be touched
by a build.
Comment 1 Richard Purdie 2010-11-12 15:30:08 UTC 
Should be fixed with ae98f7eacb9e61fe086d88dc694b4c651af9fee3
Comment 2 Gary Thomas 2010-11-12 15:56:43 UTC 
Pollution seems to be fixed, but it now fails:

NOTE: package gcc-cross-initial-4.5.0-r10: task do_fetch: Started
ERROR: Error in executing python function in: /local/poky-amltd/meta/recipes-devtools/gcc/gcc-cross-initial_4.5.0.bb
ERROR: Exception:<type 'exceptions.AttributeError'> Message:'module' object has no attribute 'exists'
NOTE: Running task 330 of 3106 (ID: 1534, /local/poky-amltd/meta/recipes-kernel/linux-libc-headers/linux-libc-headers_2.6.34.bb, do_patch)
ERROR: Traceback:
ERROR:   File "base_do_fetch", line 64, in <module>
ERROR: 
ERROR:   File "base_do_fetch", line 12, in base_do_fetch
ERROR: 
ERROR:   File "/home/local/poky-amltd/scripts/..//bitbake/lib/bb/fetch/__init__.py", line 220, in init
ERROR:     urldata[url].setup_localpath(d)
ERROR: 
ERROR:   File "/home/local/poky-amltd/scripts/..//bitbake/lib/bb/fetch/__init__.py", line 529, in setup_localpath
ERROR:     self.localpath = self.method.localpath(self.url, self, d)
ERROR: 
ERROR:   File "/home/local/poky-amltd/scripts/..//bitbake/lib/bb/fetch/local.py", line 54, in localpath
ERROR:     if os.exists(dlpath):
ERROR: 
ERROR: The lines leading to this error were:
ERROR:  0060:                                   bb.note("%s-%s: %s has no entry in conf/checksums.ini, not checking URI" % (pn,pv,uri))
ERROR:  0061:           except Exception:
ERROR:  0062:                   raise bb.build.FuncFailed("Checksum of '%s' failed" % uri)
ERROR:  0063:
ERROR: Task failed: ('Function base_do_fetch failed', '/home/local/poky-after-rp-fix/tmp/work/armv7a-poky-linux-gnueabi/gcc-cross-initial-4.5.0-r10/temp/log.do_fetch.2110')
Comment 3 Richard Purdie 2010-11-12 18:30:26 UTC 
Sorry, I'm not sure how that code was working but it did in my tests. I've pushed a fix.