After the application of the rpm upgrade series from Mark Hatle, rpm package removal doesn't appear to work from smart: # smart remove psplash-default -y Loading cache... Updating cache... ############################################################################################### [100%] Computing transaction... Removing packages (1): psplash-default-0.1+git0+88343ad23c-r15.20@i586 50.1kB will be freed. Committing transaction... Preparing... ############################################################################################### [ 0%] which exit(0) but the package is still present and the removal command can be run multiple times with no errors or effect. This manifests itself on the autobuilder where it tests removal of a package followed by installation but this fails. To reproduce, INERIT += "testimage" in local.conf and "bitbake core-image-sato -c testimage". https://autobuilder.yoctoproject.org/main/builders/nightly-arm/builds/690/steps/Running%20Sanity%20Tests/logs/stdio Adding in the usr/lib/python2.7/site-packages/smart/backends/rpm from a jethro build doesn't seem to change the issues either. The transaction is making it to rpm but doesn't seem to do anything. With rpm logging, the output is: # smart remove -y psplash-default --log-level=debug Loading cache... Updating cache... ############################################################################################### [100%] Computing transaction... Removing packages (1): psplash-default-0.1+git0+88343ad23c-r15.20@i586 50.1kB will be freed. Committing transaction... warning: Here D: opening db environment /var/lib/rpm/Packages thread:lock:log:mpool:txn D: opening db index /var/lib/rpm/Packages thread:rdonly:auto_commit mode=0x0 D: pool bf: created size 28 limit -1 flags 0 D: pool te: created size 224 limit -1 flags 0 D: pool fi: created size 320 limit -1 flags 0 warning: Here 2 warning: Here 3 D: pool tsi: created size 28 limit -1 flags 0 D: ========== --- psplash-default-0.1+git0+88343ad23c-r15.20.i586 i586/linux 0x0 D: opening db index /var/lib/rpm/Requirename thread:rdonly:auto_commit mode=0x0 D: pool ps: created size 24 limit -1 flags 0 D: opening db index /var/lib/rpm/Providename thread:rdonly:auto_commit mode=0x0 D: Requires: psplash-default YES (db provides) warning: Here 4 warning: Here 5 D: closed db index /var/lib/rpm/Requirename D: closed db index /var/lib/rpm/Providename D: closed db index /var/lib/rpm/Packages D: closed db environment /var/lib/rpm/Packages D: opening db environment /var/lib/rpm/Packages thread:lock:log:mpool:txn D: opening db index /var/lib/rpm/Packages thread:auto_commit mode=0x42 D: mounted filesystems: D: i dev bsize bavail iavail mount point D: 0 0x0000fd00 4096 11071 142096 rw / D: 1 0x00000006 4096 30566 29216 rw /dev D: 2 0x00000004 4096 1 -1 rw /proc D: 3 0x0000000f 4096 10 30659 rw /mnt/.psplash D: 4 0x00000010 4096 1 -1 rw /sys D: 5 0x00000007 4096 1 -1 rw /sys/kernel/debug D: 6 0x00000011 4096 30621 29931 rw /run D: 7 0x00000012 4096 30631 30644 rw /var/volatile D: 8 0x0000000d 4096 1 -1 rw /dev/pts D: 9 0x00000013 4096 1 -1 rw /proc/fs/nfsd D: sanity checking 1 elements D: running pre-transaction scripts D: computing 3 file fingerprints D: pool ht: created size 36 limit -1 flags 0 D: computing file dispositions ( 0%) D: opening db index /var/lib/rpm/Basenames thread:auto_commit mode=0x42 Preparing... ############################################################################################### [ 0%] D: pool psm: created size 472 limit -1 flags 0 D: ========== --- psplash-default-0.1+git0+88343ad23c-r15.20.i586 i586-linux 0x0 D: erase: psplash-default-0.1+git0+88343ad23c-r15.20.i586 has 3 files, test = 0 D: opening db index /var/lib/rpm/Name thread:auto_commit mode=0x42 D: running post-transaction scripts D: closed db index /var/lib/rpm/Filepaths D: closed db index /var/lib/rpm/Packages D: closed db environment /var/lib/rpm/Packages warning: Here 7 warning: Here 8 D: closed db index /var/lib/rpm/Basenames D: closed db index /var/lib/rpm/Name D: closed db index /var/lib/rpm/Packages D: closed db environment /var/lib/rpm/Packages (the Warning messages from debug I added to the rpm backend just to check which code paths were active).
I have identified the cause of the failure, but am still not sure how to fix it. Below is an email I've sent to the RPM5 maintainer asking for guidance. I found the cause of the python erase failure. The old code effectively did: rpmts_AddErase(rpmtsObject * s, PyObject * args, PyObject * kwds) mi = rpmtsInitIterator(s->ts, RPMTAG_NVRA, name, 0); or mi = rpmtsInitIterator(s->ts, RPMDBI_PACKAGES, &instance, sizeof(instance)); Header h; while ((h = rpmmiNext(mi)) != NULL) { uint32_t hdrNum = rpmmiInstance(mi); if (hdrNum) rpmtsAddEraseElement(s->ts, h, hdrNum); break; } The new code though is very different: (in python) it converts everything into a header. def addErase(self, item): hdrs = [] if isinstance(item, rpm.hdr): hdrs = [item] elif isinstance(item, rpm.mi): hdrs = item elif isinstance(item, int): hdrs = self.dbMatch(rpm.RPMDBI_PACKAGES, item) elif isinstance(item, str): hdrs = self.dbMatch(rpm.RPMDBI_LABEL, item) else: raise TypeError("invalid type %s" % type(item)) for h in hdrs: if not _rpmts.addErase(self, h): raise rpm.error("package not installed") The python chunk then calls rpmts_AddErase which all it does is: return PyBool_FromLong(rpmtsAddEraseElement(s->ts, h, -1) == 0); So the third item, 'hdrNum' is always set to -1, which appears to trigger the silent failure. (Looking at the current RPM 4.X code, the third parameter for calls to rpmtsAddEraseElement is always ignored.) Since the code is passing in a header already computed, I can't use the hdrNum = rpmmiInstance(mi) as far as I can tell. I did try to replace it with: return PyBool_FromLong(rpmtsAddEraseElement(s->ts, h, headerGetInstance(h)) == 0); It appears to be working, but I'm not sure if the value returned by 'headerGetInstance' is reasonable or not. So I think we have two differences here. One is at the RPM API level. One is at the python level. Is there any reason the RPM 5 API could not be modified to always define it's own 'hdrNum' -- or does RPM 5 always need a specific hdrNum, if so -- is calling headerGetInstance(h) correct to generate one or is there some other method that should be used?
Yes headerGetInstance() is correct for any header retrieved from an rpmdb. The hdrNum (always positive) is set when the header is read from an rpmdb (and is likely zero/negative for headers that did not come from an rpmdb). The underlying API issue is that the join keys which attach secondary indices to the primary Packages store need to remain opaque and should *NOT* be exposed to applications so that, say, the join key could become an uint64_t or a UUID or even a string in some relational db without having to change every application. Meanwhile, the hdrNum exposed by rpm.org (and older rpm) was a simple native endian uint32_t. rpm5.org still uses a network endian uint32_t and the value is always monotonically increasing and transactionally protected in /var/lib/rpm/Seqno.
This was fixed in the final version of the rpm upgrade patchset: http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=007c284cb83cf5d98f3e4f605244ca6f1d46caea
Verified on master: master:f7b520878babbaa7527151f22c031ae160512753 smart remove -y psplash-default --log-level=debug Loading cache... Updating cache... ############################################ [100%] Computing transaction... Removing packages (1): psplash-default-0.1+git0+88343ad23c-r15@i586 50.1kB will be freed. Committing transaction... Preparing... ############################################ [ 0%] update-alternatives: removing /usr/bin/psplash as no more alternatives exist for it 1:Removing psplash-default ############################################ [100%]