Image location: http://autobuilder.yoctoproject.org/pub/releases/yocto-2.1_M3.rc1/ Git rev: master/8debfea81e69d038bd2d56314b272cb74f5582ed Steps: 1. Download the image from the above link. 2. Launch the qemu and install a rpm package 3. Check the rpm database ########################################## root@qemuarm:~# rpm -ivh groff-1.22.3-r0.armv5e.rpm Preparing... ########################################### [100%] 1:groff ########################################### [100%] root@qemuarm:~# root@qemuarm:~# ls -lh /var/lib/rpm/__db.00* -rw-r--r-- 1 root root 65M Mar 15 08:53 /var/lib/rpm/__db.001 -rw-r--r-- 1 root root 56M Mar 15 08:53 /var/lib/rpm/__db.002 -rw-r--r-- 1 root root 1.3M Mar 15 08:53 /var/lib/rpm/__db.003 root@qemuarm:~# rpm -ivh groff-dev-1.22.3-r0.armv5e.rpm Preparing... ########################################### [100%] 1:groff-dev ########################################### [100%] root@qemuarm:~# root@qemuarm:~# ls -lh /var/lib/rpm/__db.00* -rw-r--r-- 1 root root 65M Mar 15 08:57 /var/lib/rpm/__db.001 -rw-r--r-- 1 root root 56M Mar 15 08:57 /var/lib/rpm/__db.002 -rw-r--r-- 1 root root 1.3M Mar 15 08:57 /var/lib/rpm/__db.003 root@qemuarm:~# rpm -e groff-dev root@qemuarm:~# root@qemuarm:~# ls -lh /var/lib/rpm/__db.00* -rw-r--r-- 1 root root 65M Mar 15 08:58 /var/lib/rpm/__db.001 -rw-r--r-- 1 root root 56M Mar 15 08:58 /var/lib/rpm/__db.002 -rw-r--r-- 1 root root 1.3M Mar 15 08:58 /var/lib/rpm/__db.003 root@qemuarm:~# root@qemuarm:~# rpm -e groff root@qemuarm:~# root@qemuarm:~# ls -lh /var/lib/rpm/__db.00* -rw-r--r-- 1 root root 65M Mar 15 08:58 /var/lib/rpm/__db.001 -rw-r--r-- 1 root root 56M Mar 15 08:58 /var/lib/rpm/__db.002 -rw-r--r-- 1 root root 1.3M Mar 15 08:58 /var/lib/rpm/__db.003 root@qemuarm:~# ############################## The size of some db files more than 50MB, normally it should less than 30MB. This issue happens on all qemu and BSPs. On some qemu and BSPs, the size of db file may more than 100MB: ############## -rw-r--r-- 1 root root 65M Mar 15 05:45 /var/lib/rpm/__db.001 -rw-r--r-- 1 root root 141M Mar 15 05:45 /var/lib/rpm/__db.002 -rw-r--r-- 1 root root 1.3M Mar 15 05:45 /var/lib/rpm/__db.00 ############# There are 2 old bugs with similar issue, bug 1174 and bug 1769.
Mark, as the resident RPM expert, do you have any opinion on this? My default position is "berkeley db is stupid".
These files are not part of the DB themselves, but are the journal and cache files for the DB. As long as the DB was closed properly, these files can be removed to reclaim space. The size of these files is controlled by the DB_CONFIG file in the /var/lib/rpm directory and/or the settings in the RPM macros. To attempt to reduce the size of these files you'll have to consult the relevant BerkleyDB manuals for options. Note, I do NOT recommend RPM for use in 'small' systems. It's designed for what I would categorize medium to large systems. I.e. machines that have the potential for a lot of packages to be installed (or upgraded) over the life of the device -- with a disk large enough to store the information and associated cache files. For a small system I recommend using something like opkg. The reduced functionality comes with a large disk space savings.
Sounds like "resolved notabug" to me. Mark, do you agree?
I'm torn. It's something we try to always look at. At one point we'd gotten the journal down into the 30 MB range. So it's definitely increased in size over what it used to be. I don't know if we can measurably change it -- but we likely should investigate and see if we can optimize the journal somewhat.
Is this changed at all with the move to RPM4?
(In reply to comment #5) > Is this changed at all with the move to RPM4? Yes. Now the database size less then 30MB. root@qemumips64:~# rpm -ivh groff-1.22.3-r0.mips64.rpm Preparing... ################################# [100%] Updating / installing... 1:groff-1.22.3-r0 ################################# [100%] root@qemumips64:~# root@qemumips64:~# root@qemumips64:~# du /var/lib/rpm -sh 27M /var/lib/rpm root@qemumips64:~# root@qemumips64:~# ls -lh /var/lib/rpm/ total 27M -rw-r--r-- 1 root root 4.0M Mar 21 05:38 Basenames -rw-r--r-- 1 root root 8.0K Mar 14 17:26 Conflictname -rw-r--r-- 1 root root 944K Mar 21 05:38 Dirnames -rw-r--r-- 1 root root 8.0K Mar 14 17:14 Enhancename -rw-r--r-- 1 root root 8.0K Mar 14 17:14 Filetriggername -rw-r--r-- 1 root root 40K Mar 21 05:38 Group -rw-r--r-- 1 root root 32K Mar 21 05:38 Installtid -rw-r--r-- 1 root root 152K Mar 21 05:38 Name -rw-r--r-- 1 root root 8.0K Mar 14 17:26 Obsoletename -rw-r--r-- 1 root root 19M Mar 21 05:38 Packages -rw-r--r-- 1 root root 348K Mar 21 05:38 Providename -rw-r--r-- 1 root root 168K Mar 21 05:37 Recommendname -rw-r--r-- 1 root root 224K Mar 21 05:38 Requirename -rw-r--r-- 1 root root 216K Mar 21 05:38 Sha1header -rw-r--r-- 1 root root 128K Mar 21 05:38 Sigmd5 -rw-r--r-- 1 root root 8.0K Mar 14 17:25 Suggestname -rw-r--r-- 1 root root 8.0K Mar 14 17:14 Supplementname -rw-r--r-- 1 root root 8.0K Mar 14 17:14 Transfiletriggername -rw-r--r-- 1 root root 8.0K Mar 14 17:14 Triggername -rw-r--r-- 1 root root 1.1M Mar 21 05:38 __db.001 -rw-r--r-- 1 root root 248K Mar 21 05:38 __db.002 -rw-r--r-- 1 root root 1.3M Mar 21 05:38 __db.003 root@qemumips64:~# root@qemumips64:~# rpm -e groff root@qemumips64:~# root@qemumips64:~# du /var/lib/rpm -sh 27M /var/lib/rpm root@qemumips64:~# ls -lh /var/lib/rpm/ total 27M -rw-r--r-- 1 root root 4.0M Mar 21 05:38 Basenames -rw-r--r-- 1 root root 8.0K Mar 14 17:26 Conflictname -rw-r--r-- 1 root root 944K Mar 21 05:38 Dirnames -rw-r--r-- 1 root root 8.0K Mar 14 17:14 Enhancename -rw-r--r-- 1 root root 8.0K Mar 14 17:14 Filetriggername -rw-r--r-- 1 root root 40K Mar 21 05:38 Group -rw-r--r-- 1 root root 32K Mar 21 05:38 Installtid -rw-r--r-- 1 root root 152K Mar 21 05:38 Name -rw-r--r-- 1 root root 8.0K Mar 14 17:26 Obsoletename -rw-r--r-- 1 root root 19M Mar 21 05:38 Packages -rw-r--r-- 1 root root 348K Mar 21 05:38 Providename -rw-r--r-- 1 root root 168K Mar 21 05:37 Recommendname -rw-r--r-- 1 root root 224K Mar 21 05:38 Requirename -rw-r--r-- 1 root root 216K Mar 21 05:38 Sha1header -rw-r--r-- 1 root root 128K Mar 21 05:38 Sigmd5 -rw-r--r-- 1 root root 8.0K Mar 14 17:25 Suggestname -rw-r--r-- 1 root root 8.0K Mar 14 17:14 Supplementname -rw-r--r-- 1 root root 8.0K Mar 14 17:14 Transfiletriggername -rw-r--r-- 1 root root 8.0K Mar 14 17:14 Triggername -rw-r--r-- 1 root root 1.1M Mar 21 05:38 __db.001 -rw-r--r-- 1 root root 248K Mar 21 05:38 __db.002 -rw-r--r-- 1 root root 1.3M Mar 21 05:38 __db.003 root@qemumips64:~#
I think we need to monitor this, but I suspect it will be the similar to before or better (different configuration for the database)
So is there a specific problem here? Is there something I should do?
The issue is once the user starts to 'use' the target environment to do package feed installs/upgrades/etc the RPM database (berkleydb database) in both Smart (now DNF) and RPM can grow in size that is not proportional to the activities performed. The initial ask is we need to do some basic testing on this and see what the system behavior is in these cases. If the database grows slowly or within what we believe are reasonable bounds -- then we are likely fine. The test case of installing a single package and removing it will only find the worse offenders. Its really upgrading a series of packages over time. The overall database size should remain fairly constant, along with the __db.* files (cache files) and not grow without bounds. (If they do start to grow without bounds, then it might be a BerkleyDB configuration issue or something else that can be adjusted to resolve this type of issue.)
Then this is a task for Intel's or Wind River's QA and should be assigned as such.
It appears the NEEDINFO has been cleared.
This issue disappeared after we switch to rpm4 in 2.3. We monitored the db files size in dnf/rpm runtime cases for qemu/BSPs and these files kept a small size after install/remove packages multiple times in 2.3. The following thing what QA should do is to continue to observe the rpm behavior in the next release testing (2.4). If this issue still do not happen, we can close this bug. //Yi
This issue doesn't happen in 2.4. We can close this issue.