Bug 8444 - WORKDIR/installed_pkgs.txt should be deleted after use
Summary: WORKDIR/installed_pkgs.txt should be deleted after use
Status: VERIFIED FIXED
Alias: None
Product: OE-Core
Classification: Build System, Metadata & Runtime
Component: deployment (show other bugs)
Version: 1.9
Hardware: x86 Multiple
: Medium+ normal
Target Milestone: 2.1 M2
Assignee: Mariano Lopez
QA Contact: Costin
URL:
Whiteboard:
Depends on:
Blocks:
 
Reported: 2015-10-05 21:05 UTC by brian avery
Modified: 2015-12-07 12:57 UTC (History)
5 users (show)

See Also:
OS type for building Yocto: ---
Type of Regression: ---
Verified:
Documentation change: No (bug/feature does not impact docs)


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description brian avery 2015-10-05 21:05:31 UTC
I built core-image-sato. I had a couple of different representations of what was deployed.

In the tmp/work/qemux86-poky-linux/core-image-sato/1.0-r0/installed_pkgs.txt file the package gtk+-locale-en-gb-2.24.28-r0.i586.rpm appeared. The files it contains (for example ./usr/share/locale/en_GB/LC_MESSAGES/gtk20.mo) were present in the  rootfs.

In the manifest file core-image-sato-qemux86.manifest; however, the package gtk+-locale-en-gb-2.24.28-r0.i586.rpm  was not listed.

To reproduce:
1)set up to build core-image-sato off of master
2)add INHERIT+="buildhistory" to the conf/local.conf file (I am not sure if this affects it but it's what i had when I did this)
3)build core image sato for qemu32
4) compare the installed_pkgs.txt and the core-image-sato-qemux86.manifest file.
5) the core-image-sato-qemux86.manifest file had 18 packages (all locales) that were in the installed_pkgs.txt but not in the manifest file. All the ones I checked were in the rootfs. 
6) here's a list of the rpms missing from the manifest for this build
     1  < avahi-locale-en-gb
     2  < gconf-locale-en-gb
     3  < glibc-locale-en-gb
     4  < glib-networking-locale-en-gb
     5  < gstreamer1.0-locale-en-gb
     6  < gstreamer1.0-plugins-bad-locale-en-gb
     7  < gstreamer1.0-plugins-base-locale-en-gb
     8  < gstreamer1.0-plugins-good-locale-en-gb
     9  < gtk+-locale-en-gb
    10  < libatk-1.0-locale-en-gb
    11  < libexif-locale-en-gb
    12  < libfm-locale-en-gb
    13  < libgdk-pixbuf-2.0-locale-en-gb
    14  < libglib-2.0-locale-en-gb
    15  < libsoup-2.4-locale-en-gb
    16  < pcmanfm-locale-en-gb
    17  < vte-locale-en-gb
    18  < xkeyboard-config-locale-en-gb
Comment 1 Ross Burton 2015-10-16 16:31:09 UTC
Are you sure you got the files the right way around?  I replicated this here but my manifest is right and the installed_pkgs file is missing the locales:

$ diff -U0 ~/installed_pkgs.txt  ~/core-image-base-qemux86-64.manifest
--- /home/ross/installed_pkgs.txt	2015-10-16 17:20:20.057667013 +0100
+++ /home/ross/core-image-base-qemux86-64.manifest	2015-10-16 17:19:56.249548955 +0100
@@ -7,0 +8 @@
+avahi-locale-en-gb
@@ -24,0 +26 @@
+glibc-locale-en-gb
@@ -65,0 +68 @@
+libglib-2.0-locale-en-gb
@@ -74,0 +78 @@
+libpam-locale-en-gb
Comment 2 Mariano Lopez 2015-10-16 21:23:46 UTC
I did some testing and all packages mentioned by Brian are in the manifest, actually I test also with deb and the manifest has all the packages from installed packages. 

The only different thing is that I didn't inherit the buildhistory class and the MACHINE was begleabone
Comment 3 Benjamin Esquivel 2015-10-16 22:03:37 UTC
Ross, FWIW all the files listed by Brian are locales too
Comment 4 Ross Burton 2015-10-16 22:09:16 UTC
Yes, the packages missing are definitely locales, but I'm seeing the missing packages in a different file to the report so confirmation from Brian would be good.
Comment 5 brian avery 2015-10-16 23:00:39 UTC
Brian just reran the test off of current master and it looks like ross is correct. the missing pkgs are all locales but they are missing from the installed_pkgs.txt file. They *do* appear in the core-image-sato-qemux86-<date>.rootfs.manifest file.  I must have switched the files in my diff. my bad :(
Comment 6 Ross Burton 2015-10-17 07:42:44 UTC
Thanks Brian, easy to do!
Comment 7 Mariano Lopez 2015-10-19 13:48:54 UTC
Closing as not a bug
Comment 8 Ross Burton 2015-10-19 14:04:05 UTC
The WORKDIR/installed_pkgs.txt file is still incorrect so this is still a bug, but as the file isn't the official manifest it's not a 2.0 bug anymore.

Looking into the file it appears to only exist for the installation of complementary packages (in package_manager.py) so could be deleted once that has been done.  The roofs.py class updates it to reflect packages that were subsequently removed but as the file isn't useful that can just be deleted.  The only other user of that file is the QA ptest harness, instead that should just get the list of packages from the target directly.

So I propose (for 2.1):
1) Change ptest harness to use deployed manifest or get package listing from target
2) Remove manipulation of installed_pkgs.txt in lib/oe/rootfs.py
3) Delete installed_pkgs.txt after complementary packages installed in lib/oe/package_manager.py
Comment 9 Paul Eggleton 2015-10-19 14:45:24 UTC
For the record, installed_pkgs.txt was always meant to be internal to the complementary package installation process - it's the *source* of the list of packages for that process, so naturally it does not include complementary packages such as the locales. If you want the full list of packages in the image, use the manifest, that's exactly what it's there for.
Comment 10 Mariano Lopez 2015-11-03 18:23:41 UTC
ptest overwrites WORKDIR/installed_pkgs.txt with information from RpmPkgsList, so it is already getting the packages from the target; the missing part is remove the file once is used.
Comment 11 Mariano Lopez 2015-11-04 16:03:20 UTC
Sent patch to the mailing list

http://lists.openembedded.org/pipermail/openembedded-core/2015-November/112244.html
Comment 13 Costin 2015-12-07 12:57:18 UTC
Hello,

Tested on 698c74c373110ed081a7586e21d4a27b8b44c89b. I can see that after build, the installed_pkgs.txt is erased.

Marking as verified.