Bug 8925

Summary: Custom image package list does not show kernel-related packages
Product: [Build System, Metadata & Runtime] Toaster Reporter: Belen Barros Pena <belen.barros.pena>
Component: toasterAssignee: Michael Wood <michael.g.wood>
Status: VERIFIED FIXED QA Contact: Alexandru Roman <alexandru.costinx.roman>
Severity: normal    
Priority: Medium+ CC: alexandru.costinx.roman, belen.barros.pena, brian.avery, jessica.zhang, michael.g.wood, stanciux.mihail
Version: 2.0   
Target Milestone: 2.1 M3   
Hardware: x86   
OS: Multiple   
Whiteboard:
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: No (bug/feature does not impact docs)
Attachments:
Description Flags
Screenshot showing "empty" package list for custom image
none
Design - Showing package groups in the custom image details page none

Description Belen Barros Pena 2016-01-14 12:28:36 UTC
This issue applies to the image customisation branch

http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=michaelw/toaster/ic-6

How to reproduce:

1. Start Toaster with CUSTOM_IMAGE=1
2. Create a project with release 'master'
3. Build core-image-minimal
4. Go to the 'New custom image' page
5. Search for core-image-minimal openembedded-core and click the 'customise' button
6. Give your custom image a name, then build it
7. Once it's done, select the build and in the left navigation click on the image name. The packages installed tab will show 28 packages
8. Then go to the project configuration page and on the left navigation click on the 'custom images' link and then select your custom image
9. Click the 'filter' button next to 'Add | remove' table heading and select the option "Packages in <your_image_name> (23)"

There are 23 packages on that list instead of 28. The packages missing are package groups (see bug 8901), and mostly kernel-related packages:

kernel-base
kernel-module-nvesafb
libkmod

update-alternatives-opkg is also missing. This is need to be investigated: we still don't know why.
Comment 1 Elliot Smith 2016-01-14 12:54:00 UTC
I've tried to reproduce this, but found that the number of packages shown for a custom image based on core-image-minimal matches the number of packages shown for core-image-minimal proper (i.e. 28).

However, the "Add|Remove" packages area is incorrect for the custom image, as it shows no packages at all.

Could you please try on my branch based off ic-6, elliot/toaster/ic-7? I had to fix the migrations there, so it could be that the database state is incorrect in your case, which might be why we're seeing different things.

It might also be worth starting with a clean db, to ensure we're not seeing data derived from previous builds.
Comment 2 Elliot Smith 2016-01-14 12:56:23 UTC
NB I tried to reproduce using core-image-minimal, as I wasn't sure what you meant by "Search for core-image-minimal openembedded-core".
Comment 3 Belen Barros Pena 2016-01-14 13:59:35 UTC
(In reply to comment #1)
> I've tried to reproduce this, but found that the number of packages shown
> for a custom image based on core-image-minimal matches the number of
> packages shown for core-image-minimal proper (i.e. 28).

Interesting. That was definitely not the case for me, but I'll try it again. 
> 
> However, the "Add|Remove" packages area is incorrect for the custom image,
> as it shows no packages at all.

I am not sure what you mean by this. Could you maybe post a screenshot of what you see?

> 
> Could you please try on my branch based off ic-6, elliot/toaster/ic-7?

Will do

 I had
> to fix the migrations there, so it could be that the database state is
> incorrect in your case, which might be why we're seeing different things.

I removed my build directory before building.

> 
> It might also be worth starting with a clean db, to ensure we're not seeing
> data derived from previous builds.

As per the comment above, it was a clean db.
Comment 4 Belen Barros Pena 2016-01-14 14:00:39 UTC
(In reply to comment #2)
> NB I tried to reproduce using core-image-minimal, as I wasn't sure what you
> meant by "Search for core-image-minimal openembedded-core".

I meant select the core-image-minimal provided by the openembedded-core layer as the base image for your custom image.
Comment 5 Elliot Smith 2016-01-14 14:14:41 UTC
Created attachment 2929 [details]
Screenshot showing "empty" package list for custom image
Comment 6 Elliot Smith 2016-01-14 16:04:05 UTC
I've tried from a clean build directory, and the packages shown in the build dashboard for my custom image based on core-image-minimal match the packages shown for core-image-minimal.

However, the package group packagegroup-core-boot is shown in the build dashboard, but not in the custom image "Add | Remove" packages screen. I will attempt to fix this.
Comment 7 Belen Barros Pena 2016-01-15 10:13:40 UTC
(In reply to comment #6)
> I've tried from a clean build directory, and the packages shown in the build
> dashboard for my custom image based on core-image-minimal match the packages
> shown for core-image-minimal.

I've tried this again using elliot/toaster/ic-7 and I see the same number of packages in build history and image customisation for core-image-minimal. I then built core-image-base just to be sure, and the number of packages are different: 114 in the build data, 83 in image customisation. 

> 
> However, the package group packagegroup-core-boot is shown in the build
> dashboard, but not in the custom image "Add | Remove" packages screen. I
> will attempt to fix this.

Note that the fix for the packagegroups issues is not showing the package groups in image customisation, but splitting the package groups in the build data, as explained in bug 8901
Comment 8 Elliot Smith 2016-01-15 10:35:31 UTC
I would suggest not taking on bug 8901 as well as all the other stuff required for IC right now. 8901 on its own is a few days work, as it needs a lot of UI changes + back-end changes.

I would suggest either (1) removing packagegroups wherever they are displayed, or (2) displaying them everywhere using the existing design.

My preference would be to do the latter, as it will be faster than removing them everywhere, and hopefully just involve a few tweaks to a handful of queries. Also, we do want to display packagegroups eventually (as per bug 8901), so it doesn't make sense to remove them from the display.

Once we have packages and packagegroups displayed consistently everywhere, we can then move onto bug 8901 and fix how we show them in the UI.
Comment 9 Belen Barros Pena 2016-01-15 10:57:51 UTC
This sounds reasonable, but I suspect there is a reason why we are not showing packagegroups in the list of packages for custom images. The reason can probably be explained by Michael Wood or Brian Avery.
Comment 10 Elliot Smith 2016-01-18 16:59:08 UTC
After discussion with Brian, we've decided to implement this by including packagegroups in the list of packages for a custom image, but making it so that they aren't editable (as for locale packages).
Comment 11 Belen Barros Pena 2016-01-18 18:18:20 UTC
(In reply to comment #10)
> After discussion with Brian, we've decided to implement this by including
> packagegroups in the list of packages for a custom image, but making it so
> that they aren't editable (as for locale packages).

This works for me. I will come up with something for showing those, since we'll need to explain what a packagegroup is.
Comment 12 Elliot Smith 2016-01-19 10:04:51 UTC
I've modified the query used to populate that table, as it was screening out package names with "-locale" or "packagegroup" in them. This gives the correct package count for a customised core-image-minimal (28).

However, the counts still disagree for a customised core-image-sato (562 in the custom image based on core-image-sato, 607 packages in a build created from that custom image). I will investigate this next.
Comment 13 Belen Barros Pena 2016-01-19 12:15:02 UTC
Created attachment 2933 [details]
Design - Showing package groups in the custom image details page
Comment 14 Elliot Smith 2016-01-25 16:07:02 UTC
Passing back to Michael, as he's taken over work on the elliot/toaster/ic-7-rebase branch.
Comment 15 Michael Wood 2016-02-16 10:12:49 UTC
Believe this is fixed with the package syncing work

http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=30c132bc4726be90fadee419a588a8b8f2161380
Comment 16 Alexandru Roman 2016-02-16 12:00:42 UTC
Verified on master: 7a700f59d98ffe08714582532d3edb97abc2e33e