Bug 2180

Summary: [HOB] Eliminate the by recipe grouping in the packages table
Product: [Build System, Metadata & Runtime] Hob Reporter: Shane Wang <shane.wang>
Component: hobAssignee: Cristiana Voicu <cristiana.voicu>
Status: VERIFIED FIXED QA Contact:
Severity: enhancement    
Priority: Medium CC: alexandru.damian, andrei.adrianx.dinu, belen.barros.pena, cristian.iorga, cristiana.voicu, giulia, jessica.zhang, jiajun.xu, poky.bs.watcher, poky.watcher, richard.purdie, sgw, song.liu
Version: unspecified   
Target Milestone: 1.4 M4   
Hardware: x86   
OS: Multiple   
Whiteboard:
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: ---
Attachments:
Description Flags
bug 2180 - packages table.pdf
none
hob grouping
none
Design for the packages table
none
Progress patch
none
brought-in-by-blank.png
none
size-sorting.png
none
packages sorting size none

Description Shane Wang 2012-03-27 11:37:41 UTC
Dongxiao Xu 2012-03-21 02:15:52 UTC 
In current implementation, we don't have 'Packages by recipe' tab. Actually in
the 'All packages' tab, the packages are already arranged by recipe. What's
your opinion on this item?

Belen 2012-03-21 12:14:43 UTC 
I think we really need a list of packages not grouped by recipe: a simple way
of looking at a list of packages that can be sorted by name, size, by included,
etc, without the recipe names getting on the way. How much work would that be?

I am not saying the grouping by recipe is not useful: simply that it is not
always relevant, and it shouldn't be the only way of looking at packages.

Thanks!

Belen
Comment 1 Giulia 2012-03-27 13:17:58 UTC
When the 'All packages' tab will be implemented, the tabs order should be:
1) 'Included packages' 
2) 'All packages' 
3) 'Packages by recipe'.
Comment 2 Belen Barros Pena 2012-10-08 14:22:54 UTC
Created attachment 879 [details]
bug 2180 - packages table.pdf

For a description of the packages table, see the attached document bug 2180 - packages table.pdf

For some history, see bug 2195
Comment 3 Cristian Iorga 2012-11-14 12:53:17 UTC
Hello,

I don't think this change is an improvement.

My reason is that the current way of presenting included packages is a way Hob makes clear for a developer how packages are interconnected.
Bogdan Marinescu agrees with me also.
Comment 4 Belen Barros Pena 2012-11-16 11:13:33 UTC
Hi Cristian,

I have difficulties agreeing with you on this one. This is why: 

1. The grouping by recipe breaks down the table and makes scanning the content quite hard. The "Included" tab is particularly bad, since often only one or 2 packages will be included by recipe. 

2. The grouping messes up the relationship between cell content and table headers

3. The grouping messes up the sorting. For example, sorting packages by size sorts within each recipe group, so for qemux86 core-image-minimal included packages you get a package of 244kb above one that's 0B and above another one that 16KB. 

I agree that having the ability to look at packages based on the recipe they are coming from is useful, but I don't think that should come at the expense of the table readability and manipulation. We could keep a sortable "Recipe" column in the table so that you can sort packages based on their recipe. This gives you basically the same functionality that the current grouping, without messing up the table.

In the document attached to this bug that 'Recipe' column exists only in the "All Packages" tab. Maybe we should consider adding it to the "Included" tab too.

Give me a shout if you want to talk about it.

Belen
Comment 5 Cristian Iorga 2013-01-07 13:12:22 UTC
Hello Belen,

Here are my answers:

1. As far as I can see it, the grouping is done on a logical level. I don't see listing directly localization for a certain software component as an improvement (instead of seeing it grouped under the parent-component, as it is now)

2. Maybe I am wrong, but 3. is a consequence of 2. I will answer directly to 3.

3. Well, this is a different issue. This can be fixed in another bug. Summing up the components of a recipe and sorting by total size would fix the issue. Is my assumption correct?

4. For finding easily, there is a search. If that is not functional, that is also something that needs to be fixed.
Comment 6 Andrei Dinu 2013-01-07 13:28:56 UTC
Created attachment 1000 [details]
hob grouping
Comment 7 Andrei Dinu 2013-01-07 13:31:40 UTC
Hello,

I have attached a screenshot with my suggestion for this bug.

I have grouped all the packages, without expanding them from the start. If someone wants to see certain sub components he should expand the package using the button from the left side.

However, there is the "search packages" field in which a package or sub component can be easily found. I've tested it in the environment created ( see attachment ) and the search doesn't work if the components are not expanded.

For example : opkg-1.0.8 has a sub component update-alternatives-cworth.

The search doesn't find the sub component update-alternatives-cworth if opkg-1.0.8 is not expanded.

In my opinion if the search is fully functional, finding a packages shouldn't be a problem.
Comment 8 Belen Barros Pena 2013-01-07 17:20:39 UTC
Ok, I think we have reached a stalemate. At this point I would normally give up, if it wasn't because the existing grouping causes so much trouble. A couple of other reasons why is not good:

1. The interface of Hob nowhere tells you that the grouping is by recipe. The heading of the table says "package name": how am I supposed to know that the packages are what's inside the groupings, and the group heading is a recipe? 

2. Even if we fix the search issue identified by Andrei, search won't really work in its current implementation. This is why. Our search box works not by returning search results or filtering the content of the table: it simply 'jumps' to the first instance of the search string. This behaviour sort of works when there are no groupings (for example, in the recipes table) because if sorted by 'name' all instances of the entered string will be together. But when you have things grouped, this is not the case. 

To see what I mean, you can go to the packages screen, select the "all packages" tab, then type 'lib' in the search box. The package libacl from the recipe acl-2.2.51-r3 will be highlighted. You will probably think those are the only packages starting with lib on the table, but they aren't: there are many more packages starting with lib, but they are in other recipes: libattr in in attr-2.4.46-r4, libcidn in eglibc-2.16-r15+svnr20393, libssp in gcc-runtime-4.7.2-r13 and so on and so forth. In the existing packages table, by typing 'lib' you will never find libssp (why would you, if we don't give you any hint about the existence of more packages starting with 'lib' further down?).

I am really sorry about insisting, but I really think the existing packages table is just not good enough. Beyond design reasons, is there anything else that makes you wary of removing the grouping from the table?

Belen
Comment 9 Richard Purdie 2013-01-07 20:25:19 UTC
FWIW I can certainly see Belen's points on this. To developers used to looking at this from the .bb level its natural to want to know which recipe the packages come from. I think as a compromise, the Recipes column needs to be present on all the tabs. If the user sorts by recipe, I'd really like the code to do a secondary sort by package name under those circumstances too.

With that secondary sort, it would then mean you can get the same information as the current UI but it removes the various issues Belen comments on. The challenge for Belen is how to show multiple sorts on the table ;-). I'm not sure if gtk has a way to do that or not out the box.
Comment 10 Alexandru Damian 2013-01-09 15:34:18 UTC
I agree with Belen's view, on a semantic reason.

The button triggering this view is called "Edit Packages".
The labels in the view clearly state "Packages" everywhere.
As such, the grouping, if anything, should be based on packages. 
At this stage, post build - pre-install, it makes little sense to group by recipe.

As a user:
- I want to add a column between "Size" and "Brought in by" stating the original recipe (just because it's referenced in "Brought in by".
- I'd expect to disable any kind of grouping in this list.
- I'd expect an area below the list where the dependency list (depends on, is required by) is updated for each selected package. 
- I'd expect that selecting/deselecting a package to automatically disable/enable dependencies for that package, as to always create a working image. 

Please estimate the effort to get this going.
Comment 11 Belen Barros Pena 2013-01-10 16:54:31 UTC
Created attachment 1004 [details]
Design for the packages table

Hi all,

As agreed, I've created a document to try to explain my suggested design for the packages tables (see the pdf attached). Let me know what you think.

Belen
Comment 12 Andrei Dinu 2013-01-17 12:20:47 UTC
Created attachment 1016 [details]
Progress patch

Hi all,

After i spent some time and tried several methods to solve the problem i have reached a stall. After arranging the columns and the elements in it as desired, the packages and recipes lose their connection. That happens because the data gathering and the output to the hob screen are done at the same time, not separately. I've tried different approaches but all failed in trying to keep the links between recipes and packages.

I am attaching a patch for the new recipe column, i discarded the rest of the changes made. After you apply the patch, you should have a new Recipe column with the recipe name displayed in it with the current grouping.
Comment 13 Cristiana Voicu 2013-01-30 11:15:20 UTC
Hello Belen,

I saw in the document with the suggested design, a page with a work around for the search problem. I don't think we need to implement this. I will try to  work on #3529 after i will finish this one.

Is it ok for you Belen?
Thanks,
Cristiana
Comment 14 Belen Barros Pena 2013-01-30 11:33:11 UTC
(In reply to comment #13)
> Hello Belen,
> 
> I saw in the document with the suggested design, a page with a work around
> for the search problem. I don't think we need to implement this. I will try
> to  work on #3529 after i will finish this one.
> 
> Is it ok for you Belen?
> Thanks,
> Cristiana

I totally agree.

Thanks!
Comment 15 Cristiana Voicu 2013-01-31 08:07:31 UTC
Hello Belen,

I have implemented this enhancement and I have created a branch on poky-contrib, as we agreed. Here is the link: http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=cvoicu/packagespage

Can you please take a look? 
Thanks,
Cristiana
Comment 16 Belen Barros Pena 2013-01-31 10:44:23 UTC
Cristiana: what a beautiful patch! :) It looks great. 

Couple of questions:

1. The brought in by column is blank for a couple of packages (see attached brought-in-by-blank.png). I am not sure if this is the way it's supposed to be or if there is something going on, so I thought I'd ask just in case.

2. The Size sorting seems to be sorting by text, not by numeric value. For example: 104.0 comes before 12.0 (see size-sorting.png). Will we tackle this as part of 2346?

Thanks!!

Belen

(In reply to comment #15)
> Hello Belen,
> 
> I have implemented this enhancement and I have created a branch on
> poky-contrib, as we agreed. Here is the link:
> http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=cvoicu/
> packagespage
> 
> Can you please take a look? 
> Thanks,
> Cristiana
Comment 17 Belen Barros Pena 2013-01-31 10:44:50 UTC
Created attachment 1038 [details]
brought-in-by-blank.png
Comment 18 Belen Barros Pena 2013-01-31 10:45:11 UTC
Created attachment 1039 [details]
size-sorting.png
Comment 19 Cristiana Voicu 2013-01-31 11:44:25 UTC
Hello Belen,

Thanks for your feedback! Regarding the issues you have seen, here are my comments:
1. The brought in by column is blank for a couple of packages because those packages are specified in the recipes for the image itself. These packages are not brought by some other packages. If you think that this is not ok from the desing point of view, you can open a new bug.

2. The size sorting seems to be an issue. And I saw it was the same for the old implementation(see the attachment). I propose to open a new bug for this issue.

So, I will send the patch for review.
Cristiana
Comment 20 Cristiana Voicu 2013-01-31 11:44:55 UTC
Created attachment 1040 [details]
packages sorting size
Comment 21 Belen Barros Pena 2013-01-31 12:03:39 UTC
(In reply to comment #19)
> Hello Belen,
> 
> Thanks for your feedback! Regarding the issues you have seen, here are my
> comments:
> 1. The brought in by column is blank for a couple of packages because those
> packages are specified in the recipes for the image itself. These packages
> are not brought by some other packages. If you think that this is not ok
> from the desing point of view, you can open a new bug.

Well, if they are not brought in by any other packages, everything is good.

> 
> 2. The size sorting seems to be an issue. And I saw it was the same for the
> old implementation(see the attachment). I propose to open a new bug for this
> issue.

I think this is within the scope of 2346: I'll add a comment on that bug about the size sorting issue.

> 
> So, I will send the patch for review.
> Cristiana
Comment 22 Cristiana Voicu 2013-01-31 13:00:04 UTC
patch merged to master. commit c30ef8886d5597fb2b21fb101d4485a280ba73d4
Comment 23 Belen Barros Pena 2013-02-01 10:17:57 UTC
Verified on 8d80483e2f0f755a61087be1dffaf595fbc467fd