Bug 9755 - buildhistory-diff contains loads of PKGR spam when used with PRSERV
Summary: buildhistory-diff contains loads of PKGR spam when used with PRSERV
Status: RESOLVED FIXED
Alias: None
Product: OE-Core
Classification: Build System, Metadata & Runtime
Component: Scripts and Tools (show other bugs)
Version: unspecified
Hardware: x86 Multiple
: Medium+ enhancement
Target Milestone: 2.2 M2
Assignee: Ed Bartosh
QA Contact: David Rodriguez
URL:
Whiteboard:
Depends on:
Blocks:
 
Reported: 2016-06-10 07:13 UTC by Jussi Kukkonen
Modified: 2016-07-29 15:59 UTC (History)
2 users (show)

See Also:
OS type for building Yocto: ---
Type of Regression: ---
Verified:
Documentation change: No (bug/feature does not impact docs)


Attachments
fix (797 bytes, text/plain)
2016-07-06 12:20 UTC, Ed Bartosh
no flags Details
fix v2 (1.14 KB, text/plain)
2016-07-07 09:28 UTC, Ed Bartosh
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Jussi Kukkonen 2016-06-10 07:13:03 UTC
When using PR service the buildhistory-diff output contains a lot of PKGR changes: In practice the mass of PKGR updates hide other important changes as they often account for 80% of all changes.


I realise PKGR changes are sometimes important but at least for my work the negatives outweigh the positives. I could use "grep -v PKGR" but this often hides the package name:
  packages/core2-64-poky-linux/at-spi2-core/at-spi2-core-locale-ug: PKGR changed from r0.5 to r0.4
    * PV changed from "2.18.3" to "2.20.2"
In this case I would not know which package got a PV change if I used grep.


I propose PKGR should be hidden unless "--report-all" is used (or alternatively a new flag is added to hide PKGR changes).
Comment 1 Ross Burton 2016-06-10 07:53:16 UTC
It's easy to identify PR service changes as they're all of the form rX.N -> rX.N+1, so these could be filtered out globally.  A change from r1 to r2 is something that should be in the diff as that's done by the recipes.
Comment 2 Ross Burton 2016-06-16 16:44:23 UTC
Ed wanted a bit more clarification so here we go.

Usually a PR (package revision) is of the form "r0", "r1", "r3" and so on, with r0 the default.  Historically when a recipe was edited the PR would be bumped from r0 to r1 to r2.

Now that we've said that there are two classes of users: those that care about PRs as they have package feeds and those that don't care.  The former *really* care so the PR service was created for them, this ensure that *every* rebuild of a recipe causes the PR to increase.  The latter don't care at all and the PR doesn't need to change.

The PR service simply adds a ".n" suffix to the existing PR, so if a recipe says PR=r2 then the first build becomes r2.0, then r2.1, r2.2 and so on.  These PR bumps are meaningless as they're simply markers of a build happening.

Thus, buildhistory-diff should filter out PR changes which are simply of the form rN.X -> rN.X+1, such as r1.1 to r1.2.  Changes such as r1 to r2 should still be displayed.
Comment 3 Ed Bartosh 2016-06-16 17:00:37 UTC
Thank you for the explanations!

Should filtering out be a default or should I add new flag?
I'd say it's safer to add new flag than change default behavior.

Any suggestions on option name?
Comment 4 Ross Burton 2016-06-16 21:48:19 UTC
I'd be happy with PR service bumps filtered out always, but add an option if you prefer.
Comment 5 Ed Bartosh 2016-07-06 12:20:35 UTC
Created attachment 3270 [details]
fix

Jussi, can you check if this patch works for you?

It strips only 0.1 increments and decrements as Ross suggested.
Comment 6 Jussi Kukkonen 2016-07-07 07:56:30 UTC
(In reply to comment #5)
> Created attachment 3270 [details]
> fix
> 
> Jussi, can you check if this patch works for you?
> 
> It strips only 0.1 increments and decrements as Ross suggested.

This failed with

Traceback (most recent call last):
  File "/home/jku/src/poky/scripts/buildhistory-diff", line 100, in <module>
    main()
  File "/home/jku/src/poky/scripts/buildhistory-diff", line 84, in main
    changes = oe.buildhistory_analysis.process_changes(options.buildhistory_dir, fromrev, torev, options.report_all, options.report_ver)
  File "/home/jku/src/poky/meta/lib/oe/buildhistory_analysis.py", line 392, in process_changes
    changes.extend(compare_dict_blobs(path, d.a_blob, d.b_blob, report_all, report_ver))
  File "/home/jku/src/poky/meta/lib/oe/buildhistory_analysis.py", line 372, in compare_dict_blobs
    if abs(float(vera) - float(verb)) == 0.1:
ValueError: could not convert string to float: '2.1.11'


when processing "packages/all-poky-linux/encodings/encodings-dbg". The encodings recipe has this curious thing in it:
    PR = "${INC_PR}.1"
Comment 7 Ross Burton 2016-07-07 08:16:43 UTC
"    if abs(float(vera) - float(verb)) == 0.1:"

Don't expect that versions are going to be floats, and also float arithmetic like that just doesn't work: float(0.2)-float(0.1) could easily equal 0.10000000001.

You'll have to split the version on dots and see if all the parts match expect the final fragment where one is an int and the other is the same int +1.
Comment 8 Ed Bartosh 2016-07-07 09:28:51 UTC
Created attachment 3273 [details]
fix v2

Fair enough :)
How about this one?
Comment 9 Ed Bartosh 2016-07-08 14:49:19 UTC
sent for review to the mailing list: http://lists.openembedded.org/pipermail/openembedded-core/2016-July/123654.html
Comment 10 Ed Bartosh 2016-07-21 14:53:51 UTC
implementation has been merged into oe-core master.