| Summary: | PR Service/Server behaviour is unclear in docs | ||
|---|---|---|---|
| Product: | [Documentation] Mega Manual | Reporter: | Robert Berger <pokylinux> |
| Component: | mega-manual | Assignee: | Antonin Godard <antonin.godard> |
| Status: | RESOLVED FIXED | QA Contact: | |
| Severity: | normal | ||
| Priority: | Medium+ | CC: | JPEWhacker, michael.opdenacker, randy.macleod |
| Version: | 5.2 | ||
| Target Milestone: | 6.0 M2 | ||
| Hardware: | All | ||
| OS: | Multiple | ||
| Whiteboard: | |||
| OS type for building Yocto: | --- | Type of Regression: | --- |
| Verified: | Documentation change: | Yes (doc changes required) | |
|
Description
Robert Berger
2025-01-25 16:25:47 UTC
Joshua to review and help Antonin improve docs. OK. If this behavior is intended, I would like to understand why it behaves like that. Bulk move of bugs I am assigned to from Milestone 5.2M4 to 5.3M1. ping Hi Robert, I looked into the code of the PR Server to get a sense of how it works. To my understanding there are two ways the PR could be bumped, it happens in either https://git.openembedded.org/openembedded-core/tree/meta/classes-global/package.bbclass#n301 or https://git.openembedded.org/openembedded-core/tree/meta/classes-global/package.bbclass#n304 getPR() returns either the previous value of PR, of the incremented value PR depending on the arguments passed to it. To my understanding the first getPR() call increments the value if SRCPV has changed - which it has since SRCPV contains the commit sha, which has change when AUTOREV is set and there are changes on the source code. The second getPR() changes when the taskhash has changed - which has changed because do_install() was executed. You can compare two taskhashes with 'bitbake-dumpsig -t simple-hello-world-git do_package'. The thing is I don't know if that's a bug or not - should it be called only once or twice? For now from the code I feel like bumping PR is actually the intended behavior. I don't think the documentation contradicts this. What do you think? I'm also adding Michael Opdenacker as I think was part of the development of the PRServer - maybe you can help Michael! :) Antonin Hi, I've tried to give my answer to this and try to explain how the PR service behavior works in my previous comment. As I was saying, I don't think the documentation is contradictory. So I will close the bug a week from now. If you have any suggestions on how to improve it, feel free to re-open this bug and add them here, or even better send patches to the docs list. Thanks, Antonin Hi, There are potentially 2 things changed by the PR server. One is +gitX+. The other is r0.X. It looks like +gitX+ is related to changes in the source code and r0.X is related to changes in the recipe. It's still unclear why in my test case 6 both change and not just +gitX+. (In reply to Robert Berger from comment #7) > Hi, > > There are potentially 2 things changed by the PR server. > > One is +gitX+. > The other is r0.X. > > It looks like +gitX+ is related to changes in the source code and r0.X is > related to changes in the recipe. > > It's still unclear why in my test case 6 both change and not just +gitX+. +gitX+ is indeed related to changes in the source code. r0.X is bumped each time the checksum of the do_package task of the simple-hello-world-git recipe changes. This happens here: https://git.openembedded.org/openembedded-core/tree/meta/classes-global/package.bbclass?id=235e6d49e5888ad04416219e10b6df91a738661a#n306 This line sets the value of PRAUTO and represents the number X found in r0.X. It will in the end make it into EXTENDPRAUTO, which itself makes to PKGR == r0.X. This line calls getPR(version, pkgarch, checksum). Between test case 5 and 6, only the checksum changes. This checksum is the checksum of the do_package task (gotten from get_do_package_hash() above). Now, let's dump what changed with regards to this task between two consecutive runs, using the sigdata file in build/tmp/stamps/: ``` $ bitbake-diffsigs tmp/stamps/cortexa57-poky-linux/simple-hello-world-git/1.0.1+git.do_package.sigdata.fa9806981a2978e659dae947209e0f322b27d6fc7b3cac359ecc8668daab152f tmp/stamps/cortexa57-poky-linux/simple-hello-world-git/1.0.1+git.do_package.sigdata.81f13d047d1ce85a728d83342238e153270134ba092af2746dbba9321470f7cf NOTE: Starting bitbake server... NOTE: Started PRServer with DBfile: .../build/cache/prserv.sqlite3, Address: 127.0.0.1:39781, PID: 146568 Hash for task dependency simple-hello-world-git:do_install changed from 95f6910c8ec9024318635f79b8c26ac3fe6f04f584f65d2b370921229aa4d8f3 to 6e0f164859773b701658a52d7750ba5932c122290366b2bc880127edbe8084bf Hash for task dependency simple-hello-world-git:do_compile changed from 463fab24e668b0edd1b51b05df7472077ce91e0c54debae9b3d0c33dd07d554a to d48b8cc9dc9c653cc604a551543ccb7d5588982a26085b2babd65e982c68bdc8 Hash for task dependency simple-hello-world-git:do_configure changed from 61eee51703fe0c2a8c2f4563370a2d93c9c3841f34254368324dd191d761d3a4 to e8b3944c0984a4b11a046efa4c8f07f9bac0697fb2f84f0829c6acda65fc6ce4 Hash for task dependency simple-hello-world-git:do_deploy_source_date_epoch changed from 1617b9936581d35019e83cc002ba1fd55534a2ff336d3c298d8191810aa4b743 to 7ef74490e285291463a64744c45272acc3fc3dc97e1dde71109e39586cbe7c77 Hash for task dependency simple-hello-world-git:do_patch changed from 473eca8d83aba301d1551dc159f8a81094c00d509353e445d5fa2a7fd02ef8b4 to 7d0b29381d1fde3e050026fb8b9e4d1368d9df2090c47ad8098c3420b2ade31c Hash for task dependency simple-hello-world-git:do_unpack changed from f03dae5d9cd00fc7dd335a44fbd23d4a0b132e2b65daef6b85c3b7ee619b0a06 to a9ae3c788aac0e12b3a53bb276b205c8b6c564ea01d54619ff7310afd4796be2 Hash for task dependency simple-hello-world-git:do_fetch changed from ef92d9cd7c616daee6b5511267e5d5b468785fffdf91d41cd5265f669c2e6040 to 779c157ec3172142f37287cdd6a7206bb5ab306ef33344dea97235c86d1ea1a1 basehash changed from a6399ca4bbeba168cef30919e7fd66bf18127a29d2ad6a1dcbfa6949927a31d8 to 8ff8764b0327948a8b973c28916b57f250e91afc1da075eb1502b8a32a51ab46 Variable fetcher_hashes_dummyfunc value changed from '2650ad6714c3f3248abfe9d3daf1196f307ed494' to '4af682a50174f5deb0397847da97d7cdba4ad067' ``` The last line shows that the value of fetcher_hashes_dummyfunc changed from '2650ad6714c3f3248abfe9d3daf1196f307ed494' to '4af682a50174f5deb0397847da97d7cdba4ad067'. Those are the commit hashes in the git history of the simple-hello-world-git repository. Now you can see why this 0.X gets bumped, is because of the SRCREV change. The documentation does not imply the opposite IMO. If that's clear for you, I'll close this bug. Cheers, Antonin I just realized that the example in the docs was indeed wrong, sorry. I've sent a patch here: https://lore.kernel.org/r/20260120-pr-server-increment-details-v1-1-fa5d288def24@bootlin.com Help reviewing / ack would be appreciated. Thanks, Antonin I think you are referring to this: https://docs.yoctoproject.org/dev/dev-manual/packages.html#automatically-incrementing-a-package-version-number SRCREV = "${AUTOREV}" PV = "1.0+git" This means, that with or without a PR Server running we should have something like: hello-world-git_1.0+gitX+b6558dd387-rX.X_armv7a-neon.ipk and not hello-world-git_0.0+gitX+b6558dd387-rX.X_armv7a-neon.ipk because of PV = "1.0+git" I will give it a try and will let you know what happens with a PR Server running and SRCREV = "${AUTOREV}". Generally speaking we have a couple of possibilities here, which should be reflected by gitX, commit id and rX.X. 1) recipe did not change, code did not change 2) recipe did not change, code changed 3) recipe changed (e.g. from fixed commit ID to AUTOREV), code did not change 4) recipe changed, code changed In your example we can see from the commit id change that the code changed: hello-world-git_1.0+git0+b6558dd387-r0.0_armv7a-neon.ipk hello-world-git_1.0+git1+dd2f5c3565-r0.0_armv7a-neon.ipk But we don't know if the recipe changed as well or not. I guess it did not, because of the r0.0 in both cases. Stay tuned, as soon as I'll find some time to test I'll let you know. (In reply to Robert Berger from comment #10) > I think you are referring to this: > > https://docs.yoctoproject.org/dev/dev-manual/packages.html#automatically- > incrementing-a-package-version-number > > SRCREV = "${AUTOREV}" > PV = "1.0+git" > > This means, that with or without a PR Server running we should have > something like: > > hello-world-git_1.0+gitX+b6558dd387-rX.X_armv7a-neon.ipk > > and not > > hello-world-git_0.0+gitX+b6558dd387-rX.X_armv7a-neon.ipk > > because of PV = "1.0+git" Correct, thank you, I will correct that in the v2. > I will give it a try and will let you know what happens with a PR Server > running and SRCREV = "${AUTOREV}". > > Generally speaking we have a couple of possibilities here, which should be > reflected by gitX, commit id and rX.X. > > 1) recipe did not change, code did not change Here nothing changes (commit id, r0.X, gitX), obviously, I think we can agree on that. > 2) recipe did not change, code changed That is not possible in practice. If the code changes and you want your recipe to take the changes, there are two possibilities I can think of: - You are using AUTOREV, and as explained above this will trigger a "recipe change" so gitX _and_ r0.X get bumped. - You a using a fixed SRCREV, which you need to update to the new commit id. > 3) recipe changed (e.g. from fixed commit ID to AUTOREV), code did not change This should only bump r0.X. > 4) recipe changed, code changed This should bump r0.X and gitX, and the commit id should change. > In your example we can see from the commit id change that the code changed: > > hello-world-git_1.0+git0+b6558dd387-r0.0_armv7a-neon.ipk > hello-world-git_1.0+git1+dd2f5c3565-r0.0_armv7a-neon.ipk I corrected this in my patch I sent in my previous comment as this was wrong. I changed it to: hello-world-git_1.0+git0+b6558dd387-r0.0_armv7a-neon.ipk hello-world-git_1.0+git1+dd2f5c3565-r0.1_armv7a-neon.ipk The aim of the patch was aiming at correcting these two lines specifically. Antonin (In reply to Antonin Godard from comment #11) > (In reply to Robert Berger from comment #10) > > > > Generally speaking we have a couple of possibilities here, which should be > > reflected by gitX, commit id and rX.X. > > > > 1) recipe did not change, code did not change > > Here nothing changes (commit id, r0.X, gitX), obviously, I think we can > agree on that. Yes. My point is, that you should also somehow describe the previous state and the current, since this is important for gitX, commit id and rX.X. > > > 2) recipe did not change, code changed > > That is not possible in practice. If the code changes and you want your > recipe to take the changes, there are two possibilities I can think of: > - You are using AUTOREV, and as explained above this will trigger a "recipe > change" so gitX _and_ r0.X get bumped. Yes AUTOREV and someone checked in a new version of the code. > - You a using a fixed SRCREV, which you need to update to the new commit id. A change from AUTOREV to fixed SRCREV or fixed SRCREV to AUTOREV could mean that the recipe changed, but *) you could change the SRVCREV from a class like here: https://git.yoctoproject.org/meta-yocto/tree/meta-poky/conf/distro/include/poky-floating-revisions.inc INHERIT += "poky-bleeding" POKY_AUTOREV_RECIPES = "\ libmatchbox \ opkg-utils \ matchbox-config-gtk \ matchbox-desktop \ matchbox-keyboard \ matchbox-panel-2 \ matchbox-terminal \ matchbox-theme-sato \ matchbox-wm \ pseudo \ puzzles \ sato-icon-theme \ sato-screenshot \ settings-daemon \ " *) you could change the SRCREV from a config file: e.g. in local.conf: SRCREV:pn-hello-world-git = "ab7730a2f39660b573fae9e9c20e8c33f8a642db" or SRCREV:pn-hello-world-git = "${AUTOREV}" meaning the recipe did not change *) you could use the devupstream class in your recipe: https://gitlab.com/meta-layers/meta-yocto-training/-/blob/master/recipes-training/simple-hello-world-git-devupstream/simple-hello-world-git-devupstream_git.bb?ref_type=heads # non upstream version: bitbake simple-hello-world-git-devupstream # upstream version: # we can activate the devupstream build if we enable e.g. in local.conf: PREFERRED_VERSION_simple-hello-world-git-devupstream = "1.0.2%" or bitbake simple-hello-world-git-devupstream-upstream > > > 3) recipe changed (e.g. from fixed commit ID to AUTOREV), code did not change > > This should only bump r0.X. I need to test this. > > > 4) recipe changed, code changed > > This should bump r0.X and gitX, and the commit id should change. > > > In your example we can see from the commit id change that the code changed: > > > > hello-world-git_1.0+git0+b6558dd387-r0.0_armv7a-neon.ipk > > hello-world-git_1.0+git1+dd2f5c3565-r0.0_armv7a-neon.ipk > > I corrected this in my patch I sent in my previous comment as this was > wrong. I changed it to: > > hello-world-git_1.0+git0+b6558dd387-r0.0_armv7a-neon.ipk > hello-world-git_1.0+git1+dd2f5c3565-r0.1_armv7a-neon.ipk > > The aim of the patch was aiming at correcting these two lines specifically. > > Antonin Stay tuned ;) (In reply to Robert Berger from comment #12) > A change from AUTOREV to fixed SRCREV or fixed SRCREV to AUTOREV could mean > that the recipe changed, but > > *) you could change the SRVCREV from a class like here: > > https://git.yoctoproject.org/meta-yocto/tree/meta-poky/conf/distro/include/ > poky-floating-revisions.inc > > INHERIT += "poky-bleeding" > > POKY_AUTOREV_RECIPES = "\ > libmatchbox \ > opkg-utils \ > matchbox-config-gtk \ > matchbox-desktop \ > matchbox-keyboard \ > matchbox-panel-2 \ > matchbox-terminal \ > matchbox-theme-sato \ > matchbox-wm \ > pseudo \ > puzzles \ > sato-icon-theme \ > sato-screenshot \ > settings-daemon \ > " > > *) you could change the SRCREV from a config file: > > e.g. in local.conf: > > SRCREV:pn-hello-world-git = "ab7730a2f39660b573fae9e9c20e8c33f8a642db" > > or > > SRCREV:pn-hello-world-git = "${AUTOREV}" > > meaning the recipe did not change The thing is saying "r0.X gets bumped when the recipe changes" is not exactly true… The reality is that this gets bumped when the hash of the do_package task changes. In all the cases above I believe that should be the case. This is what I tried to explain in more details in the patch I sent (https://lore.kernel.org/r/20260120-pr-server-increment-details-v1-1-fa5d288def24@bootlin.com). Thanks for putting your efforts in testing this! :) I have a PR server running locally and this in my recipe:
SRCREV = "${AUTOREV}"
SRC_URI = "git:///${COREBASE}/../meta-yocto-training-sources/simple-hello-world-git/;protocol=file;branch=master"
PV = "1.0.1+git"
1) First run:
tmp/deploy/ipk/aarch64/simple-hello-world-git_1.0.1+git0+ab7730a2f3-r0.0_aarch64.ipk
git0, ab7730a2f3, r0.0
Without a PR server running it would be -r0_aarch64.ipk and not r0.0
2) I change the sources (git add/git commit) and don't change the recipe:
tmp/deploy/ipk/aarch64/simple-hello-world-git_1.0.1+git1+8677ce399f-r0.1_aarch64.ipk
git1, 8677ce399f, r0.1
3) I remove the last commit from my git repo
git reset --hard HEAD^
QA Issue: Package version for package xxx went backwards which would break package feeds (from 0:1.0.1+git1+8677ce399f-r0.1 to 0:1.0.1+git0+ab7730a2f3-r0.0) [version-going-backwards]
Back to
tmp/deploy/ipk/aarch64/simple-hello-world-git_1.0.1+git0+ab7730a2f3-r0.0_aarch64.ipk
git0, ab7730a2f3, r0.0
4) I change the SRCREV to a fixed commit id
SRCREV = "ab7730a2f39660b573fae9e9c20e8c33f8a642db"
tmp/deploy/ipk/aarch64/simple-hello-world-git_1.0.1+git0+ab7730a2f3-r0.0_aarch64.ipk
git0, ab7730a2f3, r0.0
but no [version-going-backwards]
So yes, indeed we can not distinguish from the package name whether the recipe was changed or the code.
"r0.X gets bumped when the recipe changes" is WRONG!
Now the interesting question would be: Should it be like this? RP?
I would find it very useful if we could distinguish from the package name if the change was in the recipe or the source code. Especially for audits!
5) I change to AUTOREV and modify the sources
tmp/deploy/ipk/aarch64/simple-hello-world-git-dev_1.0.1+git2+e7506f103f-r0.2_aarch64.ipk
git2, e7506f103f, r0.2
6) I change the SRCREV to a fixed commit id
SRCREV = "ab7730a2f39660b573fae9e9c20e8c33f8a642db"
QA Issue: Package version for package XXX went backwards which would break package feeds (from 0:1.0.1+git2+e7506f103f-r0.2 to 0:1.0.1+git0+ab7730a2f3-r0.0) [version-going-backwards]
tmp/deploy/ipk/aarch64/simple-hello-world-git_1.0.1+git0+ab7730a2f3-r0.0_aarch64.ipk
git0, ab7730a2f3, r0.0
7) I change to AUTOREV and modify the sources
tmp/deploy/ipk/aarch64/simple-hello-world-git-src_1.0.1+git3+cae4dff799-r0.3_aarch64.ipk
git3, cae4dff799, r0.3
8) I change the recipe ${CFLAGS} -O3
tmp/deploy/ipk/aarch64/simple-hello-world-git-dbg_1.0.1+git3+cae4dff799-r0.4_aarch64.ipk
git3, cae4dff799, r0.4
This is the only case, where r0 gets incremented (and nothing else) where we can say that the recipe was changed.
As you say:
+gitX+ is indeed related to changes in the source code.
r0.X is bumped each time the checksum of the do_package task of the simple-hello-world-git recipe changes, but you can not say that this is always related to changes in the recipe.
In this specific case we can say it's related to changes in the meta data, since
we changed from git3, cae4dff799, r0.3 to git3, cae4dff799, r0.4.
This means that due to a new compilation the do_package task checksum changed and since git3 as well as cae4dff799 stayed the same it has to be some change in the meta data (recipe).
9) I change the SRCREV back to fixed commit id:
SRCREV = "ab7730a2f39660b573fae9e9c20e8c33f8a642db"
tmp/deploy/ipk/aarch64/simple-hello-world-git_1.0.1+git4+ab7730a2f3-r0.5_aarch64.ipk
git4, ab7730a2f3, r0.5
because we still have -O3 in the recipe.
10) I remove the -O3
QA Issue: Package version for package XXX went backwards which would break package feeds (from 0:1.0.1+git4+ab7730a2f3-r0.5 to 0:1.0.1+git0+ab7730a2f3-r0.0) [version-going-backwards]
tmp/deploy/ipk/aarch64/simple-hello-world-git_1.0.1+git0+ab7730a2f3-r0.0_aarch64.ipk
----
In case you want me to test something else, please let me know.
(In reply to Robert Berger from comment #14) > I have a PR server running locally and this in my recipe: > > SRCREV = "${AUTOREV}" > SRC_URI = > "git:///${COREBASE}/../meta-yocto-training-sources/simple-hello-world-git/; > protocol=file;branch=master" > PV = "1.0.1+git" > > 1) First run: > > tmp/deploy/ipk/aarch64/simple-hello-world-git_1.0.1+git0+ab7730a2f3-r0. > 0_aarch64.ipk > > git0, ab7730a2f3, r0.0 > > Without a PR server running it would be -r0_aarch64.ipk and not r0.0 > > 2) I change the sources (git add/git commit) and don't change the recipe: > > tmp/deploy/ipk/aarch64/simple-hello-world-git_1.0.1+git1+8677ce399f-r0. > 1_aarch64.ipk > > git1, 8677ce399f, r0.1 > > 3) I remove the last commit from my git repo > > git reset --hard HEAD^ > > QA Issue: Package version for package xxx went backwards which would break > package feeds (from 0:1.0.1+git1+8677ce399f-r0.1 to > 0:1.0.1+git0+ab7730a2f3-r0.0) [version-going-backwards] > > Back to > > tmp/deploy/ipk/aarch64/simple-hello-world-git_1.0.1+git0+ab7730a2f3-r0. > 0_aarch64.ipk > > git0, ab7730a2f3, r0.0 > > 4) I change the SRCREV to a fixed commit id > > SRCREV = "ab7730a2f39660b573fae9e9c20e8c33f8a642db" > > tmp/deploy/ipk/aarch64/simple-hello-world-git_1.0.1+git0+ab7730a2f3-r0. > 0_aarch64.ipk > > git0, ab7730a2f3, r0.0 > > but no [version-going-backwards] > > So yes, indeed we can not distinguish from the package name whether the > recipe was changed or the code. > > "r0.X gets bumped when the recipe changes" is WRONG! > > Now the interesting question would be: Should it be like this? RP? > > I would find it very useful if we could distinguish from the package name if > the change was in the recipe or the source code. Especially for audits! I would suggest filing a bug affecting the PR server directly, as this is not a docs problem anymore. > 5) I change to AUTOREV and modify the sources > > tmp/deploy/ipk/aarch64/simple-hello-world-git-dev_1.0.1+git2+e7506f103f-r0. > 2_aarch64.ipk > > git2, e7506f103f, r0.2 > > 6) I change the SRCREV to a fixed commit id > > SRCREV = "ab7730a2f39660b573fae9e9c20e8c33f8a642db" > > QA Issue: Package version for package XXX went backwards which would break > package feeds (from 0:1.0.1+git2+e7506f103f-r0.2 to > 0:1.0.1+git0+ab7730a2f3-r0.0) [version-going-backwards] > > tmp/deploy/ipk/aarch64/simple-hello-world-git_1.0.1+git0+ab7730a2f3-r0. > 0_aarch64.ipk > > git0, ab7730a2f3, r0.0 > > 7) I change to AUTOREV and modify the sources > > tmp/deploy/ipk/aarch64/simple-hello-world-git-src_1.0.1+git3+cae4dff799-r0. > 3_aarch64.ipk > > git3, cae4dff799, r0.3 > > 8) I change the recipe ${CFLAGS} -O3 > > tmp/deploy/ipk/aarch64/simple-hello-world-git-dbg_1.0.1+git3+cae4dff799-r0. > 4_aarch64.ipk > > git3, cae4dff799, r0.4 > > This is the only case, where r0 gets incremented (and nothing else) where we > can say that the recipe was changed. > > As you say: > > +gitX+ is indeed related to changes in the source code. > > r0.X is bumped each time the checksum of the do_package task of the > simple-hello-world-git recipe changes, but you can not say that this is > always related to changes in the recipe. I've sent a new version of the patchset that limits the r0.X change to being related to the do_package task hash change: https://lore.kernel.org/r/20260127-pr-server-increment-details-v2-0-b46f91df002b@bootlin.com (In reply to Antonin Godard from comment #15) > I've sent a new version of the patchset that limits the r0.X change to being > related to the do_package task hash change: > https://lore.kernel.org/r/20260127-pr-server-increment-details-v2-0-b46f91df002b@bootlin.com Patches merged: https://git.yoctoproject.org/yocto-docs/commit/?id=09f0430bc69024b9854c31ba6783ddd807aa4f19 https://git.yoctoproject.org/yocto-docs/commit/?id=7a0324b6a10e64ee250945747db10ca88040b1ce https://git.yoctoproject.org/yocto-docs/commit/?id=411122812ced4ec32127a823896a73aacf6eb97c Robert, if you think the PR server behaves incorrectly with regards to your observations, please open a new bug against the PR server directly. Thanks for your help on this! Moving to Resolved Fixed. |