| Summary: | [Performance] do_package should be PACKAGE_ARCH specific, not MACHINE | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Product: | [Build System, Metadata & Runtime] OE-Core | Reporter: | Richard Purdie <richard.purdie> | ||||||
| Component: | core | Assignee: | Richard Purdie <richard.purdie> | ||||||
| Status: | VERIFIED FIXED | QA Contact: | |||||||
| Severity: | enhancement | ||||||||
| Priority: | High | CC: | corneliux.stoicescu, Martin.Jansa, meta.mr.watcher, meta.watcher | ||||||
| Version: | unspecified | ||||||||
| Target Milestone: | 1.4 | ||||||||
| Hardware: | x86 | ||||||||
| OS: | Multiple | ||||||||
| Whiteboard: | |||||||||
| OS type for building Yocto: | --- | Type of Regression: | --- | ||||||
| Verified: | Documentation change: | --- | |||||||
| Attachments: |
|
||||||||
When this is done, the whitelist in sstate.bbclass should be improved, as per the comments there. http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=50247c8f6e0e864266c468afe8537f7fd0a87890 http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=a68c7c792ed85cfb85bcfac32dd629384f169245 I'm not sure if this is side-effect of this changes, but now I have issues with some MACHINE_ARCH recipes.
IIRC mixing PACKAGE_ARCH is not really supported (even when it worked before) or at least setting PACKAGE_ARCH to something else only for some package in recipe
Doing something like
PACKAGE_ARCH_om-gta02 = "${MACHINE_ARCH}"
in bbappend is not good idea, right?
Now I see ipk files sometimes missing during do_rootfs (and only one version MACHINE_ARCH or TUNE_PKGARCH in deploy/ipk).
So I've removed machine override so bbappend sets PACKAGE_ARCH globally:
https://github.com/shr-distribution/meta-smartphone/commit/93d8ca2d0c10e226c4a0adb484cc83ade1b54145
But what about base.bbclass changing PACKAGE_ARCH when SRC_URI_OVERRIDES_PACKAGE_ARCH is enabled and only one MACHINE has machine specific file in FILESPATH? Does it work correctly now?
(In reply to comment #3) > I'm not sure if this is side-effect of this changes, but now I have issues > with some MACHINE_ARCH recipes. > > IIRC mixing PACKAGE_ARCH is not really supported (even when it worked > before) or at least setting PACKAGE_ARCH to something else only for some > package in recipe > > Doing something like > PACKAGE_ARCH_om-gta02 = "${MACHINE_ARCH}" > in bbappend is not good idea, right? > > Now I see ipk files sometimes missing during do_rootfs (and only one version > MACHINE_ARCH or TUNE_PKGARCH in deploy/ipk). > > So I've removed machine override so bbappend sets PACKAGE_ARCH globally: > https://github.com/shr-distribution/meta-smartphone/commit/ > 93d8ca2d0c10e226c4a0adb484cc83ade1b54145 > > But what about base.bbclass changing PACKAGE_ARCH when > SRC_URI_OVERRIDES_PACKAGE_ARCH is enabled and only one MACHINE has machine > specific file in FILESPATH? Does it work correctly now? Another such example in oe-core is opkg (update-alternatives) PACKAGE_ARCH_update-alternatives-cworth = "all" WARNING: The recipe opkg is trying to install files into a shared area when those files already exist. Those files and their manifest location are: /OE/jansa-test/shr-core/tmp-eglibc/deploy/ipk/all/update-alternatives-cworth_0.1.8+svnr633-r10.8_all.ipk Matched in Please verify which package should provide the above files. NOTE: recipe opkg-1_0.1.8+svnr633-r10.8: task do_package_write_ipk: Succeeded I tested this using the following steps: 1)"bitbake less" with MACHINE = "routerstationpro" in localconf 2)"bitbake less" with MACHINE = "qemumips" in local.conf Using commit_1 9da9e5d8aeced9880e84d421492ebc451cf7fa1b(the commit right before Richard's patches), the second "bitbake less" command only did setscene tasks. Using commit_2 a68c7c792ed85cfb85bcfac32dd629384f169245(the second commit from Richard which completes this enhancement), the second command did some setscene tasks and redid about 100 other tasks(from do_package up). The second scenario which contained the patches for this performance enhancement actually took a few minutes to finish while the first scenario(without the patches for this enhancement) finished in under a minute. Richard, can you please look at my results? I will attach the cooker logs. Note_1: all settings not mentioned above(except thread and core numbers) are the default Note_2: Testing this with the latest master build was hard because the output was very different(ex: some pearl RPMs ware not built). I thought that in order to compare performance differences, the commits have to be as close as possible. Created attachment 1123 [details]
without patches
Created attachment 1124 [details]
with patches
PS: routerstationpro and qemumips both use mips32 architecture I did look into this. I think there were some issues in master where things were getting rebuilt and shouldn't have been. I ran your test with latest master and only opkg-config-base got rebuilt so I'm happy this is now working as it should, even if at the commit you tested it perhaps wasn't. Its a good test case and we should have something like this as part of the QA plan. Stoicescu please try to run openembedded-core/scripts/sstate-diff-machines.sh --targets=less --machines="qemumips routerstationpro" --tmpdir=tmp-eglibc/ And compare generated list.M files (see script header for details) Verified in master: 9fa7608ddffa85db01571fb41ba656b7e1e937eb Test case added: https://bugzilla.yoctoproject.org/tr_show_case.cgi?case_id=738 |
We currently have do_package[stamp-extra-info] = "${MACHINE}" in package.bbclass. Ideally this should become ${PACKAGE_ARCH} but at the moment it can't due to the shlibs structure placed into the individual machine specific sysroot directories. A good optimisation would be to allow multiple shlibs directories to be searched, remove these from the sysroots and then make packaging a per PACKAGE_ARCH issue rather than MACHINE specific.