14248 2021-02-23 12:33:33 +0000 Can't enable qemu-usermode for intel-skylake-64 target 2023-09-07 14:56:35 +0000 1 1 7 Build System, Metadata & Runtime BSPs bsps-meta-intel unspecified x86 Multiple RESOLVED OBSOLETE Medium enhancement 0.0.0 1 c.finck naveen.kumar.saini aehs29 alejandro c.finck naveen.kumar.saini randy.macleod ross.burton --- --- No (bug/feature does not impact docs) oldest_to_newest 89349 0 c.finck 2021-02-23 12:33:33 +0000 Using the meta-intel layer and the "intel-skylake-64" machine, tune-skylake.inc is included. This file comes with a line MACHINE_FEATURES_remove = "qemu-usermode" while at the same time checking if MACHINE_FEATURES contains "qemu-usermode" (https://git.yoctoproject.org/cgit/cgit.cgi/meta-intel/tree/conf/machine/include/tune-skylake.inc#n28). How can this condition ever be fulfilled? According to Paul Eggleton, another Intel employee, MACHINE_FEATURES_remove "removes the value no matter how it comes in" and "_remove should be used sparingly and really ought to be used only where it's very unlikely that anyone would want to override it" (https://www.openembedded.org/pipermail/bitbake-devel/2015-September/006419.html). I currently cannot build any image using "intel-skylake-64", as I'm using a read-only filesystem and base packages like systemd already pull in glib and udev-hwdb, which require qemu-usermode to build an image with read-only filesystem. If there is no specific reason for having this line, please remove it and let the user decide if qemu-usermode shall be included or not. 89382 1 alejandro 2021-02-25 01:57:26 +0000 Hey Colin, Took me a bit to figure out how that was working. (In reply to comment #0) > Using the meta-intel layer and the "intel-skylake-64" machine, > tune-skylake.inc is included. > This file comes with a line > > MACHINE_FEATURES_remove = "qemu-usermode" > > while at the same time checking if MACHINE_FEATURES contains "qemu-usermode" > (https://git.yoctoproject.org/cgit/cgit.cgi/meta-intel/tree/conf/machine/ > include/tune-skylake.inc#n28). I believe QEMU still hasn't gotten AVX2 support (fact check me on this, haven;t checked in a while), which means that if qemu-usermode isn't removed it will start breaking things for others > > How can this condition ever be fulfilled? > According to Paul Eggleton, another Intel employee, MACHINE_FEATURES_remove > "removes the value no matter how it comes in" and "_remove should be used > sparingly and really ought to be used only where it's very unlikely that > anyone would want to override it" > (https://www.openembedded.org/pipermail/bitbake-devel/2015-September/006419. > html). > You're right that condition cannot easily be fulfilled, in fact Paul was right too and normally it shouldn't work, but the way I got it to work here: https://github.com/aehs29/meta-chromebook/blob/507555aa06e89a8cd52fcce56bf255fde44dfe30/conf/machine/eve-chromebook.conf#L78 and here: https://github.com/aehs29/meta-chromebook/blob/master/classes/qemuuser-enable-chromebook.bbclass was to inherit a class from the conf file which would then run anon python to re-add qemu-usermode with the supported optimizations only. > I currently cannot build any image using "intel-skylake-64", as I'm using a > read-only filesystem and base packages like systemd already pull in glib and > udev-hwdb, which require qemu-usermode to build an image with read-only > filesystem. > > If there is no specific reason for having this line, please remove it and > let the user decide if qemu-usermode shall be included or not. I hope that helps. Alejandro 89383 2 naveen.kumar.saini 2021-02-25 03:10:31 +0000 Hi Colin, You can try to use intel-corei7-64 machine, if -mtune=skylake is not requirement! 89385 3 c.finck 2021-02-25 09:16:07 +0000 (In reply to comment #1) > I believe QEMU still hasn't gotten AVX2 support (fact check me on this, > haven;t checked in a while), which means that if qemu-usermode isn't removed > it will start breaking things for others Indeed, there is still no support for any AVX in QEMU. However, the way I understand it, qemu-usermode is only used during build when `EXTRA_IMAGE_FEATURES += "read-only-rootfs"` is set. The build process then calls qemu-usermode to perform tasks that would otherwise be handled at first boot (gobject-introspection or generating the udev-hwdb). Please correct me if this has changed in the meantime and qemu-usermode is now used in additional cases. Apart from that, things won't "break" if qemu-usermode is enabled. The intel-skylake-64 target would merely be compiled with fewer optimizations. If my theory still holds that qemu-usermode is only added for "read-only-rootfs", and "read-only-rootfs" is not a default, things should be unchanged for most users. > You're right that condition cannot easily be fulfilled, in fact Paul was > right too and normally it shouldn't work, but the way I got it to work here: > > https://github.com/aehs29/meta-chromebook/blob/ > 507555aa06e89a8cd52fcce56bf255fde44dfe30/conf/machine/eve-chromebook.conf#L78 > > and here: > https://github.com/aehs29/meta-chromebook/blob/master/classes/qemuuser- > enable-chromebook.bbclass Oh dear... Thank you very much for that workaround! But is that in any way supported, without a chance of suddenly breaking in the next release? :) (In reply to comment #2) > You can try to use intel-corei7-64 machine, if -mtune=skylake is > not requirement! I have been using intel-corei7-64 previously, but wanted to upgrade to intel-skylake-64 now, as I'm not using any machines older than Skylake. There is more than just AVX/AVX2 to gain from an upgrade from Nehalem to Skylake (e.g. AES). 89392 4 ross.burton 2021-02-25 15:39:43 +0000 qemu-usermode is not only used in read-only rootfs, it's *essential* for read-only-rootfs and an optimisation for read-write rootfs. 89396 5 randy.macleod 2021-02-25 15:52:30 +0000 Not implemented in qemu but there is a summer of code project that Ross is aware of so who knows how long this might take. 89397 6 ross.burton 2021-02-25 15:54:07 +0000 Found the bug# in qemu: https://bugs.launchpad.net/qemu/+bug/1818075 89422 7 c.finck 2021-02-26 16:31:13 +0000 (In reply to comment #4) > qemu-usermode is not only used in read-only rootfs, it's *essential* for > read-only-rootfs and an optimisation for read-write rootfs. In that case, isn't there even more reason to remove the MACHINE_FEATURES_remove = "qemu-usermode" line? This now sounds like it breaks >50% of the cases where people could use the "intel-skylake-64" machine as a Yocto target. Looking at Alejandro's workaround in https://github.com/aehs29/meta-chromebook/blob/507555aa06e89a8cd52fcce56bf255fde44dfe30/conf/machine/eve-chromebook.conf#L78, he actually had to configure a new machine and couldn't apply this on the existing "intel-skylake-64" target. Right now, I must consider "intel-skylake-64" broken by default, with no way to add back "qemu-usermode". Those people who still want maximum optimizations after removing the line above can still add MACHINE_FEATURES_append = "qemu-usermode" to some configuration file. 89429 8 alejandro 2021-02-27 00:44:57 +0000 (In reply to comment #7) > (In reply to comment #4) > > qemu-usermode is not only used in read-only rootfs, it's *essential* for > > read-only-rootfs and an optimisation for read-write rootfs. > > In that case, isn't there even more reason to remove the > > MACHINE_FEATURES_remove = "qemu-usermode" > > line? > > This now sounds like it breaks >50% of the cases where people could use the > "intel-skylake-64" machine as a Yocto target. > Looking at Alejandro's workaround in > https://github.com/aehs29/meta-chromebook/blob/ > 507555aa06e89a8cd52fcce56bf255fde44dfe30/conf/machine/eve-chromebook. > conf#L78, he actually had to configure a new machine and couldn't apply this > on the existing "intel-skylake-64" target. > > Right now, I must consider "intel-skylake-64" broken by default, with no way > to add back "qemu-usermode". > Those people who still want maximum optimizations after removing the line > above can still add > > MACHINE_FEATURES_append = "qemu-usermode" > > to some configuration file. Its not my decision to make, but I disagree, I believe intel-skylake-64 should take advantage of as many optimizations/instructions as possible by default, I'd love for us to support it fully for all components but sadly we depend on an external bug, so IMO the skylake tune should still remove qemu-usermode. I was going to create another machine regardless, IMO the provided machines should be used as reference, if you tweak the metadata a bit I'm sure you'll be able to inherit that class and keep the skylake machine conf vanilla if that's what you want. That last statement is a bit confusing, you seem to want to keep qemu-usermode by default but you're stating others can append it again?. - Removing qemu-usermode by default allows skylake to be tuned properly - Not removing qemu-usermode, will support qemu-usermode, but will build a half optimized skylake 89443 9 ross.burton 2021-02-27 21:25:38 +0000 > This now sounds like it breaks >50% of the cases where people could use the > "intel-skylake-64" machine as a Yocto target. What you seem to be missing is that adding qemu-usermode to intel-skylake-64 breaks 100% of use-cases as qemu can't execute Skylake code. 89444 10 c.finck 2021-02-28 07:18:39 +0000 (In reply to comment #8) > That last statement is a bit confusing Yep, I should have suggested for local.conf: MACHINE_FEATURES_remove = "qemu-usermode" (In reply to comment #8) > - Removing qemu-usermode by default allows skylake to be tuned properly > - Not removing qemu-usermode, will support qemu-usermode, but will build a > half optimized skylake - Keeping qemu-usermode by default allows a build for "intel-skylake-64" to succeed at all (considering that Ross said qemu-usermode is used most of the time). - If you still want maximum Skylake tuning (with AVX and AVX2) *and* don't need qemu-usermode, you can add MACHINE_FEATURES_remove = "qemu-usermode" to your local.conf. Is an "intel-skylake-64" target that can hardly be used in a Yocto build really what you want? (In reply to comment #9) > What you seem to be missing is that adding qemu-usermode to intel-skylake-64 > breaks 100% of use-cases as qemu can't execute Skylake code. Look at the actual code: https://git.yoctoproject.org/cgit/cgit.cgi/meta-intel/tree/conf/machine/include/tune-skylake.inc The "intel-skylake-64" target already checks if qemu-usermode is part of MACHINE_FEATURES and disables AVX and AVX2 in that case, just enough for qemu-usermode to work. However, the current "intel-skylake-64" target also hardcodes MACHINE_FEATURES_remove = "qemu-usermode", hence there is no way to add back qemu-usermode without creating a new target. What's the purpose of the @bb.utils.contains('MACHINE_FEATURES', 'qemu-usermode', ...) check then? 89445 11 alejandro 2021-02-28 23:43:01 +0000 (In reply to comment #10) > (In reply to comment #8) > > That last statement is a bit confusing > > Yep, I should have suggested for local.conf: > > MACHINE_FEATURES_remove = "qemu-usermode" > > > (In reply to comment #8) > > - Removing qemu-usermode by default allows skylake to be tuned properly > > - Not removing qemu-usermode, will support qemu-usermode, but will build a > > half optimized skylake > > - Keeping qemu-usermode by default allows a build for "intel-skylake-64" to > succeed at all (considering that Ross said qemu-usermode is used most of the > time). > - If you still want maximum Skylake tuning (with AVX and AVX2) *and* don't > need qemu-usermode, you can add > > MACHINE_FEATURES_remove = "qemu-usermode" > > to your local.conf. > > Is an "intel-skylake-64" target that can hardly be used in a Yocto build > really what you want? > > > (In reply to comment #9) > > What you seem to be missing is that adding qemu-usermode to intel-skylake-64 > > breaks 100% of use-cases as qemu can't execute Skylake code. > > Look at the actual code: > https://git.yoctoproject.org/cgit/cgit.cgi/meta-intel/tree/conf/machine/ > include/tune-skylake.inc > The "intel-skylake-64" target already checks if qemu-usermode is part of > MACHINE_FEATURES and disables AVX and AVX2 in that case, just enough for > qemu-usermode to work. > > However, the current "intel-skylake-64" target also hardcodes > MACHINE_FEATURES_remove = "qemu-usermode", hence there is no way to add back > qemu-usermode without creating a new target. > > What's the purpose of the @bb.utils.contains('MACHINE_FEATURES', > 'qemu-usermode', ...) check then? I'm aware what the code does, I put that code myself, the purpose of the check is that people like you and I who are able to keep qemu-usermode in some way and want to use the skylake tune along with qemu-usermode at least get a build working even though were sacrificing optimization, a non-optimized working build is better than a broken build IMO. I'm sorry this is confusing for you, I propose we add some comments to the code to explain that (I think I explained it on the commit message, but it doesn't hurt to add more information). Worst case scenario we should keep the MACHINE_FEATURES_remove = "qemu-usermode" line to keep an optimized system by default and remove the bb.utils check so that people aren't confused by it like you are, but I don't believe there's much benefit in that. 96359 12 randy.macleod 2023-09-07 14:56:35 +0000 qemu now has avx2 support.