1525 2011-09-26 12:50:21 +0000 [multilib] Triggering building a 64 bit kernel with a 32 bit userspace 2026-06-25 15:24:41 +0000 1 1 7 Build System, Metadata & Runtime Meta-yocto meta-yocto unspecified All Multiple RESOLVED WONTFIX Medium enhancement Future 1 msm-oss bruce.ashfield akuster bruce.ashfield dvhart jessica.zhang msm-oss poky.bs.watcher poky.watcher randy.macleod richard.purdie sgw shane.wang song.liu srifenbark zhe.he --- --- Yes (doc changes required) oldest_to_newest 16481 0 msm-oss 2011-09-26 12:50:21 +0000 For example, if we are building a multilib system that is 32bit and adding certain 64bit packages we most likely want to promote the kernel configuration to a 64bit kernel so it can run all the packages installed on the root file system 16482 1 richard.purdie 2011-09-26 12:54:20 +0000 I'd like to keep this specific to the kernel. Just to clarify what we're talking about here, the problem is when you build a 32 bit userspace but add 64 bit multilibs. Somehow we need to change the kernel we generate to 64 bit. I'm hoping with linux-yocto, this should be as straight foward as changing the KMACHINE from a 32 bit to a 64 bit one? If so, we just need to figure out how to trigger that change and document how to do it. Any comments Bruce? 16483 2 msm-oss 2011-09-26 13:00:42 +0000 Hopefully, this is done in such a way it works such that we can promote other recipes to 64bit as well. 16484 3 msm-oss 2011-09-26 13:01:56 +0000 (In reply to comment #2) > Hopefully, this is done in such a way it works such that we can promote other > recipes to 64bit as well. Or least other *simple* linux kernel recipes - right now don't use the linux-yocyo tree just yet and might not ever for certain releases we make. 16485 4 bruce.ashfield 2011-09-26 13:13:50 +0000 This is trivial for the linux-yocto kernel(s). But you still need to know the configuration that you'd like to add. i.e. just changing it to CONFIG_64BIT is not always enough, at times there are other changes that you may or may not need to make. If you want to bump a 32 bit kernel to 64 bit, you can either embed a 64 bit fragment in the meta branch and just add: KERNEL_FEATURES_append=" cfg/64bit.scc" And you'd be done. If you don't have it in the tree, just add a cfg fragment onto the SRC_URI that has the same contents as you'd have in the in-tree variant. This could be done via python, shell, variable, whatever you want to trigger the promotion. 16486 5 msm-oss 2011-09-26 13:20:54 +0000 (In reply to comment #4) > This is trivial for the linux-yocto kernel(s). But you still need to know the > configuration that you'd like to add. i.e. just changing it to CONFIG_64BIT is > not > always enough, at times there are other changes that you may or may not need to > make. > > If you want to bump a 32 bit kernel to 64 bit, you can either embed a 64 bit > fragment in the meta branch and just add: > > KERNEL_FEATURES_append=" cfg/64bit.scc" > > And you'd be done. If you don't have it in the tree, just add a cfg fragment > onto the SRC_URI that has the same contents as you'd have in the in-tree > variant. > > This could be done via python, shell, variable, whatever you want to > trigger the promotion. And if we just added multilib to overrides we could fixup each recipe based off that value. 25074 6 msm-oss 2012-09-12 01:15:05 +0000 Does just adding CONFIG_64BIT work for other arches? There are no toolchain issues? -M 25075 7 bruce.ashfield 2012-09-12 02:13:01 +0000 I've used the CONFIG_64 25076 8 bruce.ashfield 2012-09-12 02:14:52 +0000 scratch that last comment, it decided to post itself :) I've used the CONFIG_64BIT 'trick' on most arches and boards that support 64 bit kernels, but sometimes have base 32 bit BSPs. Same thing for endian swaps. Even if it is more involved than just CONFIG_64BIT, the arch can be picked up in the fragments and secondary options added. 25085 9 msm-oss 2012-09-12 03:59:43 +0000 (In reply to comment #8) > scratch that last comment, it decided to post itself :) > > I've used the CONFIG_64BIT 'trick' on most arches and boards that support > 64 bit kernels, but sometimes have base 32 bit BSPs. Same thing for endian > swaps. > > Even if it is more involved than just CONFIG_64BIT, the arch can be picked > up in the fragments and secondary options added. There are actually toolchain issues on powerpc with doing this... Is see there is a DISTRO_FEATURE = multiarch, and it's fixing some issues (with some recipe hacks) but now I'm hitting some linker issues... still investigating. -M 25087 10 msm-oss 2012-09-12 04:26:30 +0000 (In reply to comment #9) > (In reply to comment #8) > > scratch that last comment, it decided to post itself :) > > > > I've used the CONFIG_64BIT 'trick' on most arches and boards that support > > 64 bit kernels, but sometimes have base 32 bit BSPs. Same thing for endian > > swaps. > > > > Even if it is more involved than just CONFIG_64BIT, the arch can be picked > > up in the fragments and secondary options added. > > There are actually toolchain issues on powerpc with doing this... > > Is see there is a DISTRO_FEATURE = multiarch, and it's fixing some issues > (with some recipe hacks) but now I'm hitting some linker issues... still > investigating. > > -M Looks like I just needed this in my kernel recipe: +KERNEL_CC_append_e6500 = " -m64" +KERNEL_LD_append_e6500 = " -melf64ppc" Which is not that surprising considering the default for the 32-bit toolchain is 32 bit stuff... I'm still sort of curious how the CodeSourcery toolchains seem to get away without specifying this... -M 25088 11 msm-oss 2012-09-12 04:28:36 +0000 (In reply to comment #10) > (In reply to comment #9) > > (In reply to comment #8) > > > scratch that last comment, it decided to post itself :) > > > > > > I've used the CONFIG_64BIT 'trick' on most arches and boards that support > > > 64 bit kernels, but sometimes have base 32 bit BSPs. Same thing for endian > > > swaps. > > > > > > Even if it is more involved than just CONFIG_64BIT, the arch can be picked > > > up in the fragments and secondary options added. > > > > There are actually toolchain issues on powerpc with doing this... > > > > Is see there is a DISTRO_FEATURE = multiarch, and it's fixing some issues > > (with some recipe hacks) but now I'm hitting some linker issues... still > > investigating. > > > > -M > > Looks like I just needed this in my kernel recipe: > > +KERNEL_CC_append_e6500 = " -m64" > +KERNEL_LD_append_e6500 = " -melf64ppc" > > Which is not that surprising considering the default for the 32-bit > toolchain is 32 bit stuff... I'm still sort of curious how the CodeSourcery > toolchains seem to get away without specifying this... > > -M Ooooh spoke too soon, not we have kernel module issues... ERROR: QA Issue: Architecture did not match (20 to 21) on /work/t4240qds-poky-linux/linux-qoriq-sdk-3.0.34-r6/packages-split/kernel-module-sctp/lib/modules/3.0.34-rt55/kernel/net/sctp/sctp.ko Should be easy enough to work around. -M 25089 12 bruce.ashfield 2012-09-12 04:33:50 +0000 I've seen both of these in the past, but the solution has always depended on the kernel .. so yes, getting something generic, and kernel version independent is the trick. As for the -m64, I've added it to KERNEL_CC or we've patched the kernel Makefiles to be a bit smarter and specify it themselves. Typically it is only the arch/boot code that gets tied up with this. The QA warning, I've had much more recent experience. Since oe-core doesn't expect the 32 -> 64 bit mismatch, you can either mark all the modules with INSANE_SKIP (which isn't necessarily trivial) or you can modify the insane class to have a kernel exception in the checking (there's already one for kernel + x32) .. but that exception is a bit tricky since non kernel-packaged (external) modules will be missed and from what I've been told, the oe-core design really expects a separate 64 bit kernel to be built from the 32 bit userspace (I think I have that right). But one would think that the multilib support could trigger an exception to this test, I just haven't found a clean way to do it yet. 25091 13 msm-oss 2012-09-12 04:39:20 +0000 (In reply to comment #12) > I've seen both of these in the past, but the solution has always > depended on the kernel .. so yes, getting something generic, and > kernel version independent is the trick. > > As for the -m64, I've added it to KERNEL_CC or we've patched the > kernel Makefiles to be a bit smarter and specify it themselves. > Typically it is only the arch/boot code that gets tied up with this. Hmmmm, OK. > The QA warning, I've had much more recent experience. Since oe-core > doesn't expect the 32 -> 64 bit mismatch, you can either mark all the > modules with INSANE_SKIP (which isn't necessarily trivial) or you can > modify the insane class to have a kernel exception in the checking > (there's already one for kernel + x32) .. but that exception is a > bit tricky since non kernel-packaged (external) modules will be missed > and from what I've been told, the oe-core design really expects a > separate 64 bit kernel to be built from the 32 bit userspace (I think > I have that right). But one would think that the multilib support could > trigger an exception to this test, I just haven't found a clean way > to do it yet. This one we can probably work around with the gcc multiarch functions already present, and the qa test and add a proper exception for kernel-modules. -M 27170 14 msm-oss 2012-11-02 02:33:39 +0000 We have done some things for this in meta-fsl-ppc and they are probably hacky but seem to work OK. We added a BUILD_64BIT_KERNEL variable to promote the kernel to a 64-bit variant on a 32-bit machine. I'm just asking to look at and keep these in mind, I'm planning on joining any discussion that takes place to solve this problem permanently. -M 27210 15 bruce.ashfield 2012-11-02 14:26:43 +0000 I had a look at meta-fsl-ppc and will definitely keep that approach in mind, and it's similar to what I've done in the past: KERNEL_EXTRA_CC_OPTIONS ?= "${@bb.utils.contains("TRANSLATED_TARGET_ARCH", "powerpc64", "-m64", "", d)}" KERNEL_EXTRA_LD_OPTIONS ?= "${@bb.utils.contains("TRANSLATED_TARGET_ARCH", "powerpc64", "-melf64ppc", "", d)}" (assuming that KERNEL_EXTRA_CC_OPTIONS is available). The inhibiting of the QA check is also interesting. I also do it via variables, but had to do extra work to pickup modules. How are you inhibiting module checks ? Is it the all_qa (I'm not completely familiar with it from memory). But definitely, good input. We likely want to make the QA checks smarter, with some flags to allow certain mismatches, but this is a good starting point. 27211 16 bruce.ashfield 2012-11-02 14:28:55 +0000 Moving this to 1.4. I'm also thinking that this may get into some kernel specific areas, do we want someone from the kernel "team" to be the assignee for this in 1.4 ? Or is this already well underway ? 27212 17 msm-oss 2012-11-02 15:36:28 +0000 (In reply to comment #15) > I had a look at meta-fsl-ppc and will definitely keep that approach in mind, > and it's similar to > what I've done in the past: > > KERNEL_EXTRA_CC_OPTIONS ?= "${@bb.utils.contains("TRANSLATED_TARGET_ARCH", > "powerpc64", "-m64", "", d)}" > KERNEL_EXTRA_LD_OPTIONS ?= "${@bb.utils.contains("TRANSLATED_TARGET_ARCH", > "powerpc64", "-melf64ppc", "", d)}" > > (assuming that KERNEL_EXTRA_CC_OPTIONS is available). Probably is available, I used others probably because I saw that method first > The inhibiting of the QA check is also interesting. I also do it via > variables, but had to do > extra work to pickup modules. > > How are you inhibiting module checks ? Is it the all_qa (I'm not completely > familiar with it from > memory). Presumably by removing one of the two being removed. I stopped doing a lot of investigation after the error went away ;) > But definitely, good input. We likely want to make the QA checks smarter, > with some flags to allow > certain mismatches, but this is a good starting point. Agreed, x32 has some of the same issues with specific exceptions for it. 27455 18 radu.moisan 2012-11-14 22:40:38 +0000 (In reply to comment #16) > Moving this to 1.4. > > I'm also thinking that this may get into some kernel specific areas, do we > want someone > from the kernel "team" to be the assignee for this in 1.4 ? Or is this > already well underway ? As far as I know there is no work underway for this. Is there anybody from kernel team that can jump in? I'm trying a workload estimation for 1.4 for this task and I don't have a starting point. It seems to me more like a kernel issue, or am I wrong. 27475 19 bruce.ashfield 2012-11-15 16:47:03 +0000 It is more of a kernel issue. I'd estimate that this is 10 perfect days work, due to the time required to discuss and try the various solutions. I'm adding Darren as the assignee for now, just so he's aware of this as well. 27476 20 bruce.ashfield 2012-11-15 16:47:33 +0000 I meant "Adding Darren to the cc" Bruce 27483 21 msm-oss 2012-11-15 16:56:45 +0000 I can try to help with this, either some specific task or just testing this out... let me know. -M 29256 22 bruce.ashfield 2013-01-28 19:08:14 +0000 bumping this out. will kick up a design thread once some other issues for M4 have been dealt with (like a new kernel version). 30839 23 bruce.ashfield 2013-03-19 12:58:02 +0000 We have some ideas for this, but it's too late for 1.4, the status quo of insane skip will stay for now. The changes this late in the -dev cycle would carry enough risk to make bumping this a better idea than pushing it into the release. 36885 24 bruce.ashfield 2013-09-18 15:16:59 +0000 moving to 1.6, will do this early in the new development cycle. 51401 25 srifenbark 2015-06-01 21:33:10 +0000 Hi, This actually looks like something that will need to be documentezasd in the YP user manuals once a solution is figured out. I am noting that here mostly for myself so I don't lose sight of it. Scott 106018 26 randy.macleod 2026-06-25 15:24:41 +0000 This is not a common use-case so the user should take responsibilty for that proper kernel configuration.