Bug 4389

I need more info on this. I don't understand which scripts are you referring to here... I can find the scripts directory in /usr/src/kernel/scripts in the target rootfs. I used adt-installer with the default config file.

The scripts directory is not complete out of the box. If you try building a module
against it, you should get failures. If you don't, then that's even better and the bug
is easy to close.

If you extract the sample-module that I attached and try something like the following:

# make sure the kernel source is in the SDK (via something like the below):

TOOLCHAIN_TARGET_TASK_append = " python-dev kernel-dev"

# build and export the ADT/SDK
# install the SDK

# source the SDK

  % cd /path/to/sdk/
  % source env.sh

  # change to the kernel source:
  % cd /path/to/sdk/<BSP>/sysroots/<BSP>/usr/src/kernel/
  
  # build the sample module
  % make CROSS_COMPILE=<prefix>-linux-gnu- ARCH=x86
                   M=/path/to/sample-module modules

Does it work ? I need an intermediate step of:
  % cd /path/to/sdk/<BSP>/sysroots/<BSP>/usr/src/kernel/
  % make ARCH=x86 scripts

for the build to work for me. Maybe I'm just doing something wrong, or am missing
a setup step!

Created attachment 1208 [details]
sample/simple module

Setting this back to New, since the information was updated.

I duplicated the problem. Looking for a solution...

Yang Shi at Windriver (I added him to the cc of the bug) did a bit of
legwork at the options for this as well. I don't have the latest, but
he should be around next week to update the case with some options.

I believe the correct way to do it is to have a nativesdk kernel-scripts package. We could do a 'make scripts' at SDK install time, which would be easy, but that would imply the user has the native toolchain installed because we only provide a cross-toolchain with the SDK.

It could work. I talked with Bruce about two other approaches.

1. It's similar to your approach, but just reuse kernel-dev package

2. Create a task or function to install the scripts binaries, then add the task or function into populate_sdk.

Good Morning,

I've been asked to find out if this being worked on and to pick it up if not.  Please let me know, thanks.

M

PS: had to set "Documentation change" in order to add myself to CC.

*** Bug 5402 has been marked as a duplicate of this bug. ***

I never really wrapped my head our this one. I talked to Bruce since he did some work on it.  He replied: "our workaround is specific to Wind River and it would need more effort to be generalized."

I meant to mark this as M+ yesterday, so the High is invalid

Funny this just updated. I wanted to add a comment to this.

I spent several hours today trying to implement this, and it currently isn't possible to do this due to technical issues.

I'm working on this in my new kernel-devsrc implementation, since it is a much cleaner representation of the kernel build, and have found the following:

 - we cannot copy the scripts from the kernel build, since they will fail QA due
   to rpath issues. We also cannot easily change the kernel build to avoid the
   rpath issues, since the tools/flags/etc are filtered by the kernel.

 - the tools as built for the actual kernel compile, are using the host compiler
   so have the wrong interpreter, or are even the wrong architecture completely for
   use on the target.

So barring the ability to copy the tools from the kernel artifacts, I was attempting to implement a build of the script as part of the kernel-devsrc packaging. That failed due to the following:

 - If we pass the target LDFLAGS, CFLAGS, etc, we can build the scripts such that
   they pass QA (rpaths are good), but they won't actually run on the target, since
   you get the host's ld.so, and that is not the right intepreter on the target, so
   you can't actually run the scripts on the target.

 - If we override HOST_CC to actually be CC (aka the target compiler) and also pass
   the target CFLAGS, LDFLAGS, etc, we can't build the tools, since 'fixdep' is 
   always built *and* run as part of the scripts building. When you build it with
   the target compiler + flags, it can't run on the host, so your scripts build
   fails.

So outside of using qemu to somehow build the scripts in the kernel-devsrc build, I'm not seeing a way to make this work. I looked into using qemu_run, but couldn't figure out an effective way to do it.

If someone knows how to do this, or has another idea. Let me know.

I'm actually testing this now. The way to do this was to build the non-native binaries with oemake HOSTCC=${CC} etc, . The cross-compiled build is rather easy, but in order to run these non-native binaries I had to create feww wrappers and run the binaries via qemu-xxx (user mode qemu). Seems to work fine, I will definitely test it thoroughly.

(In reply to comment #14)
> I'm actually testing this now. The way to do this was to build the
> non-native binaries with oemake HOSTCC=${CC} etc, . The cross-compiled build
> is rather easy, but in order to run these non-native binaries I had to
> create feww wrappers and run the binaries via qemu-xxx (user mode qemu).
> Seems to work fine, I will definitely test it thoroughly.

Right. I was to the point of doing the qemu wrapper myself, since the HOSTCC=CC
is what I currently have implemented .. but got bogged down in the details.

This should really be done as part of the new implementation I've done for kernel-devsrc, how did you want to combine the two changes ?

Once you have something working, can you send it to me, or attach it here, so I can merge it into the new build steps and test ?

And obviously I had to patch some Makefiles to call the wrappers instead of the actual (original) non-native binaries. 
One important observation: if you install qemu native on your host via:

# apt install qemu-user

Then it simply adding
oe_runmake ARCH=${ARCH} HOSTCC="${CC} -static" HOSTLD="${LD}" HOSTAR="${AR}" scripts

into do_install is pretty much all you need to do, as the cross-compiled binaries are "automagically" handled by the host.

I ran into this when I noticed that otherwise identical setups on two different machines would not build identically: one would not build at all (the one missing the qemus), the other one had no problems. But whatever the host did with installed qemus I managed to reproduce with our oe qemu-native.

And obviously I had to patch some Makefiles to call the wrappers instead of the actual (original) non-native binaries. 
One important observation: if you install qemu native on your host via:

# apt install qemu-user

Then it simply adding
oe_runmake ARCH=${ARCH} HOSTCC="${CC} -static" HOSTLD="${LD}" HOSTAR="${AR}" scripts

into do_install is pretty much all you need to do, as the cross-compiled binaries are "automagically" handled by the host.

I ran into this when I noticed that otherwise identical setups on two different machines would not build identically: one would not build at all (the one missing the qemus), the other one had no problems. But whatever the host did with installed qemus I managed to reproduce with our oe qemu-native. 
(In reply to comment #15)
> (In reply to comment #14)
> > I'm actually testing this now. The way to do this was to build the
> > non-native binaries with oemake HOSTCC=${CC} etc, . The cross-compiled build
> > is rather easy, but in order to run these non-native binaries I had to
> > create feww wrappers and run the binaries via qemu-xxx (user mode qemu).
> > Seems to work fine, I will definitely test it thoroughly.
> 
> Right. I was to the point of doing the qemu wrapper myself, since the
> HOSTCC=CC
> is what I currently have implemented .. but got bogged down in the details.
> 
> This should really be done as part of the new implementation I've done for
> kernel-devsrc, how did you want to combine the two changes ?
> 
> Once you have something working, can you send it to me, or attach it here,
> so I can merge it into the new build steps and test ?

I'll wait for your changes to be merged first.

Ah, but that's the problem. Unless you get those patches upstream, the approach
is dead in the water.

Even though I'd like to, this isn't something that can only work with a patched
kernel. So at a minimum, there needs to be a test and a graceful fallback when
trying to do these cross-compiled, target-native scripts.

It gets complex, and just making people run "make scripts" starts to look like the right answer again.

I'll send you my patch tomorrow, needs a cleanup, more of a POC at this point.(In reply to comment #18)
> Ah, but that's the problem. Unless you get those patches upstream, the
> approach
> is dead in the water.
> 
> Even though I'd like to, this isn't something that can only work with a
> patched
> kernel. So at a minimum, there needs to be a test and a graceful fallback
> when
> trying to do these cross-compiled, target-native scripts.
> 
> It gets complex, and just making people run "make scripts" starts to look
> like the right answer again.

No, there is no problem. I undo the patching (restore the unmodified Makefiles) after the the "make scripts". The wrappers are not distributed either.

(In reply to comment #19)
> I'll send you my patch tomorrow, needs a cleanup, more of a POC at this
> point.(In reply to comment #18)
> > Ah, but that's the problem. Unless you get those patches upstream, the
> > approach
> > is dead in the water.
> > 
> > Even though I'd like to, this isn't something that can only work with a
> > patched
> > kernel. So at a minimum, there needs to be a test and a graceful fallback
> > when
> > trying to do these cross-compiled, target-native scripts.
> > 
> > It gets complex, and just making people run "make scripts" starts to look
> > like the right answer again.
> 
> No, there is no problem. I undo the patching (restore the unmodified
> Makefiles) after the the "make scripts". The wrappers are not distributed
> either.

That's definitely a problem, if you are somehow apply a patch during the
build to the kernel.

That's not something I ever want done to any of the kernel build workflows
that I have, and there's no guaranteed that the patching will work across
kernel versions.

How exactly are you doing this patching ?

(In reply to comment #20)
> (In reply to comment #19)
> > I'll send you my patch tomorrow, needs a cleanup, more of a POC at this
> > point.(In reply to comment #18)
> > > Ah, but that's the problem. Unless you get those patches upstream, the
> > > approach
> > > is dead in the water.
> > > 
> > > Even though I'd like to, this isn't something that can only work with a
> > > patched
> > > kernel. So at a minimum, there needs to be a test and a graceful fallback
> > > when
> > > trying to do these cross-compiled, target-native scripts.
> > > 
> > > It gets complex, and just making people run "make scripts" starts to look
> > > like the right answer again.
> > 
> > No, there is no problem. I undo the patching (restore the unmodified
> > Makefiles) after the the "make scripts". The wrappers are not distributed
> > either.
> 
> That's definitely a problem, if you are somehow apply a patch during the
> build to the kernel.
> 
> That's not something I ever want done to any of the kernel build workflows
> that I have, and there's no guaranteed that the patching will work across
> kernel versions.
> 
> How exactly are you doing this patching ?

I attached the two patches I am currently testing. The idea is to populate the kernel-devsrc with additional files that are supposed to be the same as if the user ran "make script" in native build environment. (so the user does not have to do that). Some of the files are binaries (cross-compiled) and some of the files are the artifacts created by these binaries.  Beside these additional files, the rest of the package (sources etc) remain exactly as before. No existing files should be modified.

Created attachment 4213 [details]
patches being tested.

(In reply to comment #22)
> Created attachment 4213 [details]
> patches being tested.

The approach is on-track, and is what I was considering as well. So we are aligned with that. I'm just glad I didn't go any further with my work before realizing we were looking at the same thing.

I'm going to submit my kernel-devsrc changes early next week (Monday is a holiday here), and after that, you can easily slide these changes on top of my patch into the new package build process.

Did you want a copy of the WIP patch for the new dev-src for testing ?

(In reply to comment #23)
> (In reply to comment #22)
> > Created attachment 4213 [details]
> > patches being tested.
> 
> The approach is on-track, and is what I was considering as well. So we are
> aligned with that. I'm just glad I didn't go any further with my work before
> realizing we were looking at the same thing.
> 
> I'm going to submit my kernel-devsrc changes early next week (Monday is a
> holiday here), and after that, you can easily slide these changes on top of
> my patch into the new package build process.
> 
> Did you want a copy of the WIP patch for the new dev-src for testing ?

Good to know we are on the same page here. And yes, getting an advanced copy of your patch is a good idea, as it will enable me to test both patches together on a variety of MACHINEs. (Monday is a holiday here as well).

Created attachment 4217 [details]
WIP kernel-devsrc patch

Sorry for the slow update, I was derailed with another issue yesterday.

Attached is my WIP patch for the new kernel-devsrc.

I'm still working on it today, but would be interested to know that your patches work on top of the new structure.

(In reply to comment #26)
> Sorry for the slow update, I was derailed with another issue yesterday.
> 
> Attached is my WIP patch for the new kernel-devsrc.
> 
> I'm still working on it today, but would be interested to know that your
> patches work on top of the new structure.

Thanks. I know you have been very busy lately and I am actually surprised you found some extra time to work on this one. I'll keep you posted.

Created attachment 4218 [details]
WIP second iteration, combined patches, powerpc does not build.

Just to keep you posted, I attached a new patch, (combining your work with mine).
I tested it with

$ core-image-sato-sdk -ctestimage

With the following MACHINEs:

#builds + tests OK (combined patch) MACHINE = "qemuarm64"
#builds + tests OK (combined patch) MACHINE = "qemuarm"
#builds + tests OK (combined patch) MACHINE = "qemux86-64"
#builds + tests OK (combined patch) MACHINE = "qemux86"
#builds + tests OK (combined patch) MACHINE = "qemumips64"

Some problems I have to resolve yet:

MACHINE = "qemumips"
-------------------
do_rootfs:
# error building image MACHINE = "qemumips"
#Error: 
# Problem 1: conflicting requests
#  - nothing provides gdbserver needed by packagegroup-core-eclipse-debug-1.0-r0.noarch
# Problem 2: conflicting requests
#  - nothing provides gdbserver needed by packagegroup-core-tools-debug-1.0-r3.noarch

Package builds fine, though. So chances are good this will work OK. 

MACHINE="qemuppc"
---------------
Cannot build the package at all,
qemu-ppc throws an illegal instruction:

powerpc-poky-linux-gcc  -m32 -mhard-float -mcpu=7400 -mno-spe --sysroot=/data/master/poky/build-sato-testimage/tmp/work/qemuppc-poky-linux/kernel-devsrc/1.0-r0/recipe-sysroot -Wl,-O1 -Wl,--hash-style=gnu -Wl,--as-needed -Wp,-MD,scripts/basic/.fixdep.d -Iscripts/basic -Wall -Wmissing-prototypes -Wstrict-prototypes -O2 -fomit-frame-pointer -std=gnu89 -o scripts/basic/fixdep scripts/basic/fixdep.c
| qemu: uncaught target signal 4 (Illegal instruction) - core dumped
| Illegal instruction (core dumped)

(In reply to comment #29)
> Just to keep you posted, I attached a new patch, (combining your work with
> mine).
> I tested it with
> 
> $ core-image-sato-sdk -ctestimage
> 
> With the following MACHINEs:
> 
> #builds + tests OK (combined patch) MACHINE = "qemuarm64"
> #builds + tests OK (combined patch) MACHINE = "qemuarm"
> #builds + tests OK (combined patch) MACHINE = "qemux86-64"
> #builds + tests OK (combined patch) MACHINE = "qemux86"
> #builds + tests OK (combined patch) MACHINE = "qemumips64"
> 
> Some problems I have to resolve yet:
> 
> MACHINE = "qemumips"
> -------------------
> do_rootfs:
> # error building image MACHINE = "qemumips"
> #Error: 
> # Problem 1: conflicting requests
> #  - nothing provides gdbserver needed by
> packagegroup-core-eclipse-debug-1.0-r0.noarch
> # Problem 2: conflicting requests
> #  - nothing provides gdbserver needed by
> packagegroup-core-tools-debug-1.0-r3.noarch
> 
> Package builds fine, though. So chances are good this will work OK. 

That's definitely good news.

I'll finish my testing and submit the patch to the mailing list on Friday.

On my side, this isn't the first time I've heard of qemuppc getting sigill and crashing.

I personally wouldn't want to hold up the work you are doing on ppc, so is it possible to do an arch test, and require the on-target "make scripts" for PPC ?

Bruce

> 
> MACHINE="qemuppc"
> ---------------
> Cannot build the package at all,
> qemu-ppc throws an illegal instruction:
> 
> powerpc-poky-linux-gcc  -m32 -mhard-float -mcpu=7400 -mno-spe
> --sysroot=/data/master/poky/build-sato-testimage/tmp/work/qemuppc-poky-linux/
> kernel-devsrc/1.0-r0/recipe-sysroot -Wl,-O1 -Wl,--hash-style=gnu
> -Wl,--as-needed -Wp,-MD,scripts/basic/.fixdep.d -Iscripts/basic -Wall
> -Wmissing-prototypes -Wstrict-prototypes -O2 -fomit-frame-pointer -std=gnu89
> -o scripts/basic/fixdep scripts/basic/fixdep.c
> | qemu: uncaught target signal 4 (Illegal instruction) - core dumped
> | Illegal instruction (core dumped)

(In reply to comment #30)
> (In reply to comment #29)
> > Just to keep you posted, I attached a new patch, (combining your work with
> > mine).
> > I tested it with
> > 
> > $ core-image-sato-sdk -ctestimage
> > 

> On my side, this isn't the first time I've heard of qemuppc getting sigill
> and crashing.
> 
> I personally wouldn't want to hold up the work you are doing on ppc, so is
> it possible to do an arch test, and require the on-target "make scripts" for
> PPC ?
> 

I actually managed to build the package for powerpc as well meanwhile.
I had to create the images with static linking though (passing -static to CC). So the binaries a bit bigger as a consequence. I'll try to investigate a bit deeper later on.

*** Bug 6630 has been marked as a duplicate of this bug. ***

This is still valid, and should stay around to document that we are aware of how this works (and for someone else to pick up the work in progress parts .. if that ever happens). It just isn't high priority.

Bulk move from 4.99 or 0.00 to 5.99

There was some recent discussion on IRC related to this topic and some discussion during bug triage meeting.