Bug 7441

For what it is worth, it is bad that *any* process leave the kernel directory dirty
and not commit its changes. It means that future patching will be impossible 
and bisecting / debugging issues also not possible.

I've had a change for making the  meta directory detection more robust for a
while, as part of a change unifying the detection across scripts. As you guessed,
looking within the directory for a sentinel file is the easiest thing to do.

I like the failure from the point of view that it is exposing some poor practices 
that are happening to the kernel directory .. and saving future pain, but we can
still make things more robust. There's never hard in that.

If you look at:

  git://git.yoctoproject.org/yocto-kernel-tools master-test

You'll see the in progress patches. Feel free to take them for a test drive.

Bruce,

Thanks for taking a look at this so quickly. I'll consider having our patch process commit the changes. If nothing else, it will avoid shipping builds with "-dirty" kernels.

I looked over 0f1c9d2 quickly. Forgive me if I'm wrong, my bash scripting gets rusty from lack of use.

It looks like you save the name of the metadir to the '.metadir' file. Subsequent tests use -d to check if .metadir is a directory before catting the file into a variable. If I read the code correctly, that path will never run.

Cheers,
Jeff

(In reply to comment #2)
> Bruce,
> 
> Thanks for taking a look at this so quickly. I'll consider having our patch
> process commit the changes. If nothing else, it will avoid shipping builds
> with "-dirty" kernels.
> 
> I looked over 0f1c9d2 quickly. Forgive me if I'm wrong, my bash scripting
> gets rusty from lack of use.
> 
> It looks like you save the name of the metadir to the '.metadir' file.
> Subsequent tests use -d to check if .metadir is a directory before catting
> the file into a variable. If I read the code correctly, that path will never

Yep. That's why this is under test :) That should be -f, not -d for the test.
And I made the change here.

Bruce

> run.
> 
> Cheers,
> Jeff

(In reply to comment #3)
> (In reply to comment #2)
> > Bruce,
> > 
> > Thanks for taking a look at this so quickly. I'll consider having our patch
> > process commit the changes. If nothing else, it will avoid shipping builds
> > with "-dirty" kernels.
> > 
> > I looked over 0f1c9d2 quickly. Forgive me if I'm wrong, my bash scripting
> > gets rusty from lack of use.
> > 
> > It looks like you save the name of the metadir to the '.metadir' file.
> > Subsequent tests use -d to check if .metadir is a directory before catting
> > the file into a variable. If I read the code correctly, that path will never
> 
> Yep. That's why this is under test :) That should be -f, not -d for the test.
> And I made the change here.
> 
> Bruce
> 
> > run.
> > 
> > Cheers,
> > Jeff

OK now the meta-xilinx layer is making this messy. They keep the Kernel metadata separately and add it to the build flow with SRC_URI and type=kmeta. As far as I can tell, those directories don't get copied into the kernel tree until 'updateme' gets run. When 'createme' runs, it can't detect the meta directory (because nothing exists yet) and meta_dir is empty going into metaize(). The script then tries to create /cfg and fails with permission errors.

Jeff

(In reply to comment #4)
> (In reply to comment #3)
> > (In reply to comment #2)
> > > Bruce,
> > > 
> > > Thanks for taking a look at this so quickly. I'll consider having our patch
> > > process commit the changes. If nothing else, it will avoid shipping builds
> > > with "-dirty" kernels.
> > > 
> > > I looked over 0f1c9d2 quickly. Forgive me if I'm wrong, my bash scripting
> > > gets rusty from lack of use.
> > > 
> > > It looks like you save the name of the metadir to the '.metadir' file.
> > > Subsequent tests use -d to check if .metadir is a directory before catting
> > > the file into a variable. If I read the code correctly, that path will never
> > 
> > Yep. That's why this is under test :) That should be -f, not -d for the test.
> > And I made the change here.
> > 
> > Bruce
> > 
> > > run.
> > > 
> > > Cheers,
> > > Jeff
> 
> OK now the meta-xilinx layer is making this messy. They keep the Kernel
> metadata separately and add it to the build flow with SRC_URI and
> type=kmeta. As far as I can tell, those directories don't get copied into
> the kernel tree until 'updateme' gets run. When 'createme' runs, it can't
> detect the meta directory (because nothing exists yet) and meta_dir is empty
> going into metaize(). The script then tries to create /cfg and fails with
> permission errors.
> 
> Jeff

(In reply to comment #4)
> (In reply to comment #3)
> > (In reply to comment #2)
> > > Bruce,
> > > 
> > > Thanks for taking a look at this so quickly. I'll consider having our patch
> > > process commit the changes. If nothing else, it will avoid shipping builds
> > > with "-dirty" kernels.
> > > 
> > > I looked over 0f1c9d2 quickly. Forgive me if I'm wrong, my bash scripting
> > > gets rusty from lack of use.
> > > 
> > > It looks like you save the name of the metadir to the '.metadir' file.
> > > Subsequent tests use -d to check if .metadir is a directory before catting
> > > the file into a variable. If I read the code correctly, that path will never
> > 
> > Yep. That's why this is under test :) That should be -f, not -d for the test.
> > And I made the change here.
> > 
> > Bruce
> > 
> > > run.
> > > 
> > > Cheers,
> > > Jeff
> 
> OK now the meta-xilinx layer is making this messy. They keep the Kernel
> metadata separately and add it to the build flow with SRC_URI and
> type=kmeta. As far as I can tell, those directories don't get copied into
> the kernel tree until 'updateme' gets run. When 'createme' runs, it can't
> detect the meta directory (because nothing exists yet) and meta_dir is empty
> going into metaize(). The script then tries to create /cfg and fails with
> permission errors.

Ah yes. I've run into this before as well.  We won't be getting this change into 1.8, so I'll
work on it more tomorrow, I'd rather not include any more complexity (I just removed
a lot of it) so I need to come up with more than a one off solution.

For now, the best thing to do is to make sure that some custom task runs before the 
kernel configme and cleans up, so everything works as expected.

> 
> Jeff

I pushed a new variant, with an exception case in the createme script. It will fallback
to a default and then propagate it to all following tools.

I haven't build tested this against the xilinx layers, but it does survive standard and linux-yocto-custom
builds (which have no meta-data).

(In reply to comment #6)
> I pushed a new variant, with an exception case in the createme script. It
> will fallback
> to a default and then propagate it to all following tools.
> 
> I haven't build tested this against the xilinx layers, but it does survive
> standard and linux-yocto-custom
> builds (which have no meta-data).

Thanks I will test against my build flow on Monday. I have to roll back the workaround I made to work with the old scripts.

(In reply to comment #7)
> (In reply to comment #6)
> > I pushed a new variant, with an exception case in the createme script. It
> > will fallback
> > to a default and then propagate it to all following tools.
> > 
> > I haven't build tested this against the xilinx layers, but it does survive
> > standard and linux-yocto-custom
> > builds (which have no meta-data).
> 
> Thanks I will test against my build flow on Monday. I have to roll back the
> workaround I made to work with the old scripts.

Sorry this took so long, I got around to testing it today and didn't see any issues. FWIW, I'm still going to stick with your original suggestion to not build a dirty kernel.

Thanks!

Great. Since I won't push this into 1.8 at this late point, I'll move this to 1.9 and
send it for merging once the tree thaws.