Can you test the patch that I attached to the bug ?

It is essential your patch, and I'd like to add your Signed-off-by to the commit message before I send it.

After talking with Richard, I decided to not go for a few higher maintenance ideas and instead have changed your routine to only return directories with .scc and .cfg files. That allows us to re-run if a file is modified, added or removed from the directories, which addresses the concern that Richard had.

This still will trigger a few more re-executions of the meta-data + kernel build than I'd like, but I'll wait for some of the task equivalence changes to be complete before trying to check that although the meta-data task has re-run, the output is the same and hence we don't need to rebuild or repackage the kernel.

Created attachment 4540 [details]
proposed directory tracking patch

Hi Bruce,

I got a weird result:

If I modify the testing fragment file poky/meta-yocto-bsp/recipes-kernel/linux/files/test.cfg, it works.
And if I `touch test2.cfg`, it also triggers kernel_metadata task.

But in 2 conditions it doesn't work:
1. mv test.cfg test.cfg.bak
2. rm test2.cfg

I also checked modification time of the dir poky/meta-yocto-bsp/recipes-kernel/linux/files/ using `stat` util, it changed in both conditions. Don't know why yocto didn't trigger kernel_metadata in these 2 cases.

(In reply to comment #3)
> Hi Bruce,
> 
> I got a weird result:
> 
> If I modify the testing fragment file
> poky/meta-yocto-bsp/recipes-kernel/linux/files/test.cfg, it works.
> And if I `touch test2.cfg`, it also triggers kernel_metadata task.
> 
> But in 2 conditions it doesn't work:
> 1. mv test.cfg test.cfg.bak
> 2. rm test2.cfg
> 
> I also checked modification time of the dir
> poky/meta-yocto-bsp/recipes-kernel/linux/files/ using `stat` util, it
> changed in both conditions. Don't know why yocto didn't trigger
> kernel_metadata in these 2 cases.

Thanks for the update!

I'll try and reproduce those results here and will adjust things as necessary.

Here's the results of my tests with the patch:

# all operations are in a layers kernel patch/fragment directory

% touch f.scc
% bitbake -c kernel_configme linux-yocto
# meta-data task re-runs

% rm f.scc
% bitbake -c kernel_configme linux-yocto
# meta-data task re-runs

% touch f.scc
% bitbake -c kernel_configme linux-yocto
# meta-data task re-runs
 
% mv f.scc f.scc.bak
% bitbake -c kernel_configme linux-yocto
# meta-data task does *not* run

% mv f.scc.bak f.scc
% bitbake -c kernel_configme linux-yocto
# meta-data task does *not* run

% rm f.scc
% bitbake -c kernel_configme linux-yocto
# meta-data task re-runs

So while I'm not seeing the same results as you on the 'rm' action. I am seeing the same thing with the 'mv' operation.

I've pinged Richard to comment if this is expected, since we are talking about bitbake behaviour here.

just updating this to indicate that I'm waiting for RPs comment as to whether the behaviour is expected or not.

No, at a quick glance those behaviours are not expected and we need to investigate further. Sounds like a potential bitbake fetcher issue.

I can't put a milestone on this, since I unfortunately don't understand the current behaviour and hence to fix the bug. Once my other 3.1 features are done in the M3 timeframe, I'll take a closer look.

The directory tracking variant of this has been merged into oe-core.

Corner cases will be handled in separate bugs.