13434 2019-07-09 06:16:26 +0000 checksum indirect cfg and scc flags 2020-08-28 20:07:49 +0000 1 1 7 Build System, Metadata & Runtime OE-Core kernel unspecified x86 Multiple RESOLVED FIXED Medium+ normal 3.2 M3 1 zhangzl2013 bruce.ashfield randy.macleod richard.purdie tom.zanussi --- --- No (bug/feature does not impact docs) oldest_to_newest 84393 0 zhangzl2013 2019-07-09 06:16:26 +0000 Yocto can not realize the modification of the cfg / scc files indirectly introduced by scc files in custom layers. Email thread: http://lists.openembedded.org/pipermail/openembedded-core/2019-July/284403.html 84438 1 bruce.ashfield 2019-07-17 14:03:34 +0000 Can you test the patch that I attached to the bug ? It is essential your patch, and I'd like to add your Signed-off-by to the commit message before I send it. After talking with Richard, I decided to not go for a few higher maintenance ideas and instead have changed your routine to only return directories with .scc and .cfg files. That allows us to re-run if a file is modified, added or removed from the directories, which addresses the concern that Richard had. This still will trigger a few more re-executions of the meta-data + kernel build than I'd like, but I'll wait for some of the task equivalence changes to be complete before trying to check that although the meta-data task has re-run, the output is the same and hence we don't need to rebuild or repackage the kernel. 84439 2 4540 bruce.ashfield 2019-07-17 14:04:52 +0000 Created attachment 4540 proposed directory tracking patch 84471 3 zhangzl2013 2019-07-22 10:03:51 +0000 Hi Bruce, I got a weird result: If I modify the testing fragment file poky/meta-yocto-bsp/recipes-kernel/linux/files/test.cfg, it works. And if I `touch test2.cfg`, it also triggers kernel_metadata task. But in 2 conditions it doesn't work: 1. mv test.cfg test.cfg.bak 2. rm test2.cfg I also checked modification time of the dir poky/meta-yocto-bsp/recipes-kernel/linux/files/ using `stat` util, it changed in both conditions. Don't know why yocto didn't trigger kernel_metadata in these 2 cases. 84472 4 bruce.ashfield 2019-07-22 12:46:07 +0000 (In reply to comment #3) > Hi Bruce, > > I got a weird result: > > If I modify the testing fragment file > poky/meta-yocto-bsp/recipes-kernel/linux/files/test.cfg, it works. > And if I `touch test2.cfg`, it also triggers kernel_metadata task. > > But in 2 conditions it doesn't work: > 1. mv test.cfg test.cfg.bak > 2. rm test2.cfg > > I also checked modification time of the dir > poky/meta-yocto-bsp/recipes-kernel/linux/files/ using `stat` util, it > changed in both conditions. Don't know why yocto didn't trigger > kernel_metadata in these 2 cases. Thanks for the update! I'll try and reproduce those results here and will adjust things as necessary. 84473 5 bruce.ashfield 2019-07-22 13:53:13 +0000 Here's the results of my tests with the patch: # all operations are in a layers kernel patch/fragment directory % touch f.scc % bitbake -c kernel_configme linux-yocto # meta-data task re-runs % rm f.scc % bitbake -c kernel_configme linux-yocto # meta-data task re-runs % touch f.scc % bitbake -c kernel_configme linux-yocto # meta-data task re-runs % mv f.scc f.scc.bak % bitbake -c kernel_configme linux-yocto # meta-data task does *not* run % mv f.scc.bak f.scc % bitbake -c kernel_configme linux-yocto # meta-data task does *not* run % rm f.scc % bitbake -c kernel_configme linux-yocto # meta-data task re-runs So while I'm not seeing the same results as you on the 'rm' action. I am seeing the same thing with the 'mv' operation. I've pinged Richard to comment if this is expected, since we are talking about bitbake behaviour here. 84754 6 bruce.ashfield 2019-08-27 15:18:19 +0000 just updating this to indicate that I'm waiting for RPs comment as to whether the behaviour is expected or not. 85196 7 richard.purdie 2019-10-03 15:20:22 +0000 No, at a quick glance those behaviours are not expected and we need to investigate further. Sounds like a potential bitbake fetcher issue. 85671 8 bruce.ashfield 2019-12-02 20:52:05 +0000 I can't put a milestone on this, since I unfortunately don't understand the current behaviour and hence to fix the bug. Once my other 3.1 features are done in the M3 timeframe, I'll take a closer look. 88060 9 bruce.ashfield 2020-08-28 20:07:49 +0000 The directory tracking variant of this has been merged into oe-core. Corner cases will be handled in separate bugs. 4540 2019-07-17 14:04:52 +0000 2019-07-17 14:04:52 +0000 proposed directory tracking patch 0001-kernel-yocto-checksum-all-modifications-to-available.patch application/octet-stream 3878 bruce.ashfield RnJvbSBhYTIxOTU5YTQ3NDg2Y2Q2MzAzMWZmNjhlMzg5MDE1Nzc2N2RmNmY3IE1vbiBTZXAgMTcg MDA6MDA6MDAgMjAwMQpGcm9tOiBCcnVjZSBBc2hmaWVsZCA8YnJ1Y2UuYXNoZmllbGRAZ21haWwu Y29tPgpEYXRlOiBXZWQsIDE3IEp1bCAyMDE5IDA5OjM0OjI1IC0wNDAwClN1YmplY3Q6IFtQQVRD SF0ga2VybmVsLXlvY3RvOiBjaGVja3N1bSBhbGwgbW9kaWZpY2F0aW9ucyB0byBhdmFpbGFibGUg a2VybmVsCiBmcmFnbWVudHMgZGlyZWN0b3JpZXMKClRoaXMgaXMgYmFzZWQgb24gdGhlIHBhdGNo IGZyb20gWmhhb2xvbmcgWmhhbmcgPHpoYW5nemwyMDEzQDEyNi5jb20+CltrZXJuZWwteW9jdG86 IGNoZWNrc3VtIGluZGlyZWN0IGNmZyBhbmQgc2NjIGZpbGVzXQoKV2hpbGUgdGhlIHJlY29tbWVu ZGVkIG1hbm5lciB0byBzaGFyZS9yZXVzZSBmZWF0dXJlIGZyYWdtZW50cyBpcyB0bwptYWludGFp biB0aGVtIGluIGEga2VybmVsLW1ldGEgcmVwb3NpdG9yeSBhbmQgdHJhY2sgdGhlIGNoYW5nZXMg dmlhIHRoZQpzdGFuZGFyZCBTUkNSRVYgZmV0Y2hlciBtZWNoYW5pc20sIHRoYXQgbWV0aG9kIGlz IG5vdCBhbHdheXMgcHJhY3RpY2FsCmZvciBzbWFsbCBzZXRzIG9mIGZlYXR1cmVzIG9yIGZvciBx dWljayB0ZXN0aW5nIG9mIGNoYW5nZXMuCgpUaGVzZSBvdGhlciBmbG93cyB1c2UgLnNjYyBmaWxl cyBvbiB0aGUgU1JDX1VSSS4gSXQgaGFzIGJlZW4gbm90ZWQgdGhhdApjb25maWcgZnJhZ21lbnRz IG9yIG90aGVyIGZlYXR1cmVzIGluZGlyZWN0bHkgaW5jbHVkZWQgYnkgdGhvc2UgLnNjYwpmaWxl cyB3aWxsIG5vdCB0cmlnZ2VyIHRoZSBrZXJuZWwgbWV0YS1kYXRhIHRvIGJlIHJlLXJ1biBhbmQg aGVuY2UgYQpidWlsZCBjYW4gY29udGludWUgd2l0aCBzdGFsZSBkYXRhIChvciBub3QgYmUgdHJp Z2dlcmVkIGF0IGFsbCkuCgpUbyBzb2x2ZSB0aGlzIGlzc3VlLCB3ZSBjYW4gY29sbGVjdCB0aGUg ZGlyZWN0b3JpZXMgdGhhdCBhcmUgc2VhcmNoYWJsZQp2aWEgRklMRVNFWFRSQVBBVEhTIGFuZCBh ZGQgdGhlbSB0byB0aGUgZG9fa2VybmVsX21ldGFkYXRhIHRhc2sKY2hlY2tzdW0uCgpUaGlzIGFs bG93cyBtb2RpZmljYXRpb25zLCBhZGRpdGlvbnMgYW5kIHJlbW92YWxzIGZyb20gdGhlIHBvdGVu dGlhbAprZXJuZWwgZmVhdHVyZSBkaXJlY3RvcmllcyB0byB0cmlnZ2VyIGEgcmUtZXhlY3V0aW9u IG9mIHRoZSBtZXRhIGRhdGEKdGFzay4KClNpZ25lZC1vZmYtYnk6IEJydWNlIEFzaGZpZWxkIDxi cnVjZS5hc2hmaWVsZEBnbWFpbC5jb20+Ci0tLQogbWV0YS9jbGFzc2VzL2tlcm5lbC15b2N0by5i YmNsYXNzIHwgMzkgKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKwogMSBmaWxlIGNoYW5n ZWQsIDM5IGluc2VydGlvbnMoKykKCmRpZmYgLS1naXQgYS9tZXRhL2NsYXNzZXMva2VybmVsLXlv Y3RvLmJiY2xhc3MgYi9tZXRhL2NsYXNzZXMva2VybmVsLXlvY3RvLmJiY2xhc3MKaW5kZXggZmJk MTk0YjViOS4uMDczNjg0MTY4YiAxMDA2NDQKLS0tIGEvbWV0YS9jbGFzc2VzL2tlcm5lbC15b2N0 by5iYmNsYXNzCisrKyBiL21ldGEvY2xhc3Nlcy9rZXJuZWwteW9jdG8uYmJjbGFzcwpAQCAtNjks NiArNjksNDQgQEAgZGVmIGdldF9tYWNoaW5lX2JyYW5jaChkLCBkZWZhdWx0KToKIAkgICAgCiAg ICAgcmV0dXJuIGRlZmF1bHQKIAorIyByZXR1cm5zIGEgbGlzdCBvZiBhbGwgZGlyZWN0b3JpZXMg dGhhdCBhcmUgb24gRklMRVNFWFRSQVBBVEhTIChhbmQKKyMgaGVuY2UgYXZhaWxhYmxlIHRvIHRo ZSBidWlsZCkgdGhhdCBjb250YWluIC5zY2Mgb3IgLmNmZyBmaWxlcworZGVmIGdldF9kaXJzX3dp dGhfZnJhZ21lbnRzKGQpOgorICAgIGV4dHJhcGF0aHMgPSBbXQorICAgIGV4dHJhZmlsZXMgPSBb XQorICAgIGV4dHJhcGF0aHN2YWx1ZSA9IChkLmdldFZhcigiRklMRVNFWFRSQVBBVEhTIikgb3Ig IiIpCisgICAgIyBSZW1vdmUgZGVmYXVsdCBmbGFnIHdoaWNoIHdhcyB1c2VkIGZvciBjaGVja2lu ZworICAgIGV4dHJhcGF0aHN2YWx1ZSA9IGV4dHJhcGF0aHN2YWx1ZS5yZXBsYWNlKCJfX2RlZmF1 bHQ6IiwgIiIpCisgICAgZXh0cmFwYXRocyA9IGV4dHJhcGF0aHN2YWx1ZS5zcGxpdCgiOiIpCisg ICAgZm9yIHBhdGggaW4gZXh0cmFwYXRoczoKKyAgICAgICAgZm9yIHJvb3QsIGRpcnMsIGZpbGVz IGluIG9zLndhbGsocGF0aCk6CisgICAgICAgICAgICBmb3IgbmFtZSBpbiBmaWxlczoKKyAgICAg ICAgICAgICAgICBiYXNlLCBleHQgPSBvcy5wYXRoLnNwbGl0ZXh0KG5hbWUpCisgICAgICAgICAg ICAgICAgaWYgZXh0IGFuZCBleHQgaW4gWyIuc2NjIiwgIi5jZmciXToKKyAgICAgICAgICAgICAg ICAgICAgaWYgcGF0aCArICI6VHJ1ZSIgbm90IGluIGV4dHJhZmlsZXM6CisgICAgICAgICAgICAg ICAgICAgICAgICBleHRyYWZpbGVzLmFwcGVuZChwYXRoICsgIjoiICsgc3RyKG9zLnBhdGguZXhp c3RzKHBhdGgpKSkKKworICAgIHJldHVybiAiICIuam9pbihleHRyYWZpbGVzKQorCisjIHJldHVy bnMgYSBsaXN0IG9mIGFsbCB0aGUgLnNjYyBhbmQgLmNmZyBmaWxlcyB0aGF0IGFyZSBvbiBGSUxF U0VYVFJBUEFUSFMKKyMgKGFuZCBoZW5jZSBhdmFpbGFibGUgdG8gdGhlIGJ1aWxkKQorZGVmIGdl dF9mcmFnbWVudHNfaW5fZmlsZXNleHRyYXBhdGhzKGQpOgorICAgIGV4dHJhcGF0aHMgPSBbXQor ICAgIGV4dHJhZmlsZXMgPSBbXQorICAgIGV4dHJhcGF0aHN2YWx1ZSA9IChkLmdldFZhcigiRklM RVNFWFRSQVBBVEhTIikgb3IgIiIpCisgICAgIyBSZW1vdmUgZGVmYXVsdCBmbGFnIHdoaWNoIHdh cyB1c2VkIGZvciBjaGVja2luZworICAgIGV4dHJhcGF0aHN2YWx1ZSA9IGV4dHJhcGF0aHN2YWx1 ZS5yZXBsYWNlKCJfX2RlZmF1bHQ6IiwgIiIpCisgICAgZXh0cmFwYXRocyA9IGV4dHJhcGF0aHN2 YWx1ZS5zcGxpdCgiOiIpCisgICAgZm9yIHBhdGggaW4gZXh0cmFwYXRoczoKKyAgICAgICAgZm9y IHJvb3QsIGRpcnMsIGZpbGVzIGluIG9zLndhbGsocGF0aCk6CisgICAgICAgICAgICBmb3IgbmFt ZSBpbiBmaWxlczoKKyAgICAgICAgICAgICAgICBiYXNlLCBleHQgPSBvcy5wYXRoLnNwbGl0ZXh0 KG5hbWUpCisgICAgICAgICAgICAgICAgaWYgZXh0IGFuZCBleHQgaW4gWyIuc2NjIiwgIi5jZmci XToKKyAgICAgICAgICAgICAgICAgICAgZmlsZXBhdGggPSBvcy5wYXRoLmpvaW4ocm9vdCwgbmFt ZSkKKyAgICAgICAgICAgICAgICAgICAgZXh0cmFmaWxlcy5hcHBlbmQoZmlsZXBhdGggKyAiOiIg KyBzdHIob3MucGF0aC5leGlzdHMoZmlsZXBhdGgpKSkKKworICAgIHJldHVybiAiICIuam9pbihl eHRyYWZpbGVzKQorCiBkb19rZXJuZWxfbWV0YWRhdGEoKSB7CiAJc2V0ICtlCiAJY2QgJHtTfQpA QCAtMjk2LDYgKzMzNCw3IEBAIGRvX2tlcm5lbF9jaGVja291dFtkaXJzXSA9ICIke1N9IgogYWRk dGFzayBrZXJuZWxfY2hlY2tvdXQgYmVmb3JlIGRvX2tlcm5lbF9tZXRhZGF0YSBhZnRlciBkb191 bnBhY2sKIGFkZHRhc2sga2VybmVsX21ldGFkYXRhIGFmdGVyIGRvX3ZhbGlkYXRlX2JyYW5jaGVz IGRvX3VucGFjayBiZWZvcmUgZG9fcGF0Y2gKIGRvX2tlcm5lbF9tZXRhZGF0YVtkZXBlbmRzXSA9 ICJrZXJuLXRvb2xzLW5hdGl2ZTpkb19wb3B1bGF0ZV9zeXNyb290IgorZG9fa2VybmVsX21ldGFk YXRhW2ZpbGUtY2hlY2tzdW1zXSA9ICIgJHtAZ2V0X2RpcnNfd2l0aF9mcmFnbWVudHMoZCl9Igog ZG9fdmFsaWRhdGVfYnJhbmNoZXNbZGVwZW5kc10gPSAia2Vybi10b29scy1uYXRpdmU6ZG9fcG9w dWxhdGVfc3lzcm9vdCIKIAogZG9fa2VybmVsX2NvbmZpZ21lW2RlcGVuZHNdICs9ICJ2aXJ0dWFs LyR7VEFSR0VUX1BSRUZJWH1iaW51dGlsczpkb19wb3B1bGF0ZV9zeXNyb290IgotLSAKMi4xOS4x Cgo=