9858 2016-06-30 15:07:23 +0000 Make xattrs in images work [meta] 2026-07-23 15:29:42 +0000 1 1 7 Build System, Metadata & Runtime OE-Core core 2.1 x86 Multiple NEW Medium+ enhancement 6.99 9229 9317 9857 9859 9929 15091 1 ross.burton unassigned liezhi.yang maxin.john meta.mr.watcher meta.watcher p.lobacz patrick.ohly paul randy.macleod richard.purdie sgw yi.zhao --- --- No (bug/feature does not impact docs) oldest_to_newest 63526 0 ross.burton 2016-06-30 15:07:23 +0000 It should be possible to use fsetattr or setcap inside do_install and end up with a file in an image with the relevant xattrs. 63530 1 ross.burton 2016-06-30 15:16:29 +0000 We also need a test to verify that a recipe using setcap or fsetattr in do_install ends up with the right attributes on the target. Last time I tried this I couldn't set an xattr on the target, despite using ext4. Does ext4 disable xattrs if EMBEDDED is enabled? 63531 2 ross.burton 2016-06-30 15:18:31 +0000 Note: also check CONFIG_EXT4_FS_POSIX_ACL and CONFIG_EXT4_FS_SECURITY. 64828 3 joshuagloe 2016-08-03 12:48:26 +0000 (In reply to comment #1) > We also need a test to verify that a recipe using setcap or fsetattr in > do_install ends up with the right attributes on the target. Last time I > tried this I couldn't set an xattr on the target, despite using ext4. Does > ext4 disable xattrs if EMBEDDED is enabled? Works for me, did you namespace your extended attribute? AIUI an xattr *must* be prefixed with one of the 4 defined namespaces from xattr(7): security, system, trusted or user. $ setfattr -n user.foo -v bar myfile $ getfattr -d myfile # file: myfile user.foo="bar" 64841 4 joshuagloe 2016-08-03 16:30:23 +0000 It's trivial to support xattrs up to do_package but preserving those xattr into the rootfs would ideally require support from the package manager, unfortunately neither rpm5[1] or dpkg[] currently provide that support. The obvious alternative is to set the file attributes in a postinst script but it feels like it could introduce some overhead if we want to support fsetattr in do_install. We'd need to walk the files in the package, dump any xattr and generate a script fragment to replicate detected xattr at postinst. An alternative (to the alternative) could be to have recipe authors set a variable containing filepath:xattrname:xattrvalue triplets which could be transformed into a postinst, i.e. XATTR_${PN} = "${datadir}/${PN}/datafile:system.xattrname:xattrvalue \ ${sysconfdir}/${PN}/conffile:system.xattrname:xattrvalue" I'll think about this some more… 64847 5 ross.burton 2016-08-03 19:27:34 +0000 "Dude, where's my footnotes" 64848 6 joshuagloe 2016-08-03 19:57:50 +0000 *ahem* I forgot to include references: DPKG & xattr: https://lists.debian.org/debian-dpkg/2014/09/msg00004.html https://lists.debian.org/debian-dpkg/2014/11/msg00023.html https://wiki.ubuntu.com/Security/FilesystemCapabilties https://blueprints.launchpad.net/ubuntu/+spec/security-m-dpkg-fscaps RPM5 & xattr: http://rpm5.org/community/rpm-devel/2977.html (yup, 2008) RPM4's only mention of xattr is in the IMA plugin, which sets the IMA xattr at postinst: https://github.com/rpm-software-management/rpm/blob/e9e9b667795b749e63c41f53e55839ee6e8d8bbd/plugins/ima.c#L57 65239 7 joshuagloe 2016-08-16 19:56:30 +0000 We have a report to the mailing list that an ACL set with facl at postinst is visible on the rootfs from a devshell on the image recipe, but that it isn't preserved into the image when booted via QEMU: https://lists.yoctoproject.org/pipermail/yocto/2016-August/031570.html Obviously we'll need to root cause that and fix it as part of this issue. 66077 8 patrick.ohly 2016-09-13 10:17:36 +0000 I agree that per-file capabilities set during compilation would be useful. I tried to use that at some point and had to resort to a postinst script, which is more cumbersome. OTOH, that's not very common, so I don't know how important it really is. The bigger use of xattrs in Ostro was for security.ima, the xattr storing a hash of the file content. But that already works by setting the xattr in do_image. Adding support for file xattrs in packages won't be usable for that, because the security.ima xattr also depends on a per-image signing key. Ostro also used some Smack labels, but not many, so again the approach with postinst scripts wasn't too cumbersome. The bigger problem was the lack of proper Smack semantics under pseudo: Smack supports a "transmute" flag which influences the Smack label of files created inside a directory such that they inherit the Smack label of the directory. pseudo can set that flag for a directory (it's just an xattr), but then new files created in that directory do not inherit the Smack label. meta-intel-iot-security fixes that in a post-processing step. 72865 9 ross.burton 2017-04-26 16:21:22 +0000 I think we need to bump the priority of this bug to medium+, and rescope it slightly. It's not going to be possible to fix all the package managers (dpkg, opkg, and rpm4) to support xattrs directly. But we can have a class that will write a postinst snippet that runs setcap for you, so the recipe just has to inherit the class and set a variable. 88800 10 paul 2020-12-21 14:43:03 +0000 Accidentally assigned the wrong bug to myself due to bugzilla's stupid interface. 95266 11 p.lobacz 2023-04-01 20:07:59 +0000 Actually this can be handled with opkg and ipk packages as Stefan has added support for xattr to libarchive in this https://github.com/libarchive/libarchive/pull/691 pull request. Question is, does we only need to add these options "--xattrs --acls" to opkg-build? Because, from what I have seen, this script is set in OPKGBUILDCMD. 95268 12 p.lobacz 2023-04-02 10:42:39 +0000 OK it is enough just to set this in this opkg-build script but afterwards I'm trying to call setfacl in do_install and I do not see that the file has changed its acls... 95269 13 p.lobacz 2023-04-02 11:21:01 +0000 (In reply to Piotr Łobacz from comment #12) > OK it is enough just to set this in this opkg-build script but afterwards > I'm trying to call setfacl in do_install and I do not see that the file has > changed its acls... OK I have added a test command getfacl after setfacl in do_install and verified that it actually sets the proper acls, which can be verified reading the log.do_install log: DEBUG: Executing python function extend_recipe_sysroot NOTE: Direct dependencies are ['/work/build/yocto/meta-iotedge/recipes-core/aziotd/aziotd_1.4.3.bb:do_populate_sysroot', '/work/build/yocto/meta-rust/recipes-devtools/rust/libstd-rs_1.66.0.bb:do_populate_sysroot', '/work/build/yocto/meta-rust/recipes-devtools/rust/rust-cross_1.66.0.bb:do_populate_sysroot', '/work/build/yocto/meta-virtualization/recipes-containers/docker/docker-moby_git.bb:do_populate_sysroot', '/work/build/yocto/openembedded-core/meta/recipes-connectivity/openssl/openssl_3.0.7.bb:do_populate_sysroot', '/work/build/yocto/openembedded-core/meta/recipes-core/base-files/base-files_3.0.14.bb:do_populate_sysroot', '/work/build/yocto/openembedded-core/meta/recipes-core/base-passwd/base-passwd_3.5.29.bb:do_populate_sysroot', '/work/build/yocto/openembedded-core/meta/recipes-core/glibc/glibc_2.35.bb:do_populate_sysroot', '/work/build/yocto/openembedded-core/meta/recipes-devtools/gcc/gcc-cross_11.3.bb:do_populate_sysroot', '/work/build/yocto/openembedded-core/meta/recipes-devtools/gcc/gcc-runtime_11.3.bb:do_populate_sysroot', '/work/build/yocto/openembedded-core/meta/recipes-devtools/quilt/quilt-native_0.67.bb:do_populate_sysroot', '/work/build/yocto/openembedded-core/meta/recipes-extended/shadow/shadow-sysroot_4.6.bb:do_populate_sysroot', '/work/build/yocto/openembedded-core/meta/recipes-extended/shadow/shadow_4.11.1.bb:do_populate_sysroot', 'virtual:native:/work/build/yocto/meta-rust/recipes-devtools/cargo/cargo_1.66.0.bb:do_populate_sysroot', 'virtual:native:/work/build/yocto/openembedded-core/meta/recipes-devtools/patch/patch_2.7.6.bb:do_populate_sysroot', 'virtual:native:/work/build/yocto/openembedded-core/meta/recipes-devtools/pseudo/pseudo_git.bb:do_populate_sysroot', 'virtual:native:/work/build/yocto/openembedded-core/meta/recipes-devtools/python/python3_3.10.9.bb:do_populate_sysroot', 'virtual:native:/work/build/yocto/openembedded-core/meta/recipes-extended/shadow/shadow_4.11.1.bb:do_populate_sysroot', 'virtual:native:/work/build/yocto/openembedded-core/meta/recipes-support/attr/acl_2.3.1.bb:do_populate_sysroot'] NOTE: Installed into sysroot: [] NOTE: Skipping as already exists in sysroot: ['aziotd', 'libstd-rs', 'rust-cross-core2-64-glibc', 'docker-moby', 'openssl', 'base-files', 'base-passwd', 'glibc', 'gcc-cross-x86_64', 'gcc-runtime', 'quilt-native', 'shadow-sysroot', 'shadow', 'cargo-native', 'patch-native', 'pseudo-native', 'python3-native', 'shadow-native', 'acl-native', 'opkg-utils', 'libtool-native', 'attr-native', 'linux-libc-headers', 'zlib-native', 'texinfo-dummy-native', 'xz-native', 'mpfr-native', 'gnu-config-native', 'flex-native', 'zstd-native', 'binutils-cross-x86_64', 'libmpc-native', 'gmp-native', 'libffi-native', 'readline-native', 'sqlite3-native', 'gdbm-native', 'util-linux-libuuid-native', 'bzip2-native', 'ncurses-native', 'libnsl2-native', 'openssl-native', 'libtirpc-native', 'gettext-minimal-native', 'libpam', 'libxcrypt', 'attr', 'libssh2-native', 'curl-native', 'ca-certificates-native', 'rust-native', 'aziot-keys', 'tpm2-abrmd', 'libgcc', 'tpm2-tss', 'rust-llvm-native', 'go-pty', 'go-patricia', 'go-logrus', 'go-runtime', 'libtool', 'lvm2', 'go-capability', 'go-systemd', 'go-cli', 'go-context', 'go-mux', 'compose-file', 'btrfs-tools', 'grpc-go', 'go-distribution', 'go-fsnotify', 'go-connections', 'go-dbus', 'libseccomp', 'sqlite3', 'acl', 'notary', 'm4-native', 'perl-native', 'cracklib', 'flex', 'dbus', 'autoconf-archive', 'glib-2.0', 'libgcrypt', 'json-c', 'curl', 'cmake-native', 'libaio', 'util-linux', 'systemd', 'python3', 'lzo', 'e2fsprogs', 'zlib', 'make-native', 'expat', 'libffi', 'bash-completion', 'libpcre', 'libgpg-error', 'libcap', 'libidn2', 'libcap-ng', 'util-linux-libuuid', 'ncurses', 'xz', 'kmod', 'bzip2', 'readline', 'gdbm', 'libnsl2', 'libtirpc', 'libunistring'] DEBUG: Python function extend_recipe_sysroot finished DEBUG: Executing shell function do_install getfacl: Removing leading '/' from absolute path names # file: work/build/tmp-glibc/work/core2-64-WelotecGmbH-linux/aziotctl/1.4.3.AUTOINC+cfa39b58ec-r0/image/etc/aziot/config.toml # owner: iotedge # group: iotedge user::rw- user:iotedge:rwx group::rw- mask::rwx other::r-- DEBUG: Shell function do_install finished DEBUG: Executing shell function remove_libtool_la DEBUG: Shell function remove_libtool_la finished But if I do that on my own by calling the getfacl on the same file, there are no visible acls at all. I do not know if that is normal behaviour. Could somebody explain that to me? 95270 14 p.lobacz 2023-04-02 11:23:34 +0000 I'm sorry but I should say rather that not preserved acls instead of none because there are acls visible e.g: getfacl build/tmp-glibc/work/core2-64-WelotecGmbH-linux/aziotctl/1.4.3.AUTOINC+cfa39b58ec-r0/image/etc/aziot/config.toml # file: build/tmp-glibc/work/core2-64-WelotecGmbH-linux/aziotctl/1.4.3.AUTOINC+cfa39b58ec-r0/image/etc/aziot/config.toml # owner: plobacz # group: plobacz user::rw- group::r-- other::r-- 95272 15 4943 p.lobacz 2023-04-02 21:38:51 +0000 Created attachment 4943 Add xattrs and acl support for tar archives This is a patch for setting 95273 16 p.lobacz 2023-04-03 07:59:26 +0000 OK I have verified that packages are being created from packages-split/pkg, where pkg is the dir name for package. This is what I have verified in package_ipk.bbclass. Additionally I have also verified that if I'm getting to package with `bitbake -c devshell package` command and calling `getfacl` on the file on which I'm setting ACLs with `setfacl` in do_install all privileges are indeed persisted, but still within this devshell calling on the same file but in packages-split/pkg these privileges are not persisted. So my guess is that they are disappearing on some copy command or so on. So question is, where can it be? 95275 17 p.lobacz 2023-04-03 09:42:30 +0000 OK finally I have discovered it and it works just as I wanted to :D I will create a patch for package.bbclass and share it on the mailing list with additional link in here to it 95276 18 p.lobacz 2023-04-03 10:44:45 +0000 This is the link to the patch https://lists.openembedded.org/g/openembedded-core/message/179609 I will send also other patches for opkg as well. 95278 19 p.lobacz 2023-04-03 12:33:52 +0000 OK I have submitted my patches to the list which are in here: https://lists.openembedded.org/g/openembedded-core/message/179626 https://lists.openembedded.org/g/openembedded-core/message/179627 https://lists.openembedded.org/g/openembedded-core/message/179628 https://lists.openembedded.org/g/openembedded-core/message/179629 I have also added Richard to CC as he already had some comments before regarding https://lists.openembedded.org/g/openembedded-core/message/179609 which, I have to admit, was an ugly approach of mine, to send those patches separately. Thanks Richard :) 95279 20 p.lobacz 2023-04-03 14:00:28 +0000 Hmmm unfortunately these patches still does not solve the issue. I have just verified that on rootfs I'm still having problem that the attributes are not being preserved. For ipk files they are. I have even verified by installing ipk package with ACLs but in the rootfs they are not, not even in bitbake -c devshell image-package-name... I still ned to investigate:] 95280 21 p.lobacz 2023-04-03 14:10:06 +0000 Ech it seems that there is also a completly different part for extracting ipk poackages in yocto/openembedded-core/meta/lib/oe/package_manager/ipk which I also need to adapt to this solution... 95281 22 p.lobacz 2023-04-03 14:20:16 +0000 OK finally I've got my answer: https://git.openembedded.org/openembedded-core/tree/meta/lib/oeqa/utils/package_manager.py?h=kirkstone#n163 and according to the documentation of shutil package https://docs.python.org/3/library/shutil.html Warning Even the higher-level file copying functions (shutil.copy(), shutil.copy2()) cannot copy all file metadata. On POSIX platforms, this means that file owner and group are lost as well as ACLs. On Mac OS, the resource fork and other metadata are not used. This means that resources will be lost and file type and creator codes will not be correct. On Windows, file owners, ACLs and alternate data streams are not copied. So this definitely has to be changed... 95297 23 p.lobacz 2023-04-06 10:27:22 +0000 OK I have finally fixed the problem and everything is working for me. The issue regarding not putting acls for native image was due to missing acl feautre in DISTRO_FEATURES_NATIVE even that it was set in DISTRO_FEATURE. I think that we should discuss it, how it should be solved, because right now if I choose it only in DISTRO_FEATURES, than it is not being set for DISTRO_FEATURES_NATIVE which right now I do not know if it is a proper way to go and would like to get second opinions. I had also this small talk about putting this into Kirkstone with Richard and he stated that new feature development work happens on master, not on our LTS stable release branches, which means that this will not happen for kikrstone. I can try to handle it. 95891 24 p.lobacz 2023-07-07 08:00:08 +0000 Hi all, I have finally managed to do the fixes for yocto master :-) I will try to upload the patches today evening or during the weekend. BR Piotr 95895 25 p.lobacz 2023-07-10 10:06:12 +0000 Ok I have pushed the changes for the master as Richard asked me. These are the patches: https://lists.openembedded.org/g/openembedded-core/message/184063 https://lists.openembedded.org/g/openembedded-core/message/184064 https://lists.openembedded.org/g/openembedded-core/message/184065 Additionaly keep in mind that first this one https://bugzilla.yoctoproject.org/show_bug.cgi?id=15097 has to be merged in order to have it properly working. BR Piotr 96961 26 randy.macleod 2023-10-24 14:24:51 +0000 Bulk move from 4.99 or 0.00 to 5.99 4943 2023-04-02 21:38:51 +0000 2023-04-02 21:38:51 +0000 Add xattrs and acl support for tar archives 0001-opkg-build-Add-xattrs-support-for-tar-archives.patch text/plain 2326 p.lobacz RnJvbSAwODA2ZTAyYzQzODc0MzdlNzg2ZTEwMTk4MmJkMDViNmZmN2I5YTgzIE1vbiBTZXAgMTcg MDA6MDA6MDAgMjAwMQpGcm9tOiA9P1VURi04P3E/UGlvdHI9MjA9QzU9ODFvYmFjej89IDxwLmxv YmFjekB3ZWxvdGVjLmNvbT4KRGF0ZTogU3VuLCAyIEFwciAyMDIzIDAxOjEwOjA1ICswMjAwClN1 YmplY3Q6IFtQQVRDSF0gb3BrZy1idWlsZDogQWRkIHhhdHRycyBzdXBwb3J0IGZvciB0YXIgYXJj aGl2ZXMKTUlNRS1WZXJzaW9uOiAxLjAKQ29udGVudC1UeXBlOiB0ZXh0L3BsYWluOyBjaGFyc2V0 PVVURi04CkNvbnRlbnQtVHJhbnNmZXItRW5jb2Rpbmc6IDhiaXQKCkFkZCBzdXBwb3J0IGZvciB0 YXIgYXJjaGl2ZXMgY3JlYXRlZCB3aXRoIC0teGF0dHJzIG9wdGlvbiwgUEFYIGhlYWRlciBmb3Jt YXQuCgpHTlUgdGFyIGFuZCBsaWJhcmNoaXZlIGFscmVhZHkgc3VwcG9ydHMgZXh0ZW5kZWQgYXR0 cmlidXRlcy4gV2UgY2FuIG5vdwphZGQgdGhpcyBzdXBwb3J0IGFzIHdlbGwgdG8gb3BrZy1idWls ZCBzY3JpcHQgaW4gb3JkZXIgdG8gdXNlIGZzZXRhdHRyIG9yCnNldGNhcCBpbnNpZGUgZG9faW5z dGFsbCBjb21tYW5kIGFuZCBlbmQgdXAgd2l0aCBhIGZpbGUgaW4gYW4gaW1hZ2Ugd2l0aAp0aGUg cmVsZXZhbnQgeGF0dHJzLgoKU2lnbmVkLW9mZi1ieTogUGlvdHIgxYFvYmFjeiA8cC5sb2JhY3pA d2Vsb3RlYy5jb20+CgolJSBvcmlnaW5hbCBwYXRjaDogMDAwMS1vcGtnLWJ1aWxkLUFkZC14YXR0 cnMtc3VwcG9ydC1mb3ItdGFyLWFyY2hpdmVzLnBhdGNoCi0tLQogb3BrZy1idWlsZCB8IDUgKyst LS0KIDEgZmlsZSBjaGFuZ2VkLCAyIGluc2VydGlvbnMoKyksIDMgZGVsZXRpb25zKC0pCgpkaWZm IC0tZ2l0IGEvb3BrZy1idWlsZCBiL29wa2ctYnVpbGQKaW5kZXggYTllNDVkNC4uZWI3NjRhNSAx MDA3NTUKLS0tIGEvb3BrZy1idWlsZAorKysgYi9vcGtnLWJ1aWxkCkBAIC0xNTksMTQgKzE1OSwx MyBAQCBjb21wcmVzc29yYXJncz0iIgogIyAgICAtIEJ1c3lib3ggdGFyIGRvZXNuJ3Qgc3VwcG9y dCAnLS1mb3JtYXQnCiAjICAgIC0gT24gc29tZSBMaW51eCBkaXN0cm9zLCB0YXIgbm93IGRlZmF1 bHRzIHRvIHBvc2l4IGZvcm1hdCBpZiAnLS1mb3JtYXQnCiAjICAgICAgaXNuJ3QgZXhwbGljaXRs eSBzcGVjaWZpZWQKLSMgICAgLSBPcGtnIGRvZXNuJ3QgY3VycmVudGx5IHN1cHBvcnQgcG9zaXgg Zm9ybWF0IGFyY2hpdmVzCiAjCiAjIEl0J3MgZWFzaWVyIHRvIGNoZWNrIGZvciBtZW50aW9uIG9m IHRoZSAnLS1mb3JtYXQnIG9wdGlvbiB0aGFuIHRvIGRldGVjdCB0aGUKICMgdGFyIGltcGxlbWVu dGF0aW9uIGFuZCBtYWludGFpbiBhIGxpc3Qgb2Ygd2hpY2ggc3VwcG9ydCAnLS1mb3JtYXQnLgog dGFyZm9ybWF0PSIiCiBpZiB0YXIgLS1oZWxwIDI+JjEgfCBncmVwIC0tICItLWZvcm1hdCIgPiAv ZGV2L251bGw7CiB0aGVuCi0gICAgdGFyZm9ybWF0PSItLWZvcm1hdD1nbnUiCisgICAgdGFyZm9y bWF0PSItLWZvcm1hdD1wb3NpeCIKIGZpCiAKIGNvbXByZXNzb3JfZXh0KCkgewpAQCAtMzE0LDcg KzMxMyw3IEBAIGV4cG9ydCBMQU5HPUMKIGV4cG9ydCBMQ19BTEw9QwogKCBjZCAkcGtnX2Rpci8k Q09OVFJPTCAmJiBmaW5kIC4gLXR5cGUgZiB8IHNvcnQgPiAkdG1wX2Rpci9jb250cm9sX2xpc3Qg KQogKCBjZCAkcGtnX2RpciAmJiBmaW5kIC4gLXBhdGggLi8kQ09OVFJPTCAtcHJ1bmUgLW8gLXBh dGggLiAtbyAtcHJpbnQgIHwgc29ydCA+ICR0bXBfZGlyL2ZpbGVfbGlzdCApCi0oIGNkICRwa2df ZGlyICYmIHRhciAkb2dhcmdzICR0c29ydGFyZ3MgLS1uby1yZWN1cnNpb24gJG10aW1lX2FyZ3Mg LWMgJHRhcmZvcm1hdCAtVCAkdG1wX2Rpci9maWxlX2xpc3QgfCAkY29tcHJlc3NvciAkY29tcHJl c3NvcmFyZ3MgPiAkdG1wX2Rpci9kYXRhLnRhci4kY2V4dCApCisoIGNkICRwa2dfZGlyICYmIHRh ciAtLXhhdHRycyAtLWFjbHMgJG9nYXJncyAkdHNvcnRhcmdzIC0tbm8tcmVjdXJzaW9uICRtdGlt ZV9hcmdzIC1jICR0YXJmb3JtYXQgLVQgJHRtcF9kaXIvZmlsZV9saXN0IHwgJGNvbXByZXNzb3Ig JGNvbXByZXNzb3JhcmdzID4gJHRtcF9kaXIvZGF0YS50YXIuJGNleHQgKQogKCBjZCAkcGtnX2Rp ci8kQ09OVFJPTCAmJiB0YXIgJG9nYXJncyAkdHNvcnRhcmdzIC0tbm8tcmVjdXJzaW9uIC0tbXRp bWU9QCRidWlsZF9kYXRlIC1jICR0YXJmb3JtYXQgLVQgJHRtcF9kaXIvY29udHJvbF9saXN0IHwg Z3ppcCAkemlwYXJncyA+ICR0bXBfZGlyL2NvbnRyb2wudGFyLmd6ICkKIHJtICR0bXBfZGlyL2Zp bGVfbGlzdAogcm0gJHRtcF9kaXIvY29udHJvbF9saXN0Ci0tIAoyLjM0LjEKCg==