Bug 7040

Summary: Support for /usr merge (à la systemd and Fedora)
Product: [Build System, Metadata & Runtime] OE-Core Reporter: Peter Kjellerstedt <peter.kjellerstedt>
Component: coreAssignee: Amaranth Valluri <amarnath.valluri>
Status: RESOLVED FIXED QA Contact:
Severity: enhancement    
Priority: Medium+ CC: amarnath.valluri, andrew, f.schuettler, jku, joshuagloe, matti-pekka.sivosuo, meta.mr.watcher, meta.watcher, mikko.ylinen, Qi.Chen, randy.macleod, richard.purdie, ross.burton
Version: 1.7   
Target Milestone: 2.4 M2   
Hardware: x86   
OS: Multiple   
See Also: https://bugzilla.yoctoproject.org/show_bug.cgi?id=5143
Whiteboard:
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: Don't know

Description Peter Kjellerstedt 2014-12-08 13:32:45 UTC
We would like to be able to configure our embedded system with a merged /usr à la Fedora. Preferably through a simple DISTRO_FEATURE.

Typically this would involve having /bin, /sbin and /lib* be symbolic links to their /usr counterparts. From a discussion on the oe-core mailing list I gathered that files/fs-perms.txt would need to be modified, as well as changing ${base_bindir}, ${base_sbindir} and ${base_libdir}.

According to Mark Hattle it should be pretty straightforward to redirect the /usr directories to their / counterparts. However, according to Chen Qi, the opposite is more troublesome and will require changes.

For reference: http://0pointer.de/blog/projects/the-usr-merge
Comment 1 Richard Purdie 2014-12-11 15:43:42 UTC
In the past OE has supported a model where the prefix was changed and the /usr directory was removed entirely (or could be a sumlink). This proposal seems to be different in that /usr is retained with symlinks to the usr counterparts.

This isn't something anyone I know is working on, hence the priority its being assigned. If someone is going to get it working, that would be fine, as long as the changes are not too invasive to the system. If it proves invasive, I'd much prefer the prefix approach since we know that isn't too invasive.
Comment 2 Joshua Lock 2016-06-28 16:46:40 UTC
*** Bug 9722 has been marked as a duplicate of this bug. ***
Comment 3 Joshua Lock 2016-06-28 16:49:03 UTC
A bunch of progress towards this has been made in master over recent releases and further supporting changes are being developed in my joshuagl/usrmerge branch on poky-contrib:

http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=joshuagl/usrmerge

Building that tree with the usrmerge DISTRO_FEATURE enabled results in a failure to generate the rootfs with Smart and several warnings when using other package managers (opkg and dpkg).

In both cases the root problem is the same, various of our scripts have a shebang of /bin/bash (or other utilities in /bin in some cases, i.e. sed) which is not present as we now install those binaries to /usr/bin

We need to come up with a generic solution to this problem which doesn't involve manually patching all of the shell scripts a package might include…
Comment 4 Jussi Kukkonen 2017-01-04 10:10:49 UTC
(In reply to comment #3)
> Building that tree with the usrmerge DISTRO_FEATURE enabled results in a
> failure to generate the rootfs with Smart and several warnings when using
> other package managers (opkg and dpkg).
> 
> In both cases the root problem is the same, various of our scripts have a
> shebang of /bin/bash (or other utilities in /bin in some cases, i.e. sed)
> which is not present as we now install those binaries to /usr/bin
> 
> We need to come up with a generic solution to this problem which doesn't
> involve manually patching all of the shell scripts a package might include…

I've been testing this stuff a little.

There are some minor issues I want to fix but the two major problems are:

1. build time dependencies to /bin or /lib.
I think we want to create the symlinks in the sysroots (when "usrmerge" is in DISTRO_FEATURES): otherwise things will fail and we might not even notice. Currently e.g. gobject-introspection fails to build and many postinst scripts fail at rootfs generation because prelink-rtld looks for "/lib/ld-linux.so.2".
This is not possible in a normal recipe (like base-files) because of how staging.bbclass currently works.

2. FILERDEPENDS on e.g. /bin/sh
With rpm this means install fails as nothing RPROVIDES /bin/sh. Calling /bin/sh works because the symlink is there and /usr/bin/sh exists. I'm not 100% sure what the right fix would be... Considering this:
* Make sure FILERDEPENDS is always tweaked to point to e.g. /usr/bin/sh  
* Add a QA check for not packaging things into /bin (etc) -- currently only kernel seems to break this in oe-core
Comment 5 Mikko Ylinen 2017-05-04 12:39:01 UTC
All patches were sent but some are still missing http://lists.openembedded.org/pipermail/openembedded-core/2017-March/134354.html

Can the remaining patches be prioritized for 2.4M1?
Comment 6 Ross Burton 2017-05-04 15:21:45 UTC
Mikko: can someone rebase the outstanding patches to master and publish the branch somewhere for constructive review?
Comment 7 Amaranth Valluri 2017-05-05 07:50:25 UTC
Ross, the rebased changes are on my repo on github:
https://github.com/avalluri/openembedded-core/commits/usrmerge
Comment 8 Amaranth Valluri 2017-05-22 12:22:43 UTC
Ross, did you get chance to look at the patchset
Comment 9 Jussi Kukkonen 2017-06-22 07:21:50 UTC
Amarnath, sorry for not being more active on this lately. I've assigned to you as an acknowledgement of reality... if you disagree: that's not a problem, just assign it back to me.

If there's something I can do to help with the patches, I can try but currently I'm not up-to-date on them.
Comment 10 Jussi Kukkonen 2017-06-22 07:25:40 UTC
(In reply to comment #9)
> Amarnath, sorry for not being more active on this lately. I've assigned to
> you as an acknowledgement of reality... if you disagree: that's not a
> problem, just assign it back to me.

Oh and the milestone needs to be changed if the patches are not in M1: I didn't want to decide on the milestone without being up to date on the state of patchset...
Comment 11 Amaranth Valluri 2017-06-22 07:34:25 UTC
Jussi, All the review comments are addressed, and still 7 patches needs to be accepted. The rebased changes are here: https://github.com/avalluri/openembedded-core/commits/usrmerge
Comment 12 Amaranth Valluri 2017-06-22 07:56:55 UTC
RP/Ross: Can you pleases suggest if we can take this for M1, i feel its bit unrealistic as we are nearing the milestone.
Comment 13 Amaranth Valluri 2017-07-18 08:58:48 UTC
All the needed patches landed in oe-core master.