10060 2016-08-02 01:07:33 +0000 Suggested documentation for fakeroot and Pseudo 2016-08-02 20:46:39 +0000 1 1 9 Documentation Reference handbook 2.2 x86 Multiple RESOLVED FIXED 02 August 2016: RESOLVED Undecided normal --- 1 ulfalizer srifenbark bluelightning poky.doc.watcher poky.watcher srifenbark --- --- Done (doc changes complete) oldest_to_newest 64769 0 ulfalizer 2016-08-02 01:07:33 +0000 Hello, Fakeroot and Pseudo currently seem to be almost completely undocumented. Here's some suggested documentation: Add a new section '4.4. Fakeroot and Pseudo' to the reference manual with the following contents: Some tasks are easier to implement when allowed to perform certain operations that are normally reserved for the root user. For example, the [do_install] task benefits from being able to set the UID and GID of installed files to arbitrary values. One approach to allowing tasks to perform root-only operations is to require BitBake to run as root, but this is cumbersome and has security issues. The approach that's actually used is to run tasks that benefit from root privileges in a "fake" root environment. Within this environment, the task and its child processes believe that they are running as the root user, and see an internally consistent view of the filesystem. As long as generating the final output (e.g. a package or an image) does not require root privileges, the fact that some earlier steps ran in a fake root environment does not cause problems. The capability to run tasks in a fake root environment is known as "fakeroot", from the BitBake keyword/flag that requests a fake root environment for a task. In modern Yocto versions, the program that implements fakeroot is known as Pseudo. Pseudo overrides system calls (through the LD_PRELOAD mechanism) to give the illusion of running as root. To keep track of "fake" file ownership and permissions resulting from operations that require root permissions, an sqlite3 database is used. This database is stored in ${WORKDIR}/pseudo/files.db for individual recipes and in ${STAGING_DIR_HOST}/var/pseudo/files.db for staging sysroots that need it. Storing the database in a file as opposed to in memory gives persistence between tasks, and even between builds. Caution { If you add your own task that manipulates the same files or directories as a fakeroot task, then that task should also run under fakeroot. Otherwise, it won't be able to run root-only operations, and won't see the fake file ownership and permissions set by the other task. You should also add a dependency on virtual/fakeroot-native:do_populate_sysroot, giving the following: fakeroot do_mytask () { ... } do_mytask[depends] += "virtual/fakeroot-native:do_populate_sysroot" } For more information, see the [FAKEROOT*](https://www.yoctoproject.org/docs/2.2/bitbake-user-manual/bitbake-user-manual.html#var-FAKEROOT) variables in the BitBake User Manual, and [this article about Pseudo](http://www.ibm.com/developerworks/opensource/library/os-aapseudo1/index.html). Rewrite the beginning of do_install (before the Caution) in the reference manual as follows: Copies files that are to be packaged into the holding area ${D}. Runs with the current working directory set to ${B}, which is the compilation directory. This task, as well as other tasks that either directly or indirectly depend on the installed files (e.g., do_package, do_package_write_*, and do_rootfs), run under [fakeroot](link to the new '4.4. Fakeroot and Pseudo' section). Add the following note to the end of the D glossary entry: Caution { Tasks that read from or write to this directory should run under [fakeroot](link to the new '4.4. Fakeroot and Pseudo' section). } At first I thought of mentioning fakeroot in the description of all tasks that use it. I think it might be too spammy though, and it's also easy to miss a task and accidentally give the impression that it doesn't use fakeroot. Cheers, Ulf 64770 1 ulfalizer 2016-08-02 01:32:50 +0000 Please remove the following part. It might be based on a misunderstanding. ...and in ${STAGING_DIR_HOST}/var/pseudo/files.db for staging sysroots that need it. Cheers, Ulf 64788 2 srifenbark 2016-08-02 16:51:02 +0000 Hi Ulf, See http://www.yoctoproject.org/docs/2.2/ref-manual/ref-manual.html#fakeroot-and-pseudo for the new section on Fakeroot and Pseudo. See http://www.yoctoproject.org/docs/2.2/ref-manual/ref-manual.html#ref-tasks-install for the changes to the do_install task. See http://www.yoctoproject.org/docs/2.2/ref-manual/ref-manual.html#var-D for the changes to the D variable in the glossary. Thanks, Scott 64791 3 ulfalizer 2016-08-02 17:24:43 +0000 (In reply to comment #2) > Hi Ulf, > > See > http://www.yoctoproject.org/docs/2.2/ref-manual/ref-manual.html#fakeroot-and- > pseudo for the new section on Fakeroot and Pseudo. The following part was meant to be part of the Caution (but I can see how the formatting was confusing). There's an extra } now too. You should also add a dependency on virtual/fakeroot-native:do_populate_sysroot, giving the following: fakeroot do_mytask () { ... } do_mytask[depends] += "virtual/fakeroot-native:do_populate_sysroot" > See > http://www.yoctoproject.org/docs/2.2/ref-manual/ref-manual.html#ref-tasks- > install for the changes to the do_install task. Repeating "this task" doesn't flow very well to me. Note that I also rewrote the second sentence in my version. No super strong opinions though. :P > See http://www.yoctoproject.org/docs/2.2/ref-manual/ref-manual.html#var-D > for the changes to the D variable in the glossary. > Looks good to me. Cheers, Ulf 64792 4 srifenbark 2016-08-02 17:59:18 +0000 (In reply to comment #3) > (In reply to comment #2) > > Hi Ulf, > > > > See > > http://www.yoctoproject.org/docs/2.2/ref-manual/ref-manual.html#fakeroot-and- > > pseudo for the new section on Fakeroot and Pseudo. > The following part was meant to be part of the Caution (but I can see how > the formatting was confusing). There's an extra } now too. Ahh... I see. Yes, I thought that last brace was part of the code and not closing off the Caution. Fixed - http://www.yoctoproject.org/docs/2.2/ref-manual/ref-manual.html#fakeroot-and-pseudo > > You should also add a dependency on > virtual/fakeroot-native:do_populate_sysroot, giving the following: > > fakeroot do_mytask () { > ... > } > do_mytask[depends] += "virtual/fakeroot-native:do_populate_sysroot" > > > See > > http://www.yoctoproject.org/docs/2.2/ref-manual/ref-manual.html#ref-tasks- > > install for the changes to the do_install task. > > Repeating "this task" doesn't flow very well to me. Note that I also rewrote > the second sentence in my version. > > No super strong opinions though. :P I actually thought that to myself as I was reading it. That is a good call. You are a good writer. Fixed. http://www.yoctoproject.org/docs/2.2/ref-manual/ref-manual.html#ref-tasks-install > > > See http://www.yoctoproject.org/docs/2.2/ref-manual/ref-manual.html#var-D > > for the changes to the D variable in the glossary. > > > > Looks good to me. > > Cheers, > Ulf 64793 5 ulfalizer 2016-08-02 18:08:14 +0000 Thanks! Looks good to me now. Cheers, Ulf 64794 6 srifenbark 2016-08-02 19:27:22 +0000 Thanks, Setting to RESOLVED and placing doc flag to "done." Scott 64796 7 bluelightning 2016-08-02 20:14:40 +0000 Some minor corrections: "One approach to allowing tasks to perform root-only operations is to require BitBake to run as root." -> "One approach to allowing tasks to perform root-only operations would be to require BitBake to run as root." (we're talking hypothetically here, since this is explicitly disallowed.) "In recent Yocto versions" -> "In current versions of the OpenEmbedded build system" (Please don't use "Yocto" to refer to the build system!) "the BitBake keyword/flag" -> "The BitBake function varflag" 64798 8 ulfalizer 2016-08-02 20:21:50 +0000 (In reply to comment #7) > Some minor corrections: > > "One approach to allowing tasks to perform root-only operations is to > require BitBake to run as root." -> "One approach to allowing tasks to > perform root-only operations would be to require BitBake to run as root." > (we're talking hypothetically here, since this is explicitly disallowed.) > > "In recent Yocto versions" -> "In current versions of the OpenEmbedded build > system" (Please don't use "Yocto" to refer to the build system!) Looks good to me. > "the BitBake keyword/flag" -> "The BitBake function varflag" I think this would confuse readers. How about "the BitBake keyword/variable flag"? Cheers, Ulf 64802 9 srifenbark 2016-08-02 20:34:06 +0000 (In reply to comment #8) > (In reply to comment #7) > > Some minor corrections: > > > > "One approach to allowing tasks to perform root-only operations is to > > require BitBake to run as root." -> "One approach to allowing tasks to > > perform root-only operations would be to require BitBake to run as root." > > (we're talking hypothetically here, since this is explicitly disallowed.) I try to avoid future tense if possible. However, I see your point in this case. > > > > "In recent Yocto versions" -> "In current versions of the OpenEmbedded build > > system" (Please don't use "Yocto" to refer to the build system!) Since the original specified "Yocto versions", I assumed the release in general. I do refer to the build system as the OpenEmbedded build system throughout the manual set. > > Looks good to me. > > > "the BitBake keyword/flag" -> "The BitBake function varflag" > > I think this would confuse readers. How about "the BitBake keyword/variable > flag"? Fixed > > Cheers, > Ulf See - http://www.yoctoproject.org/docs/2.2/ref-manual/ref-manual.html#fakeroot-and-pseudo 64807 10 bluelightning 2016-08-02 20:46:39 +0000 "varflag" is an official term we use in a number of other places, however I do note that in other places in the manual we say "variable flag (varflag)" so that it's clear what we mean. I won't insist upon using that here though, it doesn't matter all that much.