10141 2016-08-15 14:38:31 +0000 Suggested fleshing out of the sigdata/siginfo documentation 2016-08-22 23:02:29 +0000 1 1 9 Documentation Reference handbook 2.2 x86 Multiple RESOLVED FIXED 22 August 2016: RESOLVED Medium normal 2.2 M4 1 ulfalizer srifenbark bluelightning poky.doc.watcher poky.watcher sgw srifenbark --- --- Done (doc changes complete) oldest_to_newest 65190 0 ulfalizer 2016-08-15 14:38:31 +0000 Hello, BitBake includes a mechanism for viewing the input signature (variable dependencies, including automatically added ones) of a task, but it's hard to discover and eluded me for the longest time (hence ugly workarounds like https://lists.yoctoproject.org/pipermail/yocto/2016-April/029756.html). Here's some suggested fleshing out/rearranging of the documentation to make things easier to discover and understand: * Rename '2.3. Debugging Build Failures' to '2.3. Debugging Tools and Techniques' (Unrelated, but I think it makes sense as this section is about more general debugging than just dealing with build failures.) * Rename '2.3.3. Viewing Dependencies' to '2.3.3. Viewing Dependencies Between Recipes and Tasks' * Add a new section '2.3.4. Viewing Task Variable Dependencies' with the following contents: As mentioned in the [Checksums (Signatures)](https://www.yoctoproject.org/docs/2.2/bitbake-user-manual/bitbake-user-manual.html#checksums) section of the BitBake User Manual, BitBake tries to automatically determine what variables a task depends on so that it can rerun the task if the value of any of the variables changes. This is usually reliable, but if you do things like construct variable names at runtime, then you might have to manually declare dependencies on those variables (using the [[vardeps]](https://www.yoctoproject.org/docs/2.2/bitbake-user-manual/bitbake-user-manual.html#variable-flags) variable flag). If you are unsure whether a variable dependency is being picked up automatically for a given task, you can list the variable dependencies BitBake has determined as follows: 1. Build the recipe containing the task with $ bitbake recipename 2. Inside the [STAMPS_DIR] directory, find the sigdata file corresponding to the task. sigdata files contain a pickled Python database of all the metadata that went into creating the input checksum for the task. As an example, for the [do_fetch] task of the db recipe, the sigdata file might be found in the following location: ${BUILDDIR}/tmp/stamps/i586-poky-linux/db/6.0.30-r1.do_fetch.sigdata.7c048c18222b16ff0bcee2000ef648b1 For tasks that are accelerated through the shared state ([sstate](http://www.yoctoproject.org/docs/2.2/ref-manual/ref-manual.html#shared-state-cache)) cache, an additional siginfo file is written into [SSTATE_DIR] along with the cached task output. siginfo files contain exactly the same information as sigdata files. 3. Run 'bitbake-dumpsigs' on the sigdata or siginfo file: $ bitbake-dumpsigs ${BUILDDIR}/tmp/stamps/i586-poky-linux/db/6.0.30-r1.do_fetch.sigdata.7c048c18222b16ff0bcee2000ef648b1 In the output of the above command, you will find a line like the following, which lists all the (inferred) variable dependencies for the task. This list also includes indirect dependencies from variables depending on other variables, recursively. Task dependencies: ['PV', 'SRCREV', 'SRC_URI', 'SRC_URI[md5sum]', 'SRC_URI[sha256sum]', 'base_do_fetch'] Note that functions (e.g. base_do_fetch) also count as variable dependencies. They in turn depend on the variables they reference. The output of bitbake-dumpsigs also includes the value each variable had, a list of dependencies for each variable, and [variable whitelisting](https://www.yoctoproject.org/docs/2.2/bitbake-user-manual/bitbake-user-manual.html#var-BB_HASHBASE_WHITELIST) information. There is also a 'bitbake-diffsigs' command for comparing two siginfo/sigdata files. This can be helpful when trying to figure out what changed between two versions of a task. If 'bitbake-diffsigs' is called with just one file, it behaves like 'bitbake-dumpsigs'. You can also use BitBake to dump out the signature construction information without executing tasks by using either of the following BitBake command-line options: ‐‐dump-signatures=SIGNATURE_HANDLER -S SIGNATURE_HANDLER Note { Two common values for SIGNATURE_HANDLER are "none" and "printdiff" to only dump the signature or to compare the dumped signature with the cached one, respectively. } Using BitBake with either of these options causes BitBake to dump out .sigdata files in the stamp directory for every task it would have executed instead of building the specified target package. * Replace the contents of '4.3.4.1. Debugging' with the following: Seeing what metadata went into creating the input signature of an sstate task can be a useful debugging aid. This information is available in siginfo files in [SSTATE_DIR]. See [2.3.4. Viewing Task Variable Dependencies] for how to view and interpret the information stored in these files. '4.3.4.1. Debugging' was the only section to have sigdata/siginfo information before. Most of the information applies to all tasks and not just to sstate tasks though, so it makes sense to move it. The --dump-signatures stuff is copied verbatim from the old version. I actually couldn't get --dump-signatures to work as (I think) intended. It might be buggy. Cheers, Ulf 65191 1 ulfalizer 2016-08-15 14:43:49 +0000 One small semi-related nit: STAMPS_DIR is misspelled as STAMP_DIR in 2.8. Checksums (Signatures) in the BitBake User Manual. Cheers, Ulf 65411 2 srifenbark 2016-08-22 20:17:36 +0000 Ulf, See http://www.yoctoproject.org/docs/2.2/ref-manual/ref-manual.html#usingpoky-debugging-tools-and-techniques for the title change for 2.3. See http://www.yoctoproject.org/docs/2.2/ref-manual/ref-manual.html#usingpoky-viewing-dependencies-between-recipes-and-tasks for the title change of 2.3.3. See http://www.yoctoproject.org/docs/2.2/ref-manual/ref-manual.html#usingpoky-viewing-task-variable-dependencies for the new 2.3.4 section. See http://www.yoctoproject.org/docs/2.2/ref-manual/ref-manual.html#debugging for the updated "Debugging" section. See http://www.yoctoproject.org/docs/2.2/bitbake-user-manual/bitbake-user-manual.html#checksums for the STAMPS_DIR correction in the BB manual. Thanks, Scott 65414 3 ulfalizer 2016-08-22 20:40:43 +0000 Looks good to me! Thanks! Ulf 65415 4 ulfalizer 2016-08-22 20:44:07 +0000 Hmm... looks like the BitBake User Manual still says "STAMP_DIR" rather than "STAMPS_DIR" though. 65418 5 ulfalizer 2016-08-22 21:06:30 +0000 Found some typos in my original version as well: bitbake-dumpsigs should be bitbake-dumpsig (everywhere). Cheers, Ulf 65424 6 srifenbark 2016-08-22 22:30:30 +0000 Ulf, Fixed the typo's (4 spots). Regarding the BB manual. I forgot to push the correct HTML file. See http://www.yoctoproject.org/docs/2.2/bitbake-user-manual/bitbake-user-manual.html#checksums for the fix. Thanks, Scott 65426 7 ulfalizer 2016-08-22 22:58:43 +0000 Looks good to me. Thanks. 65429 8 srifenbark 2016-08-22 23:02:29 +0000 Thanks, Setting to RESOLVED and putting the doc flag to "done." Scott