Bug 6422

Summary: Add support for locked shared state
Product: [Build System, Metadata & Runtime] BitBake Reporter: Paul Eggleton <bluelightning>
Component: bitbakeAssignee: Richard Purdie <richard.purdie>
Status: VERIFIED FIXED QA Contact: Costin <costin.c.constantin>
Severity: enhancement    
Priority: Medium+ CC: alexandru.c.georgescu, hongxu.jia, mark.hatle, poky.bs.watcher, poky.watcher, raj.khem
Version: 1.7   
Target Milestone: 1.7 M3   
Hardware: All   
OS: Multiple   
Whiteboard: TC needed
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: Yes (doc changes required)
Bug Depends on:    
Bug Blocks: 6649, 6650, 6662    

Description Paul Eggleton 2014-06-11 09:05:01 UTC
We have had patches floating around for locked shared state support (i.e. locking the signature to a particular value for a task), but they haven't been integrated yet. 

Adding this functionality does carry some risk - in particular, if at all possible we'd want to be able to tell when something that we should have rebuilt won't be rebuilt; e.g. if a patch has been added to a recipe. It's not clear to me how we will practically do this, but support could be a problem if we think an issue should have been fixed and the fix hasn't taken effect because it was locked out.
Comment 1 Khem Raj 2014-08-08 02:56:06 UTC
Can someone add the links to patches that are related to this bug ?
Comment 2 Paul Eggleton 2014-08-08 15:01:59 UTC
I thought Richard had a branch for these somewhere but I couldn't find it. Here's the original discussion:

http://patchwork.openembedded.org/patch/62707/

Plus, a little more up-to-date than that, here is a rebased branch with Richard's patches that I have been able to find (against poky though - OE-Core and bitbake changes are intermingled at the moment so that will need to be split apart before we can go further with this at the bare minimum):

http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=paule/lockedsigs
Comment 3 Paul Eggleton 2014-08-22 14:37:00 UTC
I've retested this patchset now with master, and it still works - you can for example set the signatures up so that do_rootfs for an image can be executed alone with the rest of the tasks from sstate even if variable values get changed.

The things still blocking this from being merged:

* We need a means of restricting the targets built so that an error occurs if the build system thinks it needs to execute something outside of a whitelist despite the locked signatures. This can happen for example if a required sstate archive is missing.

* We need to report that locked signatures are in use when the build starts. (It's probably enough to just report that some have been set, a full list would be excessive.)

I've also entered a couple of other related but separate bugs and marked them dependent upon this one.
Comment 4 Hongxu Jia 2014-09-12 05:58:04 UTC
Hi Paul,

I have some patches about locked share state

http://lists.openembedded.org/pipermail/openembedded-core/2014-September/097034.html


//Hongxu
Comment 6 Costin 2014-10-24 07:02:52 UTC
Hello,

this freature is verified.

Simple steps to verify it:

1) build a package (eg. m4) 
2) go to tmp/stamps ...(details for machine used)... /m4 and observe/write down the m4 tasks related stamps
3) modify m4 recipe by adding to YOUR_YOCTO_DIR/meta/recipes-devtools/m4/m4_1.4.17.bb : 
###################################################
do_package_append() {
        bb.build.exec_func('do_test_prserv', d)
}

do_test_prserv() {
        echo "unique text here like abc"
}
##################################################

4) run:   bitbake -S none machine_type_set_in_local.conf
eg: for qemuarm:   bitbake -S none qemuarm
 
... and a locked-sigs.inc file is created. 

5) include locked-sigs.inc in local.conf by adding at the top of the file:

include full_path_to_locked-sigs.inc

eg.: /home/autobuilder/2poky/build/locked-sigs.inc

6) rebuild m4 and observe stamps present in tmp/stamps. They should coincide with stamps from locked-sigs.inc.

eg.: (present in locked-sigs.inc)
...
    m4:do_compile:c51a1533e7864e745f2f908dba00d4c3 \
    m4:do_configure:e06bf4fad2dfb6bd18491a605b55bd3a \
    m4:do_fetch:38cd097db1ba645412b1bb858c2993ae \
    m4:do_install:c7f5f53087b47d840fe868966c2f6785 \
    m4:do_package:a404cac4e4b675b59da2560bea0d805e \
    m4:do_package_write_rpm:6c9ab4e3da38d010fa20c4be72517d29 \
    m4:do_packagedata:a9bb53cfc3a679e09d6fb5002f9001aa \
    m4:do_patch:2932ecd450aaa9e97aa37a5a388c770f \
    m4:do_unpack:68e44cd283d4e828a1eb1f23730edf15 \
...


I'll mark the feature as verified.
 
Regards,

Costin