Bug 9064

Summary: do_unpack does not wipe old source tree
Product: [Build System, Metadata & Runtime] BitBake Reporter: Ross Burton <ross.burton>
Component: bitbakeAssignee: Ross Burton <ross.burton>
Status: RESOLVED FIXED QA Contact: Libertad <libertad.gonzalez.de.la.cruz>
Severity: major    
Priority: Medium+ CC: jose.perez.carranza, markus.lehtonen, poky.bs.watcher, poky.watcher, sgw
Version: 2.0   
Target Milestone: 2.2 M3   
Hardware: All   
OS: Multiple   
Whiteboard:
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: No (bug/feature does not impact docs)

Description Ross Burton 2016-02-04 22:31:10 UTC 
In the past I could do bitbake -C unpack [recipe] to force a recipe to rebuild from scratch, because the unpack task will be tainted and thus the unpack runs again.

It appears that now this is broken and if I do bitbake -C unpack it skips unpack and patch, going straight to configure.

(I'm sure this worked a few weeks ago, so a bisect shouldn't be too long)
Comment 1 Ed Bartosh 2016-02-07 19:48:10 UTC 
Works for me just fine on master@4a2f42ff9e0f4359d1ee8290582f1b29a2202372:

build (master)> bitbake -C unpack zlib -D |grep ' Executing task do_unpack'
DEBUG: Executing task do_unpack
Comment 2 Ross Burton 2016-02-08 10:49:14 UTC 
I'm *certain* I saw this not working as it was causing problems.  Of course now I can't replicate.  I'll see if I can make it break.
Comment 3 Ross Burton 2016-02-15 17:08:02 UTC 
Managed to replicate this.

1) bitbake foo
2) touch a file in foo's ${S}
3) bitbake foo -C unpack

The file that was created in (2) still exists but (3) should have removed the source tree before unpacking again.
Comment 4 Markus Lehtonen 2016-02-19 15:37:48 UTC 
To me it seems bitbake works as expected. In most cases do_unpack just calls base_do_unpack(), which in turn, just extracts the source into ${S}. Not removing anything. So if you change something in ${S} it will be "reset" back to the expected orig state, but, if you create new files they are not removed.

What recipe did you try this with?
Comment 5 Ross Burton 2016-02-19 15:51:30 UTC 
My standard test is m4.  I'm *sure* it used to erase before unpacking.  If it didn't really, then maybe it should?
Comment 6 Markus Lehtonen 2016-02-20 07:56:31 UTC 
I just tested with m4 on two random commits in poky:
commit 23083e7e5e3a8ee81b346b2ee6084d9fc83e66b5
Date:   Thu Dec 3 16:02:27 2015 -0600

commit 87c4e58dda0e2f62b0c8e4ddc6bec244643b21af
Date:   Thu Mar 19 16:05:03 2015 +0000

It didn't cleanup S{S}. So I don't think it has ever done that (unless there was some short period somewhere where it did that).

But, I agree that it probably should clean it up. It would sound more consistent if the build would always operate on a pristine source tree.
Comment 8 Markus Lehtonen 2016-03-22 07:25:49 UTC 
Moving to 2.1M4.

I re-sent the patch:
http://lists.openembedded.org/pipermail/openembedded-core/2016-March/119295.html
Comment 9 Ross Burton 2016-07-13 23:58:41 UTC 
This was implicated in causing devtool selftest failures on the autobuilder:

https://autobuilder.yoctoproject.org/main/builders/nightly-oe-selftest/builds/582/steps/Running%20oe-selftest/logs/stdio
Comment 10 Markus Lehtonen 2016-08-02 07:23:21 UTC 
Ah, it breaks usage of externalsrc :( What do you think about modifying externalsrc.bbclass so that it won't wipe ${S}
Comment 11 Markus Lehtonen 2016-08-03 08:59:35 UTC 
Version 3 of the patch:
https://patchwork.openembedded.org/patch/128383/
Comment 12 Ross Burton 2016-08-11 16:00:10 UTC 
Merged in oe-core eccae514b71394ffaed8fc45dea7942152a334a1.  Moving milestone as this missed M2.