Bug 13832

Summary: Add script to setup buildtools-extended-tarball
Product: [Build System, Metadata & Runtime] OE-Core Reporter: Tim Orling <tim.orling>
Component: Scripts and ToolsAssignee: Tim Orling <tim.orling>
Status: RESOLVED FIXED QA Contact:
Severity: normal    
Priority: High CC: kai.kang, randy.macleod, richard.purdie, ross.burton
Version: 3.1   
Target Milestone: 3.1 M4   
Hardware: x86   
OS: Multiple   
Whiteboard:
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: Don't know

Description Tim Orling 2020-03-12 15:42:38 UTC
For distros where buildtools-extended-tarball is required, such as CentOS-7 (because the default buildtools are too old), we need a script to make it straightforward for users to install. While it is tempting to have this be installed by bitbake, as is done with uninative, the consensus is that it must already be present before the oe-init-buildenv script is run. Hence the need for a standalone script.
Comment 1 Tim Orling 2020-03-12 15:47:57 UTC
Reference to original buildtools-extended-tarball bug:
https://bugzilla.yoctoproject.org/show_bug.cgi?id=13714
Comment 2 Tim Orling 2020-03-26 04:30:31 UTC
Patch submitted:
https://patchwork.openembedded.org/patch/171323/

Tested for the example usage in the commit log (and in the script itself) on CentOS-7 and Fedora-29
Comment 3 Tim Orling 2020-03-26 04:38:44 UTC
Also on poky-contrib:
http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=timo/install-buildtools_13832

NOTE:
Because os.system or subprocess calls run in a new session, it does not appear possible to have this script run the environment setup script in the shell session in which it was called. The user will still have to source the environment script to make the buildtools enabled.
Comment 4 Tim Orling 2020-03-26 14:55:55 UTC
The default for the installer is to install in /opt/poky/<installer-version>, this is not desired since it requires the user to put in their sudo password. Instead, like what is done on the AutoBuilder, install in a "buildtools" directory next to the "scripts" directory by default (users can set directory with the -d/--directory argument).

The error in sanity checker that says "your version of tar is too old" (paraphrasing) should be updated to recommend running the install-buildtools script.

Typo in the first patch in the example usage (--install-version should be --installer-version) and also the commit message.
Comment 5 Tim Orling 2020-03-26 15:12:44 UTC
The "buildtools" directory that is going to be used/created by default needs to be added to .gitignore
Comment 6 Tim Orling 2020-03-26 15:17:02 UTC
./meta/classes/sanity.bbclass:        return "Your version of tar is older than 1.28 and does not have the support needed to enable reproducible builds. Please install a newer version of tar (you could use the projects buildtools-tarball from our last release).\n"
Comment 7 Tim Orling 2020-03-26 15:25:28 UTC
message in sanity.bbclass if git is too old or tar is too old

add something to build-environnment checks earlier, where it checks if the python version is too old (<3.5)

gcc > 5.0, RP thinks there is a check somewhere the version just needs to be bumped. Look in the history of sanity.bbclass.
Comment 8 Tim Orling 2020-03-30 19:51:44 UTC
The original patch has been merged, but the additional changes discussed above are now submitted:
https://patchwork.openembedded.org/series/23493/

Also available on openembedded-core-contrib:
https://git.openembedded.org/openembedded-core-contrib/log/?h=timo/install-buildtools_13832_fixes
Comment 9 Tim Orling 2020-03-31 20:06:29 UTC
v2 patches submitted (addressing review comments, fixes, and bumping to M3 release by default):
https://patchwork.openembedded.org/series/23509/

Also available on openembedded-core-contrib:
https://git.openembedded.org/openembedded-core-contrib/log/?h=timo/install-buildtools_13832_fixes_v2
Comment 10 Tim Orling 2020-04-03 23:16:53 UTC
Thanks to testing by and input from Andre McCurdy, we have refactored the script to work with Python 3.4+

Patch submitted:
https://patchwork.openembedded.org/patch/171593/

Also available on openembedded-core-contrib:
https://git.openembedded.org/openembedded-core-contrib/log/?h=timo/install-buildtools_py34
Comment 11 Tim Orling 2020-04-08 17:55:02 UTC
merged to dunfell/3.1.rc1. If there are any new issues, please file new bugs.