Bug 10909

Summary: build-appliance-image requires pip3 but it isn't installed everywhere
Product: [Build System, Metadata & Runtime] OE-Core Reporter: Armin Kuster <akuster>
Component: virtual machinesAssignee: Scott Rifenbark <srifenbark>
Status: RESOLVED FIXED QA Contact:
Severity: normal    
Priority: Medium+ CC: meta.mr.watcher, meta.watcher, mhalstead, richard.purdie, ross.burton
Version: unspecified   
Target Milestone: 2.3 M2   
Hardware: x86   
OS: Multiple   
Whiteboard: 25 Jan 2017: RESOLVED
OS type for building Yocto: --- Type of Regression: Regression (Used to work)
Verified: Documentation change: Done (doc changes complete)

Description Armin Kuster 2017-01-11 02:55:51 UTC
| NOTE: Executing do_populate_poky_src ...
| DEBUG: Executing shell function do_populate_poky_src
| /home/pokybuild/yocto-autobuilder/yocto-worker/build-appliance/build/build/tmp/work/qemux86_64-poky-linux/build-appliance-image/15.0.0-r0/temp/run.do_populate_poky_src.17845: 173: /home/pokybuild/yocto-autobuilder/yocto-worker/build-appliance/build/build/tmp/work/qemux86_64-poky-linux/build-appliance-image/15.0.0-r0/temp/run.do_populate_poky_src.17845: pip3: not found
| WARNING: exit code 127 from a shell command.
Comment 1 Ross Burton 2017-01-11 12:35:48 UTC
Reassigning to Juro as he broke it.  CCing RP for his opinion as I've never looked at this before.

Do we need to document pip3 as a requirement?  Should this task be using python3native so it uses the Python3 we built instead of the host python?
Comment 2 Juro Bystricky 2017-01-11 18:20:41 UTC
Yes, I suppose I broke it.
Was never sure pip3 is part of Linux distros, was hoping for it.
Anyway, I also tried to use the native pip3 
AFAIK, all that is needed should be something like

EXTRANATIVEPATH += "python3-pip-native"

but this lead to various build errors, so I placed my bet on pip3 being on the host machine.

Having said this, it is probably worth implementing.
Another alternative is NOT having the toaster requirements pre-installed,
and have user manually type in:

$ pip3 install --user -r /home/builder/poky/bitbake/toaster-requirements.txt
Comment 3 Juro Bystricky 2017-01-12 00:26:09 UTC
But if we are going to simply update build requirements to include pip3,
please also see https://bugzilla.yoctoproject.org/show_bug.cgi?id=10889
Comment 4 Juro Bystricky 2017-01-12 23:21:25 UTC
So I believe the consensus was to include "pip3" among requirements.
Hence, punting to Scott as the this requires to change the documentation.
Comment 5 Ross Burton 2017-01-12 23:27:57 UTC
And Michael will need to ensure it's on all the builders.  Michael: can you ensure that pip3 is on all the builders?  So far it appears to be missing on ubuntu1604.
Comment 6 Michael Halstead 2017-01-13 01:18:35 UTC
I've installed pip3 on all the build workers. I used package management where is wasn't yet installed so the versions are not the same. Is this desired or do we need the latest pip3 available?

Versions include:

ubuntu1404 /usr/local/bin/pip3 pip 8.1.2 from /usr/local/lib/python3.4/dist-packages (python 3.4)
fedora24 /usr/bin/pip3 pip 8.0.2 from /usr/lib/python3.5/site-packages (python 3.5)
debian8 /usr/local/bin/pip3 pip 8.1.2 from /usr/local/lib/python3.4/dist-packages (python 3.4)
opensuse132 /usr/bin/pip3 pip 1.5.6 from /usr/lib/python3.4/site-packages (python 3.4)
centos7 /usr/bin/pip3 pip 8.1.0 from /usr/lib/python3.4/site-packages (python 3.4)
ubuntu1604 /usr/bin/pip3 pip 8.1.1 from /usr/lib/python3/dist-packages (python 3.5)

debian-testing /usr/bin/pip3 pip 9.0.1 from /usr/lib/python3/dist-packages (python 3.5)
debian8.io /usr/bin/pip3 pip 1.5.6 from /usr/lib/python3/dist-packages (python 3.4)
Comment 7 Ross Burton 2017-01-13 08:58:52 UTC
Matching the host python is good, so that's great thanks Michael.  Shifting back to ACCEPTED as the final it of this is documentation.
Comment 8 Scott Rifenbark 2017-01-20 00:10:59 UTC
Hi, 

I will add the "pip3" package to the "Essentials" grouping for all supported distros.  See http://www.yoctoproject.org/docs/2.3/ref-manual/ref-manual.html#required-packages-for-the-host-development-system for the host package installation requirements section in the ref-manual.  Note, the implementation is through variables - thus, the YP Quick Start will have the same changes.

Let me know if things are cool and I will RESOLVE the bug and set the doc flag to "done."

Thanks,
Scott
Comment 9 Scott Rifenbark 2017-01-25 18:31:07 UTC
Hi, 

The "pip3" package is added as an essential package for all supported distributions.  I am marking this issue RESOLVED and setting the doc flag to "done."

Scott