Bug 11310

cat /etc/issue
Ubuntu 16.04 LTS \n \l

Hi, 

I recently worked Bug 10854 and worked with Brian Avery on this.  I am going to copy him on the bug.  This package requirement stuff for the hosts seems to be in constant flux... depending on various distro versions etc.  

I would like for Brian to look at this bug and your comments before I change stuff.

Thanks, 
Scott

yipes. either I messed up (possible) or things moved.
The broken pkgs should be:

 python3-pexpect
 debianutils
 xz-utils

The broken versions were:
python3-expect
which
xz

------
tested on official ubuntu docker contianer ubuntu:16.04 
# cat /etc/issue
Ubuntu Xenial Xerus (development branch) \n \l

Brian, 

I don't know which distros this correction applies to (Ubuntu....all?)  So, I am cutting and pasting the four "Essentials" packages that are listed right now in the YP Reference Manual.  It would really help me clear this up if you could edit this comment with some substitutions for each of these distros.

Please tell me exactly how each of these should be.

Thanks,
Scott

================================================


This is for Ubuntu (see http://www.yoctoproject.org/docs/2.3/ref-manual/ref-manual.html#ubuntu-packages)

Essentials: Packages needed to build an image on a headless system:

         $ sudo apt-get install gawk wget git-core diffstat unzip texinfo \
         gcc-multilib build-essential chrpath socat cpio python python3 \
         python3-pip python3-expect xz which


================================================


This is for Fedora (see http://www.yoctoproject.org/docs/2.3/ref-manual/ref-manual.html#fedora-packages)

Essentials: Packages needed to build an image for a headless system:

     $ sudo dnf install gawk make wget tar bzip2 gzip python3 unzip perl \
       patch diffutils diffstat git cpp gcc gcc-c++ glibc-devel texinfo \
       chrpath ccache perl-Data-Dumper perl-Text-ParseWords \
       perl-Thread-Queue perl-bignum socat findutils which file cpio \
       python python3-pip python3-expect xz which


================================================



This is for OpenSUSE (see http://www.yoctoproject.org/docs/2.3/ref-manual/ref-manual.html#opensuse-packages)

Essentials: Packages needed to build an image for a headless system:

     $ sudo zypper install python gcc gcc-c++ git chrpath make wget \
       python-xml diffstat makeinfo python-curses patch socat python3 \
       python3-curses tar python3-pip python3-expect xz which


===============================================



This is for CentOS (see http://www.yoctoproject.org/docs/2.3/ref-manual/ref-manual.html#centos-packages)

Essentials: Packages needed to build an image for a headless system:

     $ sudo yum install -y epel-release
     $ sudo yum makecache
     $ sudo yum install gawk make wget tar bzip2 gzip python unzip perl \
     patch diffutils diffstat git cpp gcc gcc-c++ glibc-devel texinfo \
     chrpath socat perl-Data-Dumper perl-Text-ParseWords perl-Thread-Queue \
     python34-pip xz which SDL-devel xterm

I can confirm that for Ubuntu 16 those install:

$ sudo apt-get install gawk wget git-core diffstat unzip texinfo \
         gcc-multilib build-essential chrpath socat cpio python python3 \
         python3-pip python3-pexpect xz-utils debianutils

instead of 

$ sudo apt-get install gawk wget git-core diffstat unzip texinfo \
         gcc-multilib build-essential chrpath socat cpio python python3 \
         python3-pip python3-expect xz which

BTW in order to get bitbake core-image-minimal -g -u depexp to work with Morty on Ubuntu 16 I had to add:

sudo apt-get install -y libgtk-3-0 libgtk-3-bin libgtk-3-common libgtk-3-dev python3-gi

yep, the "essentials" are just to make images.

Brian, 

What about Comment 4?  Can you give me some feedback on those lists to be sure I am changing the right packages?

Also, for Robert's Comment 5... those packages do not appear anywhere.  What distros and groupings would those apply to?

thanks,
Scott

Rebecca, could you ask someone on the team to go through each of these and verify?

Easiest approach:
run the std distro container for each of the os's scott has below

do the apt-get/dnf/zypper/yum install that scott lists. - make sure they all install.

$useradd -m tester

$su - tester
$ git clone git://git.yoctoproject.org/poky
$ ./poky/oe-init-buildenv
$ bitbake core-image-minimal

- check that it works.
testimage still has issues inside a container, so don't worry about that right now

ty!!

Scott, my Comment 3 correction was from a quick test on the ubuntu-16.04 container. I asked Rebecca's team for help to verify the matrix.

PG team will take over the verification. I will post the progress over here after the checking.

We have ran tests for all 4 distros, python3-expect was not found in all the package management repos.

Without installing python3-expect, bitbake image build still can be executed flawlessly. So python3-expect should not be inside the essential list since it is not discoverable and not needed.

I believe pexpect was added because the code to support testimage on real hardware uses it in order to interact with the bootloader. That isn't functionality that everyone will use, but it does still exist.

Ok - I am going to remove "python3-expect" from the list of the essentials for Ubuntu, Fedora, and OpenSUSE.  Leaving all else alone unless I hear further. 

Scott

Hi, 

See http://www.yoctoproject.org/docs/2.3/ref-manual/ref-manual.html#required-packages-for-the-host-development-system in the ref-manual and http://www.yoctoproject.org/docs/2.3/yocto-project-qs/yocto-project-qs.html#packages in the QS for the latest rendition of this stuff. 

Thanks, 
Scott

Reviewed doc changes.

Hi, 

Since the docs have been reviewed with no comments, I am assuming the changes and the state of the packages are good.  I am marking the bug as RESOLVED and setting the doc flag to "done".

Scott

Sadly nope.
Rebecca, 
apt-get install python3-pexpect works on ubuntu 16.04, for example. And dnf install python3-pexpect works on fedora:24. The names change periodically, which is what is causing this to be soo frustrating.  

As Paul mentioned, python expect is being used to talk to the real hardware in testing.

At the top of the Poky tree, if you do 
$ grep -rH expect . | egrep import 
you will see:
./meta-yocto-bsp/lib/oeqa/controllers/grubtarget.py:import pexpect
./meta-yocto-bsp/lib/oeqa/controllers/edgeroutertarget.py:import pexpect
./meta-yocto-bsp/lib/oeqa/controllers/beaglebonetarget.py:import pexpect

Ok - thanks for reopening the bug.  I will need some instruction on what we want for packages and for what distributions they apply. 

Scott

Brian, 

I am changing the owner to you because I NEEDINFO for this bug.  Need someone to give me the definitive packages for the four distros.  

Thanks, 
Scott

adding saul, 
Rebecca did you all have a chance to follow up on this?

(In reply to comment #21)
> adding saul, 
> Rebecca did you all have a chance to follow up on this?

I will follow up on this. I have just synced up with Saul, PG team will take the next step on checking "python3-pexpect" and find out its replacement on all 4 distros and report out.

For Ubuntu 16.04, OpenSUSE, Fedora:
python3-pexpect is working. The broken pkg was python3-expect.

CentOS:
We are able to install pexpect package with yum. python3-pexpect was not found for CentOS.

Created attachment 3741 [details]
script to test install for different distros

I ended up writing a script to test installs (attached)
Here;s the results:
DEBIAN_HOST_PACKAGES_ESSENTIAL="gawk wget git-core diffstat unzip texinfo gcc-multilib \                                                                                          
     build-essential chrpath socat cpio python python3 python3-pip python3-pexpect\                                                                                               
     xz-utils debianutils"                                            |
UBUNTU_HOST_PACKAGES_ESSENTIAL="gawk wget git-core diffstat unzip texinfo gcc-multilib \                                                                                          
     build-essential chrpath socat cpio python python3 python3-pip python3-pexpect\                                                                                               
     xz-utils debianutils"                                            |
FEDORA_HOST_PACKAGES_ESSENTIAL="gawk make wget tar bzip2 gzip python3 unzip perl patch \                                                                                          
     diffutils diffstat git cpp gcc gcc-c++ glibc-devel texinfo chrpath \                                                                                                         
     ccache perl-Data-Dumper perl-Text-ParseWords perl-Thread-Queue perl-bignum socat python3-pexpect \                                                                           
     findutils which file cpio python python3-pip xz which"           |
OPENSUSE_HOST_PACKAGES_ESSENTIAL="python gcc gcc-c++ git chrpath make wget python-xml \                                                                                           
     diffstat makeinfo python-curses patch socat python3 python3-curses tar python3-pip python3-pexpect \                                                                         
     xz which"                                                        |
CENTOS_HOST_PACKAGES_ESSENTIAL="gawk make wget tar bzip2 gzip python unzip perl patch \                                                                                           
     diffutils diffstat git cpp gcc gcc-c++ glibc-devel texinfo chrpath socat \                                                                                                   
     perl-Data-Dumper perl-Text-ParseWords perl-Thread-Queue python34-pip xz \                                                                                                    
     which"                                                           |
-----
I also see python3 expect not being available in centos-7 . didn't test building any images as that is more complex to set up...
--
Reassigning back to scott. is ok?

Hi,

Thanks for the effort testing these out. See http://www.yoctoproject.org/docs/2.3/ref-manual/ref-manual.html#required-packages-for-the-host-development-system for the link into the ref-manual.  Essentially, they stayed the same except for adding "python3-pexpect" into Ubuntu/Debian, Fedora, and OpenSUSE.

See http://www.yoctoproject.org/docs/2.3/yocto-project-qs/yocto-project-qs.html#packages for the same in the QS Manual. 

I am setting the bug to IN PROGRESS REVIEW again.

Thanks, 
Scott

Just started testing on the pyro branch and the first error I get is:

ERROR: The following required tools (as specified by HOSTTOOLS) appear to be unavailable in PATH, please install them in order to proceed:
  ping

----

Package ping is a virtual package provided by:
  inetutils-ping 2:1.9.4-1build1
  iputils-ping 3:20121221-5ubuntu2
You should explicitly select one to install.

-----

sudo apt insatll iputils-ping 

fixes the problem on Ubuntu 16.x

sudo apt install iputils-ping

Hi Robert, 

See http://www.yoctoproject.org/docs/2.3/ref-manual/ref-manual.html#ubuntu-packages for the change for Ubuntu/Debian in the ref-manual.  See http://www.yoctoproject.org/docs/2.3/yocto-project-qs/yocto-project-qs.html#packages for the same change in the YP Quick Start. 

Let me know of any other failing stuff regarding packages for Pyro.

Thanks, 
Scott

Hi Scott,

I am happy with what's in your latest links.

Thanks!

Regards,

Robert

Thanks Robert.

Setting to RESOLVED and putting the doc flag to "done."

Scott