Bug 1732

Summary: [Build Appliance] Build core-image-minimal on sugarbay using Self Hosted Image
Product: [Build System, Metadata & Runtime] Meta-yocto Reporter: Edwin Zhai <edwin.zhai>
Component: meta-yoctoAssignee: Dexuan Cui <dexuan.cui>
Status: RESOLVED FIXED QA Contact:
Severity: enhancement    
Priority: Low CC: dexuan.cui, edwin.zhai, jiajun.xu, poky.bs.watcher, poky.watcher, sgw, shane.wang
Version: unspecified   
Target Milestone: 1.2 M1   
Hardware: x86   
OS: Multiple   
Whiteboard: (P1) (QA Testing Completed) Development (100% completed) : core-image-minimal can build on sugarbay
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: ---

Description Edwin Zhai 2011-11-04 03:26:07 UTC
build an qemu image, in which "bitbake core-image-minimal" work.

e.g. on x86-64 host, we build a image for L1 target(qemux86). Inside of L1 target, we build one core-image-minimal for L2 target(say qemuarm).
Comment 1 Edwin Zhai 2011-11-04 03:27:09 UTC
Saul did initial works and populate one layer in one contrib branch to start working. We are trying to reproduce it.
Comment 2 Dexuan Cui 2011-11-14 23:33:21 UTC
(In reply to comment #1)
> Saul did initial works and populate one layer in one contrib branch to start
> working. We are trying to reproduce it.

I found n450 is pretty slow so later I changed to use sugarbay.

Based on Saul's base work, with many fixings and workarounds, "bitbake core-image-minimal" can build fine.

The fixings and workarounds are:
1) fix the mis-built/install python module grp;

2) disable buuilding qemu-native since libGL.so, libGLU.so, libSDL.so are missing. Need to add the bits.

3) eglibc_initial do_install failure: I suspect this is related to the incompatibility between eglibc's build scripts and make-3.82 -- after I downgraded to make-3.81, the do_install issus disappeared. I'll continue to investigate this.

4) eglibc do_configure ERROR: no rpcgen found I had to install eglibc-utils_2.13-r17+svnr14157_x86_64.ipk into the target to resolve this issue.
	
5) eglibc: do_install ERROR: cannot stat bootparam_prot.h  The cause is: rpcgen doesn't work properly --> /lib/cpp doesn't exist: we need to ln -s /usr/bin/cpp /lib/cpp(need to fix the gcc's do_package for cpp-symlinks).

6) do_rootfs failure: ImportError: No module named tarfile I had to install python-compression_2.7.2-r0.2_x86_64.ipk into target.

7) do_rootfs failure:
...
ERROR: ld.so: object 'libpseudo.so' from LD_PRELOAD cannot be preloaded: ignored.
...
log_check: There were error messages in the logfile
log_check: Matched keyword: [ERR]

This is cause by the pseudo ERROR line(why the pseudo ERROR is caused needs further investigation)

BTW, "runqemu qemux86 core-image-minimal" still fails: libGL.so, libGLU.so, libSDL.so are missing. Need to make qemu-native pass the build as I mentioned in previous mail.


I'll make the patches for the issues 1), 4), 5) and 6).
I'm debugging issues 2), 3) and 7). I think I'll fix 2) and 3) soon, but 7) may take more time.
Comment 3 Dexuan Cui 2011-11-21 01:12:58 UTC
(In reply to comment #2)
> (In reply to comment #1)
> > Saul did initial works and populate one layer in one contrib branch to start
> > working. We are trying to reproduce it.
> 
> I found n450 is pretty slow so later I changed to use sugarbay.
> 
> Based on Saul's base work, with many fixings and workarounds, "bitbake
> core-image-minimal" can build fine.
> 
> The fixings and workarounds are:
> 1) fix the mis-built/install python module grp;
> 
> 2) disable buuilding qemu-native since libGL.so, libGLU.so, libSDL.so are
> missing. Need to add the bits.
> 
> 3) eglibc_initial do_install failure: I suspect this is related to the
> incompatibility between eglibc's build scripts and make-3.82 -- after I
> downgraded to make-3.81, the do_install issus disappeared. I'll continue to
> investigate this.
> 
> 4) eglibc do_configure ERROR: no rpcgen found I had to install
> eglibc-utils_2.13-r17+svnr14157_x86_64.ipk into the target to resolve this
> issue.
> 
> 5) eglibc: do_install ERROR: cannot stat bootparam_prot.h  The cause is: rpcgen
> doesn't work properly --> /lib/cpp doesn't exist: we need to ln -s /usr/bin/cpp
> /lib/cpp(need to fix the gcc's do_package for cpp-symlinks).
> 
> 6) do_rootfs failure: ImportError: No module named tarfile I had to install
> python-compression_2.7.2-r0.2_x86_64.ipk into target.
> 
> 7) do_rootfs failure:
> ...
> ERROR: ld.so: object 'libpseudo.so' from LD_PRELOAD cannot be preloaded:
> ignored.
> ...
> log_check: There were error messages in the logfile
> log_check: Matched keyword: [ERR]
> 
> This is cause by the pseudo ERROR line(why the pseudo ERROR is caused needs
> further investigation)
> 
> BTW, "runqemu qemux86 core-image-minimal" still fails: libGL.so, libGLU.so,
> libSDL.so are missing. Need to make qemu-native pass the build as I mentioned
> in previous mail.
> 
> 
> I'll make the patches for the issues 1), 4), 5) and 6).
> I'm debugging issues 2), 3) and 7). I think I'll fix 2) and 3) soon, but 7) may
> take more time.

I've merged the current v1 patches into http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=dcui/self-hosted-v1.

1) eglibc-initial do_install issue.still haven't figured out which place of the complex eglibc makefiles is wrong as to make-3.82. :-(

2) do_rootfs "ERROR: ld.so: object 'libpseudo.so' from LD_PRELOAD cannot be preloaded: ignored." Working on this.
Comment 4 Jiajun Xu 2011-11-22 08:27:51 UTC
QA will try minimal build on sugarbay with the self hosted image.
So test plan for the feature is ready.
Comment 5 Dexuan Cui 2011-12-04 18:24:05 UTC
(In reply to comment #3)
> 
> I've merged the current v1 patches into
> http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=dcui/self-hosted-v1.
> 
> 1) eglibc-initial do_install issue.still haven't figured out which place of the
> complex eglibc makefiles is wrong as to make-3.82. :-(
This turned to be a bug of make-3.82:
http://savannah.gnu.org/bugs/?30723
http://cvs.savannah.gnu.org/viewvc/make/main.c?root=make&r1=1.243&r2=1.244
http://old.nabble.com/-bug--30723--implicit-re-executing-of-subdirs-breaks-$(origin)-with-make-3.82-td29394353.html

The patch has been in poky master now.

> 
> 2) do_rootfs "ERROR: ld.so: object 'libpseudo.so' from LD_PRELOAD cannot be
> preloaded: ignored." Working on this.
This seems to be a bug of busybox: I reported a new bug 1784 for this.
Now we use non-busybox-version utilities to work this issue around.

The latest patches are at
http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=dcui/self-hosted-v4
All the patches are in poky master now, except the patch "runqemu-ifup, runqemu-internal: be wiser when locating the network tools": RP asked us to improve the patch by defining a shell function to share code.

We'l setup a semi-formal branch the self-hosted-image. And will update the Build  Appliance wiki later.
Comment 6 Jiajun Xu 2011-12-21 19:11:38 UTC
I tested the feature with 1.2_M1 branch, now I could build a self-hosted image for sugarbay and build a minimal image with it.
Comment 7 Dexuan Cui 2012-01-10 13:05:58 UTC
All the patches are in poky master.
So let me mark it to RESOLVED|FIXED.