Bug 941

Summary: gnu-configize fails to run
Product: [Build System, Metadata & Runtime] Meta-yocto Reporter: Gary Thomas <gary>
Component: meta-yoctoAssignee: Dexuan Cui <dexuan.cui>
Status: RESOLVED FIXED QA Contact:
Severity: major    
Priority: High CC: dexuan.cui, edwin.zhai, poky.bs.watcher, poky.watcher, sgw
Version: unspecified   
Target Milestone: 1.1 M2   
Hardware: x86   
OS: Multiple   
Whiteboard: The patch series submitted "looks good to" RP and being reviewed by the community (01/Jun/2011)
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: ---
Bug Depends on:    
Bug Blocks: 968    
Attachments:
Description Flags
gnu-config.patch none

Description Gary Thomas 2011-03-31 04:46:59 UTC
Any recipe which uses gnu-configize may have problems since this step currently fails.

Tested against master (6f4fbfe272f5d85cee95660a9278f3031533aec4)

Looking at log.do_configure for 'bitbake bash' shows these errors:
Perl lib version (5.12.2) doesn't match executable version (v5.10.1) at /home/local/p60_poky/tmp/sysroots/i686-linux/usr/lib/perl/5.12.2/Config.pm line 50.
Compilation failed in require at /home/local/p60_poky/tmp/sysroots/i686-linux/usr/lib/perl/5.12.2/File/Copy.pm line 14.
BEGIN failed--compilation aborted at /home/local/p60_poky/tmp/sysroots/i686-linux/usr/lib/perl/5.12.2/File/Copy.pm line 14.
Compilation failed in require at /home/local/p60_poky/tmp/sysroots/i686-linux/usr/share/autoconf/Autom4te/FileUtils.pm line 166.
BEGIN failed--compilation aborted at /home/local/p60_poky/tmp/sysroots/i686-linux/usr/share/autoconf/Autom4te/FileUtils.pm line 166.
Compilation failed in require at /home/local/p60_poky/tmp/sysroots/i686-linux/usr/bin/autom4te line 41.
BEGIN failed--compilation aborted at /home/local/p60_poky/tmp/sysroots/i686-linux/usr/bin/autom4te line 41.
Comment 1 Dexuan Cui 2011-04-11 21:52:23 UTC
Coul (In reply to comment #0)
> Any recipe which uses gnu-configize may have problems since this step currently
> fails.
> Tested against master (6f4fbfe272f5d85cee95660a9278f3031533aec4)
> Looking at log.do_configure for 'bitbake bash' shows these errors:
> Perl lib version (5.12.2) doesn't match executable version (v5.10.1) at
> /home/local/p60_poky/tmp/sysroots/i686-linux/usr/lib/perl/5.12.2/Config.pm line
> 50.
> Compilation failed in require at
> /home/local/p60_poky/tmp/sysroots/i686-linux/usr/lib/perl/5.12.2/File/Copy.pm
> line 14.
> BEGIN failed--compilation aborted at
> /home/local/p60_poky/tmp/sysroots/i686-linux/usr/lib/perl/5.12.2/File/Copy.pm
> line 14.
> Compilation failed in require at
> /home/local/p60_poky/tmp/sysroots/i686-linux/usr/share/autoconf/Autom4te/FileUtils.pm
> line 166.
> BEGIN failed--compilation aborted at
> /home/local/p60_poky/tmp/sysroots/i686-linux/usr/share/autoconf/Autom4te/FileUtils.pm
> line 166.
> Compilation failed in require at
> /home/local/p60_poky/tmp/sysroots/i686-linux/usr/bin/autom4te line 41.
> BEGIN failed--compilation aborted at
> /home/local/p60_poky/tmp/sysroots/i686-linux/usr/bin/autom4te line 41.

Could this bug be a duplicate of bug 968?
Hi Gary, can you please check in the build log if perl-native's and gnu-config-native's do_populate_sysroot succeed or not before the first ERROR?
Comment 2 Gary Thomas 2011-04-12 16:58:59 UTC
I don't think it's related to bug #968

As for the ordering, here are the relevant time stamps

-rw-rw-r-- 1 gthomas gthomas 0 Mar 30 06:23 tmp/stamps/i686-linux/gnu-config-native-0.1+cvs20080123-r2.do_populate_sysroot
-rw-rw-r-- 1 gthomas gthomas 0 Mar 30 06:40 tmp/stamps/i686-linux/perl-native-5.12.2-r8.do_populate_sysroot
-rw-rw-r-- 1 gthomas gthomas 0 Mar 31 05:16 tmp/stamps/armv7a-poky-linux-gnueabi/bash-4.1-r1.do_configure
Comment 3 Edwin Zhai 2011-04-27 01:59:47 UTC
gary,
I can't reproduce it on 927d33c170f062c0a59d2a321679d5fe22e80a97 @ master by 'bitbake bash'.

Seems it got fixed. Can you have a try, or I need do something else?
Thanks,
edwin
Comment 4 Dexuan Cui 2011-04-27 02:03:28 UTC
(In reply to comment #3)
> gary,
> I can't reproduce it on 927d33c170f062c0a59d2a321679d5fe22e80a97 @ master by
> 'bitbake bash'.
> Seems it got fixed. Can you have a try, or I need do something else?
> Thanks,
> edwin

Edwin, please see the tmo/workdir/.../bash*/temp/log.do_configure that has the error msgs. I suppost this is What Gary meant.
Comment 5 Gary Thomas 2011-04-27 03:54:41 UTC
Yes, you only see this error message in log.do_configure

A closer look shows that it seems to be running Perl scripts and
libraries from sysroots, but the system host Perl is being called,
thus causing the mismatch.

On my system, here's the main host version:
  $ perl --version
  This is perl, v5.10.1 (*) built for i386-linux-thread-multi

This is the one in sysroots that _should_ be used instead:
  $ tmp/sysroots/i686-linux/usr/bin/perl --version
  This is perl 5, version 12, subversion 2 (v5.12.2) built for i686-linux-thread-multi

So however Perl is being executed in the gnu-configize, it's getting
the host program (5.10.1), not the correct sysroots version (5.12.2)
Comment 6 Dexuan Cui 2011-04-29 01:40:45 UTC
(In reply to comment #5)
> Yes, you only see this error message in log.do_configure
> A closer look shows that it seems to be running Perl scripts and
> libraries from sysroots, but the system host Perl is being called,
> thus causing the mismatch.
> On my system, here's the main host version:
>   $ perl --version
>   This is perl, v5.10.1 (*) built for i386-linux-thread-multi
> This is the one in sysroots that _should_ be used instead:
>   $ tmp/sysroots/i686-linux/usr/bin/perl --version
>   This is perl 5, version 12, subversion 2 (v5.12.2) built for
> i686-linux-thread-multi
> So however Perl is being executed in the gnu-configize, it's getting
> the host program (5.10.1), not the correct sysroots version (5.12.2)

autoconf-native's do_configure detects perl from PATH and reflect the path
into the scripts generated, e.g., ${STAGING_BINDIR_NATIVE}/autom4te.

If perl-native's does populate_sysroot later than autoconf, the host's perl
will be used -- this is not what we desire.

I used the following patch to meta/recipes-devtools/autoconf/autoconf_2.65.bb

-PR = "r2"
+PR = "r3"
-DEPENDS_virtclass-native = "m4-native gnu-config-native"
+DEPENDS_virtclass-native = "m4-native gnu-config-native perl-native"

This fixes the bug in my side.
Comment 7 Dexuan Cui 2011-04-29 01:43:29 UTC
(In reply to comment #6)
> (In reply to comment #5)
> > Yes, you only see this error message in log.do_configure
> > A closer look shows that it seems to be running Perl scripts and
> > libraries from sysroots, but the system host Perl is being called,
> > thus causing the mismatch.
> > On my system, here's the main host version:
> >   $ perl --version
> >   This is perl, v5.10.1 (*) built for i386-linux-thread-multi
> > This is the one in sysroots that _should_ be used instead:
> >   $ tmp/sysroots/i686-linux/usr/bin/perl --version
> >   This is perl 5, version 12, subversion 2 (v5.12.2) built for
> > i686-linux-thread-multi
> > So however Perl is being executed in the gnu-configize, it's getting
> > the host program (5.10.1), not the correct sysroots version (5.12.2)
> autoconf-native's do_configure detects perl from PATH and reflect the path
> into the scripts generated, e.g., ${STAGING_BINDIR_NATIVE}/autom4te.
> If perl-native's does populate_sysroot later than autoconf, the host's perl
> will be used -- this is not what we desire.
> I used the following patch to meta/recipes-devtools/autoconf/autoconf_2.65.bb
> -PR = "r2"
> +PR = "r3"
> -DEPENDS_virtclass-native = "m4-native gnu-config-native"
> +DEPENDS_virtclass-native = "m4-native gnu-config-native perl-native"
> This fixes the bug in my side.
mised one more piece of patch:
in meta/recipes-devtools/gnu-config/gnu-config/gnu-configize.in, should change the hardcoded "/usr/bin/perl"  with "perl".

-eval 'case $# in 0) exec /usr/bin/perl -S "$0";; *) exec /usr/bin/perl -S "$0" "$@";; esac'
+eval 'case $# in 0) exec perl -S "$0";; *) exec perl -S "$0" "$@";; esac'
Comment 8 Gary Thomas 2011-04-29 07:11:01 UTC
Those two changes (which probably need to be separate in a series) do seem to fix the issue.  Now when I run 'bitbake bash -c configure' there are no indications that gnu-configize failed in any way.
Comment 9 Dexuan Cui 2011-05-04 23:19:07 UTC
(In reply to comment #6)
> (In reply to comment #5)
> > Yes, you only see this error message in log.do_configure
> > A closer look shows that it seems to be running Perl scripts and
> > libraries from sysroots, but the system host Perl is being called,
> > thus causing the mismatch.
> > On my system, here's the main host version:
> >   $ perl --version
> >   This is perl, v5.10.1 (*) built for i386-linux-thread-multi
> > This is the one in sysroots that _should_ be used instead:
> >   $ tmp/sysroots/i686-linux/usr/bin/perl --version
> >   This is perl 5, version 12, subversion 2 (v5.12.2) built for
> > i686-linux-thread-multi
> > So however Perl is being executed in the gnu-configize, it's getting
> > the host program (5.10.1), not the correct sysroots version (5.12.2)
> autoconf-native's do_configure detects perl from PATH and reflect the path
> into the scripts generated, e.g., ${STAGING_BINDIR_NATIVE}/autom4te.
> If perl-native's does populate_sysroot later than autoconf, the host's perl
> will be used -- this is not what we desire.
Actually, not only autoconf-native but also automake-native, quilt-native, glib-2.0-native, util-linux-native, quilt-native have the same issue (they all don't cause this bug)
{quilt, glib-2.0, util-linux}-native depend on {autoconf,automake}-native, and {autoconf,automake}-native depend on gnu-config-native.

So I think the proper fix should be making quilt-native and gnu-config-native depend on perl-native (surely we still need to "bitbake -c clean" {autoconf,automake,glib-2.0,util-linux} and bitbake them again)

> mised one more piece of patch:
> in meta/recipes-devtools/gnu-config/gnu-config/gnu-configize.in, should change
> the hardcoded "/usr/bin/perl"  with "perl".
> -eval 'case $# in 0) exec /usr/bin/perl -S "$0";; *) exec /usr/bin/perl -S "$0"
> "$@";; esac'
> +eval 'case $# in 0) exec perl -S "$0";; *) exec perl -S "$0" "$@";; esac'
This change is still necessary.
Comment 10 Gary Thomas 2011-05-05 03:48:30 UTC
Do you have a patch which makes all of the changes for me to test?
Comment 11 Dexuan Cui 2011-05-05 06:05:11 UTC
Created attachment 144 [details]
gnu-config.patch
Comment 12 Dexuan Cui 2011-05-05 06:08:10 UTC
(In reply to comment #10)
> Do you have a patch which makes all of the changes for me to test?
Yes, please help to test the attached patch in comment #11.
After applying the patch, at least we also need to do
bitbake -c cleanall autoconf-native automake-native.

(In reply to comment #11)
> Created attachment 144 [details]
> gnu-config.patch
Comment 13 Gary Thomas 2011-05-05 06:21:50 UTC
What about the previous change to autoconf-native?  Is this not necessary (it's not in this patch)?
Comment 14 Dexuan Cui 2011-05-05 07:29:41 UTC
(In reply to comment #13)
> What about the previous change to autoconf-native?  Is this not necessary (it's
> not in this patch)?

Yes, I think it's not necessary as autoconf-native depends on gnu-config-native. If we add perl-native into gnu-config-native's DEPENDS, we can make sure autoconf-native also depends on perl-native. Certainly we need to re-build and re-populate autoconf-native. 
Does this make sense?
Comment 15 Gary Thomas 2011-05-05 07:31:52 UTC
I'll give it a try with only this patch.
Comment 16 Dexuan Cui 2011-05-05 07:33:40 UTC
(In reply to comment #9)
> (In reply to comment #6)
> > (In reply to comment #5)
> > > Yes, you only see this error message in log.do_configure
> > > A closer look shows that it seems to be running Perl scripts and
> > > libraries from sysroots, but the system host Perl is being called,
> > > thus causing the mismatch.
> > > On my system, here's the main host version:
> > >   $ perl --version
> > >   This is perl, v5.10.1 (*) built for i386-linux-thread-multi
> > > This is the one in sysroots that _should_ be used instead:
> > >   $ tmp/sysroots/i686-linux/usr/bin/perl --version
> > >   This is perl 5, version 12, subversion 2 (v5.12.2) built for
> > > i686-linux-thread-multi
> > > So however Perl is being executed in the gnu-configize, it's getting
> > > the host program (5.10.1), not the correct sysroots version (5.12.2)
> > autoconf-native's do_configure detects perl from PATH and reflect the path
> > into the scripts generated, e.g., ${STAGING_BINDIR_NATIVE}/autom4te.
> > If perl-native's does populate_sysroot later than autoconf, the host's perl
> > will be used -- this is not what we desire.
> Actually, not only autoconf-native but also automake-native, quilt-native,
> glib-2.0-native, util-linux-native, quilt-native have the same issue (they all
> don't cause this bug)
> {quilt, glib-2.0, util-linux}-native depend on {autoconf,automake}-native, and
> {autoconf,automake}-native depend on gnu-config-native.
> So I think the proper fix should be making quilt-native and gnu-config-native
> depend on perl-native (surely we still need to "bitbake -c clean"
> {autoconf,automake,glib-2.0,util-linux} and bitbake them again)
> > mised one more piece of patch:
> > in meta/recipes-devtools/gnu-config/gnu-config/gnu-configize.in, should change
> > the hardcoded "/usr/bin/perl"  with "perl".
> > -eval 'case $# in 0) exec /usr/bin/perl -S "$0";; *) exec /usr/bin/perl -S "$0"
> > "$@";; esac'
> > +eval 'case $# in 0) exec perl -S "$0";; *) exec perl -S "$0" "$@";; esac'

> This change is still necessary.

Maybe doing sed replacement for gnu-config-native is a better way:
sed -i -e 's,/usr/bin/perl,${STAGING_BINDIR_NATIVE}/perl,g'
${D}${bindir}/gnu-configize
Comment 17 Gary Thomas 2011-05-05 07:44:25 UTC
Seems to work with only the patch to gnu-config
Comment 18 Dexuan Cui 2011-05-05 07:46:39 UTC
(In reply to comment #17)
> Seems to work with only the patch to gnu-config

Gary, thank you very much for the verification! 

I'll do more tests and try to send out the patch to oe-core mailing list tomorrow.
Comment 19 Dexuan Cui 2011-05-05 21:04:00 UTC
(In reply to comment #16)
> (In reply to comment #9)
> > (In reply to comment #6)
> > > (In reply to comment #5)
> > > > Yes, you only see this error message in log.do_configure
> > > > A closer look shows that it seems to be running Perl scripts and
> > > > libraries from sysroots, but the system host Perl is being called,
> > > > thus causing the mismatch.
> > > > On my system, here's the main host version:
> > > >   $ perl --version
> > > >   This is perl, v5.10.1 (*) built for i386-linux-thread-multi
> > > > This is the one in sysroots that _should_ be used instead:
> > > >   $ tmp/sysroots/i686-linux/usr/bin/perl --version
> > > >   This is perl 5, version 12, subversion 2 (v5.12.2) built for
> > > > i686-linux-thread-multi
> > > > So however Perl is being executed in the gnu-configize, it's getting
> > > > the host program (5.10.1), not the correct sysroots version (5.12.2)
> > > autoconf-native's do_configure detects perl from PATH and reflect the path
> > > into the scripts generated, e.g., ${STAGING_BINDIR_NATIVE}/autom4te.
> > > If perl-native's does populate_sysroot later than autoconf, the host's perl
> > > will be used -- this is not what we desire.
> > Actually, not only autoconf-native but also automake-native, quilt-native,
> > glib-2.0-native, util-linux-native, quilt-native have the same issue (they all
> > don't cause this bug)
> > {quilt, glib-2.0, util-linux}-native depend on {autoconf,automake}-native, and
> > {autoconf,automake}-native depend on gnu-config-native.
> > So I think the proper fix should be making quilt-native and gnu-config-native
> > depend on perl-native (surely we still need to "bitbake -c clean"
> > {autoconf,automake,glib-2.0,util-linux} and bitbake them again)
> > > mised one more piece of patch:
> > > in meta/recipes-devtools/gnu-config/gnu-config/gnu-configize.in, should change
> > > the hardcoded "/usr/bin/perl"  with "perl".
> > > -eval 'case $# in 0) exec /usr/bin/perl -S "$0";; *) exec /usr/bin/perl -S "$0"
> > > "$@";; esac'
> > > +eval 'case $# in 0) exec perl -S "$0";; *) exec perl -S "$0" "$@";; esac'
> > This change is still necessary.
> Maybe doing sed replacement for gnu-config-native is a better way:
> sed -i -e 's,/usr/bin/perl,${STAGING_BINDIR_NATIVE}/perl,g'
> ${D}${bindir}/gnu-configize

RP doesn't think this is ok. We're discussining.
Comment 20 Dexuan Cui 2011-05-10 18:02:10 UTC
A workaround patch has been in poky master:
http://git.pokylinux.org/cgit/cgit.cgi/poky/commit/?id=605141a93443df042634b2219a8628a9004be023

And I'm working to tidy the usages of perl-native/perl-native-runtime to get a real fix.
Comment 22 Saul Wold 2011-06-09 19:04:59 UTC
Patch series has been applied to master and oe-core