Bug 671

Summary: source poky-init-build-dev build-x86/ will cause error
Product: [Build System, Metadata & Runtime] Meta-yocto Reporter: liping Ke <liping.ke>
Component: meta-yoctoAssignee: Dexuan Cui <dexuan.cui>
Status: RESOLVED FIXED QA Contact:
Severity: minor    
Priority: Medium CC: bluelightning, dexuan.cui, dvhart, jiajun.xu, josh, lianhao.lu, poky.bs.watcher, poky.watcher, sgw, shane.wang
Version: 1.0   
Target Milestone: 1.1   
Hardware: x86   
OS: Multiple   
Whiteboard:
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: ---
Attachments:
Description Flags
error screen when source a folder build-arm/ none

Description liping Ke 2011-01-27 17:18:34 UTC
if we source to a totally new folder build-x86, and have "/" at the end
$source poky-init-build-dev build-x86/
$bitbake XXX
the command will use {POKYBASE} as build root folder, instead of {POKYBASE}/build-x86/
note: build-x86 should be a new folder here. "/" cause error here.

Thanks& Regards,
criping
Comment 1 Joshua Lock - Disabled 2011-02-24 06:40:47 UTC
I can't reproduce this with master, I took the following steps:

joshual@scimitar:~/Projects/Yocto/poky[master☠]
$ source poky-init-build-env build-x86/

joshual@scimitar:~/Projects/Yocto/poky/build-x86[master☠]
$ bitbake sudo

Then with the build running:

joshual@scimitar:~/Projects/Yocto/poky/build-x86[master☠]
$ tree -L 2 tmp/work/
tmp/work/
`-- i686-linux
    |-- autoconf-native-2.65-r2
    |-- automake-native-1.11.1-r1
    |-- curl-native-7.21.2-r1
    |-- expat-native-2.0.1-r0
    |-- gettext-native-0.17-r5
    |-- git-native-1.7.3.4-r0
    |-- gnu-config-native-0.1+cvs20080123-r1
    |-- gzip-native-1.4-r0
    |-- libtool-native-2.4-r2
    |-- m4-native-1.4.15-r0
    |-- openssl-native-0.9.8p-r3
    |-- perl-native-5.12.2-r5
    |-- pkgconfig-native-0.25-r0
    |-- pseudo-native-1.0-r0
    |-- quilt-native-0.48-r0
    |-- sqlite3-native-3.7.3-r1
    |-- tar-replacement-native-1.25-r0
    |-- tcl-native-8.5.9-r0
    `-- zlib-native-1.2.5-r0

20 directories, 0 files

joshual@scimitar:~/Projects/Yocto/poky/build-x86[master☠]
$ tree -L 2 tmp/sysroots/
tmp/sysroots/
`-- i686-linux
    |-- bin
    |-- etc
    |-- usr
    `-- var

5 directories, 0 files

If you can still reproduce this issue can you walk me through how you produce it?
Comment 2 liping Ke 2011-02-24 17:05:07 UTC
Hi, Josh

It's easy to reproduce.
1) Clone a new tree. git clone git://git.pokylinux.org/poky.git poky.git (Actually, make sure that the newly created build folder build-x86 does not exist before is ok)
2) cd poky.git
3) source poky-init-built-env build-x86/

You will find no build-x86 folder is created. Instead, conf folder is under 
the {POKYBASE}/conf/local.conf and {POKYBASE}/conf/layer.conf


It was caused by the "/" after build-x86


any problem, just let me know.

Thanks& Regards,
criping
Comment 3 Joshua Lock - Disabled 2011-02-25 02:11:18 UTC
Can you tell me which shell you are using? When I follow your steps the build-x86 directory is created and I am in it, following is a paste of my bash history when trying to reproduce this issue.

For reference I am using bash 4.1.7(1) on Fedora 14.

-------------


joshual@scimitar:~/mess/08/poky.git[master☠]
$ source poky-init-build-env build-x86/
You had no conf/local.conf file. Poky has created this configuration file for
you with some default values. You may wish to edit it to use a different MACHINE
(target hardware) or enable parallel build options to take advantage of multiple
cores for example. See the file for more information as common configuration
options are commented.

Also, for more information see the Poky Reference Manual:
    http://yoctoproject.org/community/documentation

You had no conf/bblayers.conf file. Poky has created this configuration file for
you with some default values. To add additional metadata layers into your
configuration please add entries to this file.

For more information see the Poky Reference Manual:
    http://yoctoproject.org/community/documentation


### Shell environment set up for Poky builds. ###

You can now run 'bitbake <target>'

Common targets are:
    poky-image-minimal
    poky-image-sato
    meta-toolchain
    meta-toolchain-sdk

You can also run generated qemu images with a command like 'poky-qemu qemux86'

joshual@scimitar:~/mess/08/poky.git/build-x86[master☠]
$ pwd
/home/joshual/mess/08/poky.git/build-x86

joshual@scimitar:~/mess/08/poky.git/build-x86[master☠]
$ tree conf/
conf/
|-- bblayers.conf
`-- local.conf

0 directories, 2 files
Comment 4 Paul Eggleton 2011-02-25 02:43:44 UTC
criping, FWIW I also tried following your instructions on my Ubuntu 10.10 system (with bash 4.1.5(1)) and it also worked as expected, creating a build-x86 directory containing the usual conf subdirectory and contents.

Perhaps there is more at work here than the trailing slash?
Comment 5 Joshua Lock - Disabled 2011-07-26 22:31:27 UTC
No info provided, let's close this bug.
Comment 6 liping Ke 2011-07-26 22:59:17 UTC
Created attachment 198 [details]
error screen when source a folder build-arm/
Comment 7 liping Ke 2011-07-26 23:00:04 UTC
Oh, Sorry for very late response...

I found the problem still exists. I paste a graph
1) clone latest poky.git to /media/poky/test.git
2) source oe-init-build-env build-arm/

error happens, build-arm does not exist. and conf appears in the folder of 
/media/poky/test.git/conf
Comment 8 liping Ke 2011-07-26 23:00:47 UTC
My machine is ubuntu 10.04...


Thanks a lot for your help!
criping
Comment 9 Joshua Lock - Disabled 2011-07-27 08:30:16 UTC
I still really can't reproduce this on Fedora or Ubuntu, are you running with the dash shell?

Can you try running 'readlink /bin/sh' ?
Comment 10 Joshua Lock - Disabled 2011-07-27 09:32:45 UTC
Clearly something isn't right for Liping but we need to figure out what that is, I suspect dash.
Comment 11 liping Ke 2011-07-27 18:24:11 UTC
Hi, Josh

I am using bash. Seems with "/" ended directory on machine, the first-time creating the folder will cause the problem (if we created the folder build-arm, then the problem will be gone, we can change to thise build-arm folder even if with "\" ended). I am not sure whether it relates to Ubuntu 10.04. I will ask Lianhao verify this problem and paste his result too.

Thanks a lot!

criping



(In reply to comment #10)
> Clearly something isn't right for Liping but we need to figure out what that
> is, I suspect dash.
Comment 12 liping Ke 2011-07-28 18:22:04 UTC
This problem exists on ubuntu10.04 x64 bit machine too.

Thanks& Regards,
criping
Comment 13 Darren Hart 2011-07-29 15:33:05 UTC
Can you provide the output of the following on a machine that exhibits this failure:

$ echo $BDIR
Comment 14 Darren Hart 2011-07-29 15:36:08 UTC
If BDIR is not EMPTY and is the same as the top level poky dir - that's the problem. If it is EMPTY, can you add some echo statements to lines 27-44 of oe-buildenv-internal to see what it's doing with BDIR and BUILDDIR. Also a run with "set -x" added to the top of the same file would be helpful.
Comment 15 liping Ke 2011-07-31 18:31:23 UTC
Hi, Darren
$BDIR is empty. I will get the needed information for you a little later.

Thanks!
criping
Comment 16 liping Ke 2011-07-31 19:24:13 UTC
Hi, Darren

I found if I source oe-XXXXX test-dir (no \ append),

BDIR = "/media/poky/poky.git/test-dir"
BUILDDIR = "/media/poky/poky.git/test-dir"


Then delete test-dir, execute following cmd,
source oe-XXXXX test-dir\ (with \ append)
BDIR = 
BUILDDIR = "/media/poky/poky.git/"

I guess this is the problem.

The print happens after line 38.

Seems most of guys in Shanghai met the problem ( we mostly installed ubuntu)


You can connect to my machine if you need it. I can send you my credentials.

Thanks a lot!
criping
Comment 17 Darren Hart 2011-07-31 21:28:23 UTC
(In reply to comment #16)
> Hi, Darren
> 
> I found if I source oe-XXXXX test-dir (no \ append),
> 
> BDIR = "/media/poky/poky.git/test-dir"
> BUILDDIR = "/media/poky/poky.git/test-dir"
> 
> 
> Then delete test-dir, execute following cmd,
> source oe-XXXXX test-dir\ (with \ append)

Can you confirm if you are using \ or / at the end of the directory name? \ should give you a new line with a ">" prompting you to complete the previous commands as the \ escapes the next character which it is waiting for. The / works as expected for all environments we've tested.
Comment 18 liping Ke 2011-07-31 23:11:10 UTC
Hi, Darren

Sorry for the typo. Yes, it should be "/" at the end of the directory name.
So it's a problem for all of us here who're using ubuntu 10.04.

Thanks& Regards,
criping
Comment 19 Darren Hart 2011-08-01 08:55:42 UTC
(In reply to comment #18)
> Hi, Darren
> 
> Sorry for the typo. Yes, it should be "/" at the end of the directory name.
> So it's a problem for all of us here who're using ubuntu 10.04.
> 
> Thanks& Regards,
> criping

OK, can you run with "set -x" added to the beginning the oe-buildenv-internal script and attach the results please.

--
Darren
Comment 20 Saul Wold 2011-08-01 17:23:17 UTC
Also include the output of the "env" command, I am wondering if there is a LANG setting or something similar that is in the PRC env, but not in the US environment.
Comment 21 Shane Wang 2011-08-01 21:54:04 UTC
The problem can't be reproduced on Shane's machine and Dexuan's machine, but we saw that on Lianhao's machine and Jiajun's machine. The reason is they are using a different version of readlink.

Dexuan debugged it on Lianhao's machine and worked out a patch. So, this bug was assigned to Dexuan.
Comment 22 Dexuan Cui 2011-08-01 23:02:23 UTC
(In reply to comment #21)
> The problem can't be reproduced on Shane's machine and Dexuan's machine, but we
> saw that on Lianhao's machine and Jiajun's machine. The reason is they are
> using a different version of readlink.
> 
> Dexuan debugged it on Lianhao's machine and worked out a patch. So, this bug
> was assigned to Dexuan.

"readlink -f" in Ubuntu 10.04 is buggy: it doesn't ignore a trailing / (e.g.,
"readlink -f /tmp/non-existent-dir/" returns nothing, but according to
http://www.gnu.org/s/coreutils/manual/coreutils.pdf it should do that --
hence we get bug 671. It seems Ubuntu 10.10 or even later Ubuntu 11.04,
and other Linux distributions(e.g., Open Suse 11.4) haven't such an issue.

So I think we should detect this and ask Ubuntu 10.04 users to avoid supply
a path with trailing slash here.

I made a patch at and will send it out soon:
http://git.pokylinux.org/cgit/cgit.cgi/poky-contrib/commit/?h=dcui/bug-671&id=6def4624e63c8c7cf439dff32cb155dd2bba0ebe
Comment 23 Dexuan Cui 2011-08-02 17:49:06 UTC
(In reply to comment #22)
> (In reply to comment #21)
> > The problem can't be reproduced on Shane's machine and Dexuan's machine, but we
> > saw that on Lianhao's machine and Jiajun's machine. The reason is they are
> > using a different version of readlink.
> > 
> > Dexuan debugged it on Lianhao's machine and worked out a patch. So, this bug
> > was assigned to Dexuan.
> 
> "readlink -f" in Ubuntu 10.04 is buggy: it doesn't ignore a trailing / (e.g.,
> "readlink -f /tmp/non-existent-dir/" returns nothing, but according to
> http://www.gnu.org/s/coreutils/manual/coreutils.pdf it should do that --
> hence we get bug 671. It seems Ubuntu 10.10 or even later Ubuntu 11.04,
> and other Linux distributions(e.g., Open Suse 11.4) haven't such an issue.
> 
> So I think we should detect this and ask Ubuntu 10.04 users to avoid supply
> a path with trailing slash here.
> 
> I made a patch at and will send it out soon:
> http://git.pokylinux.org/cgit/cgit.cgi/poky-contrib/commit/?h=dcui/bug-671&id=6def4624e63c8c7cf439dff32cb155dd2bba0ebe

The patch is in poky master already:
http://git.pokylinux.org/cgit/cgit.cgi/poky/commit/?id=be2a2764d8ceb398d81714661e6f199c8b11946c
Comment 24 Darren Hart 2011-08-02 21:01:48 UTC
+ if [ -z "$BDIR" ]; then
+ if expr "$1" : '.*/$' >/dev/null; then
+ echo >&2 "Error: please remove any trailing / in the argument."

This portion of the patch is really not necessary. There is no meaningful difference between the path with or without the trailing the slash, a much simpler and less noisy solution would be to simply strip the trailing slash from the user input before doing the rest of the input validation.
Comment 25 Dexuan Cui 2011-08-10 08:10:36 UTC
(In reply to comment #24)
> + if [ -z "$BDIR" ]; then
> + if expr "$1" : '.*/$' >/dev/null; then
> + echo >&2 "Error: please remove any trailing / in the argument."
> 
> This portion of the patch is really not necessary. There is no meaningful
> difference between the path with or without the trailing the slash, a much
> simpler and less noisy solution would be to simply strip the trailing slash
> from the user input before doing the rest of the input validation.

http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=b0761479e46a99b3b96e20fd142881e2a0a63136

scripts/oe-buildenv-internal: improve the error detecting for $BDIR
The previous fix for a bug in Ubuntu 10.04 readlink,
be2a2764d8ceb398d81714661e6f199c8b11946c, notified the user when a trailing
slash was used. As there is no semantic difference, simply remove any
trailing slashes and proceed without nagging the user.