Bug 5442

Created attachment 1738 [details]
v1

Here is my proposal. I chose to use oe.utils.getstatusoutput instead of subprocess.Popen for more portability.

(In reply to comment #2)
> Here is my proposal. I chose to use oe.utils.getstatusoutput instead of
> subprocess.Popen for more portability.

Hi Matthieu,

Thank you very much for taking care of this, it seems that the "stat -f -c" is a little better to check the nfs, I had sent a patch for it on Nov. 1st, 2013, but it had been pending because of where to put the checking, I will update it a little and re-send it again.

http://lists.openembedded.org/pipermail/openembedded-core/2013-November/085833.html

Thank you very much again.

Hi Robert,

stat is indeed better. But I was wondering if NFS was the only network
filesystem with troubles (for example Coda or CIFS?).

It's not actually "NFS" that the problem occurs.  It's any filesystem that doesn't have consistent inodes and/or doesn't honor all of the locking and related items that POSIX dictates for a Unix-like filesystem.

NFS just seems to be the biggest offender as it usually violates both locking and inode consistency.

I'm not sure if any of the other network filesystems have a similar problem or not, but until someone has a problem -- it's not easy to check for.

(The old reiserfs had the inode issue as well, at least original versions of it did..)

Merged into master:
http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=47208175027686339dd18df52b60ec2c43594918

I'm having a hard time understanding what was the initial problem that was solved by checking nfs is not used for TMPDIR. I've created various poky based images on tnf without apparent issue so far. 
Thank you.
  Tudor

(In reply to comment #7)

Most NFS systems have significant issues with inode number re-use as well as issues with POSIX locking.

When doing a build where the TMPDIR is on NFS, pseudo (the fake root user wrapper) can fail to properly identify the files being used and their faked permissions, owners and groups.  This will lead to the file itself being properly created, but when packaged the permissions can be incorrect.

If you inspect the logs, you will find that various pseudo diagnostic warnings will occur where it was unable to identify the initial creator of the file, so it has to assume something based on either the inode number or the file path.  (Since inode numbers are reused, this doesn't always work.)

Locking is a similar issue in that RPM, bitbake, pseudo and other components require functional POSIX locking.  Some NFS systems appear to work, while others don't work reliably.  There does not appear to be any way to reliably detect if the locking is working before it simply fails.

Thus you can run your builds with your configured 'build' directory on NFS, but the TMPDIR -must- be on a filesystem with proper

Hi, 

I dropped in a note in the insane.bbclass section of the reference manual stating that the class did not support tests on TMPDIR when it is on NFS.  I don't know if this note is sufficient or not to take care of the documentation part of this issue. 

Mark.. could you review the note and see if I need to add any more detail or clarity to what is going on?  For one thing, I don't know the consequences if TMPDIR is on NFS.  Do the tests just not get run?  I would like to state something about that in the note.

See the change here - http://www.yoctoproject.org/docs/1.6/ref-manual/ref-manual.html#ref-classes-insane

Thanks, 
Scott

(In reply to comment #9)
> See the change here -
> http://www.yoctoproject.org/docs/1.6/ref-manual/ref-manual.html#ref-classes-
> insane


Thats not correct..  It's actually the sanity class that this change affects.

Specifically:

sanity.bbclass:
+    # Check that TMPDIR isn't located on nfs
+    status.addresult(check_not_nfs(tmpdir, "TMPDIR"))

So I think you should revert your documentation change.  Instead make reference somewhere in the docs (1.3 maybe? or maybe 3.1) that the TMPDIR must reside on a local disk as NFS isn't supported.  This is due to limitations and bugs within the NFS protocol and many NFS implementations.

Mark, 

Thanks for the correction.  I took the change out of the insane class but left an explanation in the TMPDIR variable in the glossary at http://www.yoctoproject.org/docs/1.6/ref-manual/ref-manual.html#var-TMPDIR.

Regarding another important place where we might like this note I was thinking of where we define the term "Build Directory" since that is where TMPDIR will ultimately be.  Throughout the YP manual set, the term "Build Directory" is linked liberally back to this definition.

http://www.yoctoproject.org/docs/1.6/dev-manual/dev-manual.html#build-directory

How does this sound?

Scott

(In reply to comment #11)
> Mark, 
> 
> Thanks for the correction.  I took the change out of the insane class but
> left an explanation in the TMPDIR variable in the glossary at
> http://www.yoctoproject.org/docs/1.6/ref-manual/ref-manual.html#var-TMPDIR.
> 
> Regarding another important place where we might like this note I was
> thinking of where we define the term "Build Directory" since that is where
> TMPDIR will ultimately be.  Throughout the YP manual set, the term "Build
> Directory" is linked liberally back to this definition.
> 
> http://www.yoctoproject.org/docs/1.6/dev-manual/dev-manual.html#build-
> directory

I think that is correct.  The on caveat is that the TMPDIR is by default within the build-directory -- but does not have to be.  This is where the notion of setting the TMPDIR to a local disk, and using NFS for the build-directory comes into play.

--Mark

Mark, 

Good point on TMPDIR purposely getting set outside the build directory.  I will bring that caveat up in the explanation... probably both in the term and in the TMPDIR glossary entry.  

When I get the changes in I will comment with some links for review. 

thanks, 
Scott

Mark, 

Please look at these two areas now and see if we are good for this doc part of the bug:

http://www.yoctoproject.org/docs/1.6/ref-manual/ref-manual.html#var-TMPDIR

http://www.yoctoproject.org/docs/1.6/dev-manual/dev-manual.html#build-directory

Thanks, 
Scott

(In reply to comment #14)
> Please look at these two areas now and see if we are good for this doc part
> of the bug:
> 
> http://www.yoctoproject.org/docs/1.6/ref-manual/ref-manual.html#var-TMPDIR
> 
> http://www.yoctoproject.org/docs/1.6/dev-manual/dev-manual.html#build-
> directory

I think the above properly explains the issue.  New users may still be a bit confused -- but I don't see an easier way of explaining things.

Thanks Mark, 

New users in a state of confusion is a constant :)

I am resetting the doc flag to "Done."

Scott