Bug 6803

We should also remove all relevant types from meta/site/ as we don't want the wrong values to be taken from the cache.

(Yes, like ac_cv_sizeof_off_t).

See bug 6813 for a recent issue relating to this.

The Large File Issue

1. Introduce *Inodes*
1) Generally speaking, a Unix file is "stored" in two different
   parts of the disk
   - The data blocks.
     It contains the "contents" of the file.
   - The inodes.
     Where the information about the file is stored
Both the inodes and data blocks are stored in a "filesystem"
which is how a disk partition is organized.

2) What is in an inode?
   struct stat {
       dev_t     st_dev;     /* ID of device containing file */
       ino_t     st_ino;     /* inode number */
       mode_t    st_mode;    /* protection */
       nlink_t   st_nlink;   /* number of hard links */
       uid_t     st_uid;     /* user ID of owner */
       gid_t     st_gid;     /* group ID of owner */
       dev_t     st_rdev;    /* device ID (if special file) */
       off_t     st_size;    /* total size, in bytes */
       blksize_t st_blksize; /* blocksize for filesystem I/O */
       blkcnt_t  st_blocks;  /* number of 512B blocks allocated */
       time_t    st_atime;   /* time of last access */
       time_t    st_mtime;   /* time of last modification */
       time_t    st_ctime;   /* time of last status change */
   };
The inode number indexes a table of inodes in a known location on the device;

3) The maximum number of inode 
For 32 bit linux, sizeof(ino_t) is 4 byte; for 64 bit, sizeof(ino_t) is 8 byte


2. Root cause
While NFS server on a 64bit host, and file's inode number is
larger than 32bit(4G, 4294967295), the 32bit host could not
parse 64bit inode through NFS client. It affects two very
common system calls, stat() and readdir() which return an
inode number with error EOVERFLOW

sh-4.3# ls -i /etc/dbus-1/system.conf
49514586668 /etc/dbus-1/system.conf

sh-4.3# cp /etc/dbus-1/system.conf /tmp/
sh-4.3# ls -i /tmp/system.conf 
16024 /tmp/system.conf

3. Solutions
1) Recompiling 64 bit should be safe.

2) Recompiling 32 bit with AC_SYS_LARGEFILE in configure.ac.
   The Autoconf provides macro AC_SYS_LARGEFILE to check that with/without
   -D_FILE_OFFSET_BITS=64, the off_t can represent 2**63 - 1 correctly 

   Also could use '--disable-largefile' to disable the detection.

vim configure
...
  { $as_echo "$as_me:${as_lineno-$LINENO}: checking for _FILE_OFFSET_BITS value needed for large files" >&5
$as_echo_n "checking for _FILE_OFFSET_BITS value needed for large files... " >&6; }
if ${ac_cv_sys_file_offset_bits+:} false; then :
  $as_echo_n "(cached) " >&6
else
  while :; do
  cat confdefs.h - <<_ACEOF >conftest.$ac_ext
/* end confdefs.h.  */
#include <sys/types.h>
 /* Check that off_t can represent 2**63 - 1 correctly.
    We can't simply define LARGE_OFF_T to be 9223372036854775807,
    since some C++ compilers masquerading as C compilers
    incorrectly reject 9223372036854775807.  */
#define LARGE_OFF_T (((off_t) 1 << 62) - 1 + ((off_t) 1 << 62))
  int off_t_is_large[(LARGE_OFF_T % 2147483629 == 721
                       && LARGE_OFF_T % 2147483647 == 1)
                      ? 1 : -1];
int
main ()
{

  ;
  return 0;
}
_ACEOF
...

How marco _FILE_OFFSET_BITS works:

In <features.h>
...
#if defined _FILE_OFFSET_BITS && _FILE_OFFSET_BITS == 64
# define __USE_FILE_OFFSET64    1
#endif
...

In <sys/types.h>
...
# ifndef __USE_FILE_OFFSET64
typedef __ino_t ino_t;
# else
typedef __ino64_t ino_t;
# endif
...

In <bits/types.h>
...
__STD_TYPE __INO_T_TYPE __ino_t;        /* Type of file serial numbers.  */
__STD_TYPE __INO64_T_TYPE __ino64_t;    /* Type of file serial numbers (LFS).*/
...

In <bits/typesizes.h>
...
#define __INO_T_TYPE            __SYSCALL_ULONG_TYPE
#define __INO64_T_TYPE          __UQUAD_TYPE
...

3) Ask the kernel to lie
You can now enable/disable 32-bit inode numbers. To do so, use the
kernel parameter nfs.enable_ino64=. Setting 'nfs.enable_ino64=0' will
instruct the NFS client to return 32-bit inode numbers for readdir()
and stat() system calls (instead of the full 64-bit inode numbers).

By default, this kernel parameter is set to return the actual 64-bit inode numbers.

vim ./fs/nfs/inode.c
...
/**
 * nfs_compat_user_ino64 - returns the user-visible inode number
 * @fileid: 64-bit fileid
 *
 * This function returns a 32-bit inode number if the boot parameter
 * nfs.enable_ino64 is zero.
 */
u64 nfs_compat_user_ino64(u64 fileid)
{
#ifdef CONFIG_COMPAT
        compat_ulong_t ino; 
#else   
        unsigned long ino; 
#endif

        if (enable_ino64)
                return fileid;
        ino = fileid;
        if (sizeof(ino) < sizeof(fileid))
                ino ^= fileid >> (sizeof(fileid)-sizeof(ino)) * 8; 
        return ino; 
}
...

4) Ask nfs server to lie. Take unfs3 for example:
diff --git a/attr.c b/attr.c
--- a/attr.c
+++ b/attr.c
@@ -189,7 +189,7 @@ post_op_attr get_post_buf(backend_statstruct buf, struct svc_req * req)
            result.post_op_attr_u.attributes.fsid = export_fsid;
        }
     }
-#if defined(WIN32) || defined(AFS_SUPPORT)
+
     /* Recent Linux kernels (2.6.24 and newer) expose large fileids even to
        non-LFS 32-bit applications, unless kernel parameter
        nfs.enable_ino64=0. This means that applications will fail with
@@ -197,9 +197,7 @@ post_op_attr get_post_buf(backend_statstruct buf, struct svc_req * req)
        trouble, we truncate to 32 bits */
     result.post_op_attr_u.attributes.fileid =
        (buf.st_ino >> 32) ^ (buf.st_ino & 0xffffffff);
-#else
-    result.post_op_attr_u.attributes.fileid = buf.st_ino;
-#endif
+
     result.post_op_attr_u.attributes.atime.seconds = buf.st_atime;
     result.post_op_attr_u.attributes.atime.nseconds = 0;
     result.post_op_attr_u.attributes.mtime.seconds = buf.st_mtime;

diff --git a/readdir.c b/readdir.c
--- a/readdir.c
+++ b/readdir.c
@@ -166,12 +166,9 @@ READDIR3res read_dir(const char *path, cookie3 cookie, cookieverf3 verf,
 
            strcpy(&obj[i * NFS_MAXPATHLEN], this->d_name);
 
-#if defined(WIN32) || defined(AFS_SUPPORT)
            /* See comment in attr.c:get_post_buf */
            entry[i].fileid = (buf.st_ino >> 32) ^ (buf.st_ino & 0xffffffff);
-#else
-           entry[i].fileid = buf.st_ino;
-#endif
+
            entry[i].name = &obj[i * NFS_MAXPATHLEN];
            entry[i].cookie = (cookie + 1 + i) | rcookie;
            entry[i].nextentry = NULL;


Refers:
http://www.tcm.phy.cam.ac.uk/sw/inodes64.html
http://www.grymoire.com/Unix/Inodes.html
https://en.wikipedia.org/wiki/Inode

//Hongxu

So we just changed the scope of this by saying large files are basically mandatory and anything that doesn't use them is buggy.

Anything targetting 64-bit is safe out of the box.  For 32-bit hosts we should be able to do something similar to Debian, which looks for binaries that link to any of these symbols: https://github.com/Debian/lintian/blob/master/data/binaries/lfs-symbols

Link to anything in that list, and you're not using largefile APIs.

Got a proof of concept:

- poky-contrib:ross/elf  has an improved ELF parser that I'm using in...
- github:rossburton/meta-ross, insanity-lfs.bbclass.

This uses the same symbol list as Debian, and warns if a binary is linking to any of those symbols.

Lots of oe-core fails with qemux86...

Almost but not quite implemented by check_32bit_symbols() in insane.bbclass.

Bulk move from 4.99 or 0.00 to 5.99