Bug 13216

Hi,

Using oe-core in standalone mode (bitbake) inside a lxc container using a zfs file system, we get the following error:

Log data follows:
| DEBUG: Executing python function set_image_size
| DEBUG: 130757.900000 = 100583 * 1.300000
| DEBUG: 130757.900000 = max(130757.900000, 8192)[130757.900000] + 1
| DEBUG: 130758.000000 = int(130757.900000)
| DEBUG: 130758 = aligned(130758)
| DEBUG: returning 130758
| DEBUG: Python function set_image_size finished
| DEBUG: Executing python function extend_recipe_sysroot
| NOTE: Direct dependencies are ['virtual:native:/root/oe-core/meta/recipes-devtools/makedevs/makedevs_1.0.1.bb:do_populate_sysroot', 'virtual:native:/root/oe-core/meta/recipes-devtools/e2fsprogs/e2fsprogs_1.44.5.bb:do_populate_sysroot', 'virtual:native:/root/oe-core/meta/recipes-devtools/pseudo/pseudo_git.bb:do_populate_sysroot', '/root/oe-core/meta/recipes-kernel/kmod/depmodwrapper-cross_1.0.bb:do_populate_sysroot', 'virtual:native:/root/oe-core/meta/recipes-core/update-rc.d/update-rc.d_0.8.bb:do_populate_sysroot', 'virtual:native:/root/oe-core/meta/recipes-devtools/opkg/opkg_0.4.0.bb:do_populate_sysroot', '/root/oe-core/meta/recipes-devtools/qemu/qemuwrapper-cross_1.0.bb:do_populate_sysroot', 'virtual:native:/root/oe-core/meta/recipes-devtools/opkg-utils/opkg-utils_0.4.0.bb:do_populate_sysroot', '/root/oe-core/meta/recipes-devtools/mklibs/mklibs-native_0.1.44.bb:do_populate_sysroot', '/root/oe-core/meta/recipes-core/glibc/cross-localedef-native_2.29.bb:do_populate_sysroot', 'virtual:native:/root/oe-core/meta/recipes-extended/pbzip2/pbzip2_1.1.13.bb:do_populate_sysroot', 'virtual:native:/root/oe-core/meta/recipes-devtools/prelink/prelink_git.bb:do_populate_sysroot', '/root/oe-core/meta/recipes-core/glibc/ldconfig-native_2.12.1.bb:do_populate_sysroot', 'virtual:native:/root/oe-core/meta/recipes-extended/pigz/pigz_2.4.bb:do_populate_sysroot']
| NOTE: Installed into sysroot: []
| NOTE: Skipping as already exists in sysroot: ['makedevs-native', 'e2fsprogs-native', 'pseudo-native', 'depmodwrapper-cross', 'update-rc.d-native', 'opkg-native', 'qemuwrapper-cross', 'opkg-utils-native', 'mklibs-native', 'cross-localedef-native', 'pbzip2-native', 'prelink-native', 'ldconfig-native', 'pigz-native', 'quilt-native', 'xz-native', 'attr-native', 'automake-native', 'pkgconfig-native', 'libtool-native', 'gnu-config-native', 'texinfo-dummy-native', 'util-linux-native', 'autoconf-native', 'gettext-minimal-native', 'kmod-native', 'qemu-native', 'debianutils-native', 'openssl-native', 'gtk+3-native', 'gdk-pixbuf-native', 'libarchive-native', 'libsolv-native', 'shadow-native', 'glib-2.0-native', 'bzip2-native', 'binutils-native', 'elfutils-native', 'zlib-native', 'ncurses-native', 'm4-native', 'python3-native', 'gtk-doc-native', 'libxcursor-native', 'cairo-native', 'at-spi2-atk-native', 'libxdamage-native', 'libxrender-native', 'libpng-native', 'pango-native', 'atk-native', 'gobject-introspection-native', 'libxi-native', 'libxext-native', 'libxrandr-native', 'libxfixes-native', 'fontconfig-native', 'libx11-native', 'libxcomposite-native', 'libjpeg-turbo-native', 'meson-native', 'shared-mime-info-native', 'ninja-native', 'lzo-native', 'cmake-native', 'expat-native', 'rpm-native', 'gettext-native', 'libpcre-native', 'libffi-native', 'flex-native', 'gdbm-native', 'sqlite3-native', 'libnsl2-native', 'readline-native', 'libtirpc-native', 'util-macros-native', 'libxcb-native', 'pixman-native', 'libsm-native', 'at-spi2-core-native', 'libxml2-native', 'dbus-native', 'xorgproto-native', 'harfbuzz-native', 'fribidi-native', 'freetype-native', 'libxft-native', 'gperf-native', 'xtrans-native', 'python3-setuptools-native', 'intltool-native', 're2c-native', 'curl-native', 'file-native', 'nss-native', 'popt-native', 'perl-native', 'db-native', 'libxau-native', 'xcb-proto-native', 'libpthread-stubs-native', 'libxdmcp-native', 'libice-native', 'libxtst-native', 'icu-native', 'unzip-native', 'libxml-parser-perl-native', 'nspr-native']
| DEBUG: Python function extend_recipe_sysroot finished
| DEBUG: Executing shell function do_image_ext4
| DEBUG: Executing dd if=/dev/zero of=/root/oe-core/build/tmp-glibc/work/qemux86_64-oe-linux/core-image-minimal/1.0-r0/deploy-core-image-minimal-image-complete/core-image-minimal-qemux86-64-20190305222515.rootfs.ext4 seek=130758 count=0 bs=1024
| 0+0 records in
| 0+0 records out
| 0 bytes copied, 0.000166729 s, 0.0 kB/s
| DEBUG: Actual Rootfs size:  100583	/root/oe-core/build/tmp-glibc/work/qemux86_64-oe-linux/core-image-minimal/1.0-r0/rootfs
| DEBUG: Actual Partion size: 133896192
| DEBUG: Executing mkfs.ext4 -F -i 4096 -O 64bit /root/oe-core/build/tmp-glibc/work/qemux86_64-oe-linux/core-image-minimal/1.0-r0/deploy-core-image-minimal-image-complete/core-image-minimal-qemux86-64-20190305222515.rootfs.ext4 -d /root/oe-core/build/tmp-glibc/work/qemux86_64-oe-linux/core-image-minimal/1.0-r0/rootfs
| mke2fs 1.44.5 (15-Dec-2018)
| Discarding device blocks: done         
| Creating filesystem with 130756 1k blocks and 32768 inodes
| Filesystem UUID: a136fb3e-10e0-4847-a107-bc1926327ae9
| Superblock backups stored on blocks:
| 	8193, 24577, 40961, 57345, 73729
| 
| Allocating group tables: done 
| Writing inode tables: done 
| Creating journal (4096 blocks): done
| Copying files into the device: __populate_fs: Could not allocate block in ext2 filesystem while writing file "libnl-genl-3.so.200.26.0"
| mkfs.ext4: Could not allocate block in ext2 filesystem while populating file system
| WARNING: exit code 1 from a shell command.
| ERROR: Function failed: do_image_ext4 (log file is located at /root/oe-core/build/tmp-glibc/work/qemux86_64-oe-linux/core-image-minimal/1.0-r0/temp/log.do_image_ext4.8133)
ERROR: Task (/root/oe-core/meta/recipes-core/images/core-image-minimal.bb:do_image_ext4) failed with exit code '1'


After a quick discussion with Richard Purdie on irc [1], it seems that the size calculation is wrong. The current calculation is based on "du -ks", using "--apparent-size" fixes the issue. Still, I'm not convinced that this is a valid longterm solution since even when using --apparent-size du can under-report the actual size.

Ross Burton already has a branch that tries to fix this issue [2]. We tested it and it seems to do the job!


[1] https://www.yoctoproject.org/irc/%23yocto.2019-03-05.log.html#t2019-03-05T22:28:52
[2] https://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=ross/du