Bug 12988

Summary: wic: sparse_copy does not work on aufs in a Docker container
Product: [Build System, Metadata & Runtime] OE-Core Reporter: Ming Liu <liu.ming50>
Component: Scripts and ToolsAssignee: Kevin Hao <kexin.hao>
Status: RESOLVED FIXED QA Contact:
Severity: normal    
Priority: Medium CC: liu.ming50, randy.macleod, ross.burton
Version: 5.99   
Target Milestone: 3.2 M2   
Hardware: x86   
OS: Multiple   
Whiteboard:
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: No (bug/feature does not impact docs)

Description Ming Liu 2018-11-08 08:43:36 UTC
Preconditions/Environment
-------------------------

Poky tip is: commit 411184bfaa6269bf2926bb2a576c0922958cbbb3
Running in a Docker container derived from Ubuntu 16.04 image.
Docker host is using AUFS filesystem for Docker volumes.

WIC is enabled in build/conf/local.conf using IMAGE_FSTYPES, WKS_SEARCH_PATH and WKS_FILES


Triggering Action/Cause
-----------------------
When I execute 'wic rm'

For example: wic rm ${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.wic:1/EFI/BOOT/${ACRN_UEFI_IMAGE_NAME}.efi --native-sysroot ${STAGING_DIR_NATIVE}


Expectation
-----------
The file should be removed from the partition of a wic image 
 

Actual Result
-------------
Build stops with error:

ERROR: sos-image-weston-1.0-r0 do_image_wic: Function failed: do_image_wic (log file is located at /opt/acrn-yocto/build/tmp/work/apl_nuc-poky-linux/sos-image-weston/1.0-r0/temp/log.do_image_wic.12779)
ERROR: Logfile of failure stored in: /opt/acrn-yocto/build/tmp/work/apl_nuc-poky-linux/sos-image-weston/1.0-r0/temp/log.do_image_wic.12779
Log data follows:
| DEBUG: Executing python function set_image_size
| DEBUG: 3625793.600000 = 2789072 * 1.300000
| DEBUG: 3728193.600000 = max(3625793.600000, 65536)[3625793.600000] + 1
| DEBUG: 3728194.000000 = int(3728193.600000)
| DEBUG: 3728194 = aligned(3728194)
| DEBUG: returning 3728194
| DEBUG: Python function set_image_size finished
| DEBUG: Executing python function extend_recipe_sysroot
| NOTE: Direct dependencies are ['/opt/acrn-yocto/sources/poky/meta/recipes-core/glibc/glibc_2.28.bb:do_populate_sysroot', 'virtual:native:/opt/acrn-yocto/sources/poky/meta/recipes-devtools/ninja/ninja_1.8.2.bb:do_populate_sysroot', '/opt/acrn-yocto/sources/poky/meta/recipes-devtools/qemu/qemuwrapper-cross_1.0.bb:do_populate_sysroot', 'virtual:native:/opt/acrn-yocto/sources/poky/meta/recipes-extended/bc/bc_1.07.1.bb:do_populate_sysroot', 'virtual:native:/opt/acrn-yocto/sources/poky/meta/recipes-devtools/meson/meson_0.47.2.bb:do_populate_sysroot', '/opt/acrn-yocto/sources/poky/meta/recipes-devtools/quilt/quilt-native_0.65.bb:do_populate_sysroot', 'virtual:native:/opt/acrn-yocto/sources/poky/../meta-intel/recipes-core/microcode/iucode-tool_2.3.1.bb:do_populate_sysroot', 'virtual:native:/opt/acrn-yocto/sources/poky/meta/recipes-support/lz4/lz4_1.8.2.bb:do_populate_sysroot', '/opt/acrn-yocto/sources/poky/meta/recipes-core/glibc/ldconfig-native_2.12.1.bb:do_populate_sysroot', 'virtual:native:/opt/acrn-yocto/sources/poky/meta/recipes-devtools/squashfs-tools/squashfs-tools_git.bb:do_populate_sysroot', 'virtual:native:/opt/acrn-yocto/sources/poky/meta/recipes-devtools/qemu/qemu_3.0.0.bb:do_populate_sysroot', 'virtual:native:/opt/acrn-yocto/sources/poky/meta/recipes-extended/gperf/gperf_3.1.bb:do_populate_sysroot', 'virtual:native:/opt/acrn-yocto/sources/poky/meta/recipes-devtools/createrepo-c/createrepo-c_git.bb:do_populate_sysroot', 'virtual:native:/opt/acrn-yocto/sources/poky/meta/recipes-extended/acpica/acpica_20180508.bb:do_populate_sysroot', 'virtual:native:/opt/acrn-yocto/sources/poky/meta/recipes-devtools/opkg-utils/opkg-utils_0.3.6.bb:do_populate_sysroot', '/opt/acrn-yocto/sources/poky/meta/recipes-bsp/grub/grub-efi_2.02.bb:do_populate_sysroot', 'virtual:native:/opt/acrn-yocto/sources/poky/meta/recipes-devtools/fdisk/gptfdisk_1.0.4.bb:do_populate_sysroot', 'virtual:native:/opt/acrn-yocto/sources/poky/meta/recipes-support/lzop/lzop_1.03.bb:do_populate_sysroot', 'virtual:native:/opt/acrn-yocto/sources/poky/meta/recipes-devtools/pseudo/pseudo_git.bb:do_populate_sysroot', '/opt/acrn-yocto/sources/poky/meta/recipes-devtools/python/python3-native_3.5.6.bb:do_populate_sysroot', '/opt/acrn-yocto/sources/poky/meta/recipes-bsp/gnu-efi/gnu-efi_3.0.8.bb:do_populate_sysroot', 'virtual:native:/opt/acrn-yocto/sources/poky/meta/recipes-devtools/bison/bison_3.0.4.bb:do_populate_sysroot', '/opt/acrn-yocto/sources/poky/meta/recipes-core/systemd/systemd-boot_239.bb:do_populate_sysroot', '/opt/acrn-yocto/sources/poky/../meta-intel/recipes-core/microcode/intel-microcode_20180807.bb:do_populate_sysroot', '/opt/acrn-yocto/sources/poky/meta/recipes-devtools/cdrtools/cdrtools-native_3.01.bb:do_populate_sysroot', '/opt/acrn-yocto/sources/poky/../meta-acrn/recipes-kernel/linux/linux-acrn_4.19.0.bb:do_populate_sysroot', 'virtual:native:/opt/acrn-yocto/sources/poky/meta/recipes-devtools/dnf/dnf_2.7.5.bb:do_populate_sysroot', '/opt/acrn-yocto/sources/poky/meta/recipes-core/util-linux/util-linux_2.32.1.bb:do_populate_sysroot', '/opt/acrn-yocto/sources/poky/meta/recipes-kernel/kmod/kmod-native_git.bb:do_populate_sysroot', '/opt/acrn-yocto/sources/poky/meta/recipes-devtools/binutils/binutils-cross_2.31.bb:do_populate_sysroot', 'virtual:native:/opt/acrn-yocto/sources/poky/meta/recipes-devtools/mtools/mtools_4.0.18.bb:do_populate_sysroot', 'virtual:native:/opt/acrn-yocto/sources/poky/meta/recipes-devtools/dosfstools/dosfstools_4.1.bb:do_populate_sysroot', '/opt/acrn-yocto/sources/poky/meta/recipes-kernel/kmod/depmodwrapper-cross_1.0.bb:do_populate_sysroot', 'virtual:native:/opt/acrn-yocto/sources/poky/meta/recipes-devtools/rpm/rpm_4.14.2.bb:do_populate_sysroot', '/opt/acrn-yocto/sources/poky/meta/recipes-core/gettext/gettext-minimal-native_0.19.8.1.bb:do_populate_sysroot', 'virtual:native:/opt/acrn-yocto/sources/poky/meta/recipes-devtools/elfutils/elfutils_0.173.bb:do_populate_sysroot', 'virtual:native:/opt/acrn-yocto/sources/poky/meta/recipes-devtools/e2fsprogs/e2fsprogs_1.44.3.bb:do_populate_sysroot', 'virtual:native:/opt/acrn-yocto/sources/poky/meta/recipes-devtools/btrfs-tools/btrfs-tools_4.17.1.bb:do_populate_sysroot', 'virtual:native:/opt/acrn-yocto/sources/poky/meta/recipes-extended/pigz/pigz_2.4.bb:do_populate_sysroot', 'virtual:native:/opt/acrn-yocto/sources/poky/../meta-acrn/recipes-devtools/python/python3-kconfiglib_10.7.0.bb:do_populate_sysroot', 'virtual:native:/opt/acrn-yocto/sources/poky/meta/recipes-core/update-rc.d/update-rc.d_0.8.bb:do_populate_sysroot', '/opt/acrn-yocto/sources/poky/meta/recipes-devtools/gcc/gcc-runtime_8.2.bb:do_populate_sysroot', '/opt/acrn-yocto/sources/poky/../meta-acrn/recipes-kernel/linux/linux-acrn-uos_4.19.0.bb:do_populate_sysroot', 'virtual:native:/opt/acrn-yocto/sources/poky/meta/recipes-extended/parted/parted_3.2.bb:do_populate_sysroot', 'virtual:native:/opt/acrn-yocto/sources/poky/meta/recipes-support/bmap-tools/bmap-tools_3.4.bb:do_populate_sysroot', 'virtual:native:/opt/acrn-yocto/sources/poky/meta/recipes-core/ovmf/ovmf_git.bb:do_populate_sysroot', 'virtual:native:/opt/acrn-yocto/sources/poky/meta/recipes-devtools/python/python3-pip_18.0.bb:do_populate_sysroot', 'virtual:native:/opt/acrn-yocto/sources/poky/meta/recipes-devtools/prelink/prelink_git.bb:do_populate_sysroot', '/opt/acrn-yocto/sources/poky/meta/recipes-support/libcap/libcap_2.25.bb:do_populate_sysroot', '/opt/acrn-yocto/sources/poky/meta/recipes-devtools/gcc/gcc-cross_8.2.bb:do_populate_sysroot', 'virtual:native:/opt/acrn-yocto/sources/poky/meta/recipes-devtools/opkg/opkg_0.3.6.bb:do_populate_sysroot', '/opt/acrn-yocto/sources/poky/meta/recipes-devtools/syslinux/syslinux_6.03.bb:do_populate_sysroot', '/opt/acrn-yocto/sources/poky/meta/recipes-devtools/mklibs/mklibs-native_0.1.43.bb:do_populate_sysroot', '/opt/acrn-yocto/sources/poky/meta/recipes-devtools/libtool/libtool-native_2.4.6.bb:do_populate_sysroot', 'virtual:native:/opt/acrn-yocto/sources/poky/meta/recipes-core/gettext/gettext_0.19.8.1.bb:do_populate_sysroot', '/opt/acrn-yocto/sources/poky/meta/recipes-extended/texinfo-dummy-native/texinfo-dummy-native.bb:do_populate_sysroot', 'virtual:native:/opt/acrn-yocto/sources/poky/meta/recipes-devtools/automake/automake_1.16.1.bb:do_populate_sysroot', 'virtual:native:/opt/acrn-yocto/sources/poky/meta/recipes-devtools/makedevs/makedevs_1.0.1.bb:do_populate_sysroot', 'virtual:native:/opt/acrn-yocto/sources/poky/meta/recipes-devtools/syslinux/syslinux_6.03.bb:do_populate_sysroot', 'virtual:native:/opt/acrn-yocto/sources/poky/meta/recipes-devtools/nasm/nasm_2.13.03.bb:do_populate_sysroot', 'virtual:native:/opt/acrn-yocto/sources/poky/meta/recipes-devtools/intltool/intltool_0.51.0.bb:do_populate_sysroot', 'virtual:native:/opt/acrn-yocto/sources/poky/meta/recipes-devtools/pkgconfig/pkgconfig_git.bb:do_populate_sysroot', 'virtual:native:/opt/acrn-yocto/sources/poky/meta/recipes-devtools/autoconf/autoconf_2.69.bb:do_populate_sysroot', 'virtual:native:/opt/acrn-yocto/sources/poky/meta/recipes-devtools/flex/flex_2.6.0.bb:do_populate_sysroot', 'virtual:native:/opt/acrn-yocto/sources/poky/meta/recipes-core/util-linux/util-linux_2.32.1.bb:do_populate_sysroot', '/opt/acrn-yocto/sources/poky/meta/recipes-core/glibc/cross-localedef-native_2.28.bb:do_populate_sysroot', 'virtual:native:/opt/acrn-yocto/sources/poky/meta/recipes-devtools/gnu-config/gnu-config_git.bb:do_populate_sysroot', 'virtual:native:/opt/acrn-yocto/sources/poky/meta/recipes-bsp/grub/grub-efi_2.02.bb:do_populate_sysroot', '/opt/acrn-yocto/sources/poky/meta/recipes-devtools/libtool/libtool-cross_2.4.6.bb:do_populate_sysroot']
| NOTE: Installed into sysroot: []
| NOTE: Skipping as already exists in sysroot: ['glibc', 'ninja-native', 'qemuwrapper-cross', 'bc-native', 'meson-native', 'quilt-native', 'iucode-tool-native', 'lz4-native', 'ldconfig-native', 'squashfs-tools-native', 'qemu-native', 'gperf-native', 'createrepo-c-native', 'acpica-native', 'opkg-utils-native', 'grub-efi', 'gptfdisk-native', 'lzop-native', 'pseudo-native', 'python3-native', 'gnu-efi', 'bison-native', 'systemd-boot', 'intel-microcode', 'cdrtools-native', 'linux-acrn', 'dnf-native', 'util-linux', 'kmod-native', 'binutils-cross-x86_64', 'mtools-native', 'dosfstools-native', 'depmodwrapper-cross', 'rpm-native', 'gettext-minimal-native', 'elfutils-native', 'e2fsprogs-native', 'btrfs-tools-native', 'pigz-native', 'python3-kconfiglib-native', 'update-rc.d-native', 'gcc-runtime', 'linux-acrn-uos', 'parted-native', 'bmap-tools-native', 'ovmf-native', 'python3-pip-native', 'prelink-native', 'libcap', 'gcc-cross-x86_64', 'opkg-native', 'syslinux', 'mklibs-native', 'libtool-native', 'gettext-native', 'texinfo-dummy-native', 'automake-native', 'makedevs-native', 'syslinux-native', 'nasm-native', 'intltool-native', 'pkgconfig-native', 'autoconf-native', 'flex-native', 'util-linux-native', 'cross-localedef-native', 'gnu-config-native', 'grub-efi-native', 'libtool-cross', 'linux-libc-headers', 'systemd-systemctl-native', 're2c-native', 'readline-native', 'python3-setuptools-native', 'shadow-native', 'xz-native', 'attr-native', 'zlib-native', 'lzo-native', 'libsdl2-native', 'alsa-lib-native', 'pixman-native', 'dtc-native', 'glib-2.0-native', 'curl-native', 'openssl-native', 'expat-native', 'sqlite3-native', 'libxml2-native', 'file-native', 'cmake-native', 'debianutils-native', 'ncurses-native', 'popt-native', 'mkfontscale-native', 'mkfontdir-native', 'gdbm-native', 'bzip2-native', 'librepo-native', 'python3-iniparse-native', 'libdnf-native', 'libcomps-native', 'opkg-utils', 'ncurses', 'libxcrypt', 'bash-completion', 'zlib', 'gdk-pixbuf-native', 'gtk-doc-native', 'db-native', 'dbus-native', 'nss-native', 'libarchive-native', 'acl-native', 'gtk-icon-utils-native', 'libgcc', 'binutils-native', 'libmpc-native', 'mpfr-native', 'gmp-native', 'libsolv-native', 'e2fsprogs', 'groff-native', 'libxml-parser-perl-native', 'm4-native', 'ovmf', 'qemu-helper-native', 'unzip-native', 'libx11-native', 'libxrandr-native', 'libxext-native', 'libxrender-native', 'util-macros-native', 'libpng-native', 'libpcre-native', 'libffi-native', 'freetype-native', 'libfontenc-native', 'xorgproto-native', 'gpgme-native', 'python3-six-native', 'libcheck-native', 'gobject-introspection-native', 'libjpeg-turbo-native', 'shared-mime-info-native', 'nspr-native', 'librsvg-native', 'attr', 'perl-native', 'libxcb-native', 'xtrans-native', 'font-util-native', 'swig-native', 'libassuan-native', 'libgpg-error-native', 'pango-native', 'libcroco-native', 'cairo-native', 'xcb-proto-native', 'libpthread-stubs-native', 'libxau-native', 'libxdmcp-native', 'fontconfig-native', 'fribidi-native', 'harfbuzz-native', 'libxft-native', 'libsm-native', 'icu-native', 'libice-native']
| DEBUG: Python function extend_recipe_sysroot finished
| DEBUG: Executing shell function do_image_wic
| INFO: Creating image(s)...
| 
| WARNING: overhead-factor was specified, but size was not, so bitbake variables will be used for the size. In this case both IMAGE_OVERHEAD_FACTOR and --overhead-factor will be applied
| INFO: The new image(s) can be found here:
|   /opt/acrn-yocto/build/tmp/work/apl_nuc-poky-linux/sos-image-weston/1.0-r0/deploy-sos-image-weston-image-complete/sos-image-weston-apl-nuc-20181108070812/acrn-systemd-bootdisk-microcode-201811080709-sda.direct
| 
| The following build artifacts were used to create the image(s):
|   ROOTFS_DIR:                   /opt/acrn-yocto/build/tmp/work/apl_nuc-poky-linux/sos-image-weston/1.0-r0/rootfs
|   BOOTIMG_DIR:                  /opt/acrn-yocto/build/tmp/work/apl_nuc-poky-linux/sos-image-weston/1.0-r0/recipe-sysroot/usr/share
|   KERNEL_DIR:                   /opt/acrn-yocto/build/tmp/deploy/images/apl-nuc
|   NATIVE_SYSROOT:               /opt/acrn-yocto/build/tmp/work/apl_nuc-poky-linux/sos-image-weston/1.0-r0/recipe-sysroot-native
| 
| INFO: The image(s) were created using OE kickstart file:
|   /opt/acrn-yocto/sources/poky/../meta-acrn/scripts/lib/wic/canned-wks/acrn-systemd-bootdisk-microcode.wks
| Traceback (most recent call last):
|   File "/opt/acrn-yocto/sources/poky/scripts/lib/wic/filemap.py", line 393, in _invoke_fiemap
|     fcntl.ioctl(self._f_image, _FIEMAP_IOCTL, self._buf, 1)
| OSError: [Errno 95] Operation not supported
| 
| During handling of the above exception, another exception occurred:
| 
| Traceback (most recent call last):
|   File "/opt/acrn-yocto/sources/poky/scripts/lib/wic/filemap.py", line 533, in filemap
|     return FilemapFiemap(image, log)
|   File "/opt/acrn-yocto/sources/poky/scripts/lib/wic/filemap.py", line 367, in __init__
|     self.block_is_mapped(0)
|   File "/opt/acrn-yocto/sources/poky/scripts/lib/wic/filemap.py", line 414, in block_is_mapped
|     struct_fiemap = self._invoke_fiemap(block, 1)
|   File "/opt/acrn-yocto/sources/poky/scripts/lib/wic/filemap.py", line 401, in _invoke_fiemap
|     raise ErrorNotSupp(errstr)
| wic.filemap.ErrorNotSupp: FilemapFiemap: the FIEMAP ioctl is not supported by the file-system
| 
| During handling of the above exception, another exception occurred:
| 
| Traceback (most recent call last):
|   File "/opt/acrn-yocto/sources/poky/scripts/wic", line 540, in <module>
|     sys.exit(main(sys.argv[1:]))
|   File "/opt/acrn-yocto/sources/poky/scripts/wic", line 535, in main
|     return hlp.invoke_subcommand(args, parser, hlp.wic_help_usage, subcommands)
|   File "/opt/acrn-yocto/sources/poky/scripts/lib/wic/help.py", line 98, in invoke_subcommand
|     subcmd[0](args, usage)
|   File "/opt/acrn-yocto/sources/poky/scripts/wic", line 271, in wic_rm_subcommand
|     engine.wic_rm(args, args.native_sysroot)
|   File "/opt/acrn-yocto/sources/poky/scripts/lib/wic/engine.py", line 544, in wic_rm
|     disk.remove(args.path.part, args.path.path)
|   File "/opt/acrn-yocto/sources/poky/scripts/lib/wic/engine.py", line 376, in remove
|     self._put_part_image(pnum)
|   File "/opt/acrn-yocto/sources/poky/scripts/lib/wic/engine.py", line 319, in _put_part_image
|     seek=self.partitions[pnum].start)
|   File "/opt/acrn-yocto/sources/poky/scripts/lib/wic/filemap.py", line 551, in sparse_copy
|     fmap = api(src_fname)
|   File "/opt/acrn-yocto/sources/poky/scripts/lib/wic/filemap.py", line 535, in filemap
|     return FilemapSeek(image, log)
|   File "/opt/acrn-yocto/sources/poky/scripts/lib/wic/filemap.py", line 214, in __init__
|     self._probe_seek_hole()
|   File "/opt/acrn-yocto/sources/poky/scripts/lib/wic/filemap.py", line 249, in _probe_seek_hole
|     raise ErrorNotSupp("the file-system does not support "
| wic.filemap.ErrorNotSupp: the file-system does not support "SEEK_HOLE" and "SEEK_DATA" but only provides a stub implementation
| WARNING: /opt/acrn-yocto/build/tmp/work/apl_nuc-poky-linux/sos-image-weston/1.0-r0/temp/run.do_image_wic.12779:1 exit 1 from 'wic rm sos-image-weston-apl-nuc-20181108070812.rootfs.wic:1/EFI/BOOT/bootx64.efi --native-sysroot /opt/acrn-yocto/build/tmp/work/apl_nuc-poky-linux/sos-image-weston/1.0-r0/recipe-sysroot-native'
| ERROR: Function failed: do_image_wic (log file is located at /opt/acrn-yocto/build/tmp/work/apl_nuc-poky-linux/sos-image-weston/1.0-r0/temp/log.do_image_wic.12779)
ERROR: Task (/opt/acrn-yocto/sources/poky/../meta-acrn/recipes-graphics/images/sos-image-weston.bb:do_image_wic) failed with exit code '1'
NOTE: Tasks Summary: Attempted 5414 tasks of which 5365 didn't need to be rerun and 1 failed.

Summary: 1 task failed:
  /opt/acrn-yocto/sources/poky/../meta-acrn/recipes-graphics/images/sos-image-weston.bb:do_image_wic
Summary: There was 1 ERROR message shown, returning a non-zero exit code.


Reproducibility
---------------
5/5