Bug 12319

Summary: WIC: ZeroDivisionError when writing to Docker container volume residing on AUFS filesystem
Product: [Build System, Metadata & Runtime] OE-Core Reporter: James Kelly <jamespeterkelly+yocto>
Component: Scripts and ToolsAssignee: Armin Kuster <akuster>
Status: CLOSED FIXED QA Contact:
Severity: normal    
Priority: Medium CC: akuster, anuj.mittal, ross.burton
Version: 2.2.2   
Target Milestone: 4.99   
Hardware: Other   
OS: arm   
Whiteboard: Backport 2.2.3, 2.3.3, and 2.4.1?
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: No (bug/feature does not impact docs)

Description James Kelly 2017-11-08 00:58:28 UTC
Preconditions/Environment
-------------------------
Yocto build environment obtained from https://github.com/Xilinx/poky which is
based on Yocto 2.2.2.
Running in a Docker container derived from Ubuntu 16.04 image.
Docker host is using AUFS filesystem for Docker volumes.
Docker host is running Boot2Docker version 17.10.0-ce.Boot2Docker version 17.10.0-ce, build HEAD : 34fe485 - Wed Oct 18 17:16:34 UTC 2017 in a VM.

WIC is enabled in build/conf/local.conf using IMAGE_FSTYPES, WKS_SEARCH_PATH and WKS_FILES

Triggering Action/Cause
-----------------------
bitbake core-image-minimal

Expectation
-----------
A disk image is created by WIC from the build 

Actual Result
-------------
Build stops in meta/recipes-core/images/core-image-minimal.bb:do_image_wic
recipe with error:

| Traceback (most recent call last):
|   File "/home/user/sources/core/scripts/wic", line 319, in <module>
|     sys.exit(main(sys.argv[1:]))
|   File "/home/user/sources/core/scripts/wic", line 314, in main
|     return hlp.invoke_subcommand(args, parser, hlp.wic_help_usage, subcommands)
|   File "/home/user/sources/core/scripts/lib/wic/help.py", line 95, in invoke_subcommand
|     subcommands.get(args[0], subcommand_error)[0](args[1:], usage)
|   File "/home/user/sources/core/scripts/wic", line 247, in wic_create_subcommand
|     options.compressor, options.bmap, options.debug)
|   File "/home/user/sources/core/scripts/lib/wic/engine.py", line 195, in wic_create
|     crobj.main(cmdline)
|   File "/home/user/sources/core/scripts/lib/wic/creator.py", line 125, in main
|     return self._subcmds[pname](options, *args[1:])
|   File "/home/user/sources/core/scripts/lib/wic/plugins/imager/direct_plugin.py", line 94, in do_create
|     creator.assemble()
|   File "/home/user/sources/core/scripts/lib/wic/imager/direct.py", line 330, in assemble
|     self.__image.assemble(full_path)
|   File "/home/user/sources/core/scripts/lib/wic/utils/partitionedfs.py", line 361, in assemble
|     sparse_copy(source, image_file, part['start'] * self.sector_size)
|   File "/home/user/sources/core/scripts/lib/wic/filemap.py", line 535, in sparse_copy
|     fmap = filemap(src_fname)
|   File "/home/user/sources/core/scripts/lib/wic/filemap.py", line 529, in filemap
|     return FilemapFiemap(image, log)
|   File "/home/user/sources/core/scripts/lib/wic/filemap.py", line 347, in __init__
|     _FilemapBase.__init__(self, image, log)
|   File "/home/user/sources/core/scripts/lib/wic/filemap.py", line 98, in __init__
|     self.blocks_cnt //= self.block_size
| ZeroDivisionError: integer division or modulo by zero

Reproducibility
---------------
5/5

Root Cause
----------
In function get_block_size in scripts/lib/wic/filemap.py the FIGETBSZ IOCTL is used to get the block size of the file system where WIC is creating the image.

In a Docker container with volumes created in a AUFS file system this IOCTL returns zero resulting in a divide by zero when the block size is subsequently used.

Suggested Fix
-------------

The Intel/bmap-tools project on Github had the same issue (it seems to have VERY similar code) and it was fixed with this commit that uses fstat to get the
block size if the FIGETBSZ IOCTL returns zero.

https://github.com/intel/bmap-tools/commit/17365f4fe9089df7ee9800a2a0ced177ec4798a4
Comment 1 Anuj Mittal 2018-07-17 01:39:38 UTC
Fixed back in Jan 2018: https://git.yoctoproject.org/cgit/cgit.cgi/poky/commit?id=bab3b2e9390d632a190bb96fa161bd5554a72948

It's in sumo. Should this be backported to rocko/pyro as well?
Comment 2 Armin Kuster 2018-07-30 17:52:02 UTC
(In reply to comment #1)
> Fixed back in Jan 2018:
> https://git.yoctoproject.org/cgit/cgit.cgi/poky/
> commit?id=bab3b2e9390d632a190bb96fa161bd5554a72948
> 
> It's in sumo. Should this be backported to rocko/pyro as well?

got it in rocko-nmut
Comment 3 Armin Kuster 2018-12-12 15:59:33 UTC
(In reply to comment #2)
> (In reply to comment #1)
> > Fixed back in Jan 2018:
> > https://git.yoctoproject.org/cgit/cgit.cgi/poky/
> > commit?id=bab3b2e9390d632a190bb96fa161bd5554a72948
> > 
> > It's in sumo. Should this be backported to rocko/pyro as well?
> 
> got it in rocko-nmut

in rocko proper

http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?h=rocko&id=53d5f01d2dc34ac140f985aef0fbe0c407394a2c
Comment 4 Armin Kuster 2019-04-09 08:11:34 UTC
not fixing anything past rocko.
Comment 5 Armin Kuster 2019-04-09 08:12:18 UTC
closing