Bug 10783

Summary: wic should not be writing out to /var/tmp/wic
Product: [Build System, Metadata & Runtime] OE-Core Reporter: Paul Eggleton <bluelightning>
Component: Scripts and ToolsAssignee: Ed Bartosh <eduard.bartosh>
Status: RESOLVED FIXED QA Contact:
Severity: minor    
Priority: Medium+ CC: brian.avery, mikko.ylinen, olev.kartau, ross.burton, sgw
Version: 2.2   
Target Milestone: 2.3 M3   
Hardware: x86   
OS: Multiple   
Whiteboard:
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: Yes (doc changes required)

Description Paul Eggleton 2016-12-11 22:49:21 UTC
wic by default wants to write to /var/tmp/wic. Hardcoding this slightly unusual temp path doesn't seem right to me, and in any case it breaks if two users try to write to it - the first one that creates it will own it preventing the second from writing to it.
Comment 1 Ed Bartosh 2016-12-12 11:39:33 UTC
I can fix it by making specifying output directory mandatory.
However, it can hurt users who run wic without -o option and expect results in /var/tmp/wic

Do you have a better idea how to fix this?
Comment 2 brian avery 2016-12-15 15:42:12 UTC
what about using mktemp?

i.e. write to /var/tmp/wic-XXX

via mktemp -d --tmpdir=/var/tmp wic-XXXX
Comment 3 Paul Eggleton 2016-12-15 20:22:48 UTC
(In reply to comment #2)
> what about using mktemp?
> 
> i.e. write to /var/tmp/wic-XXX

If /var/tmp itself is owned by someone else that'll still break.

I'm not sure how to reconcile this with "existing users expect /var/tmp". We do make backwards-incompatible changes elsewhere from time to time though, if we have to warn people now and then change it in a couple of releases that would be fine. We'd also naturally document it in the migration guide when it does happen.
Comment 4 Ed Bartosh 2017-01-24 14:10:37 UTC
please, review the fix: http://lists.openembedded.org/pipermail/openembedded-core/2017-January/131679.html
Comment 5 Ed Bartosh 2017-01-30 19:10:13 UTC
moved to M3. we're late for M2.
Comment 6 Ed Bartosh 2017-02-01 10:04:38 UTC
The patch has been merged to master.
Comment 7 Mikko Ylinen 2017-02-01 10:38:47 UTC
(In reply to comment #6)
> The patch has been merged to master.

There's some problem with it:

00:40:53.815 [build_intel-corei7-64] | During handling of the above exception, another exception occurred:
00:40:53.815 [build_intel-corei7-64] | 
00:40:53.815 [build_intel-corei7-64] | Traceback (most recent call last):
00:40:53.815 [build_intel-corei7-64] |   File "/srv/jenkins/workspace/builder-slot-0/openembedded-core/scripts/wic", line 330, in <module>
00:40:53.815 [build_intel-corei7-64] |     sys.exit(main(sys.argv[1:]))
00:40:53.815 [build_intel-corei7-64] |   File "/srv/jenkins/workspace/builder-slot-0/openembedded-core/scripts/wic", line 325, in main
00:40:53.815 [build_intel-corei7-64] |     return hlp.invoke_subcommand(args, parser, hlp.wic_help_usage, subcommands)
00:40:53.815 [build_intel-corei7-64] |   File "/srv/jenkins/workspace/builder-slot-0/openembedded-core/scripts/lib/wic/help.py", line 95, in invoke_subcommand
00:40:53.815 [build_intel-corei7-64] |     subcommands.get(args[0], subcommand_error)[0](args[1:], usage)
00:40:53.815 [build_intel-corei7-64] |   File "/srv/jenkins/workspace/builder-slot-0/openembedded-core/scripts/wic", line 258, in wic_create_subcommand
00:40:53.815 [build_intel-corei7-64] |     options.compressor, options.bmap, options.debug)
00:40:53.815 [build_intel-corei7-64] |   File "/srv/jenkins/workspace/builder-slot-0/openembedded-core/scripts/lib/wic/engine.py", line 198, in wic_create
00:40:53.815 [build_intel-corei7-64] |     crobj.main(cmdline)
00:40:53.815 [build_intel-corei7-64] |   File "/srv/jenkins/workspace/builder-slot-0/openembedded-core/scripts/lib/wic/creator.py", line 125, in main
00:40:53.815 [build_intel-corei7-64] |     return self._subcmds[pname](options, *args[1:])
00:40:53.815 [build_intel-corei7-64] |   File "/srv/jenkins/workspace/builder-slot-0/openembedded-core/scripts/lib/wic/plugins/imager/direct_plugin.py", line 101, in do_create
00:40:53.815 [build_intel-corei7-64] |     creator.cleanup()
00:40:53.815 [build_intel-corei7-64] |   File "/srv/jenkins/workspace/builder-slot-0/openembedded-core/scripts/lib/wic/imager/direct.py", line 395, in cleanup
00:40:53.816 [build_intel-corei7-64] |     for fname in os.listdir(self.workdir):
00:40:53.816 [build_intel-corei7-64] | FileNotFoundError: [Errno 2] No such file or directory: '/tmp/wicxbm1fyav'
Comment 8 Olev Kartau 2017-02-01 12:04:46 UTC
I guess that
tempfile.mktemp(prefix='wic')

creates a file, where we actually need directory?
Comment 9 Olev Kartau 2017-02-01 12:08:58 UTC
I think Mikko copied (2 comments above) a piece which is not most relevant,
the immediate issue reported in build was:

mkdosfs: unable to create /tmp/wicxbm1fyav/rootfs_primary_uefi.1.vfat

and that leads to my guess that tmpdir was not created
Comment 10 Olev Kartau 2017-02-02 11:38:23 UTC
correcting my comment, tempfile.mktemp() seems to create nothing.
Even by doc, this function only returns temporary file name,
without creating, and is also deprecated.
mkdtemp should be used to create a directory,
this has fixed issue in local build
Comment 11 Ed Bartosh 2017-02-02 14:06:17 UTC
Should be fixed by this patchset: http://lists.openembedded.org/pipermail/openembedded-core/2017-February/132189.html

Please, review.