<?xml version="1.0" encoding="UTF-8" standalone="yes" ?>
<!DOCTYPE bugzilla SYSTEM "https://bugzilla.yoctoproject.org/page.cgi?id=bugzilla.dtd">

<bugzilla version="5.0.6"
          urlbase="https://bugzilla.yoctoproject.org/"
          
          maintainer="it-coreprojects-helpdesk@linuxfoundation.org"
>

    <bug>
          <bug_id>13810</bug_id>
          
          <creation_ts>2020-02-22 17:05:49 +0000</creation_ts>
          <short_desc>systemd file executable bits disappearing</short_desc>
          <delta_ts>2021-03-16 19:04:10 +0000</delta_ts>
          <reporter_accessible>1</reporter_accessible>
          <cclist_accessible>1</cclist_accessible>
          <classification_id>7</classification_id>
          <classification>Build System, Metadata &amp; Runtime</classification>
          <product>OE-Core</product>
          <component>core</component>
          <version>3.0</version>
          <rep_platform>x86</rep_platform>
          <op_sys>Multiple</op_sys>
          <bug_status>RESOLVED</bug_status>
          <resolution>FIXED</resolution>
          
          
          <bug_file_loc></bug_file_loc>
          <status_whiteboard></status_whiteboard>
          <keywords></keywords>
          <priority>Medium+</priority>
          <bug_severity>normal</bug_severity>
          <target_milestone>3.3 M3</target_milestone>
          
          
          <everconfirmed>1</everconfirmed>
          <reporter name="Richard Purdie">richard.purdie</reporter>
          <assigned_to name="Randy MacLeod">randy.macleod</assigned_to>
          <cc>anuj.mittal</cc>
    
    <cc>flowergom</cc>
    
    <cc>meta.mr.watcher</cc>
    
    <cc>meta.watcher</cc>
    
    <cc>randy.macleod</cc>
    
    <cc>richard.purdie</cc>
    
    <cc>steve</cc>
    
    <cc>yang.wang</cc>
          
          
          <cf_os>---</cf_os>
          <cf_regression_type>---</cf_regression_type>
          
          <cf_docchange>No (bug/feature does not impact docs)</cf_docchange>

      

      

      

          <comment_sort_order>oldest_to_newest</comment_sort_order>  
          <long_desc isprivate="0" >
    <commentid>86559</commentid>
    <comment_count>0</comment_count>
    <who name="Richard Purdie">richard.purdie</who>
    <bug_when>2020-02-22 17:05:49 +0000</bug_when>
    <thetext>We have a set of autobuilder failures:

https://autobuilder.yoctoproject.org/typhoon/#/builders/72/builds/1637
https://autobuilder.yoctoproject.org/typhoon/#/builders/72/builds/1638
https://autobuilder.yoctoproject.org/typhoon/#/builders/72/builds/1639

where a service fails to run at boot. The issue is that /lib/systemd/systemd-remount-fs does not have its executable bit set.

pokybuild@ubuntu1804-ty-2:~/yocto-worker/qa-extras2/build/build/tmp/work/core2-64-poky-linux/systemd/1_244.3-r0/package/lib/systemd$ ls -la
total 7768
drwxr-xr-x 10 pokybuild pokybuild    4096 Feb 22 13:23 .
drwxr-xr-x  6 pokybuild pokybuild    4096 Feb 22 13:23 ..
drwxr-xr-x  2 pokybuild pokybuild    4096 Feb 22 13:23 .debug
-rwxr-xr-x  2 pokybuild pokybuild 2208952 Feb 22 13:23 libsystemd-shared-244.so
drwxr-xr-x  2 pokybuild pokybuild    4096 Feb 22 13:23 network
drwxr-xr-x  2 pokybuild pokybuild    4096 Feb 22 13:23 ntp-units.d
-rw-r--r--  2 pokybuild pokybuild     692 Feb 22 12:37 resolv.conf
drwxr-xr-x 19 pokybuild pokybuild   12288 Feb 22 13:23 system
-rwxr-xr-x  2 pokybuild pokybuild 1562680 Feb 22 13:23 systemd
-rw-r--r--  2 pokybuild pokybuild   33616 Feb 22 13:23 systemd-ac-power
-rwxr-xr-x  2 pokybuild pokybuild   22488 Feb 22 13:23 systemd-backlight
-rwxr-xr-x  2 pokybuild pokybuild   14368 Feb 22 13:23 systemd-binfmt
-rwxr-xr-x  2 pokybuild pokybuild   14384 Feb 22 13:23 systemd-boot-check-no-failures
-rwxr-xr-x  2 pokybuild pokybuild   14296 Feb 22 13:23 systemd-cgroups-agent
-rwxr-xr-x  2 pokybuild pokybuild   18576 Feb 22 13:23 systemd-dissect
-rwxr-xr-x  2 pokybuild pokybuild   26584 Feb 22 13:23 systemd-fsck
-rwxr-xr-x  2 pokybuild pokybuild   14368 Feb 22 13:23 systemd-growfs
-rwxr-xr-x  2 pokybuild pokybuild   14304 Feb 22 13:23 systemd-hibernate-resume
-rwxr-xr-x  2 pokybuild pokybuild   30760 Feb 22 13:23 systemd-hostnamed
-rw-r--r--  2 pokybuild pokybuild   54944 Feb 22 13:23 systemd-initctl
-rwxr-xr-x  2 pokybuild pokybuild  153656 Feb 22 13:23 systemd-journald
-rwxr-xr-x  2 pokybuild pokybuild   47136 Feb 22 13:23 systemd-localed
-rwxr-xr-x  2 pokybuild pokybuild  260200 Feb 22 13:23 systemd-logind
-rw-r--r--  2 pokybuild pokybuild  525056 Feb 22 13:23 systemd-machined
-rwxr-xr-x  2 pokybuild pokybuild   14288 Feb 22 13:23 systemd-makefs
-rwxr-xr-x  2 pokybuild pokybuild   18576 Feb 22 13:23 systemd-modules-load
-rwxr-xr-x  2 pokybuild pokybuild  891184 Feb 22 13:23 systemd-networkd
-rwxr-xr-x  2 pokybuild pokybuild   30904 Feb 22 13:23 systemd-networkd-wait-online
-rwxr-xr-x  2 pokybuild pokybuild   34864 Feb 22 13:23 systemd-network-generator
-rwxr-xr-x  2 pokybuild pokybuild   22584 Feb 22 13:23 systemd-pstore
-rwxr-xr-x  2 pokybuild pokybuild   14376 Feb 22 13:23 systemd-quotacheck
-rwxr-xr-x  2 pokybuild pokybuild   18392 Feb 22 13:23 systemd-random-seed
-rw-r--r--  2 pokybuild pokybuild   80792 Feb 22 13:23 systemd-remount-fs
-rwxr-xr-x  2 pokybuild pokybuild   14296 Feb 22 13:23 systemd-reply-password
-rwxr-xr-x  2 pokybuild pokybuild  399400 Feb 22 13:23 systemd-resolved
-rwxr-xr-x  2 pokybuild pokybuild   22560 Feb 22 13:23 systemd-rfkill
-rwxr-xr-x  2 pokybuild pokybuild   51248 Feb 22 13:23 systemd-shutdown
-rwxr-xr-x  2 pokybuild pokybuild   26760 Feb 22 13:23 systemd-sleep
-rwxr-xr-x  2 pokybuild pokybuild   30768 Feb 22 13:23 systemd-socket-proxyd
-rwxr-xr-x  2 pokybuild pokybuild   14296 Feb 22 13:23 systemd-sulogin-shell
-rwxr-xr-x  2 pokybuild pokybuild   22664 Feb 22 13:23 systemd-sysctl
-rwxr-xr-x  2 pokybuild pokybuild    1258 Feb 22 13:23 systemd-sysv-install
-rw-r--r--  2 pokybuild pokybuild  148824 Feb 22 13:23 systemd-timedated
-rwxr-xr-x  2 pokybuild pokybuild   55336 Feb 22 13:23 systemd-timesyncd
-rwxr-xr-x  2 pokybuild pokybuild   14296 Feb 22 13:23 systemd-time-wait-sync
-rwxr-xr-x  2 pokybuild pokybuild  784920 Feb 22 13:23 systemd-udevd
-rwxr-xr-x  2 pokybuild pokybuild   14296 Feb 22 13:23 systemd-update-done
-rwxr-xr-x  2 pokybuild pokybuild   14376 Feb 22 13:23 systemd-update-utmp
-rwxr-xr-x  2 pokybuild pokybuild   18400 Feb 22 13:23 systemd-user-runtime-dir
-rwxr-xr-x  2 pokybuild pokybuild   18392 Feb 22 13:23 systemd-vconsole-setup
-rwxr-xr-x  2 pokybuild pokybuild   14296 Feb 22 13:23 systemd-volatile-root
drwxr-xr-x  3 pokybuild pokybuild    4096 Feb 22 13:23 system-generators
drwxr-xr-x  2 pokybuild pokybuild    4096 Feb 22 13:23 system-preset
drwxr-xr-x  2 pokybuild pokybuild    4096 Feb 22 13:23 system-shutdown
drwxr-xr-x  2 pokybuild pokybuild    4096 Feb 22 13:23 system-sleep

There are a few other files in that directory that aren&apos;t executable and probably should be. I&apos;ve tracked this back to a do_package sstate artefact. We don&apos;t have the original build that created this unfortunately but its clearly something that went wrong in the build of systemd.</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>86560</commentid>
    <comment_count>1</comment_count>
    <who name="Richard Purdie">richard.purdie</who>
    <bug_when>2020-02-22 17:06:30 +0000</bug_when>
    <thetext>I appreciate I listed this outside pseudo but it doesn&apos;t matter, the broken permissions are the same inside pseudo.</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>86561</commentid>
    <comment_count>2</comment_count>
    <who name="Richard Purdie">richard.purdie</who>
    <bug_when>2020-02-22 17:25:31 +0000</bug_when>
    <thetext>I believe it was built on centos7-ty-1 but trying a build there didn&apos;t result in the same permissions issue. Had to resort to a cleansstate on systemd on the infrastructure to clear the corrupt sstate out. Still no idea how this got in :(</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>86613</commentid>
    <comment_count>3</comment_count>
    <who name="Randy MacLeod">randy.macleod</who>
    <bug_when>2020-02-27 16:27:08 +0000</bug_when>
    <thetext>The plan is to see if this happens again.
If anyone has seen this sort of file permissions failure, please comment.</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>86625</commentid>
    <comment_count>4</comment_count>
    <who name="Randy MacLeod">randy.macleod</who>
    <bug_when>2020-02-28 18:32:50 +0000</bug_when>
    <thetext>I check with people who know the WR testsuite and results and while we don&apos;t have a specific test, we do have test code that would fail if this problem occurred so we are almost certainly not seeing the issue.

We will add a specific test that would catch this error in the coming month or so.</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>86982</commentid>
    <comment_count>5</comment_count>
    <who name="Randy MacLeod">randy.macleod</who>
    <bug_when>2020-04-13 12:44:34 +0000</bug_when>
    <thetext>We&apos;ll see if this happens again during 3.2 but if not, I&apos;m tempted to close it as can&apos;t reproduce.</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>87199</commentid>
    <comment_count>6</comment_count>
    <who name="Anuj Mittal">anuj.mittal</who>
    <bug_when>2020-05-14 17:30:17 +0000</bug_when>
    <thetext>Something similar happened to me today while building an image. gio-querymodules from glib-2.0 didn&apos;t have executable bit set which resulted in qemu failing with a permissions denied error while generating rootfs.

Doing -c install -f glib-2.0 also didn&apos;t help. The build directory had the correct permissions set but the image directory didn&apos;t. Meson logs didn&apos;t show anything helpful either.

Forcing configure to be re-run and then install helped. Any suggestions on how to debug?</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>88543</commentid>
    <comment_count>7</comment_count>
    <who name="Randy MacLeod">randy.macleod</who>
    <bug_when>2020-11-03 10:45:13 +0000</bug_when>
    <thetext>This is likely fixed by the changes made to oe-core to:
 1. reduce the number of directories/files that pseudo manages 
cba4f51f01 pseudo: Upgrade to include mkostemp64 wrapper
ea22fb3aa1 pseudo: Update to account for patches merged on branch
016ee90e21 abi_version,sanity: Tell users TMPDIR must be clean after pseudo changes
...

f9ae80aee9 pseudo: Fix statx function usage
13500f5234 wic: Handle new PSEUDO_IGNORE_PATHS variable
ad8f5532ff base/bitbake.conf: Enable pseudo path filtering
3006bbe776 pseudo: Add pathfix patch
e1e481f360 pseudo: Add may unlink patch
929a27bf6c pseudo: Fix xattr segfault
9ce621fa20 psuedo: Add tracking of linked files for fds
2db491d97d pseudo: Abort on mismatch patch
8defb687a2 pseudo: Add support for ignoring paths from the pseudo DB
1c13149b81 pseudo: Ignore mismatched inodes from the db


and

 2. change associated pseudo warnings into errors.

https://git.openembedded.org/openembedded-core/commit/?id=2db491d97da08d44ebd257f98489550a82a7935c

I&apos;m going to leave the bug open until we review it and confirm that the error hasn&apos;t happened since these changes in late September.</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>89197</commentid>
    <comment_count>8</comment_count>
    <who name="Minjae Kim">flowergom</who>
    <bug_when>2021-02-10 14:39:36 +0000</bug_when>
    <thetext>A similar failure happened again:
https://autobuilder.yoctoproject.org/typhoon/#/builders/72/builds/3070/steps/24/logs/stdio</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>89199</commentid>
    <comment_count>9</comment_count>
    <who name="Anuj Mittal">anuj.mittal</who>
    <bug_when>2021-02-10 16:22:09 +0000</bug_when>
    <thetext>(In reply to comment #8)
&gt; A similar failure happened again:
&gt; https://autobuilder.yoctoproject.org/typhoon/#/builders/72/builds/3070/steps/
&gt; 24/logs/stdio

This looks different from the permissions problem that this bug is for.</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>89232</commentid>
    <comment_count>10</comment_count>
    <who name="Randy MacLeod">randy.macleod</who>
    <bug_when>2021-02-11 16:54:36 +0000</bug_when>
    <thetext>Anuj, I agree that:
  https://bugzilla.yoctoproject.org/show_bug.cgi?id=13810#c8
is a different issue. I&apos;m raising this bug to a M+ for 3.3-M3 so that we can 
discuss it in the coming weeks and hopefully agree that the changes to pseudo paths made late last year seem to have fixed this issue. I don&apos;t thing the same issue has happened in the last few months but since it&apos;s a rare error, I&apos;m not sure how long people want to leave the bug opened to look for it.</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>89632</commentid>
    <comment_count>11</comment_count>
    <who name="Randy MacLeod">randy.macleod</who>
    <bug_when>2021-03-16 19:04:10 +0000</bug_when>
    <thetext>I don&apos;t think we&apos;ve seen this since the series of patches mentioned earlier including:
  https://git.openembedded.org/openembedded-core/commit/?id=2db491d97d
were merged. Please re-open if you still see this happening on master or 3.3 otherwise.

Added Richard Purdie and Steve Sakoman to CC to give them a chance to object.</thetext>
  </long_desc>
      
      

    </bug>

</bugzilla>