Bug 14999

Summary: devtool fails finish upgrade when gitmodules file is empty
Product: [Build System, Metadata & Runtime] OE-Core Reporter: Thomas Roos <thomas>
Component: devtools / tool chainAssignee: Thomas Roos <thomas>
Status: RESOLVED FIXED QA Contact:
Severity: normal    
Priority: Medium+ CC: meta.mr.watcher, meta.watcher, randy.macleod
Version: 5.99   
Target Milestone: 4.2 M2   
Hardware: x86   
OS: Multiple   
Whiteboard:
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: No (bug/feature does not impact docs)

Description Thomas Roos 2023-01-05 09:49:44 UTC
when doing an devtool upgrade to an recipe getting it's source from git and contains an empty .gitmodules file the upgrade itself to the workspace works fine, but then devtool finish xxx xxx fails with this - probaly just a check if file is empty is enough...:

ERROR: ExpansionError during parsing /home/ubuntu/yocto/build/workspace/recipes/aws-c-common/aws-c-common_0.8.6.bb#####################################                                                                                                                                | ETA:  0:00:00
Traceback (most recent call last):
  File "Var <do_compile[file-checksums]>", line 1, in <module>
  File "/home/ubuntu/yocto/poky/meta/classes/externalsrc.bbclass", line 234, in srctree_hash_files(d=<bb.data_smart.DataSmart object at 0x7f034de03460>, srcdir=None):
                 if os.path.exists(os.path.join(s_dir, ".gitmodules")):
    >                submodule_helper = subprocess.check_output(["git", "config", "--file", ".gitmodules", "--get-regexp", "path"], cwd=s_dir, env=env).decode("utf-8")
                     for line in submodule_helper.splitlines():
  File "/usr/lib/python3.8/subprocess.py", line 415, in check_output(timeout=None, *popenargs=(['git', 'config', '--file', '.gitmodules', '--get-regexp', 'path'],), **kwargs={'cwd': '/home/ubuntu/yocto/build/workspace/sources/aws-c-common', 'env': {'SHELL': '/bin/bash', 'BB_ENV_PASSTHROUGH_ADDITIONS': 'ALL_PROXY BBPATH_EXTRA BB_LOGCONFIG BB_NO_NETWORK BB_NUMBER_THREADS BB_SETSCENE_ENFORCE BB_SRCREV_POLICY DISTRO FTPS_PROXY FTP_PROXY GIT_PROXY_COMMAND HTTPS_PROXY HTTP_PROXY MACHINE NO_PROXY PARALLEL_MAKE SCREENDIR SDKMACHINE SOCKS5_PASSWD SOCKS5_USER SSH_AGENT_PID SSH_AUTH_SOCK STAMPS_DIR TCLIBC TCMODE all_proxy ftp_proxy ftps_proxy http_proxy https_proxy no_proxy ', 'PWD': '/home/ubuntu/yocto/build', 'LOGNAME': 'ubuntu', 'HOME': '/home/ubuntu', 'BBPATH': '/home/ubuntu/yocto/build', 'USER': 'ubuntu', 'PATH': '/home/ubuntu/yocto/poky/scripts:/home/ubuntu/yocto/poky/bitbake/bin:/home/ubuntu/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin', 'LC_ALL': 'en_US.UTF-8', 'GIT_INDEX_FILE': '/tmp/oe-devtool-indexn1tmjq6b'}}):
     
    >    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
                    **kwargs).stdout
  File "/usr/lib/python3.8/subprocess.py", line 516, in run(input=None, capture_output=False, timeout=None, check=True, *popenargs=(['git', 'config', '--file', '.gitmodules', '--get-regexp', 'path'],), **kwargs={'stdout': -1, 'cwd': '/home/ubuntu/yocto/build/workspace/sources/aws-c-common', 'env': {'SHELL': '/bin/bash', 'BB_ENV_PASSTHROUGH_ADDITIONS': 'ALL_PROXY BBPATH_EXTRA BB_LOGCONFIG BB_NO_NETWORK BB_NUMBER_THREADS BB_SETSCENE_ENFORCE BB_SRCREV_POLICY DISTRO FTPS_PROXY FTP_PROXY GIT_PROXY_COMMAND HTTPS_PROXY HTTP_PROXY MACHINE NO_PROXY PARALLEL_MAKE SCREENDIR SDKMACHINE SOCKS5_PASSWD SOCKS5_USER SSH_AGENT_PID SSH_AUTH_SOCK STAMPS_DIR TCLIBC TCMODE all_proxy ftp_proxy ftps_proxy http_proxy https_proxy no_proxy ', 'PWD': '/home/ubuntu/yocto/build', 'LOGNAME': 'ubuntu', 'HOME': '/home/ubuntu', 'BBPATH': '/home/ubuntu/yocto/build', 'USER': 'ubuntu', 'PATH': '/home/ubuntu/yocto/poky/scripts:/home/ubuntu/yocto/poky/bitbake/bin:/home/ubuntu/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin', 'LC_ALL': 'en_US.UTF-8', 'GIT_INDEX_FILE': '/tmp/oe-devtool-indexn1tmjq6b'}}):
             if check and retcode:
    >            raise CalledProcessError(retcode, process.args,
                                          output=stdout, stderr=stderr)
bb.data_smart.ExpansionError: Failure expanding variable do_compile[file-checksums], expression was ${@srctree_hash_files(d)} which triggered exception CalledProcessError: Command '['git', 'config', '--file', '.gitmodules', '--get-regexp', 'path']' returned non-zero exit status 1.
The variable dependency chain for the failure is: do_compile[file-checksums]
Comment 1 Randy MacLeod 2023-01-05 15:42:06 UTC
Thomas, 
Are you able to submit a fix and a test case?
Comment 2 Thomas Roos 2023-01-05 16:10:30 UTC
Hi,
will try to fix this bug next week - busy till then.
Testcase is just have an repo with an empty ".gitmodules" file. Then devtool finish will throw this exception...
Comment 3 Thomas Roos 2023-01-31 08:12:29 UTC
this should have been fixed by this patch: https://lore.kernel.org/openembedded-core/20230116143022.203889-1-throos@amazon.de/

I don't know why it's not merged yet?