Bug 13085

Summary: test_signing_packages failed
Product: [QA/Testing] Functional (self) Testing Reporter: Chen Qi <Qi.Chen>
Component: oe-selftestAssignee: Richard Purdie <richard.purdie>
Status: RESOLVED FIXED QA Contact:
Severity: normal    
Priority: Medium+ CC: randy.macleod, richard.purdie
Version: 2.7   
Target Milestone: 2.8   
Hardware: x86   
OS: Multiple   
Whiteboard:
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: No (bug/feature does not impact docs)

Description Chen Qi 2018-12-14 09:38:53 UTC
https://autobuilder.yoctoproject.org/typhoon/#/builders/56/builds/39/steps/7/logs/step2d

ERROR: python-2.7.15-r1 do_package_write_rpm: Function failed: sign_rpm
ERROR: bzip2-1.0.6-r5 do_package_write_rpm: Error executing a python function in exec_python_func() autogenerated:
The stack trace of python calls that resulted in this exception/failure was:
File: 'exec_python_func() autogenerated', lineno: 2, function: <module>
     0001:
 *** 0002:sign_rpm(d)
     0003:
File: '/home/pokybuild/yocto-worker/oe-selftest/build/meta/classes/sign_rpm.bbclass', lineno: 65, function: sign_rpm
     0061:                     d.getVar('RPM_GPG_PASSPHRASE'),
     0062:                     d.getVar('RPM_FILE_CHECKSUM_DIGEST'),
     0063:                     int(d.getVar('RPM_GPG_SIGN_CHUNK')),
     0064:                     d.getVar('RPM_FSK_PATH'),
 *** 0065:                     d.getVar('RPM_FSK_PASSWORD'))
     0066:}
     0067:
     0068:do_package_index[depends] += "signing-keys:do_deploy"
     0069:do_rootfs[depends] += "signing-keys:do_populate_sysroot"
File: '/home/pokybuild/yocto-worker/oe-selftest/build/meta/lib/oe/gpg_sign.py', lineno: 50, function: sign_rpms
     0046:                cmd += "--define '_file_signing_key_password %s' " % fsk_password
     0047:
     0048:        # Sign in chunks
     0049:        for i in range(0, len(files), sign_chunk):
 *** 0050:            subprocess.check_output(shlex.split(cmd + ' '.join(files[i:i+sign_chunk])), stderr=subprocess.STDOUT)
     0051:
     0052:    def detach_sign(self, input_file, keyid, passphrase_file, passphrase=None, armor=True):
     0053:        """Create a detached signature of a file"""
     0054:
File: '/usr/lib64/python3.6/subprocess.py', lineno: 336, function: check_output
     0332:        # empty string. That is maintained here for backwards compatibility.
     0333:        kwargs['input'] = '' if kwargs.get('universal_newlines', False) else b''
     0334:
     0335:    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
 *** 0336:               **kwargs).stdout
     0337:
     0338:
     0339:class CompletedProcess(object):
     0340:    """A process that has finished running.
File: '/usr/lib64/python3.6/subprocess.py', lineno: 418, function: run
     0414:            raise
     0415:        retcode = process.poll()
     0416:        if check and retcode:
     0417:            raise CalledProcessError(retcode, process.args,
 *** 0418:                                     output=stdout, stderr=stderr)
     0419:    return CompletedProcess(process.args, retcode, stdout, stderr)
     0420:
     0421:
     0422:def list2cmdline(seq):
Exception: subprocess.CalledProcessError: Command '['/home/pokybuild/yocto-worker/oe-selftest/build/build-st-17416/tmp/work/core2-64-poky-linux/bzip2/1.0.6-r5/recipe-sysroot-native/usr/bin/rpmsign', '--addsign', '--define', '_gpg_name testuser', '--define', '_gpg_sign_cmd_extra_args --no-permission-warning --batch --passphrase=test123 --agent-program=/home/pokybuild/yocto-worker/oe-selftest/build/build-st-17416/tmp/work/core2-64-poky-linux/bzip2/1.0.6-r5/recipe-sysroot-native/usr/bin/gpg-agent|--auto-expand-secmem --pinentry-mode=loopback', '--define', '_binary_filedigest_algorithm 8', '--define', '__gpg /home/pokybuild/yocto-worker/oe-selftest/build/build-st-17416/tmp/work/core2-64-poky-linux/bzip2/1.0.6-r5/recipe-sysroot-native/usr/bin/gpg', '--define', '_gpg_path /tmp/oeqa-signing-i7scv7z9', '/home/pokybuild/yocto-worker/oe-selftest/build/build-st-17416/tmp/work/core2-64-poky-linux/bzip2/1.0.6-r5/deploy-rpms/core2_64/bzip2-doc-1.0.6-r5.0.core2_64.rpm']' returned non-zero exit status 255.
Subprocess output:
gpg: signing failed: Cannot allocate memory
gpg: signing failed: Cannot allocate memory
error: gpg exec failed (2)
Comment 1 Randy MacLeod 2018-12-20 15:58:31 UTC
Robert, the amount of locked memory, from: ulimit -a | grep lock
is 16MB on my system and likely 32MB on larger servers. It's not clear how even with say 16 parallel users of gpg, we can end up with that much locked memory to result in an allocation error. It would be best to understand and document how the allocation is getting out of hand and arrange via locks perhaps to reduce contention for this scarce resource so that the error is avoided.
Comment 2 Richard Purdie 2019-06-07 22:13:34 UTC
Finally tracked this down: http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=1334b064d3f04a976a42fe9dad64831b33d93755