Bug 15690 - AB-INT: package_write_rpm gpg exec failed
Summary: AB-INT: package_write_rpm gpg exec failed
Status: RESOLVED OBSOLETE
Alias: None
Product: OE-Core
Classification: Build System, Metadata & Runtime
Component: core (show other bugs)
Version: 5.99
Hardware: x86 Multiple
: Medium normal
Target Milestone: 6.1
Assignee: Unassigned
QA Contact:
URL:
Whiteboard: AB-INT
Depends on:
Blocks:
 
Reported: 2024-12-12 09:50 UTC by Mathieu Dubois-Briand
Modified: 2026-07-30 15:19 UTC (History)
7 users (show)

See Also:
OS type for building Yocto: ---
Type of Regression: ---
Verified:
Documentation change: No (bug/feature does not impact docs)


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Mathieu Dubois-Briand 2024-12-12 09:50:53 UTC
> 2063 ERROR: ed-1.20.2-r0 do_package_write_rpm: Error executing a python function in exec_func_python() autogenerated:
> ...
> 2112 Subprocess output:
> 2113 warning: Could not set GPG_TTY to stdin: Inappropriate ioctl for device
> 2114 gpg: signing failed: Corrupted protection
> 2115 gpg: signing failed: Corrupted protection
> 2116 error: gpg exec failed (2)

https://valkyrie.yoctoproject.org/#/builders/76/builds/566
Comment 1 Mathieu Dubois-Briand 2024-12-12 09:53:19 UTC
oe-selftest-centos alma9-vk-1 master completed at 2024-12-08T02:22:17Z
https://valkyrie.yoctoproject.org/#/builders/76/builds/566/steps/14/logs/stdio
Comment 2 Randy MacLeod 2024-12-12 15:54:08 UTC
gpg can run as a server so that may be a factor.
There were also changes in the gpg recipe.

Wait to see if the happens again.
Comment 3 Randy MacLeod 2025-05-01 14:07:11 UTC
Bulk move of all unassigned 5.2 medium importance bugs to 5.3.
Comment 4 Mathieu Dubois-Briand 2025-06-04 07:45:03 UTC
oe-selftest-fedora fedora39-vk-2 master-next completed at 2025-06-03T16:11:02Z
https://autobuilder.yoctoproject.org/valkyrie/#/builders/48/builds/1624/steps/14/logs/stdio
Comment 5 Clayton Casciato 2025-06-06 13:34:17 UTC
Our team has been seeing a similar intermittent issue for both Kirkstone and Scarthgap:
"warning: Could not set GPG_TTY to stdin: Inappropriate ioctl for device"

If we keep reattempting the build, it eventually succeeds.

Sample with redactions via [...]:
ERROR: [...] do_package_write_rpm: Error executing a python function in exec_func_python() autogenerated:
    
The stack trace of python calls that resulted in this exception/failure was:
File: 'exec_func_python() autogenerated', lineno: 2, function: <module>    
     0001:
 *** 0002:sign_rpm(d)
     0003:
File: '[...]/openembedded-core/meta/classes/sign_rpm.bbclass', lineno: 59, function: sign_rpm
     0055:
     0056:    signer = get_signer(d, d.getVar('RPM_GPG_BACKEND'))
     0057:    rpms = glob.glob(d.getVar('RPM_PKGWRITEDIR') + '/*')
     0058:
 *** 0059:    signer.sign_rpms(rpms,
     0060:                     d.getVar('RPM_GPG_NAME'),
     0061:                     d.getVar('RPM_GPG_PASSPHRASE'),
     0062:                     d.getVar('RPM_FILE_CHECKSUM_DIGEST'),
     0063:                     int(d.getVar('RPM_GPG_SIGN_CHUNK')),
File: '[...]/openembedded-core/meta/lib/oe/gpg_sign.py', lineno: 59, function: sign_rpms
     0055:                cmd += "--define '_file_signing_key_password %s' " % fsk_password
     0056:
     0057:        # Sign in chunks
     0058:        for i in range(0, len(files), sign_chunk):
 *** 0059:            subprocess.check_output(shlex.split(cmd + ' '.join(files[i:i+sign_chunk])), stderr=subprocess.STDOUT)
     0060:
     0061:    def detach_sign(self, input_file, keyid, passphrase_file, passphrase=None, armor=True, output_suffix=None, use_sha256=False):
     0062:        """Create a detached signature of a file"""
     0063:
File: '/usr/lib/python3.10/subprocess.py', lineno: 421, function: check_output
     0417:        else:
     0418:            empty = b'' 
     0419:        kwargs['input'] = empty
     0420:
 *** 0421:    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
     0422:               **kwargs).stdout
     0423:
     0424:
     0425:class CompletedProcess(object):
File: '/usr/lib/python3.10/subprocess.py', lineno: 526, function: run 
     0522:            # We don't call process.wait() as .__exit__ does that for us. 
     0523:            raise
     0524:        retcode = process.poll()
     0525:        if check and retcode: 
 *** 0526:            raise CalledProcessError(retcode, process.args,
     0527:                                     output=stdout, stderr=stderr)
     0528:    return CompletedProcess(process.args, retcode, stdout, stderr)
     0529:
     0530:
Exception: subprocess.CalledProcessError: Command '['[...]/build/tmp-beaglebone-glibc/work/beaglebone-[...]-linux-gnueabi/packagegroup-[...]-security/1.0-r0/recipe-sysroot-native/usr/bin/rpmsign', '--addsign', '--define', '_gpg_name [...]', '--define', '_gpg_sign_cmd_extra_args --no-permission-warning --batch --passphrase=[...] --agent-program=[...]/build/tmp-beaglebone-glibc/work/beaglebone-[...]-linux-gnueabi/packagegroup-[...]-security/1.0-r0/recipe-sysroot-native/usr/bin/gpg-agent|--auto-expand-secmem --pinentry-mode=loopback', '--define', '_binary_filedigest_algorithm 8', '--define', '__gpg [...]/build/tmp-beaglebone-glibc/work/beaglebone-[...]-linux-gnueabi/packagegroup-[...]-security/1.0-r0/recipe-sysroot-native/usr/bin/gpg', '--define', '_gpg_path [...]', '[...]/build/tmp-beaglebone-glibc/work/beaglebone-[...]-linux-gnueabi/packagegroup-[...]-security/1.0-r0/deploy-rpms/beaglebone/packagegroup-[...]-security-dev-1.0-r0.beaglebone.rpm', '[...]/build/tmp-beaglebone-glibc/work/beaglebone-[...]-linux-gnueabi/packagegroup-[...]-security/1.0-r0/deploy-rpms/beaglebone/packagegroup-[...]-security-dbg-1.0-r0.beaglebone.rpm', '[...]/build/tmp-beaglebone-glibc/work/beaglebone-[...]-linux-gnueabi/packagegroup-[...]-security/1.0-r0/deploy-rpms/beaglebone/packagegroup-[...]-security-1.0-r0.beaglebone.rpm']' returned non-zero exit status 1.

Subprocess output:
warning: Could not set GPG_TTY to stdin: Inappropriate ioctl for device
warning: Could not set GPG_TTY to stdin: Inappropriate ioctl for device
warning: Could not set GPG_TTY to stdin: Inappropriate ioctl for device
gpg: signing failed: Connection reset by peer
gpg: signing failed: Connection reset by peer
error: gpg exec failed (2) 
[...]/build/tmp-beaglebone-glibc/work/beaglebone-[...]-linux-gnueabi/packagegroup-[...]-security/1.0-r0/deploy-rpms/beaglebone/packagegroup-[...]-security-dev-1.0-r0.beaglebone.rpm:
[...]/build/tmp-beaglebone-glibc/work/beaglebone-[...]-linux-gnueabi/packagegroup-[...]-security/1.0-r0/deploy-rpms/beaglebone/packagegroup-[...]-security-dbg-1.0-r0.beaglebone.rpm:
[...]/build/tmp-beaglebone-glibc/work/beaglebone-[...]-linux-gnueabi/packagegroup-[...]-security/1.0-r0/deploy-rpms/beaglebone/packagegroup-[...]-security-1.0-r0.beaglebone.rpm:

ERROR: Logfile of failure stored in: [...]/build/tmp-beaglebone-glibc/work/beaglebone-[...]-linux-gnueabi/packagegroup-21sw-security/1.0-r0/temp/log.do_package_write_rpm.1067898
ERROR: Task ([...]/meta-[...]/recipes-core/packagegroups/packagegroup-[...]-security.bb:do_package_write_rpm) failed with exit code '1'
Comment 6 Randy MacLeod 2025-06-18 19:05:43 UTC
Clayton,

How often does it happen out of say 10 or 100 builds for you?
What is your build host distro and HW (cpus, mem, storage)?

Have you tried stracing all or part of the failure and
comparing it to when the build works? :-)
Comment 7 Clayton Casciato 2025-07-03 13:52:06 UTC
Hi, Randy

We've seen this on multiple Ubuntu systems (typically 22.04) including:
Ryzen 7000 Series, 96 GB RAM, PCIe 4.0 NVMe storage
Intel 14th gen mobile, 96 GB RAM, PCIe 4.0 NVMe storage

I haven't seen this for a while, though signed builds are ad-hoc (partially due to this issue). From my memory, it could happen 6 build attempts in a row (or work just fine).

Debugging attempts have mostly been focused on GPG (no luck); we haven't tried strace (but will keep it in mind for the future).

I'm wondering if this problem surfaces with rpm-sequoia: https://git.openembedded.org/openembedded-core/tree/meta/recipes-devtools/rpm-sequoia?h=walnascar
Comment 8 levi.shafter@elder-tomes.com 2025-09-16 21:32:58 UTC
I have been investigating this issue, but I'm stumped.

I noticed that the command generated by the `LocalSigner.sign_rpms()' method in `openembedded-core: gpg_sign.py` was erroneous:


--agent-program=[...]/build/tmp-beaglebone-glibc/work/armv7at2hf-neon-21software-linux-gnueabi/p11-kit/0.25.3/recipe-sysroot-native/usr/bin/gpg-agent|--auto-expand-secmem

Upon replacing the | character with a space in the script, the same error was still intermittently observed.

I took an `strace` of our build process, but the only relevant logging I could find was in `bitbake-cookerdaemon.log`:


0:openat(AT_FDCWD,
         "[...]/build/bitbake-cookerdaemon.log",
         O_RDWR|O_CREAT|O_APPEND|O_CLOEXEC, 0666) = 6

1580578:11:fstat(6, {st_mode=S_IFREG|0664, st_size=411537764, ...}) = 0

1580578:12:lseek(6, 0, SEEK_END) = 411537764

1580578:13:ioctl(6, TCGETS, 0x7fff62fbd650) = -1 ENOTTY (Inappropriate ioctl for device)

Upon investigating `bitbake-cookerdaemon.log`:

11270 12:59:30.506658 Exception in server main event loop running command [] (Traceback (most recent call last):
  File "[...]/bitbake/lib/bb/server/process.py", line 298, in main
    self.command_channel_reply.send(reply)
  File "[...]/bitbake/lib/bb/server/process.py", line 890, in send
    self._send(obj)
  File "[...]/bitbake/lib/bb/server/process.py", line 869, in _send
    self.writer.send_bytes(obj)
  File "/usr/lib/python3.12/multiprocessing/connection.py", line 200, in send_bytes
    self._send_bytes(m[offset:offset + size])
  File "/usr/lib/python3.12/multiprocessing/connection.py", line 427, in _send_bytes
    self._send(header + buf)
  File "/usr/lib/python3.12/multiprocessing/connection.py", line 384, in _send
    n = write(self._handle, buf)
        ^^^^^^^^^^^^^^^^^^^^^^^^
BrokenPipeError: [Errno 32] Broken pipe

Tracing code execution, the only method that appears to be capable of throwing a BrokenPipeError is `ConnectionWriter._send` in `bitbake/lib/bb/server/process.py`. Thinking perhaps the issue is being caused by concurrent signing tasks during the build process, I tried serializing the `do_package_write_rpm` tasks, but the same issue intermittently persists.
Comment 9 Clayton Casciato 2025-09-26 16:46:19 UTC
After backporting rpm-sequoia to a local Scarthgap build, there are still intermittent failures.

The stack trace is the same as with rpm.bb's "internal-openpgp".

However, there is a new line at the start of the subprocess output:

couldn't allocate absolute path for 'null'.
warning: Could not set GPG_TTY to stdin: Inappropriate ioctl for device
gpg: problem with fast path key listing: End of file - ignored
gpg: keydb_search failed: Broken pipe
gpg: skipped "[REDACTED_KEY]": Broken pipe
gpg: signing failed: Broken pipe
error: gpg exec failed (2)

NB: the specific subprocess output varies and we've seen this one before (with "internal-openpgp").
Comment 10 Clayton Casciato 2026-04-02 15:20:14 UTC
https://rpm.org/releases/6.0.0:
"rpmsign can use either GnuPG or Sequoia-sq for signing"

"[PATCH v8 7/9] rpm: 4.20.1 -> 6.0.1":
https://lists.openembedded.org/g/openembedded-core/message/232961
Comment 11 Paul Barker 2026-07-30 15:19:02 UTC
Hi Clayton,

We discussed in YP Bug review call, we are no longer seeing the AB-INT issue in the Yocto autobuilder.

There appear to be two different failure modes:
- On the autobuilder we saw "gpg: signing failed: Corrupted protection"
- Clayton saw "signing failed: Connection reset by peer" and "signing failed: Broken pipe"

I don't think the GPG_TTY warning is likely to be the cause of the failure.

We're going to close this issue as the original failure is resolved.

Clayton, please re-open a new bug if you're still seeing failures. We've made a lot of fixes in pseudo which hopefully will resolve your issue if you update to the latest scarthgap commit.