Bug 15415

Summary: Compiler option "-fzero-call-used-regs=used" fails for openssh-9.6p1 during linking stage on qemuppc64
Product: [Build System, Metadata & Runtime] OE-Core Reporter: William Lyu <william.lyu>
Component: connectivityAssignee: William Lyu <william.lyu>
Status: RESOLVED FIXED QA Contact:
Severity: normal    
Priority: Medium CC: meta.mr.watcher, meta.watcher, randy.macleod, sundeep.kokkonda, SunilKumar.Dora, tim.orling
Version: unspecified   
Target Milestone: 5.3   
Hardware: x86   
OS: ppc64   
Whiteboard:
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: Don't know
Attachments:
Description Flags
"openssh-9.5p1_log.do_configure": log.do_configure generated for building openssh-9.5p1
none
"openssh-9.5p1_log.do_compile": log.do_compile generated for building openssh-9.5p1
none
"openssh-9.6p1_log.do_configure": log.do_configure generated for building openssh-9.6p1
none
"openssh-9.6p1_log.do_compile": log.do_compile generated for building openssh-9.6p1 none

Description William Lyu 2024-02-29 13:56:30 UTC
From the openssh upstream, there are already several bug reports about build failure due to compiler option "-fzero-call-used-regs" during compilation stage or linking stage. This error is reported for several different architectures: ppc64el, mips64el, and x86_64-apple-darwin:
-   https://bugzilla.mindrot.org/show_bug.cgi?id=3645 (ppc64el)
-   https://bugzilla.mindrot.org/show_bug.cgi?id=3629 (x86_64-apple-darwin)
-   https://bugzilla.mindrot.org/show_bug.cgi?id=3394 (ppc64el and mips64el)

The error messages emitted in the build log are always:
    unimplemented: argument 'used' is not supported for '-fzero-call-used-regs' on this target

I believe that this is a compiler bug for the following reasons:
-   The openssh upstream has a bug report claiming that build failure with error message about "-fzero-call-used-regs" compiler option is possibly due to buggy clang (https://bugzilla.mindrot.org/show_bug.cgi?id=3475)
-   The "unimplemented" error message is only reported for certain arguably trivial functions.
-   I do not encounter build errors when doing "bitbake openssh" with 'MACHINE = "qemuppc64"' using poky version (339d362a70f4523a4b296901f5c391ea8b90019e). Note that the openssh recipe has version openssh-9.5p1 under this poky. The compiler option "-fzero-call-used-regs=used" is even used when compiling and linking, and there are no errors. I have attached the do_configure log ("openssh-9.5p1_log.do_configure") and the do_compile log ("openssh-9.5p1_log.do_compile"). Notice that, in the do_configure log, there is line "checking if powerpc64le-poky-linux-gcc ... supports compile flag -fzero-call-used-regs=all... yes" which indicates that autoconf thinks that the compiler supports the offending compiler option. As for the do_compile log, one can easily check that the offending compiler option is used.
-   All openssh upstream fixes for the aforementioned bug reports are meant to improve the autoconf compiler option checks so that such compiler option gets disabled by autoconf during checking. As explained in the previous bullet point, these autoconf compiler option checks are not really working - the check still passes for openssh-9.5p1 and openssh-9.6p1.

I am refraining from "improving" (adding more tests) in the autoconf compiler option checker, because it would be hard to come up with test cases that trigger the compiler option error due to lacking understanding the nature of the root cause of this error. This error message regarding "unimplemented" compiler option only gets triggered by some functions. The pattern of the functions that may trigger this error remains unclear.
Comment 1 William Lyu 2024-02-29 13:57:27 UTC
Created attachment 5019 [details]
"openssh-9.5p1_log.do_configure": log.do_configure generated for building openssh-9.5p1
Comment 2 William Lyu 2024-02-29 13:59:18 UTC
Created attachment 5020 [details]
"openssh-9.5p1_log.do_compile": log.do_compile generated for building openssh-9.5p1
Comment 3 William Lyu 2024-02-29 14:00:12 UTC
Created attachment 5021 [details]
"openssh-9.6p1_log.do_configure": log.do_configure generated for building openssh-9.6p1
Comment 4 William Lyu 2024-02-29 14:00:43 UTC
Created attachment 5022 [details]
"openssh-9.6p1_log.do_compile": log.do_compile generated for building openssh-9.6p1
Comment 5 Tim Orling 2024-02-29 16:00:51 UTC
We saw issues with mips32 but they do not seem the same:

https://lists.openembedded.org/g/openembedded-core/topic/103546397#194895

In this case it was fixed by turning off hardening:

https://lists.openembedded.org/g/openembedded-core/topic/104381904#195704

It is not clear this is even slightly related.
Comment 6 William Lyu 2024-02-29 16:05:13 UTC
After adding the following line to the openssh recipe, I can build openssh package on ppc64el without issues:

EXTRA_OECONF:append:powerpc64le = " --without-hardening"
Comment 7 William Lyu 2024-03-08 16:39:29 UTC
The patch disabling the compiler option is accepted by the upstream:
https://git.openembedded.org/openembedded-core/commit/meta/recipes-connectivity/openssh/openssh_9.6p1.bb?id=bc793fa9d1fe24c102d91e97b7002b6e637cbfa5
Comment 8 William Lyu 2024-03-08 16:43:43 UTC
I have built core-image-minimal (MACHINE=qemuppc64) on poky/master (b5624ee5643d881afa004571a096a189ab5389b5) with openssh and openssh-ptest.

There are no build errors now. I have ran the ptest of openssh to completion and all tests passed.
Comment 9 William Lyu 2024-03-08 20:24:51 UTC
We have a workaround that suppresses the build error at the cost of potentially degraded security of this package. Reopening this bug to track the compiler bug that impacted the usage of compiler option "-fzero-call-used-regs".
Comment 10 Randy MacLeod 2024-03-08 21:24:36 UTC
CC Sundeep in case William wants to hand the bug to him rather than have the pleasure of digging into the gcc code, email list, etc.
Comment 11 Randy MacLeod 2024-03-14 15:07:21 UTC
William will look into this initially.
Comment 12 Randy MacLeod 2025-05-15 18:40:05 UTC
Sunil, can you review this issue, before Thursday May 22?
Please talk with William if needed and make a recommendation about whether we 
should keep this bug open or accept that we have to disabling the compiler option.
Comment 13 Sermsity Sunil Kumar Dora 2025-05-16 08:18:09 UTC
Sure, Randy - I'm looking into this and will update the thread soon.
Comment 14 Sermsity Sunil Kumar Dora 2025-05-22 12:17:30 UTC
The compiler option -fzero-call-used-regs=used fails during linking of openssh-9.6p1 on qemuppc64. This is a known GCC limitation for PowerPC architectures, as confirmed by:

GCC test cases explicitly skipping these tests on PowerPC 
(https://gcc.gnu.org/git/?p=gcc.git;a=commitdiff;h=96fc91785ef1e4d52487e6b29787954d1df042a8).

Expected error: 
unimplemented: argument 'used' is not supported for '-fzero-call-used-regs' on this target.

Supported Architectures:
GCC currently supports this flag(fzero-call-used-regs=used) only for:
---------------------------
32/64-bit x86 (i?86, x86_64)
SPARC
AArch64
NVIDIA PTX
IBM s390
LoongArch64

OpenSSH Context:
---------------
OpenSSH 9.6p1+: (From Release Notes)
Improved detection of unstable flags (including -fzero-call-used-regs).

Earlier versions (e.g., 9.5p1): May have "passed" due to weaker autoconf checks (To reconfirm).
Upstream fixes: Focus on autoconf improvements, not compiler workarounds.

Current Status:
Workaround: --without-hardening (merged upstream, but reduces security).
Root Cause: Architectural limitation in GCC’s PowerPC backend.

ACCEPTED status retained (OE-Core cannot fix GCC limitations).

Engagement with GCC: I am discussing with PowerPC maintainers to clarify future support feasibility. Updates will follow.
Comment 15 Randy MacLeod 2025-05-22 15:03:52 UTC
Move to 5.3 based on Sunil's helpful comment and the fact that he is asking upstream if a fix is possible or likely.
Comment 16 Sermsity Sunil Kumar Dora 2025-05-23 05:37:55 UTC
Reply from PowerPC maintainer:
-----------------------------
If you look at the Git history, it was disabled on Darwin because of an "unimplemented, sorry" failure.  Iain limited the testsuite change to Darwin, but the feature is not implemented in the PowerPC backend and there are no plans to implement it.

Thanks, David
Comment 17 Randy MacLeod 2025-05-23 14:36:01 UTC
Thanks Sunil.

We have the work-around so I'm going to resolve the defect since William's
work-around was accepted long ago:

https://git.openembedded.org/openembedded-core/commit/meta/recipes-connectivity/openssh/openssh_9.6p1.bb?id=bc793fa9d1fe24c102d91e97b7002b6e637cbfa5

oe-core.git on master
❯ git branch -a --contains bc793fa9d1fe24c102d91e97b7002b6e637cbfa5
* master
  remotes/origin/HEAD -> origin/master
  remotes/origin/master
  remotes/origin/master-next
  remotes/origin/scarthgap
  remotes/origin/styhead
  remotes/origin/walnascar
  remotes/origin/walnascar-next