Bug 12136 - go-dep and go-helloworld fail to link with MUSL on x86-64 machines typesj
Summary: go-dep and go-helloworld fail to link with MUSL on x86-64 machines typesj
Status: RESOLVED FIXED
Alias: None
Product: OE-Core
Classification: Build System, Metadata & Runtime
Component: devtools / tool chain (show other bugs)
Version: 2.3.2
Hardware: x86 x86_64
: High critical
Target Milestone: 2.4 M4
Assignee: Matt Madison
QA Contact:
URL:
Whiteboard:
Depends on:
Blocks:
 
Reported: 2017-09-28 12:20 UTC by Saul Wold
Modified: 2017-10-04 18:26 UTC (History)
9 users (show)

See Also:
OS type for building Yocto: ---
Type of Regression: New (Never tested)
Verified:
Documentation change: No (bug/feature does not impact docs)


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Saul Wold 2017-09-28 12:20:10 UTC
Both go-dep and go-helloworld fail in the same manner when

TCLIBC = "musl"
MACHINE = "qemux86-64" or "genericx86-64" or "intel-corei7-64" (when using meta-intel layer)

the following error is observed:

| # github.com/golang/example/hello
| build/tmp/work/corei7-64-poky-linux-musl/go-helloworld/0.1-r0/recipe-sysroot-native/usr/lib/x86_64-poky-linux-musl/go/pkg/tool/linux_amd64/link: running x86_64-poky-linux-musl-gcc failed: exit status 1
| build/tmp/work/corei7-64-poky-linux-musl/go-helloworld/0.1-r0/recipe-sysroot/usr/lib/go/pkg/linux_amd64_dynlink//libstd.so: undefined reference to `__libc_malloc'
| build/tmp/work/corei7-64-poky-linux-musl/go-helloworld/0.1-r0/recipe-sysroot/usr/lib/go/pkg/linux_amd64_dynlink//libstd.so: undefined reference to `__libc_realloc'
| build/tmp/work/corei7-64-poky-linux-musl/go-helloworld/0.1-r0/recipe-sysroot/usr/lib/go/pkg/linux_amd64_dynlink//libstd.so: undefined reference to `__libc_stack_end'
| /build/tmp/work/corei7-64-poky-linux-musl/go-helloworld/0.1-r0/recipe-sysroot/usr/lib/go/pkg/linux_amd64_dynlink//libstd.so: undefined reference to `__libc_free'
| collect2: error: ld returned 1 exit status

khem had a comment from last night:


(10:06:39 PM) khem: sgw: yes I see that problem on musl+go, this is a golang issue, they check for isAlpineLinux which actually should have been isMusl, but at this point, I think there is no solution done yet
Comment 1 Otavio Salvador 2017-09-28 21:40:14 UTC
It seems like:

https://github.com/madisongh/openembedded-core/commit/b5359413d6e272bdb35f885b468548a4143cc799

might address this.

Saul, can you confirm it fixes this issue?
Comment 2 Matt Madison 2017-09-29 14:06:25 UTC
There was a different problem with the patch Otavio referred to above, and I've got an updated one on the way to the list.
Comment 3 Saul Wold 2017-10-03 22:09:36 UTC
Confirmed that Matt's first fix still fails to build with MUSL, please let me know when the new patch is provided.  I am going to assign this to matt directly.
Comment 4 Paul Barker 2017-10-04 11:08:21 UTC
I'm getting a similar set of errors building runc-opencontainers from the meta-virtualization layer for qemux86-64 with musl libc:

| DEBUG: Executing shell function do_compile
| NOTE: make -j 8 BUILDTAGS= static
| CGO_ENABLED=1 go build -i  -tags " cgo static_build" -ldflags "-w -extldflags -static -X main.gitCommit="2e7cfe036e2c6dc51ccca6eb7fa3ee6b63976dcd" -X main.version=1.0.0-rc4 " -o runc .
| # _/home/pbarker/oryx/build/tmp/work/core2-64-oe-linux-musl/runc-opencontainers/1.0.0-rc4+git2e7cfe036e2c6dc51ccca6eb7fa3ee6b63976dcd-r0/git/src/import
| /home/pbarker/oryx/build/tmp/work/core2-64-oe-linux-musl/runc-opencontainers/1.0.0-rc4+git2e7cfe036e2c6dc51ccca6eb7fa3ee6b63976dcd-r0/recipe-sysroot-native/usr/lib/x86_64-oe-linux-musl/go/pkg/tool/linux_amd64/link: running x86_64-oe-linux-musl-gcc failed: exit status 1
| /tmp/go-link-014987806/000004.o: In function `vfprintf':
| /usr/include/x86_64-linux-gnu/bits/stdio2.h:127: undefined reference to `__vfprintf_chk'
| /tmp/go-link-014987806/000004.o: In function `fprintf':
| /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
| collect2: error: ld returned 1 exit status
|
| ERROR: oe_runmake failed
| Makefile:40: recipe for target 'static' failed
| make: *** [static] Error 2
| WARNING: exit code 1 from a shell command.

This is with Matt's patch from the oe-core mailing list applied.
Comment 5 Matt Madison 2017-10-04 15:16:04 UTC
(In reply to comment #4)
> I'm getting a similar set of errors building runc-opencontainers from the
> meta-virtualization layer for qemux86-64 with musl libc:
> 
> | DEBUG: Executing shell function do_compile
> | NOTE: make -j 8 BUILDTAGS= static
> | CGO_ENABLED=1 go build -i  -tags " cgo static_build" -ldflags "-w
> -extldflags -static -X
> main.gitCommit="2e7cfe036e2c6dc51ccca6eb7fa3ee6b63976dcd" -X
> main.version=1.0.0-rc4 " -o runc .
> | #
> _/home/pbarker/oryx/build/tmp/work/core2-64-oe-linux-musl/runc-
> opencontainers/1.0.0-rc4+git2e7cfe036e2c6dc51ccca6eb7fa3ee6b63976dcd-r0/git/
> src/import
> |
> /home/pbarker/oryx/build/tmp/work/core2-64-oe-linux-musl/runc-opencontainers/
> 1.0.0-rc4+git2e7cfe036e2c6dc51ccca6eb7fa3ee6b63976dcd-r0/recipe-sysroot-
> native/usr/lib/x86_64-oe-linux-musl/go/pkg/tool/linux_amd64/link: running
> x86_64-oe-linux-musl-gcc failed: exit status 1
> | /tmp/go-link-014987806/000004.o: In function `vfprintf':
> | /usr/include/x86_64-linux-gnu/bits/stdio2.h:127: undefined reference to
> `__vfprintf_chk'
> | /tmp/go-link-014987806/000004.o: In function `fprintf':
> | /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to
> `__fprintf_chk'
> | collect2: error: ld returned 1 exit status
> |
> | ERROR: oe_runmake failed
> | Makefile:40: recipe for target 'static' failed
> | make: *** [static] Error 2
> | WARNING: exit code 1 from a shell command.
> 
> This is with Matt's patch from the oe-core mailing list applied.

In this case, it looks like the makefile needs some adjustment, since it's using 'go' (which runs go-native) instead of 'XXX-go' (where XXX is the appropriate target tuple).
Comment 6 Paul Barker 2017-10-04 15:32:45 UTC
(In reply to comment #5)
> (In reply to comment #4)
> > I'm getting a similar set of errors building runc-opencontainers from the
> > meta-virtualization layer for qemux86-64 with musl libc:
> > 
> > | DEBUG: Executing shell function do_compile
> > | NOTE: make -j 8 BUILDTAGS= static
> > | CGO_ENABLED=1 go build -i  -tags " cgo static_build" -ldflags "-w
> > -extldflags -static -X
> > main.gitCommit="2e7cfe036e2c6dc51ccca6eb7fa3ee6b63976dcd" -X
> > main.version=1.0.0-rc4 " -o runc .
> > | #
> > _/home/pbarker/oryx/build/tmp/work/core2-64-oe-linux-musl/runc-
> > opencontainers/1.0.0-rc4+git2e7cfe036e2c6dc51ccca6eb7fa3ee6b63976dcd-r0/git/
> > src/import
> > |
> > /home/pbarker/oryx/build/tmp/work/core2-64-oe-linux-musl/runc-opencontainers/
> > 1.0.0-rc4+git2e7cfe036e2c6dc51ccca6eb7fa3ee6b63976dcd-r0/recipe-sysroot-
> > native/usr/lib/x86_64-oe-linux-musl/go/pkg/tool/linux_amd64/link: running
> > x86_64-oe-linux-musl-gcc failed: exit status 1
> > | /tmp/go-link-014987806/000004.o: In function `vfprintf':
> > | /usr/include/x86_64-linux-gnu/bits/stdio2.h:127: undefined reference to
> > `__vfprintf_chk'
> > | /tmp/go-link-014987806/000004.o: In function `fprintf':
> > | /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to
> > `__fprintf_chk'
> > | collect2: error: ld returned 1 exit status
> > |
> > | ERROR: oe_runmake failed
> > | Makefile:40: recipe for target 'static' failed
> > | make: *** [static] Error 2
> > | WARNING: exit code 1 from a shell command.
> > 
> > This is with Matt's patch from the oe-core mailing list applied.
> 
> In this case, it looks like the makefile needs some adjustment, since it's
> using 'go' (which runs go-native) instead of 'XXX-go' (where XXX is the
> appropriate target tuple).

Ah that makes sense! I'll see if I can patch it to use '${GO}' instead of 'go'.
Comment 7 Matt Madison 2017-10-04 16:05:56 UTC
I have sent an updated patch to the list that addresses the undefined references issue that Saul was seeing.

Not sure how I missed it the first time around, but for musl targets we have to remove the pre-built objects for the race detector from the Go source tree before building the runtime.  Those objects are built against glibc, which was leading to the undefined references when using musl.
Comment 8 Paul Barker 2017-10-04 16:16:31 UTC
(In reply to comment #6)
> (In reply to comment #5)
> > 
> > In this case, it looks like the makefile needs some adjustment, since it's
> > using 'go' (which runs go-native) instead of 'XXX-go' (where XXX is the
> > appropriate target tuple).
> 
> Ah that makes sense! I'll see if I can patch it to use '${GO}' instead of
> 'go'.

Correct 'x86_64-oe-linux-musl-go' tool, still fails :(

| DEBUG: Executing shell function do_compile
| NOTE: make -j 8 BUILDTAGS= static
| CGO_ENABLED=1 x86_64-oe-linux-musl-go build -i  -tags " cgo static_build" -ldflags "-w -extldflags -static -X main.gitCommit="2e7cfe036e2c6dc51ccca6eb7fa3ee6b63976dcd-dirty" -X main.version=1.0.0-rc4 " -o runc .
| # _/home/pbarker/oryx/build/tmp/work/core2-64-oe-linux-musl/runc-opencontainers/1.0.0-rc4+git2e7cfe036e2c6dc51ccca6eb7fa3ee6b63976dcd-r0/git/src/import
| /home/pbarker/oryx/build/tmp/work/core2-64-oe-linux-musl/runc-opencontainers/1.0.0-rc4+git2e7cfe036e2c6dc51ccca6eb7fa3ee6b63976dcd-r0/recipe-sysroot-native/usr/lib/x86_64-oe-linux-musl/go/pkg/tool/linux_amd64/link: running x86_64-oe-linux-musl-gcc failed: exit status 1
| /tmp/go-link-660689674/000004.o: In function `vfprintf':
| /usr/include/x86_64-linux-gnu/bits/stdio2.h:127: undefined reference to `__vfprintf_chk'
| /tmp/go-link-660689674/000004.o: In function `fprintf':
| /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
| collect2: error: ld returned 1 exit status
|
| Makefile:40: recipe for target 'static' failed
| make: *** [static] Error 2
| ERROR: oe_runmake failed
| WARNING: exit code 1 from a shell command.
Comment 9 Matt Madison 2017-10-04 16:43:04 UTC
(In reply to comment #8)
> (In reply to comment #6)
> > (In reply to comment #5)
> > > 
> > > In this case, it looks like the makefile needs some adjustment, since it's
> > > using 'go' (which runs go-native) instead of 'XXX-go' (where XXX is the
> > > appropriate target tuple).
> > 
> > Ah that makes sense! I'll see if I can patch it to use '${GO}' instead of
> > 'go'.
> 
> Correct 'x86_64-oe-linux-musl-go' tool, still fails :(
> 
> | DEBUG: Executing shell function do_compile
> | NOTE: make -j 8 BUILDTAGS= static
> | CGO_ENABLED=1 x86_64-oe-linux-musl-go build -i  -tags " cgo static_build"
> -ldflags "-w -extldflags -static -X
> main.gitCommit="2e7cfe036e2c6dc51ccca6eb7fa3ee6b63976dcd-dirty" -X
> main.version=1.0.0-rc4 " -o runc .
> | #
> _/home/pbarker/oryx/build/tmp/work/core2-64-oe-linux-musl/runc-
> opencontainers/1.0.0-rc4+git2e7cfe036e2c6dc51ccca6eb7fa3ee6b63976dcd-r0/git/
> src/import
> |
> /home/pbarker/oryx/build/tmp/work/core2-64-oe-linux-musl/runc-opencontainers/
> 1.0.0-rc4+git2e7cfe036e2c6dc51ccca6eb7fa3ee6b63976dcd-r0/recipe-sysroot-
> native/usr/lib/x86_64-oe-linux-musl/go/pkg/tool/linux_amd64/link: running
> x86_64-oe-linux-musl-gcc failed: exit status 1
> | /tmp/go-link-660689674/000004.o: In function `vfprintf':
> | /usr/include/x86_64-linux-gnu/bits/stdio2.h:127: undefined reference to
> `__vfprintf_chk'
> | /tmp/go-link-660689674/000004.o: In function `fprintf':
> | /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to
> `__fprintf_chk'
> | collect2: error: ld returned 1 exit status
> |
> | Makefile:40: recipe for target 'static' failed
> | make: *** [static] Error 2
> | ERROR: oe_runmake failed
> | WARNING: exit code 1 from a shell command.

Is this using the recipe at tip of master in meta-virtualization?  If so, I think it may need some fixing (beyond what was committed there a couple of weeks back) - for one thing, it's setting the GOROOT environment variable in its do_compile(), which it should not do.  And the CGO_CFLAGS and CGO_LDFLAGS settings look redundant.  It would help to pass '-v -x' to the go build command in the makefile to get more complete logs of what it's doing.
Comment 10 Paul Barker 2017-10-04 17:00:05 UTC
(In reply to comment #9)
> (In reply to comment #8)
> > (In reply to comment #6)
> > > (In reply to comment #5)
> > > > 
> > > > In this case, it looks like the makefile needs some adjustment, since it's
> > > > using 'go' (which runs go-native) instead of 'XXX-go' (where XXX is the
> > > > appropriate target tuple).
> > > 
> > > Ah that makes sense! I'll see if I can patch it to use '${GO}' instead of
> > > 'go'.
> > 
> > Correct 'x86_64-oe-linux-musl-go' tool, still fails :(
> > 
> > | DEBUG: Executing shell function do_compile
> > | NOTE: make -j 8 BUILDTAGS= static
> > | CGO_ENABLED=1 x86_64-oe-linux-musl-go build -i  -tags " cgo static_build"
> > -ldflags "-w -extldflags -static -X
> > main.gitCommit="2e7cfe036e2c6dc51ccca6eb7fa3ee6b63976dcd-dirty" -X
> > main.version=1.0.0-rc4 " -o runc .
> > | #
> > _/home/pbarker/oryx/build/tmp/work/core2-64-oe-linux-musl/runc-
> > opencontainers/1.0.0-rc4+git2e7cfe036e2c6dc51ccca6eb7fa3ee6b63976dcd-r0/git/
> > src/import
> > |
> > /home/pbarker/oryx/build/tmp/work/core2-64-oe-linux-musl/runc-opencontainers/
> > 1.0.0-rc4+git2e7cfe036e2c6dc51ccca6eb7fa3ee6b63976dcd-r0/recipe-sysroot-
> > native/usr/lib/x86_64-oe-linux-musl/go/pkg/tool/linux_amd64/link: running
> > x86_64-oe-linux-musl-gcc failed: exit status 1
> > | /tmp/go-link-660689674/000004.o: In function `vfprintf':
> > | /usr/include/x86_64-linux-gnu/bits/stdio2.h:127: undefined reference to
> > `__vfprintf_chk'
> > | /tmp/go-link-660689674/000004.o: In function `fprintf':
> > | /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to
> > `__fprintf_chk'
> > | collect2: error: ld returned 1 exit status
> > |
> > | Makefile:40: recipe for target 'static' failed
> > | make: *** [static] Error 2
> > | ERROR: oe_runmake failed
> > | WARNING: exit code 1 from a shell command.
> 
> Is this using the recipe at tip of master in meta-virtualization?  If so, I
> think it may need some fixing (beyond what was committed there a couple of
> weeks back) - for one thing, it's setting the GOROOT environment variable in
> its do_compile(), which it should not do.  And the CGO_CFLAGS and
> CGO_LDFLAGS settings look redundant.  It would help to pass '-v -x' to the
> go build command in the makefile to get more complete logs of what it's
> doing.

I'm using commit 49a65220 from here: https://gitlab.com/oryx/meta-virtualization/commits/pbarker/runc. Along with latest oe-core plus your latest patch on the mailing list.

I'll have another go at adding '-v -x -work' to go build so that it shows more info and keeps most of the intermediate files around for analysis. Sadly I can't get the /tmp/go-link-NNNNNNNN directory to stick around - the deletion of that is unconditional at the minute in go :(