Bug 12631

Summary: Go 1.10 generates invalid code
Product: [Build System, Metadata & Runtime] OE-Core Reporter: Otavio Salvador <otavio>
Component: devtools / tool chainAssignee: Khem Raj <raj.khem>
Status: RESOLVED FIXED QA Contact:
Severity: critical    
Priority: Medium+ CC: matt, meta.mr.watcher, meta.watcher, otavio, raj.khem, richard.purdie, ross.burton, stephano
Version: 2.5   
Target Milestone: 2.6 M4   
Hardware: x86   
OS: Multiple   
Whiteboard:
OS type for building Yocto: --- Type of Regression: Regression (Used to work)
Verified: Documentation change: No (bug/feature does not impact docs)
Attachments:
Description Flags
Patch for testing - not final version none

Description Otavio Salvador 2018-03-28 08:04:37 UTC
The following code:

<pre>
package main                                                                                                                                           
                                                                                                                                                       
import (                                                                                                                                               
    "net/http"                                                                                                                                         
                                                                                                                                                       
    "github.com/sirupsen/logrus"                                                                                                                       
)                                                                                                                                                      
                                                                                                                                                       
func main() {                                                                                                                                          
    logrus.Info("starting")                                                                                                                            
                                                                                                                                                       
    client := &http.Client{}                                                                                                                           
    r, _ := http.NewRequest("GET", "https://google.com", nil)                                                                                          
                                                                                                                                                       
    client.Do(r)                                                                                                                                       
                                                                                                                                                       
    logrus.Info("started")                                                                                                                             
}
</pre>

Works when using 1.9.4 and fails with 1.10. However, when using Go from our host to cross build it for same target, it works fine.
Comment 1 Juro Bystricky 2018-03-28 08:27:01 UTC
This bug is filed under "BuildAppliance". Does it mean the problem is observed in BuildAppliance?
Comment 2 Otavio Salvador 2018-03-28 08:57:46 UTC
(In reply to comment #1)
> This bug is filed under "BuildAppliance". Does it mean the problem is
> observed in BuildAppliance?

I chose runtime.
Comment 3 Matt Madison 2018-03-28 09:49:50 UTC
I tested the sample code, and it worked fine for me, but I may have been building it differently than Otavio.

MACHINE: qemuarm
- created recipe for the sample code
- added recipe for logrus, set DEPENDS in recipe for sample code to point to it

If you're using glide or godep or some other Go dependency manager, though, you might be running into an issue with the change in how go1.10 handles build flags. Used to be that if you used the `-linkshared' build flag, the go tool would automatically propagate that build modifier to all packages built.  With go1.10, it now only applies that modifier to packages explicitly listed on the command line.

So go.bbclass might need to be changed to add "-gcflags=all:linkshared -asmflags=all:linkshared" to GOBUILDFLAGS when building against the shared runtime, so that any dependencies being handled internally by godep/glide/go get/etc. get compiled with the correct matching settings.
Comment 4 Khem Raj 2018-03-28 09:53:25 UTC
I think this seems a good thing to do. I have few apps which are using other go build systems. 

Matt can you post this change ?
Comment 5 Matt Madison 2018-03-28 09:59:14 UTC
Yes, I'll work up a patch for that change.  Hopefully Otavio can verify that it actually helps with this particular issue.
Comment 6 Otavio Salvador 2018-03-28 10:56:34 UTC
The suggested change does not build.

Log data follows:
| DEBUG: Executing shell function do_compile
| invalid value "all:linkshared" for flag -gcflags: missing =<value> in <pattern>=<value>
| usage: list [-e] [-f format] [-json] [build flags] [packages]
| Run 'go help list' for details.
| invalid value "all:linkshared" for flag -gcflags: missing =<value> in <pattern>=<value>
| usage: install [-i] [build flags] [packages]
Comment 7 Otavio Salvador 2018-03-28 13:51:53 UTC
Using the recipe in:

https://gist.github.com/gustavosbarreto/38216d2c50f9af589b91989dbc9916f3

You can reproduce this.
Comment 8 Matt Madison 2018-03-29 04:58:07 UTC
Created attachment 4243 [details]
Patch for testing - not final version

Sorry, I fat-fingered the needed compiler/assembler flags.

I still can't reproduce the problem with a qemuarm target, but that could be due to compiler output being identical for armv5 in the static vs. dynamic linkage case.  I might be able to with another target, will give that a try.

In the meantime, could you try with the attached patch to see if it helps?  I think I'll need to rework the patch a bit before submitting since the bbclass needs to work with both 1.9.4 and 1.10, but only 1.10 supports the new flag format, but this should at least tell me whether I'm on the right track.

Thanks,
-Matt
Comment 9 Otavio Salvador 2018-03-29 07:20:31 UTC
It does not seem to help.

The issue seems to happen in ARM using hardfp. Also, we found that if we use "http" instead of "http", it "works".
Comment 10 Matt Madison 2018-03-30 15:13:21 UTC
I can reproduce this now on an ARMv7 target, provided the image also includes the ca-certificates package.  Also, you have to wait until /dev/random has enough entropy.

I can also reproduce this natively by building go 1.10 from source, building a shared runtime with 'go install -linkshared -buildmode=shared std',  and doing a 'go get -linkshared github.com/gustavosbarreto/go-crazy'.  So it's not ARM-specific.

It's a panic due to a segfault, always appearing to happen around a sync.Pool request in logrus's Entry.log function.  However, it only seems to happen if the https request is also getting processed in other goroutines.  Panic tracebacks always point to the logrus package as the running thread, sometimes with and sometimes without other goroutines being active.

It does seem to happen only when the program is linked against shared runtime, not when linked statically.

Definitely an upstream issue, just not 100% sure that it's really in the Go runtime, or if logrus is misusing the sync package.
Comment 11 Otavio Salvador 2018-04-02 06:41:14 UTC
I am glad you succeed in reproducing it. Did you find any upstream issue related to it?
Comment 12 Matt Madison 2018-04-02 07:44:47 UTC
I haven't found an upstream bug report that matches this exactly, and it's hard to know whether the root cause is in the go shared runtime support or in the logrus package.

Some possibly related issues:
https://github.com/golang/go/issues/23199
https://github.com/sirupsen/logrus/issues/740
https://bugzilla.redhat.com/show_bug.cgi?format=multiple&id=1561619
Comment 13 Otavio Salvador 2018-04-02 09:03:07 UTC
I reported this upstream: https://github.com/golang/go/issues/24640
Comment 14 Richard Purdie 2018-04-19 08:29:56 UTC
Move to 2.6 M1 since this is now not the default for the 2.5 release.
Comment 15 Khem Raj 2018-06-18 17:32:07 UTC
Otavio

Can you see if 

https://github.com/sirupsen/logrus/pull/752/commits/8369e2f077642788e2ef31f342a0a4621b2b4607

when applied to logrus fixes the issue or not?
Comment 16 Khem Raj 2018-06-18 17:33:56 UTC
We might have to wait until 1.11 release  ( tentative Due by July 31, 2018)
Comment 17 Stephen K Jolley 2018-06-29 15:04:07 UTC
I have put this into Accepted, since it is not waiting on information, but waiting for an upgrade to be available.
Comment 18 Khem Raj 2018-10-08 16:31:48 UTC
I think we have to move this to 2.7 which 1.11 has been released, we might not be able to integrate that into 2.6 as of now, given the release is so close.
Comment 19 Khem Raj 2018-10-19 03:05:32 UTC
Posted patch to ml.

https://patchwork.openembedded.org/series/14602/
Comment 20 Khem Raj 2018-10-29 15:17:40 UTC
this is fixed