Bug 3898 - kernel menuconfig unusable
Summary: kernel menuconfig unusable
Status: RESOLVED FIXED
Alias: None
Product: Kernel
Classification: Yocto Project Subprojects
Component: kernel-tooling (show other bugs)
Version: 1.4
Hardware: x86 Multiple
: Medium normal
Target Milestone: 2.2
Assignee: Juro Bystricky
QA Contact:
URL:
Whiteboard:
Depends on:
Blocks:
 
Reported: 2013-02-18 01:50 UTC by Trevor Woerner
Modified: 2016-10-25 20:34 UTC (History)
16 users (show)

See Also:
OS type for building Yocto: ---
Type of Regression: ---
Verified:
Documentation change: Don't know


Attachments
this is what I see by default (9.72 KB, image/png)
2013-02-18 01:50 UTC, Trevor Woerner
no flags Details
"bitbake linux-yocto -c menuconfig" on master (9.72 KB, image/png)
2013-02-28 13:18 UTC, Trevor Woerner
no flags Details
"bitbake linux-yocto -c menuconfig" on danny (59.37 KB, image/png)
2013-02-28 13:19 UTC, Trevor Woerner
no flags Details
"make menuconfig" without yocto (13.70 KB, image/png)
2013-02-28 13:21 UTC, Trevor Woerner
no flags Details
"bitbake linux-yocto -c menuconfig" on MUT (9.72 KB, image/png)
2013-03-01 02:50 UTC, Trevor Woerner
no flags Details
log from ncurses-native of MUT (38.95 KB, text/plain)
2013-03-01 02:59 UTC, Trevor Woerner
no flags Details
my xterm environment (5.10 KB, text/plain)
2013-03-01 03:56 UTC, Trevor Woerner
no flags Details
my gnome-terminal environment (5.15 KB, text/plain)
2013-03-01 03:57 UTC, Trevor Woerner
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Trevor Woerner 2013-02-18 01:50:54 UTC
Created attachment 1073 [details]
this is what I see by default

When I run

    $ bitbake linux-yocto -c menuconfig

I end up with a kernel menuconfig that is completely unusable (please see
attachment).

This issue was brought up around August 2012 on the OE-core mailing list.
Around that time Liang Li proposed the following patch which solves this
problem for me.

http://lists.linuxtogo.org/pipermail/openembedded-core/2012-August/027130.html

diff --git meta/classes/cml1.bbclass meta/classes/cml1.bbclass
index bd25311..948cfad 100644
--- meta/classes/cml1.bbclass
+++ meta/classes/cml1.bbclass
@@ -15,6 +15,7 @@ HOSTLDFLAGS = "${BUILD_LDFLAGS}"
 HOST_LOADLIBES = "-lncurses"
 
 python do_menuconfig() {
+        d.setVar("HOSTLDFLAGS", "")
         oe_terminal("${SHELL} -c \"make menuconfig; echo 'Pausing for
         5 seconds'; sleep 5\"", '${PN} Configuration', d)
 }
 do_menuconfig[depends] += "ncurses-native:do_populate_sysroot"

For some reason this issue died without any resolution.

According to Liang this patch solved the problem for a Fedora 17 system.
I'm running openSuSE 12.2.
Comment 1 Trevor Woerner 2013-02-18 01:52:50 UTC
Is there any reason a new, separate terminal needs to be opened?
Is it possible to have menuconfig run within the calling terminal?
This could possibly help someone logging in to a build machine remotely.
Comment 2 Richard Purdie 2013-02-18 05:17:40 UTC
Can you please tell us which revision of master this was tested with? If it wasn't with the lastest master could you please test that as there were some changes in that area recently which may improve this.
Comment 3 Richard Purdie 2013-02-18 05:19:24 UTC
btw, the workaround you propose is just that, a workaround. There appears to be something wrong with our ncurses-native and setting HOSTLDFLAGS avoids using. We do not assume the system provides an ncurses-native so that workaround cannot be merged into master, we need to find out what is wrong with ncurses-native and fix it.
Comment 4 Trevor Woerner 2013-02-18 15:35:36 UTC
(In reply to comment #2)
> Can you please tell us which revision of master this was tested with?

Yes, sorry. It is with the latest master. I just did a pull of yocto
and all layers and tried again with the exact same results.


Build Configuration:
BB_VERSION        = "1.17.1"
BUILD_SYS         = "x86_64-linux"
NATIVELSBSTRING   = "SUSE-LINUX-12.2"
TARGET_SYS        = "i586-poky-linux"
MACHINE           = "qemux86"
DISTRO            = "poky"
DISTRO_VERSION    = "1.3+snapshot-20130218"
TUNE_FEATURES     = "m32 i586"
TARGET_FPU        = ""
meta              
meta-yocto        
meta-yocto-bsp    = "master:c7b23ab68aafc04d9830ef318015912e5d4f0672"
Comment 5 Darren Hart 2013-02-28 04:41:47 UTC
I have not experienced this before. Did this used to work for you? Does it work on the same system trying to do menuconfig from the last release (danny)? Please try danny and let us know, that will identify whether this is likely a host issue or an oe-core issue.

Another useful datapoint would be whether make menuconfig works from a local set Linux sources (independent of bitbake).

There are a number of factors that can influence the display, there is some discussion of those here:

https://bugzilla.redhat.com/show_bug.cgi?id=166291

And probably some in more recent reports.
Comment 6 Bruce Ashfield 2013-02-28 05:00:49 UTC
I don't have any experience with ncurses-native .. and with my current
load of M5 kernel items, chances are that I'm not going to get to this.

I do have one extra question, does menuconfig have the same issues
with busybox ? It should, since the code in question is exactly the 
same.

I'd suggest that someone familiar with the build system side of
things or userspace is actually a better person to look at this, it
doesn't have anything to do with the kernel itself, or even the 
kernel build.
Comment 7 Darren Hart 2013-02-28 05:11:22 UTC
I agree. I'm bouncing this to Saul who may have someone in mind. Paul seems a likely candidate?
Comment 8 Trevor Woerner 2013-02-28 13:18:27 UTC
Created attachment 1090 [details]
"bitbake linux-yocto -c menuconfig" on master

Build Configuration:
BB_VERSION        = "1.17.1"
BUILD_SYS         = "x86_64-linux"
NATIVELSBSTRING   = "SUSE-LINUX-12.2"
TARGET_SYS        = "i586-poky-linux"
MACHINE           = "qemux86"
DISTRO            = "poky"
DISTRO_VERSION    = "1.3+snapshot-20130228"
TUNE_FEATURES     = "m32 i586"
TARGET_FPU        = ""
meta              
meta-yocto        
meta-yocto-bsp    = "master:d7b248e715d99766bf8602ff9f038f8b0afa5e78"
Comment 9 Trevor Woerner 2013-02-28 13:19:38 UTC
Created attachment 1091 [details]
"bitbake linux-yocto -c menuconfig" on danny

Build Configuration:
BB_VERSION        = "1.16.0"
TARGET_ARCH       = "i586"
TARGET_OS         = "linux"
MACHINE           = "qemux86"
DISTRO            = "poky"
DISTRO_VERSION    = "1.3"
TUNE_FEATURES     = "m32 i586"
TARGET_FPU        = ""
meta              
meta-yocto        
meta-yocto-bsp    = "danny:98292d1ef1cb44cf96f0a28fbca6439eda55ce1b"
Comment 10 Trevor Woerner 2013-02-28 13:21:19 UTC
Created attachment 1092 [details]
"make menuconfig" without yocto

$ git clone file:///home/trevor/devel/Downloads/git2/git.yoctoproject.org.linux-yocto-3.2
$ cd linux-yocto-3.2
$ make menuconfig


NOTES:
$ echo $TERM
xterm-256color
Comment 11 Trevor Woerner 2013-02-28 13:24:06 UTC
(In reply to comment #5)
> I have not experienced this before. Did this used to work for you?

Don't know. When I tried it the other day it was my first time.

> Does it
> work on the same system trying to do menuconfig from the last release
> (danny)? Please try danny and let us know, that will identify whether this
> is likely a host issue or an oe-core issue.

I came across this issue while following the recently-released kernel lab,
so the first time this happened I was on danny.

(see attachment #1091 [details])

> Another useful datapoint would be whether make menuconfig works from a local
> set Linux sources (independent of bitbake).

(see attachment #1092 [details])
 
> There are a number of factors that can influence the display, there is some
> discussion of those here:
> 
> https://bugzilla.redhat.com/show_bug.cgi?id=166291
> 
> And probably some in more recent reports.

Thanks. I'll have a look at those.
Comment 12 Tom Zanussi 2013-02-28 14:32:36 UTC
Just one point that may have been missed and may be important:

This doesn't appear to be a problem on Ubuntu systems i.e. 12.04 (Precise Pangonlin) works fine, while on Fedora Core 17 the problems appears.

My guess is that most people use Ubuntu, and few use F17, maybe because the docs and exiting tutorials are geared toward Ubuntu so it hasn't been run into or reported much.

Having a case that works vs the same thing that doesn't on two different systems with the same code should help narrow things down.
Comment 13 Saul Wold 2013-02-28 21:23:52 UTC
Another datapoint, I just tried this on my FC18 machine and the menuconfig starts correctly with MUT (master_under_test) (3.8 kernel), I will work on getting an FC17 VM started and see if I can reproduce this.

It might be helpful to get your ncurses-native config.log file. It might be show something installed on your system that's triggering the failure.
Comment 14 Trevor Woerner 2013-03-01 02:50:52 UTC
Created attachment 1095 [details]
"bitbake linux-yocto -c menuconfig" on MUT

Build Configuration:
BB_VERSION        = "1.17.1"
BUILD_SYS         = "x86_64-linux"
NATIVELSBSTRING   = "SUSE-LINUX-12.2"
TARGET_SYS        = "i586-poky-linux"
MACHINE           = "qemux86"
DISTRO            = "poky"
DISTRO_VERSION    = "1.3+snapshot-20130301"
TUNE_FEATURES     = "m32 i586"
TARGET_FPU        = ""
meta              
meta-yocto        
meta-yocto-bsp    = "stage/master_under_test:feb3a55170e7c0a9df3552607e0a62dce2b3a160"
Comment 15 Trevor Woerner 2013-03-01 02:55:00 UTC
(In reply to comment #13)
> Another datapoint, I just tried this on my FC18 machine and the menuconfig
> starts correctly with MUT (master_under_test) (3.8 kernel), I will work on
> getting an FC17 VM started and see if I can reproduce this.

I just tried it with MUT and still get bad results (see attachment #1095 [details]).

By the way, others have mentioned issues with Fedora systems; I'd like
to reiterate that I'm working with openSUSE 12.2.

> It might be helpful to get your ncurses-native config.log file. It might be
> show something installed on your system that's triggering the failure.

Okay, I'll attach that as well.
Comment 16 Trevor Woerner 2013-03-01 02:59:06 UTC
Created attachment 1096 [details]
log from ncurses-native of MUT

This is my do_configure log for my master_under_test build.
Comment 17 Tom Zanussi 2013-03-01 03:09:02 UTC
I'm no longer able to reproduce this on F17 with either danny or master with linux-yocto 3.4 or 3.8 or stable 3.4.

I'll have to try fresh, as I definitely did see this doing the kernel lab on F17.
Comment 18 Tom Zanussi 2013-03-01 03:11:57 UTC
Actually, looking at tour pictures, mine was somewhat different - everything was basically double and shifted but not large unreadable blocks like in your pics.
Comment 19 Trevor Woerner 2013-03-01 03:48:37 UTC
Saul and I had a great IRC session and discovered a couple more very
interesting data points:

1) by default my local.conf does not contain any OE_TERMINAL setting. according
to bitbake-env my setting for OE_TERMINAL (by default) is "auto"

2) out of habit, I always use an xterm as my terminal

3) if I do explicitly set OE_TERMINAL in my local.conf to either "screen"
or "xterm" it works just fine. In the case of "screen", my existing xterm
is used for the menuconfig. In the case of "xterm" a new terminal is started
in a new window and runs menuconfig just fine

4) if I start up a gnome-terminal, and set it up to perform Yocto builds,
and run the "-c menuconfig" command, a new gnome-terminal pops up, runs
menuconfig, and looks perfectly fine

5) Saul pointed me to meta/lib/oe/terminal.py where we noticed all the
options have a priority... except Screen

So the problem appears to be related to the fact I'm using an xterm to
run the "-c menuconfig" command which, by default, starts up a gnome-terminal.

If the terminal that is used is the same as the one from which the command
is issued, everything works fine.

If an xterm is used to issue the command and a gnome-terminal is started,
it is messed up. Incidentally, if I use a gnome-terminal to issue the
command and set OE_TERMINAL to "xterm" explicitly, that works fine too.
Comment 20 Trevor Woerner 2013-03-01 03:56:40 UTC
Created attachment 1097 [details]
my xterm environment

(as requested)
Comment 21 Trevor Woerner 2013-03-01 03:57:09 UTC
Created attachment 1098 [details]
my gnome-terminal environment
Comment 22 Jason Wessel 2013-03-04 22:20:34 UTC
Indepently I had this problem reported to me and I researched the root cause to compiling against a different curses.h, ncurses.h or ncursesw.h vs the correct library in the sysroot from ncurses-native.

The only way to fix this properly is to patch all 3 places that there are problems.  See: http://lists.linuxtogo.org/pipermail/openembedded-core/2013-March/036672.html

1) busybox's the curses.h check needs an override
2) The kernels's curses.h check  need an override
3) The cml1.bbclass needs to specify the override for the Yocto Project ncurses-native
Comment 23 Saul Wold 2013-03-22 20:46:16 UTC
Assigning to Jason, since he has a patch pending (on hold) pending upstream acceptance, once we know for sure, we can then deal with merging this.
Comment 24 Richard Purdie 2013-04-16 08:49:14 UTC
Jason: Did you get any feedback from upstream on this?
Comment 25 Jason Wessel 2013-04-16 14:15:40 UTC
Not yet.  I'll ping the upstream maintainers again.
Comment 26 Darren Hart 2014-07-31 15:02:03 UTC
(In reply to comment #25)
> Not yet.  I'll ping the upstream maintainers again.

Jason, any update?
Comment 27 Paul Eggleton 2014-11-06 16:13:53 UTC
Jason, do you have an update on the status of this issue?
Comment 28 Jason Wessel 2015-01-19 16:21:48 UTC
(In reply to comment #27)
> Jason, do you have an update on the status of this issue?

I'll submit the patch to the kbuild folks and if I don't hear from them again, I'll ping AKPM.  I'd like to see this resolved once and for all.
Comment 29 Juro Bystricky 2016-10-25 20:34:36 UTC
I tried to reproduce the problem with krogoth/morty, but to no avail.
Over the time I tested Ubuntu 14,15,16, Fedora22, Centos7 and probably other distros as well. I also tested several terminals 
OE_TERMINAL = "xxx"
(xxx one of auto, gnome, konsole, screen, rxvt and even xterm).

Hence I am closing this problem as RESOLVED/FIXED. If by any chance we run into this problem at some point again, we can re-open the bug.