Bug 1100

Sounds like it could be a duplicate of 1072 (machine-specific xorg.confs not being picked up) e.g. crownbay shouldn't even be looking for module "intel".

(In reply to comment #0)
> Tree/Branch: Poky/1.1_M1
> Poky Commit: fc55b224caa3eeac4abda099ec9ed505db59fb28
> Meta Branch: 1.1_M1
> Meta Commit: 1b227f8ed2df529cc13f7169b0c08ff3d0f8c812
Hi Jiajun, how did you know the meta commit 1b227f? The autobuilder's file gitinfo only shows the commit number of poky master.

(In reply to comment #1)
> Sounds like it could be a duplicate of 1072 (machine-specific xorg.confs not
> being picked up) e.g. crownbay shouldn't even be looking for module "intel".
Agree.
I checked the file core-image-sato-sdk-live-emenlow-20110521070906.hddimg.bz2 and the file xorg.conf in it is from
meta/recipes-graphics/xorg-xserver/xserver-xf86-config/xorg.conf
rather than
meta-intel/meta-emenlow/recipes-graphics/xorg-xserver/xserver-xf86-config/emenlow/xorg.conf.
I'm not sure what caused this, however, with MACHINE="emenlow", when I did "bitbake xserver-xf86-config -c unpack -f", I found tmp/work/emenlow-poky-linux/xserver-xf86-config-0.1-r9/xorg.conf is just from 
meta-intel/meta-emenlow/recipes-graphics/xorg-xserver/xserver-xf86-config/emenlow/xorg.conf! This means I can't reproduce this bug.
BTW: I used the same Poky Commit and Meta Commit as mentioned above by Jiajun.
This is really weird... Maybe this bug only happens on autobuilder somehow???

As to Bug 1072, Tom, do you still remember the commit numbers of poky master/mete-intel based on which you reported the bug? Can you please use "bitbake xserver-xf86-config -c unpack -f" to verify the bug is reproducible?

(In reply to comment #2)
> (In reply to comment #0)
> > Tree/Branch: Poky/1.1_M1
> > Poky Commit: fc55b224caa3eeac4abda099ec9ed505db59fb28
> > Meta Branch: 1.1_M1
> > Meta Commit: 1b227f8ed2df529cc13f7169b0c08ff3d0f8c812
> Hi Jiajun, how did you know the meta commit 1b227f? The autobuilder's file
> gitinfo only shows the commit number of poky master.
> 

Dexuan,
I could get both poky and meta-intel commit from http://autobuilder.pokylinux.org:8010/builders/emenlow. There will be several build task on the page. You could click hyperlink of each build id. Any by checking the build time (RC1 build is triggered on May 20th), we could find which build is what we want. In the build page, http://autobuilder.pokylinux.org:8010/builders/emenlow/builds/47, there are detailed information for both poky and meta-intel commit.

Update the title since blacksand could start X with live boot. And only crownbay-noemgd is tested since crownbay image build failed.

Tom or Darren to look at ASAP and ask Dexuan to assist if not root-caused today or tomorrow.

(In reply to comment #2)
> (In reply to comment #0)
> > Tree/Branch: Poky/1.1_M1
> > Poky Commit: fc55b224caa3eeac4abda099ec9ed505db59fb28
> > Meta Branch: 1.1_M1
> > Meta Commit: 1b227f8ed2df529cc13f7169b0c08ff3d0f8c812
> Hi Jiajun, how did you know the meta commit 1b227f? The autobuilder's file
> gitinfo only shows the commit number of poky master.
> 
> (In reply to comment #1)
> > Sounds like it could be a duplicate of 1072 (machine-specific xorg.confs not
> > being picked up) e.g. crownbay shouldn't even be looking for module "intel".
> Agree.
> I checked the file core-image-sato-sdk-live-emenlow-20110521070906.hddimg.bz2
> and the file xorg.conf in it is from
> meta/recipes-graphics/xorg-xserver/xserver-xf86-config/xorg.conf
> rather than
> meta-intel/meta-emenlow/recipes-graphics/xorg-xserver/xserver-xf86-config/emenlow/xorg.conf.
> I'm not sure what caused this, however, with MACHINE="emenlow", when I did
> "bitbake xserver-xf86-config -c unpack -f", I found
> tmp/work/emenlow-poky-linux/xserver-xf86-config-0.1-r9/xorg.conf is just from 
> meta-intel/meta-emenlow/recipes-graphics/xorg-xserver/xserver-xf86-config/emenlow/xorg.conf!
> This means I can't reproduce this bug.
> BTW: I used the same Poky Commit and Meta Commit as mentioned above by Jiajun.
> This is really weird... Maybe this bug only happens on autobuilder somehow???
> 
> As to Bug 1072, Tom, do you still remember the commit numbers of poky
> master/mete-intel based on which you reported the bug? Can you please use
> "bitbake xserver-xf86-config -c unpack -f" to verify the bug is reproducible?

I just updated bug 1072 with information that allows me to reproduce the xorg.conf problem.

bitbake xserver-xf86-config -c unpack -f

for crownbay-noemgd

with the named commits shows a correct xorg.conf.

(In reply to comment #7)
> bitbake xserver-xf86-config -c unpack -f
> for crownbay-noemgd
> with the named commits shows a correct xorg.conf.

But in bug 1072, doing the bitbake for sugarbay  shows a wrong xorg.conf in your side.


In my side, doing the bitbake for menlow and sugarbay both shows a correct xorg.conf.

I'm trying to figure out why I can't reproduce the bug...

(In reply to comment #8)
> (In reply to comment #7)
> > bitbake xserver-xf86-config -c unpack -f
> > for crownbay-noemgd
> > with the named commits shows a correct xorg.conf.
> 
> But in bug 1072, doing the bitbake for sugarbay  shows a wrong xorg.conf in
> your side.
> 
> 
> In my side, doing the bitbake for menlow and sugarbay both shows a correct
> xorg.conf.
> 
> I'm trying to figure out why I can't reproduce the bug...

Because I thought there was no problem with crownbay noemgd, I did a complete new build of core-image-sato-live assuming the xorg.conf would be correct, but checking the xorg.conf I see it's the wrong one again.  I get this in xorg.conf:

Section "Device"
    Identifier    "Intel Graphics Driver"
    Driver        "intel"
EndSection


When I should get this:

Section "Device"
    Identifier  "Generic VESA"
    Driver      "vesa"
EndSection

The only difference I can see was that this was a fresh build of a whole image, where the other just built the package alone.

In this case, bitbake -c cleanall xserver-xf86-config followed by
bitbake xserver-xf86-config -c unpack -f still shows the problem

Tom noticed that there was a difference between the ordering of the bbappend LOAD in the logs between a good and a bad unpack. In a good case it looks like this (using -DDDD):

DEBUG: Appending .bbappend file /home/dvhart/source/poky.git/meta-yocto/recipes-graphics/xorg-xserver/xserver-xf86-config_0.1.bbappend to /home/dvhart/source/poky.git/meta/recipes-graphics/xorg-xserver/xserver-xf86-config_0.1.bb
DEBUG: BB /home/dvhart/source/poky.git/meta-yocto/recipes-graphics/xorg-xserver/xserver-xf86-config_0.1.bbappend: handle(data, include)
DEBUG: LOAD /home/dvhart/source/poky.git/meta-yocto/recipes-graphics/xorg-xserver/xserver-xf86-config_0.1.bbappend
DEBUG: Appending .bbappend file /home/dvhart/source/poky.git/layers/meta-intel/meta-crownbay/recipes-graphics/xorg-xserver/xserver-xf86-config_0.1.bbappend to /home/dvhart/source/poky.git/meta/recipes-graphics/xorg-xserver/xserver-xf86-config_0.1.bb
DEBUG: BB /home/dvhart/source/poky.git/layers/meta-intel/meta-crownbay/recipes-graphics/xorg-xserver/xserver-xf86-config_0.1.bbappend: handle(data, include)
DEBUG: LOAD /home/dvhart/source/poky.git/layers/meta-intel/meta-crownbay/recipes-graphics/xorg-xserver/xserver-xf86-config_0.1.bbappend


Where the meta-crownbay bbappend is added last. In a bad unpack, it looks like this:

DEBUG: Appending .bbappend file /usr/local/src/yocto/junk/meta-intel/meta-crownbay/recipes-graphics/xorg-xserver/xserver-xf86-config_0.1.b\
bappend to /usr/local/src/yocto/junk/meta/recipes-graphics/xorg-xserver/xserver-xf86-config_0.1.bb                                         
DEBUG: BB /usr/local/src/yocto/junk/meta-intel/meta-crownbay/recipes-graphics/xorg-xserver/xserver-xf86-config_0.1.bbappend: handle(data, \
include)                                                                                                                                   
DEBUG: LOAD /usr/local/src/yocto/junk/meta-intel/meta-crownbay/recipes-graphics/xorg-xserver/xserver-xf86-config_0.1.bbappend              
DEBUG: Appending .bbappend file /usr/local/src/yocto/junk/meta-yocto/recipes-graphics/xorg-xserver/xserver-xf86-config_0.1.bbappend to /us\
r/local/src/yocto/junk/meta/recipes-graphics/xorg-xserver/xserver-xf86-config_0.1.bb                                                       
DEBUG: BB /usr/local/src/yocto/junk/meta-yocto/recipes-graphics/xorg-xserver/xserver-xf86-config_0.1.bbappend: handle(data, include)       
DEBUG: LOAD /usr/local/src/yocto/junk/meta-yocto/recipes-graphics/xorg-xserver/xserver-xf86-config_0.1.bbappend  

Note that meta-yocto is added second. In each case, the priority of meta-yocto was 5 and the priority of meta-crownbay was 6.

Apparently the pathnames for the layers impact the order in which they are loaded. If I setup symlinks to mirror tomz's setup, I can reproduce the bug:

Good bblayers.conf:
BBLAYERS = " \
  /home/dvhart/source/poky.git/meta \
  /home/dvhart/source/poky.git/meta-yocto \
  /home/dvhart/source/poky.git/layers/meta-intel/meta-crownbay \

Bad bblayers.conf:
BBLAYERS = " \
  /usr/local/src/yocto/junk/meta \
  /usr/local/src/yocto/junk/meta-yocto \
  /usr/local/src/yocto/junk/meta-intel/meta-crownbay \

And note that these are indeed pointing to the same place:

$ ls -la /usr/local/src/yocto/junk/
total 8
drwxr-xr-x 2 root root 4096 2011-05-26 21:55 .
drwxr-xr-x 3 root root 4096 2011-05-26 21:52 ..
lrwxrwxrwx 1 root root   33 2011-05-26 21:55 meta -> /home/dvhart/source/poky.git/meta
lrwxrwxrwx 1 root root   46 2011-05-26 21:54 meta-intel -> /home/dvhart/source/poky.git/layers/meta-intel
lrwxrwxrwx 1 root root   39 2011-05-26 21:53 meta-yocto -> /home/dvhart/source/poky.git/meta-yocto

I've arrived at root cause. After some instrumenting (patch to be attached in case anyone is interested) I learned that the root of the problem is that cooker.py BBCooker.collect_bbfiles() doesn't take priority or bblayers.conf ordering into account when preparing the files lists. The most relevant bit follows:

       newfiles = set()
        for f in files:
            bb.plain("parsing %s" % f)
            if os.path.isdir(f):
                dirfiles = self.find_bbfiles(f)
                newfiles.update(dirfiles)
            else:
                globbed = glob.glob(f)
                if not globbed and os.path.exists(f):
                    globbed = [f]
                newfiles.update(globbed)
...
       bb.plain("xserver-xf86-config files:")
        for f in newfiles:
            if f.count("xserver-xf86-config") > 0:
                bb.plain("  %s" % f)


With the known good paths, this results in:
xserver-xf86-config files:
  /home/dvhart/source/poky.git/meta/recipes-graphics/xorg-xserver/xserver-xf86-config_0.1.bb
  /home/dvhart/source/poky.git/meta-yocto/recipes-graphics/xorg-xserver/xserver-xf86-config_0.1.bbappend
  /home/dvhart/source/poky.git/layers/meta-intel/meta-crownbay/recipes-graphics/xorg-xserver/xserver-xf86-config_0.1.bbappend

With the known bad paths, this results in:
xserver-xf86-config files:
  /usr/local/src/yocto/junk/meta-intel/meta-crownbay/recipes-graphics/xorg-xserver/xserver-xf86-config_0.1.bbappend
  /usr/local/src/yocto/junk/meta-yocto/recipes-graphics/xorg-xserver/xserver-xf86-config_0.1.bbappend
  /usr/local/src/yocto/junk/meta/recipes-graphics/xorg-xserver/xserver-xf86-config_0.1.bb

Note that in the bad path, the meta recipe is last in the list (taking precedence over the other files).

By using a python set type (which I gather is convenient for the bbmask mechanism) we lose any ordering that existed simply from the ordering of the layers in bblayers.conf. Consider the following:

In [1]: a=set()
In [2]: b=set()
In [3]: a.add("a")
In [4]: a.add("b")
In [5]: a.add("c")
In [6]: print list(a)
['a', 'c', 'b']

In [7]: b.add("z")
In [8]: b.add("y")
In [9]: b.add("x")
In [10]: print list(b)
['y', 'x', 'z']

Notice the difference in output order versus input order. If you replace the letters with the layer paths, you get the same sort of result. A proper fix for this probably involves ensuring files are enumerated in layer priority order.

Created attachment 168 [details]
dvhart's bitbake instrumentation patch

Here is the instrumentation patch I used to track down the bbappend ordering issue in case it is of any use to anyone.

Removing Blacksand and setting to All as this is a core bitbake issue and really not machine specific.

Master has a fix for this:
http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=00c71132d5a326546bbb7fdf173995c76a9e9803

Commited to master and cherry-picked to M1

Please note that this is a workaround. It makes it so the order of repositories are honored in bblayers.conf, but it does not address the missing layer priority ordering. I've opened a new bug for that: Bug 1125

Verify the bug with Yocto 1.1 M1 RC2 build, the bug is fixed. Detailed commit information:
Tree/Branch: Poky/1.1_M1
Poky Commit: 4ff7af11ef69849ef9c16f585eae58ac920b222b
Meta Branch: 1.1_M1
Meta Commit: fc719f0cd6530ce15148b4aa274f1644b461b298