Bug 14676 - Unable to add a layer to the layer index: meta-lat
Summary: Unable to add a layer to the layer index: meta-lat
Status: RESOLVED FIXED
Alias: None
Product: Layer Index
Classification: Yocto Project Subprojects
Component: Layer Index (show other bugs)
Version: 4.0
Hardware: x86 Multiple
: Medium+ major
Target Milestone: 4.1 M1
Assignee: Tim Orling
QA Contact: apoorv sangal
URL:
Whiteboard:
Depends on:
Blocks:
 
Reported: 2022-01-07 22:05 UTC by Randy MacLeod
Modified: 2022-04-11 02:58 UTC (History)
6 users (show)

See Also:
OS type for building Yocto: ---
Type of Regression: ---
Verified:
Documentation change: No (bug/feature does not impact docs)


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Randy MacLeod 2022-01-07 22:05:36 UTC
Hongux sent an email in December to have https://github.com/Wind-River/meta-lat added to the layer index. Tim Orling and others have tried to update the layer index and are seeing problems as described in the irc logs below.


[11:46] <vmeson> Several weeks ago, Hongxu asked to have meta-lat added to the layer index but there hasn't been any response. Who is taking care of that now? I'd ask Paul E but he's likely sleeping now.
[11:48] <vmeson> We _might_ be able to volunteer someone since we make use of the LI code for our product.
[11:49] <RP> vmeson: Paul E is the best we have right now, or maybe moto-timo
[11:49] <RP> sakoman: not sure
[11:49] <RP> sakoman: probably still early for maint
[11:50] <moto-timo> I saw the layer request but forgot to follow up
[11:50] <moto-timo> ross is also an admin
[11:50] <rburton> hm i thought i pressed approve on that
[11:52] <rburton> so the layer index says no unpublished layers to review
[11:52] <rburton> but if you go to it, the layer index crashes
[11:55] <moto-timo> halstead: I'm getting 'Internal Error' logging in to layer index. can we restart it?
[11:56] <abelloni> RP: in https://autobuilder.yoctoproject.org/typhoon/#/builders/87/builds/2862/steps/14/logs/stdio
[11:56] <abelloni> you have seen the gdb failure
[11:56] <abelloni> but what about Inconsistency detected by ld.so: dl-open.c: 560: dl_open_worker: Assertion `_dl_debug_initialize (0, args->nsid)->r_state == RT_CONSISTENT' failed!
[11:56] <abelloni> it seems to be a different issue
[11:56] <moto-timo> rburton: yeah, now that I tried I remember the error and decided to wait until after holidays... which means I forgot about it
[12:02] <RP> abelloni: I didn't spot that, no. That sounds bad :(
[12:02] <RP> abelloni: not sure on a cause :/
[12:03] <rburton> vmeson: i suggest filing a bug that http://layers.openembedded.org/layerindex/branch/master/layer/meta-lat/ produces an error, and copy me/paul/tim
[12:03] <rburton> and halstead
[12:04] <rburton> and hopefully someone can get to the django logs, as i don't know how to
[12:06] <halstead> Yes, I'll restart it and pull logs. 
[12:06] <halstead> And check on what's up with the docs
[12:08] <RP> abelloni: I think that is some kind of comment against all of them now. Hopefully the others I did are more accurate that the assertion one! 
[12:16] <abelloni> well I checked before triaging ;)
[12:20] <RP> abelloni: cool :)
[12:55] <vmeson> rburton: I'll do that after I get  lunch.
[12:57] <rburton> cool
[12:57] <rburton> I'll be gone shortly so will look monday unless some beats me
[13:03] <halstead> moto-timo: I have captured logs and logged in successfully myself. Watching logs now. I'm curious if you can reproduce now.
[13:06] <moto-timo> halstead: https://layers.openembedded.org/layerindex/review/meta-lat/
[13:07] <moto-timo> halstead: as I recall if a layer was already added it used to go to a screen saying so...
[13:08] <moto-timo> halstead: sometimes the Internal Server warnings came from Django (which is likely the case here and a bug) or from nginx (which is usually "Bad Gateway")
[13:09] <halstead> moto-timo: That page response if big enough it starts buffering to a tmpfile and then I'm not sure why it fails after that.
[13:10] <halstead> moto-timo: So you've logged in successfully at least?
[13:12] <moto-timo> halstead: indeed
[13:12] <moto-timo> halstead: the original reason for today's query was vmeson asking about meta-lat
[13:13] <halstead> moto-timo: So disabling the tmpfile it's immediately returning a 500. I'll need to raise the debug log level.
[13:13] <moto-timo> halstead: I still don't see it in the layers, so I wonder if the data input was malformed somehow
[13:14] <moto-timo> halstead: strange, it's in Layers in the admin interface...
[13:14] <halstead> The layer update process is looping with "FileExistsError: [Errno 17] File exists: 'python2' -> '/tmp/tmpc2hwzx78/tmp-glibc/hosttools/python'"
[13:14] <moto-timo> oh gee isn't that fantastic?
[13:14] <moto-timo> python2 die die die
[13:18] <moto-timo> halstead: any thing in the log to indicate which layer is causing that or more hints?
[13:19] <halstead> Still reading.
[13:19] <moto-timo> this is likely in a part of the code I haven't touched much
[15:12] <moto-timo> fwiw I have acces to login to the vm running layer index now. Some of the inner workings are still only in Paul’s head though
[15:14] <moto-timo> There are some wonky things from older releases. The ‘python2’ file already exists is come from base.bbclass
[15:15] <moto-timo> https://git.yoctoproject.org/poky/tree/meta/classes/base.bbclass?h=zeus#n141
Comment 1 Tim Orling 2022-01-07 22:39:27 UTC
The layer exists in the database (via the 'admin' interface of Django), but something is causing the update layers job to fail.
Comment 2 Tim Orling 2022-01-14 06:47:51 UTC
One problem here is that the layerindex uses "master" to both represent the oe-core branch _and_ the branch of the layer. Since the branch for meta-lat is "main" this breaks the current data model.
Comment 3 Tim Orling 2022-01-14 07:00:48 UTC
Attempt to force the layer to load:
$ docker exec <container_id> sh -c "/opt/layerindex/layerindex/update.py -l meta-lat -b main -r"

https://layers.openembedded.org/layerindex/updates/33936/
Comment 4 Tim Orling 2022-04-07 16:25:57 UTC
Assigning to Randy to have someone within WR work on it.
Comment 5 Randy MacLeod 2022-04-07 21:04:05 UTC
Robert, Please rebase our local patches that make sense to be upstreamed and work with Tim to try to get this issue resolved.
Comment 6 Tim Orling 2022-04-10 05:10:23 UTC
Through the admin interface, you can set the actual branch name (if it is different than "master").

Changed layer branch meta-lat: master layer collection to 'lat-layer', actual branch to 'main'
Changed layer meta-lat status to 'New'
Comment 8 Randy MacLeod 2022-04-10 16:33:06 UTC
Thanks Tim!
Comment 9 Tim Orling 2022-04-11 02:58:39 UTC
NOTE: we are still working through some issues with layer updates, so this might take a couple days before it finally populates (but that is a meta-data problem).