The layer index update script is tripping over this every time it runs: -------------- snip -------------- Traceback (most recent call last): File "/var/www/html/layerindex/layerindex/update.py", line 304, in main File "/usr/local/lib/python2.7/dist-packages/git/repo/base.py", line 420, in iter_commits File "/usr/local/lib/python2.7/dist-packages/git/objects/commit.py", line 192, in iter_items File "/usr/local/lib/python2.7/dist-packages/git/cmd.py", line 219, in <lambda> File "/usr/local/lib/python2.7/dist-packages/git/cmd.py", line 430, in _call_process File "/usr/local/lib/python2.7/dist-packages/git/cmd.py", line 309, in execute File "/usr/lib/python2.7/subprocess.py", line 679, in __init__ File "/usr/lib/python2.7/subprocess.py", line 1135, in _execute_child File "/usr/lib/python2.7/subprocess.py", line 1091, in pipe_cloexec OSError: [Errno 24] Too many open files -------------- snip -------------- This may be related to the fact that we aren't processing any events generated during parsing and thus these are stacking up in memory; perhaps there are open file handles associated with these as well.
I should mention that this is not a new issue, it's been happening for some time now and is only getting worse as the database has grown in size.
It seems an known issue [1] and it's fixed since v0.3.5, i think with an upgrade should be enough to solve the problem the current version is 1.0.1 [2]. [1] https://github.com/gitpython-developers/GitPython/issues/60 [2] https://pypi.python.org/pypi/GitPython/1.0.1
I had a look into this since we were not only leaking file handles, but also memory. I've applied the following fix: http://git.yoctoproject.org/cgit/cgit.cgi/layerindex-web/commit/?id=f4b45258589185fd8bb7f220470a72650dc792f8 After this everything should now be working much better - at least a refresh can now be run successfully. I'll mark this as fixed for now, if we see the problem again we can reopen.
An additional fix for the fix for compatibility with daisy and earlier: http://git.yoctoproject.org/cgit/cgit.cgi/layerindex-web/commit/?id=40f1d51de67645506cbe7becb625cbcaeaa9f716
(In reply to comment #4) > An additional fix for the fix for compatibility with daisy and earlier: > > http://git.yoctoproject.org/cgit/cgit.cgi/layerindex-web/commit/ > ?id=40f1d51de67645506cbe7becb625cbcaeaa9f716 Seems that the issue is fixed after Paul's confirmation.