Bug 8364

'verify_prereq' in bitbake/bin/toaster will also need fixing to reflect any version change. It would be good if this function checked the dependencies from the requirements.txt rather than hardcoding the values.

I sent an RFC about this to the mailing list:https://lists.yoctoproject.org/pipermail/toaster/2015-November/003347.html

Assigned to Ed, as he's bisected changes to Django and identified the culprit commit which is causing the issue.

2 comments/questions:
1) Why did we choose Django 1.8.6? Should we do a toaster-requirements like Django < 1.9?  that way we would get the most up to date 1.8 available

2) On ed's branch: ed/toaster/django_upgrade-8364  I had 170 errors like WARNING database is locked, model: <class 'orm.models.Task'>, args: (), kwargs: {'using': 'default', 'force_insert': True} from the final save attempt.  If I used the patch Ed had suggested we try applied to 1.8.6 + this branch I had 0 errors.

My diff against the standard 1.8.6:
(t-Django-1.8)bavery@whisper:~/src/intel/yocto-builds/TOASTER/myBugs/t-Django-1.8-ED_FIX$ diff ./lib/python2.7/site-packages/django/db/models/query.py ../t-Django-1.8/./lib/python2.7/site-packages/django/db/models/query.py
348,349c348
<         with transaction.atomic(using=self.db):
<             obj.save(force_insert=True, using=self.db)
---
>         obj.save(force_insert=True, using=self.db)
439c438,439
<             obj = self.create(**params)
---
>             with transaction.atomic(using=self.db):
>                 obj = self.create(**params)

-- So, Basically, from my testing we need more of the original fix in the create calls...

the build that i used as a test case was core-sato-image-dev in my previous comment.

Brian, thank you for the review and testing! I'll try to fix what you pointed out.

As for the locking issue, I based my fix on the Elliot's feedback. He wasn't able to reproduce the issue with only this patch applied: https://gist.github.com/bartosh/8c188444e349a76cc1b9

Looks like monkey patching didn't work properly or I missed something.


Ellioth, can you confirm that the issue was gone in your setup with the above patch applied?

Yes, the issue is gone with that patch.

I tested poky-contrib/ed/toaster/django_upgrade-8364

In 3 different settings:
1) as is
2) with the monkeypatch reverted (revert 6b997fd0cb9d913ef734659386cc97985e428119)
3) additionally with the retry patch reverted (revert d27cdb20eebe14f532ec726e769cad399ccdedf5)

Results:
1) - worked flawlessly for me under 1.8.7 & 1.8.6.
2) had over 100 "database is locked" errors in first 10%, no additional for most of the build and another 255 at the end (last 5%? ) of the build.  Additionally, because the retry is a .2 second loop, it would periodically make the ui unresponsive and I'd get an error page briefly if I repeatedly hit "all builds" while it was doing this.
3) had ~ 100 "database is locked" errors.  From what I can tell, though I didn't verify this, when we have a database is locked error we lose data (namely the task we were storing).

Since #1 works, I'd like to put it in early and see if the monkey's generate problems for anyone over the course of 2.1 testing. #2 hurts the UI and I'm pretty sure #3 loses info...

Furthermore, it would be nice/good to submit the monkeypatch to Django to get the django people to comment on it. I seriously doubt they'd accept it but their comments may very well be illuminating.

The patchset is under review by upstream: http://lists.openembedded.org/pipermail/bitbake-devel/2015-December/006751.html

both patchset have been accepted upstream: http://cgit.openembedded.org/bitbake/commit/?id=49976eca4a6e37e7653814c569badcd3e0fb719a

Verified on master.