7981 2015-07-09 16:48:57 +0000 Set up procedure for layer index database upgrades 2016-07-14 17:10:21 +0000 1 1 6 Yocto Project Subprojects Layer Index Layer Index Metadata unspecified x86 Multiple RESOLVED FIXED Medium+ enhancement 2.2 M1 6618 7855 1 bluelightning mhalstead belen.barros.pena benjamin.esquivel mhalstead michael.g.wood libertad.gonzalez.de.la.cruz --- --- No (bug/feature does not impact docs) oldest_to_newest 52252 0 bluelightning 2015-07-09 16:48:57 +0000 When we make changes to the layer index we usually have to upgrade the database; there is a risk that the upgrade will fail and thus we need to do a backup of the database first, then do the upgrade, and then restore the backup if it fails. There's the added complication that the database user used by the layer index application normally has restricted access to the database, which is not sufficient for the upgrade. What we need is a documented procedure to follow for doing the upgrade on our own instance (this is not really user documentation because it's specific to our setup). 52333 1 belen.barros.pena 2015-07-14 09:39:56 +0000 This kind of blocks the Toaster image customisation feature ... Paul disagrees, but I still think it does ;) 53295 2 2670 alejandro.franco 2015-08-20 20:01:26 +0000 Created attachment 2670 Update process outline For the full document contact Alex Franco 53296 3 alejandro.franco 2015-08-20 20:04:04 +0000 Need more info from Michael Halstead: (1) production set up using docker, (2) feedback on update process - stripped down version attached. 54562 4 michael.g.wood 2015-09-23 16:01:53 +0000 Any update on this, we're getting close to requiring the blocking bug #7575 fixed for information needed in live toaster instances. Is there an intermediate / less complicated solution that can be done in the mean time to unblock this bug? 54980 5 mhalstead 2015-10-06 03:45:09 +0000 We are very close currently the migration fails. I've been in touch with Alex Franco about this. I see that the update script is currently run with just the -q switch while the update also uses the -r switch to reload the data. Does -r only needs to be set on the first update or is this required on all calls to update.py now? 55005 6 alejandro.franco 2015-10-06 22:59:39 +0000 Been in touch with Michael about this, now since this is an enhancement it probably won't go into 2.0 so I am moving this to 2.1 as we continue working on it 55013 7 belen.barros.pena 2015-10-07 08:14:19 +0000 (In reply to comment #6) > Been in touch with Michael about this, now since this is an enhancement it > probably won't go into 2.0 so I am moving this to 2.1 as we continue working > on it We really need this in Toaster for 2.0. We have split the image recipes from all other recipes as a usability enhancement and in preparation for image customisation. But Toaster's ability to properly identify image recipes is severely flawed without the parsing information the layer index provides. We have been asking for this since the beginning of this release. Can we please (please, please) get this done within 2.0? 55015 8 mhalstead 2015-10-07 09:15:47 +0000 I sat down with Paul and he fixed out the fix. The migration revision history is missing on production so we ran the following commands to set the versions manually. Now we're moving forward again. python /opt/layerindex/manage.py migrate captcha 0001 --fake python /opt/layerindex/manage.py migrate reversion 0005 --fake If all goes well we should have this done today. 55016 9 belen.barros.pena 2015-10-07 09:30:36 +0000 > If all goes well we should have this done today. Remind me to buy you a drink when I get to Dublin :) Thank you! 55105 10 mhalstead 2015-10-09 11:28:15 +0000 The new version based on the paule/inherits2 layer is now running in production. Please review the site and report any errors. I did have to make several changes to the Dockerfile and support files to make it production ready. I'll push these to my own branch soon if that's acceptable. 55107 11 belen.barros.pena 2015-10-09 11:39:12 +0000 The layer index looked ok to me and just tested with Toaster, and things seem to be working as expected. Once again, thank you so much for getting this done! 55465 12 alejandro.franco 2015-10-19 22:58:20 +0000 7575 was closed, so for this we are now discussing a public DB snapshot, beyond that it can probably be closed unless Dockerfile changes need to happen as well, discussion ongoing with Michael and Paul. 62517 13 bluelightning 2016-05-30 02:05:45 +0000 I think this one is now resolved - Michael has put in place procedures for upgrades and we've been through at least two upgrades with that now. 2670 2015-08-20 20:01:26 +0000 2015-08-20 20:01:26 +0000 Update process outline file_7981.txt text/plain 2465 alejandro.franco WW9jdG8gdXBkYXRlIHByb2Nlc3MgZm9yIGxheWVyaW5kZXgtd2ViIChzZW5zaXRpdmUvaW50ZXJu YWwvc3BlY2lmaWMgaW5mbyByZW1vdmVkKQ0KDQpJbnRybw0KDQpVcGRhdGluZyB0aGUgbGF5ZXIg aW5kZXggcmVxdWlyZXMgaW4gZ2VuZXJhbCB0aGUgZm9sbG93aW5nIHN0ZXBzOg0KLSBQdWxsIGxh dGVzdCBjb2RlIGZyb20gdGhlIHJlcG9zaXRvcnkNCi0gVXBkYXRlIGRhdGFiYXNlIG1vZGVscyBp ZiByZXF1aXJlZCAobWFuYWdlLnB5IG1pZ3JhdGUgbGF5ZXJpbmRleCkNCi0gVXBkYXRlIGxheWVy IGluZm9ybWF0aW9uLCBpbmNsdWRpbmcgYW55IG5ld2x5IGFkZGVkIGZpZWxkcyAodXBkYXRlLnB5 IC1yIC1xKQ0KICAtIE9wdGlvbiAtciBtZWFucyB0aGUgZGF0YSB3aWxsIGJlIHJlbG9hZGVkLCB0 aGlzIHNob3VsZCBwcmVzZXJ2ZSBleGlzdGluZyBJRHMNCiAgLSBPcHRpb24gLXEgbWVhbnMgdGhl IHVwZGF0ZSB3aWxsIGJlIHF1aWV0LCB0aGlzIGlzIGdlbmVyYWxseSByZWNvbW1lbmRlZA0KICAt IFRoZSB1cGRhdGUgbXVzdCBiZSBpc3N1ZWQgZm9yIGFkZGl0aW9uYWwgYnJhbmNoZXMgKGNvcnJl c3BvbmRpbmcgd2l0aCBZb2N0byB2ZXJzaW9ucykgYmVzaWRlcyBtYXN0ZXIgKHRoZSBkZWZhdWx0 KSwgaW4gdGhvc2UgY2FzZXMgdXBkYXRlLnB5IGlzIGFsc28gY2FsbGVkIHdpdGggb3B0aW9uczoN CiAgLWIgPGJyYW5jaG5hbWU+IA0KICAteCBzbyBubyBuZXcgZGF0YSBpcyBmZXRjaGVkIHVubmVj ZXNzYXJpbHkNCg0KUHJvZHVjdGlvbg0KDQpJbiBwcm9kdWN0aW9uIGxheWVyaW5kZXgtd2ViIHJ1 bnMgb24gdG9wIG9mIERvY2tlciBjb250YWluZXJzLCByZXF1aXJpbmcgdGhlIGZvbGxvd2luZyB0 byBiZSB1cGRhdGVkIG9uIHByb2R1Y3Rpb24gc2VydmVyKHMpLiBEb2NrZXIgY29udGFpbmVycyB0 byB1cGRhdGU6DQogIGxheWVyc2RiOiBydW5zIHRoZSBtYXJpYWRiIGRhdGFiYXNlDQogIGxheWVy c3dlYjogcnVucyB0aGUgbmdpbnggd2ViIHNlcnZlcg0KICBsYXllcnNhcHA6IGhvc3RzIGxheWVy aW5kZXgtd2ViIGFwcGxpY2F0aW9uDQoNClN0ZXBzIHRvIHVwZGF0ZToNCihEb3dudGltZSB3aW5k b3cgbWF5IGJlIG5lY2Vzc2FyeSwgc3RlcCAjNSB0YWtlcyB+MSBob3VyIG9uIGl0cyBvd24uKQ0K DQotIFJlYWQgdGhlIGxheWVyaW5kZXgtd2ViL1JFQURNRSBhbmQgbGF5ZXJpbmRleC13ZWIvZG9j a2VyL1JFQURNRSBpbnN0cnVjdGlvbnMNCi0gT2J0YWluIG9yIGlzc3VlIGEgc25hcHNob3Qgb2Yg dGhlIGRhdGFiYXNlIChpbiBjYXNlIHdlIG5lZWQgdG8gcm9sbCBiYWNrKQ0KLSBMb2cgaW50byBw cm9kdWN0aW9uIHNlcnZlcihzKSBob3N0aW5nIHRoZSBsYXllcmluZGV4IERvY2tlciBjb250YWlu ZXJzDQotIE9uIHRoZSBsYXllcnNhcHAgY29udGFpbmVyOg0KLSBQdWxsIGxhdGVzdCBjb2RlIChv ciByZWdlbmVyYXRlIGNvbnRhaW5lcj8pDQogICAgZ2l0IGZldGNoDQogICAgZ2l0IHJlYmFzZSBv cmlnaW4vbWFzdGVyDQoNCk5PVEU6IHNldHRpbmdzIGZpbGVzIGFyZSBnZW5lcmljLCBhbmQgZG8g bm90IGNvbnRhaW4gYW55IHNldHRpbmdzIHJlcXVpcmVkIGZvciBsYXllcmluZGV4IHRvIHdvcmsg aW4gcHJvZHVjdGlvbi4gTW9yZSBzcGVjaWZpYyBzZXR0aW5ncyBpbiB0aGUgZnVsbCBkb2N1bWVu dC4NCg0KLSBSdW4gbWlncmF0aW9uIChtYW5hZ2UucHkgbWlncmF0ZSBsYXllcmluZGV4KQ0KLSBD dXN0b20gc2NyaXB0IHNob3VsZCBiZSBhdCAvb3B0L21pZ3JhdGUuc2gsIGFuZCBjYW4gYmUgcnVu IGZyb20gaG9zdCBhczoNCmRvY2tlciBydW4gLS1ybSAtLWxpbmsgbGF5ZXJkYjpsYXllcnNkYiBo YWxzdGVhZC9sYXllcmluZGV4LWFwcCAvb3B0L21pZ3JhdGUuc2gNCi0gUnVuIGxheWVyIHVwZGF0 ZSAodXBkYXRlLnB5KQ0KU2VlIGludHJvZHVjdGlvbiBpbmZvIGFib3ZlIGFuZCByZWZlciB0byBS RUFETUUgZmlsZXMgZm9yIGJldHRlciB1bmRlcnN0YW5kaW5nDQotIEN1c3RvbSBzY3JpcHQgc2hv dWxkIGJlIGF0IC9vcHQvdXBkYXRlbGF5ZXJzLnNoLCBhbmQgY2FuIGJlIHJ1biBmcm9tIGhvc3Qg YXM6DQpkb2NrZXIgcnVuIC0tcm0gLS1saW5rIGxheWVyZGI6bGF5ZXJzZGIgaGFsc3RlYWQvbGF5 ZXJpbmRleC1hcHAgL29wdC9taWdyYXRlLnNoDQoNClJvbGxpbmcgYmFjayBzaG91bGQgYmUgYXMg c2ltcGxlIGFzOg0KLSBSZWxvYWRpbmcgZGF0YWJhc2Ugd2l0aCBwcmV2aW91c2x5IGNhcHR1cmVk IHNuYXBzaG90DQotIFJlc2V0dGluZyBsYXllcmluZGV4IGNvZGUgdG8gYSBwcmV2aW91cyBjb21t aXQNCg0KKGZvciByZWZlcmVuY2UpIHNldHRpbmcgdXAgbmV3IGluc3RhbmNlczoNCkNsb25lIHRo ZSByZXBvIGFuZCBmb2xsb3cgaW5zdHJ1Y3Rpb25zIGluIHRoZSBSRUFETUUgZmlsZXM7IHVzaW5n IERvY2tlciBjb250YWluZXJzIGlzIG9wdGlvbmFsLiBSZXZpZXcgTk9URVMgdXBkYXRlIHN0ZXAg NSAoc2VlIGFib3ZlKS4=