Error: RuntimeError: Cannot reverse this migration. 'Project.bitbake_version' and its values cannot be restored. Steps to reproduce: 0. cd opt/ A. git clone git://git.yoctoproject.org/poky B. virtualenv venv C. source venv/bin/activate D. pip install -r /opt/poky/bitbake/toaster-requirements.txt E. pip install mysql-python F. create a mysql database and setup access in /opt/poky/bitbake/lib/toaster/toastermain/settings.py G. /opt/poky/bitbake/lib/toaster/manage.py syncdb H. /opt/poky/bitbake/lib/toaster/manage.py migrate orm
If you're following the instructions for the poky 1.8 release "fido" please make sure to switch this after cloning. (or specify the branch in the clone command git clone -b fido git://git.yoctoproject.org/poky.git ) We need to update the documentation to fix these instructions as it seems to be missing.
References to "git clone" should also include the corresponding release of poky that the documentation is written for. http://www.yoctoproject.org/docs/1.8/toaster-manual/toaster-manual.html#setting-up-a-hosted-service-and-running-in-build-mode Instances of: " $ git clone git://git.yoctoproject.org/poky" Should be: " $ git clone -b fido git://git.yoctoproject.org/poky" Where "fido" is the release for which the documentation is written for.
I see the same issue on poky master @ a533776. The error doesn't occur if using SQLite. I'm running MySQL as part of XAMPP. `/opt/lampp/bin/mysql --version` gives: ``` /opt/lampp/bin/mysql Ver 14.14 Distrib 5.6.16, for Linux (x86_64) using EditLine wrapper ``` `toastermain/settings.py` is like this: ``` DATABASES = { 'default': { 'ENGINE': 'django.db.backends.mysql', 'NAME': 'toaster', 'USER': 'toaster', 'PASSWORD': '*******', 'HOST': '/opt/lampp/var/mysql/mysql.sock', 'PORT': '' } } ``` I did the setup of the basic database structure using the mysql command line: ``` $ /opt/lampp/bin/mysql -u root > CREATE USER 'toaster'@'localhost' IDENTIFIED BY '*******'; > CREATE DATABASE toaster; > GRANT ALL PRIVILEGES ON toaster.* TO 'toaster'@'localhost'; ``` Note that the MySQL server is configured to use InnoDB tables by default, which support transactions (the error I was getting suggested lack of DDL support, so I thought the use of MyISAM tables might be causing my error). I confirmed InnoDB support with: ``` > SELECT TABLE_NAME, ENGINE FROM information_schema.TABLES where TABLE_SCHEMA = 'toaster'; +--------------------------------+--------+ | TABLE_NAME | ENGINE | +--------------------------------+--------+ | auth_group | InnoDB | | auth_group_permissions | InnoDB | ... ``` I then started toaster in managed mode. When the migrations run (against a clean db) the migrations fail with this error: ``` $ TOASTER_MANAGED=1 source toaster start The system will start. Syncing... Creating tables ... Creating table auth_permission Creating table auth_group_permissions Creating table auth_group Creating table auth_user_groups Creating table auth_user_user_permissions Creating table auth_user Creating table django_content_type Creating table django_session Creating table django_admin_log Creating table south_migrationhistory You just installed Django's auth system, which means you don't have any superusers defined. Would you like to create one now? (yes/no): no Installing custom SQL ... Installing indexes ... Installed 0 object(s) from 0 fixture(s) Synced: > django.contrib.auth > django.contrib.contenttypes > django.contrib.messages > django.contrib.sessions > django.contrib.admin > django.contrib.staticfiles > django.contrib.humanize > south Not synced (use migrations): - orm - bldcontrol (use ./manage.py migrate to migrate these) Running migrations for orm: - Migrating forwards to 0024_auto__add_field_recipe_is_image. > orm:0001_initial > orm:0002_auto__add_field_build_timespent > orm:0003_timespent - Migration 'orm:0003_timespent' is marked for no-dry-run. > orm:0004_auto__add_field_package_installed_name > orm:0005_auto__add_target_image_file__add_target_file__add_field_variablehistor > orm:0006_auto__add_field_target_image_size__add_field_target_license_manifest_p > orm:0007_auto__add_helptext > orm:0008_auto__chg_field_variablehistory_operation__chg_field_recipe_descriptio > orm:0009_auto__add_projectvariable__add_projectlayer__add_projecttarget__add_pr > orm:0010_auto__add_field_project_branch__add_field_project_short_description__a > orm:0011_auto__add_field_projectlayer_dirpath > orm:0012_auto__add_field_projectlayer_optional__add_field_projecttarget_task > orm:0013_auto__add_release__add_layerversiondependency__add_unique_layerversion > orm:0014_auto__chg_field_package_summary__chg_field_layer_summary__chg_field_re > orm:0015_auto__add_field_layer_vcs_web_url__add_field_layer_vcs_web_tree_base_u > orm:0016_auto__add_field_release_helptext__chg_field_release_branch__add_index_ > orm:0017_auto__del_toastersettingdefaultlayer__add_releaselayersourcepriority__ > orm:0018_auto__add_field_layer_version_project > orm:0019_auto__add_buildartifact > orm:0020_auto__add_field_layer_version_local_path__add_field_recipe_pathflags__ > orm:0021_auto__chg_field_build_project__chg_field_project_bitbake_version__chg_ Traceback (most recent call last): File "/home/ell/dev/toaster/poky/bitbake/bin/../lib/toaster/manage.py", line 10, in <module> execute_from_command_line(sys.argv) File "/home/ell/toaster-run/toaster-env/lib/python2.7/site-packages/django/core/management/__init__.py", line 399, in execute_from_command_line utility.execute() File "/home/ell/toaster-run/toaster-env/lib/python2.7/site-packages/django/core/management/__init__.py", line 392, in execute self.fetch_command(subcommand).run_from_argv(self.argv) File "/home/ell/toaster-run/toaster-env/lib/python2.7/site-packages/django/core/management/base.py", line 242, in run_from_argv self.execute(*args, **options.__dict__) File "/home/ell/toaster-run/toaster-env/lib/python2.7/site-packages/django/core/management/base.py", line 285, in execute output = self.handle(*args, **options) File "/home/ell/toaster-run/toaster-env/lib/python2.7/site-packages/south/management/commands/migrate.py", line 111, in handle ignore_ghosts = ignore_ghosts, File "/home/ell/toaster-run/toaster-env/lib/python2.7/site-packages/south/migration/__init__.py", line 220, in migrate_app success = migrator.migrate_many(target, workplan, database) File "/home/ell/toaster-run/toaster-env/lib/python2.7/site-packages/south/migration/migrators.py", line 254, in migrate_many result = migrator.__class__.migrate_many(migrator, target, migrations, database) File "/home/ell/toaster-run/toaster-env/lib/python2.7/site-packages/south/migration/migrators.py", line 329, in migrate_many result = self.migrate(migration, database) File "/home/ell/toaster-run/toaster-env/lib/python2.7/site-packages/south/migration/migrators.py", line 133, in migrate result = self.run(migration, database) File "/home/ell/toaster-run/toaster-env/lib/python2.7/site-packages/south/migration/migrators.py", line 114, in run return self.run_migration(migration, database) File "/home/ell/toaster-run/toaster-env/lib/python2.7/site-packages/south/migration/migrators.py", line 93, in run_migration print(self.run_migration_error(migration)) File "/home/ell/toaster-run/toaster-env/lib/python2.7/site-packages/south/migration/migrators.py", line 323, in run_migration_error (self.format_backwards(migration), extra_info)) File "/home/ell/toaster-run/toaster-env/lib/python2.7/site-packages/south/migration/migrators.py", line 310, in format_backwards self.backwards(migration)() File "/home/ell/toaster-run/toaster-env/lib/python2.7/site-packages/south/migration/migrators.py", line 60, in <lambda> return (lambda: direction(orm)) File "/home/ell/dev/toaster/poky/bitbake/lib/toaster/orm/migrations/0021_auto__chg_field_build_project__chg_field_project_bitbake_version__chg_.py", line 33, in backwards raise RuntimeError("Cannot reverse this migration. 'Project.bitbake_version' and its values cannot be restored.") RuntimeError: Cannot reverse this migration. 'Project.bitbake_version' and its values cannot be restored. Error on migration, trying to recover... Running migrations for orm: - Migrating backwards to just after 0001_initial. < orm:0020_auto__add_field_layer_version_local_path__add_field_recipe_pathflags__ (faked) < orm:0019_auto__add_buildartifact (faked) < orm:0018_auto__add_field_layer_version_project (faked) < orm:0017_auto__del_toastersettingdefaultlayer__add_releaselayersourcepriority__ (faked) < orm:0016_auto__add_field_release_helptext__chg_field_release_branch__add_index_ (faked) < orm:0015_auto__add_field_layer_vcs_web_url__add_field_layer_vcs_web_tree_base_u (faked) < orm:0014_auto__chg_field_package_summary__chg_field_layer_summary__chg_field_re (faked) < orm:0013_auto__add_release__add_layerversiondependency__add_unique_layerversion (faked) < orm:0012_auto__add_field_projectlayer_optional__add_field_projecttarget_task (faked) < orm:0011_auto__add_field_projectlayer_dirpath (faked) < orm:0010_auto__add_field_project_branch__add_field_project_short_description__a (faked) < orm:0009_auto__add_projectvariable__add_projectlayer__add_projecttarget__add_pr (faked) < orm:0008_auto__chg_field_variablehistory_operation__chg_field_recipe_descriptio (faked) < orm:0007_auto__add_helptext (faked) < orm:0006_auto__add_field_target_image_size__add_field_target_license_manifest_p (faked) < orm:0005_auto__add_target_image_file__add_target_file__add_field_variablehistor (faked) < orm:0004_auto__add_field_package_installed_name (faked) < orm:0003_timespent (faked) < orm:0002_auto__add_field_build_timespent (faked) Running migrations for orm: - Migrating forwards to 0024_auto__add_field_recipe_is_image. > orm:0002_auto__add_field_build_timespent FATAL ERROR - The following SQL query failed: ALTER TABLE `orm_build` ADD COLUMN `timespent` integer NOT NULL DEFAULT 0; The error was: (1060, "Duplicate column name 'timespent'") ! Error found during real run of migration! Aborting. ! Since you have a database that does not support running ! schema-altering statements in transactions, we have had ! to leave it in an interim state between migrations. ! You *might* be able to recover with: - no dry run output for delete_foreign_key() due to dynamic DDL, sorry = ALTER TABLE `orm_build` DROP COLUMN `timespent` CASCADE; [] ! The South developers regret this has happened, and would ! like to gently persuade you to consider a slightly ! easier-to-deal-with DBMS (one that supports DDL transactions) ! NOTE: The error which caused the migration to fail is further up. Error in migration: orm:0002_auto__add_field_build_timespent Traceback (most recent call last): File "/home/ell/dev/toaster/poky/bitbake/bin/../lib/toaster/manage.py", line 10, in <module> execute_from_command_line(sys.argv) File "/home/ell/toaster-run/toaster-env/lib/python2.7/site-packages/django/core/management/__init__.py", line 399, in execute_from_command_line utility.execute() File "/home/ell/toaster-run/toaster-env/lib/python2.7/site-packages/django/core/management/__init__.py", line 392, in execute self.fetch_command(subcommand).run_from_argv(self.argv) File "/home/ell/toaster-run/toaster-env/lib/python2.7/site-packages/django/core/management/base.py", line 242, in run_from_argv self.execute(*args, **options.__dict__) File "/home/ell/toaster-run/toaster-env/lib/python2.7/site-packages/django/core/management/base.py", line 285, in execute output = self.handle(*args, **options) File "/home/ell/toaster-run/toaster-env/lib/python2.7/site-packages/south/management/commands/migrate.py", line 111, in handle ignore_ghosts = ignore_ghosts, File "/home/ell/toaster-run/toaster-env/lib/python2.7/site-packages/south/migration/__init__.py", line 220, in migrate_app success = migrator.migrate_many(target, workplan, database) File "/home/ell/toaster-run/toaster-env/lib/python2.7/site-packages/south/migration/migrators.py", line 254, in migrate_many result = migrator.__class__.migrate_many(migrator, target, migrations, database) File "/home/ell/toaster-run/toaster-env/lib/python2.7/site-packages/south/migration/migrators.py", line 329, in migrate_many result = self.migrate(migration, database) File "/home/ell/toaster-run/toaster-env/lib/python2.7/site-packages/south/migration/migrators.py", line 133, in migrate result = self.run(migration, database) File "/home/ell/toaster-run/toaster-env/lib/python2.7/site-packages/south/migration/migrators.py", line 114, in run return self.run_migration(migration, database) File "/home/ell/toaster-run/toaster-env/lib/python2.7/site-packages/south/migration/migrators.py", line 84, in run_migration migration_function() File "/home/ell/toaster-run/toaster-env/lib/python2.7/site-packages/south/migration/migrators.py", line 60, in <lambda> return (lambda: direction(orm)) File "/home/ell/dev/toaster/poky/bitbake/lib/toaster/orm/migrations/0002_auto__add_field_build_timespent.py", line 14, in forwards keep_default=False) File "/home/ell/toaster-run/toaster-env/lib/python2.7/site-packages/south/db/generic.py", line 47, in _cache_clear return func(self, table, *args, **opts) File "/home/ell/toaster-run/toaster-env/lib/python2.7/site-packages/south/db/generic.py", line 418, in add_column self.execute(sql) File "/home/ell/toaster-run/toaster-env/lib/python2.7/site-packages/south/db/generic.py", line 282, in execute cursor.execute(sql, params) File "/home/ell/toaster-run/toaster-env/lib/python2.7/site-packages/django/db/backends/util.py", line 69, in execute return super(CursorDebugWrapper, self).execute(sql, params) File "/home/ell/toaster-run/toaster-env/lib/python2.7/site-packages/django/db/backends/util.py", line 53, in execute return self.cursor.execute(sql, params) File "/home/ell/toaster-run/toaster-env/lib/python2.7/site-packages/django/db/utils.py", line 99, in __exit__ six.reraise(dj_exc_type, dj_exc_value, traceback) File "/home/ell/toaster-run/toaster-env/lib/python2.7/site-packages/django/db/backends/util.py", line 53, in execute return self.cursor.execute(sql, params) File "/home/ell/toaster-run/toaster-env/lib/python2.7/site-packages/django/db/backends/mysql/base.py", line 124, in execute return self.cursor.execute(query, args) File "/home/ell/toaster-run/toaster-env/lib/python2.7/site-packages/MySQLdb/cursors.py", line 205, in execute self.errorhandler(self, exc, value) File "/home/ell/toaster-run/toaster-env/lib/python2.7/site-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler raise errorclass, errorvalue django.db.utils.OperationalError: (1060, "Duplicate column name 'timespent'") ```
This is a blocker for me, as I need it fixed to be able to run Toaster (I can't get it to work with SQLite, as described in bug 8147). Which branch of poky should be referenced in the docs change?
I think this is just an issue with MySQL performing a dry run of the SQL statements, which throws an error with some of the migrations as they can't be run backwards. (When using SQLite, migrations aren't dry run: see http://south.aeracode.org/wiki/Tutorial3.)
Created attachment 2686 [details] Design - Showing the default project storing the Knotty builds
Do we need to consider other mechanisms for triggering builds apart from the command line? Just wondering whether they should be called "Command line builds" or something like "External builds" instead. (I don't think "External builds" is correct either, but there could be a way to trigger a bitbake build, but not from the command line.)
(In reply to comment #7) > Do we need to consider other mechanisms for triggering builds apart from the > command line? Just wondering whether they should be called "Command line > builds" or something like "External builds" instead. (I don't think > "External builds" is correct either, but there could be a way to trigger a > bitbake build, but not from the command line.) The only case I can think of is something like the Autobuilder (CI sending information about its builds to a Toaster instance). An easy workaround would be to allow people to change the name of the project. 'Command line builds' reflects what we believe is the most likely use case for this project. If it doesn't fit your use case, change the project name! :) Would that work?
Yes, I think that would be fine. Should the UI therefore retain the option to edit the "default project"'s name? Note that I've mostly implemented a better default project. I have tried adding the default project to the db during initial setup by migration, rather than at runtime when the first command line build appears. However, this has the effect of putting a project into the database, which means the landing page is skipped (because there's a project). Should I keep the old behaviour for now (where the default project is created at runtime), even though it feels incorrect?
Alternatively, I could modify the landing page (to only redirect if there is a project other than the default one) and modify the projects page to only show projects with a build count > 0.
(In reply to comment #9) > Yes, I think that would be fine. Should the UI therefore retain the option > to edit the "default project"'s name? yes, please! > > Note that I've mostly implemented a better default project. > > I have tried adding the default project to the db during initial setup by > migration, rather than at runtime when the first command line build appears. > However, this has the effect of putting a project into the database, which > means the landing page is skipped (because there's a project). > > Should I keep the old behaviour for now (where the default project is > created at runtime), even though it feels incorrect?
(In reply to comment #10) > Alternatively, I could modify the landing page (to only redirect if there is > a project other than the default one) This sounds better to me, particularly if creating the project at runtime is the wrong thing to do. and modify the projects page to only > show projects with a build count > 0. This will be strange for normal projects. It means that if I've just created a project, and then go to the all projects page, the project I just created won't be listed there, which will make me think that something has gone wrong. What's worse: there would be no way for me to access the project, since the only entry point for a project with no builds is the list of all projects. I'm afraid we'll need to do something like "if this is the default project and the number of builds in it is 0, do now show it".
Created attachment 2693 [details] Design - Showing the default project storing the Knotty builds Adding some additional help text based on Tiago's feedback (pages 5 and 8)
Belen, could we move the new design for the command line builds out of this bug and into a fresh one? It would be good if we could just get the "default project" working close to how it works with SQLite right now, and think about a new design for that part of the application in future.
GUI work is now covered by bug 8231
To test, set up Toaster with a MySQL database, as per instructions in http://www.yoctoproject.org/docs/1.8/toaster-manual/toaster-manual.html#setting-up-a-hosted-service-and-running-in-analysis-mode.
Verified while installing MySQL instance for GDC.