| Summary: | Can not migrate orm (mysql, virtualenv) master branch (2015-06-29) #2578ff3 | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Product: | [Build System, Metadata & Runtime] Toaster | Reporter: | Daniel Kuhne <dakuit00> | ||||||
| Component: | toaster | Assignee: | Elliot Smith <elliot.smith> | ||||||
| Status: | VERIFIED FIXED | QA Contact: | Mihail Stanciu <stanciux.mihail> | ||||||
| Severity: | normal | ||||||||
| Priority: | Medium | CC: | alexandru.damian, belen.barros.pena, cristiana.voicu, elliot.smith, jessica.zhang, michael.g.wood | ||||||
| Version: | unspecified | ||||||||
| Target Milestone: | 1.9 M3 | ||||||||
| Hardware: | x86 | ||||||||
| OS: | x86_64 | ||||||||
| Whiteboard: | GUI design available | ||||||||
| OS type for building Yocto: | --- | Type of Regression: | --- | ||||||
| Verified: | Documentation change: | Yes (doc changes required) | |||||||
| Attachments: |
|
||||||||
|
Description
Daniel Kuhne
2015-06-29 12:47:03 UTC
If you're following the instructions for the poky 1.8 release "fido" please make sure to switch this after cloning. (or specify the branch in the clone command git clone -b fido git://git.yoctoproject.org/poky.git ) We need to update the documentation to fix these instructions as it seems to be missing. References to "git clone" should also include the corresponding release of poky that the documentation is written for. http://www.yoctoproject.org/docs/1.8/toaster-manual/toaster-manual.html#setting-up-a-hosted-service-and-running-in-build-mode Instances of: " $ git clone git://git.yoctoproject.org/poky" Should be: " $ git clone -b fido git://git.yoctoproject.org/poky" Where "fido" is the release for which the documentation is written for. I see the same issue on poky master @ a533776. The error doesn't occur if using SQLite.
I'm running MySQL as part of XAMPP. `/opt/lampp/bin/mysql --version` gives:
```
/opt/lampp/bin/mysql Ver 14.14 Distrib 5.6.16, for Linux (x86_64) using EditLine wrapper
```
`toastermain/settings.py` is like this:
```
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql',
'NAME': 'toaster',
'USER': 'toaster',
'PASSWORD': '*******',
'HOST': '/opt/lampp/var/mysql/mysql.sock',
'PORT': ''
}
}
```
I did the setup of the basic database structure using the mysql command line:
```
$ /opt/lampp/bin/mysql -u root
> CREATE USER 'toaster'@'localhost' IDENTIFIED BY '*******';
> CREATE DATABASE toaster;
> GRANT ALL PRIVILEGES ON toaster.* TO 'toaster'@'localhost';
```
Note that the MySQL server is configured to use InnoDB tables by default, which support transactions (the error I was getting suggested lack of DDL support, so I thought the use of MyISAM tables might be causing my error). I confirmed InnoDB support with:
```
> SELECT TABLE_NAME, ENGINE FROM information_schema.TABLES where TABLE_SCHEMA = 'toaster';
+--------------------------------+--------+
| TABLE_NAME | ENGINE |
+--------------------------------+--------+
| auth_group | InnoDB |
| auth_group_permissions | InnoDB |
...
```
I then started toaster in managed mode. When the migrations run (against a clean db) the migrations fail with this error:
```
$ TOASTER_MANAGED=1 source toaster start
The system will start.
Syncing...
Creating tables ...
Creating table auth_permission
Creating table auth_group_permissions
Creating table auth_group
Creating table auth_user_groups
Creating table auth_user_user_permissions
Creating table auth_user
Creating table django_content_type
Creating table django_session
Creating table django_admin_log
Creating table south_migrationhistory
You just installed Django's auth system, which means you don't have any superusers defined.
Would you like to create one now? (yes/no): no
Installing custom SQL ...
Installing indexes ...
Installed 0 object(s) from 0 fixture(s)
Synced:
> django.contrib.auth
> django.contrib.contenttypes
> django.contrib.messages
> django.contrib.sessions
> django.contrib.admin
> django.contrib.staticfiles
> django.contrib.humanize
> south
Not synced (use migrations):
- orm
- bldcontrol
(use ./manage.py migrate to migrate these)
Running migrations for orm:
- Migrating forwards to 0024_auto__add_field_recipe_is_image.
> orm:0001_initial
> orm:0002_auto__add_field_build_timespent
> orm:0003_timespent
- Migration 'orm:0003_timespent' is marked for no-dry-run.
> orm:0004_auto__add_field_package_installed_name
> orm:0005_auto__add_target_image_file__add_target_file__add_field_variablehistor
> orm:0006_auto__add_field_target_image_size__add_field_target_license_manifest_p
> orm:0007_auto__add_helptext
> orm:0008_auto__chg_field_variablehistory_operation__chg_field_recipe_descriptio
> orm:0009_auto__add_projectvariable__add_projectlayer__add_projecttarget__add_pr
> orm:0010_auto__add_field_project_branch__add_field_project_short_description__a
> orm:0011_auto__add_field_projectlayer_dirpath
> orm:0012_auto__add_field_projectlayer_optional__add_field_projecttarget_task
> orm:0013_auto__add_release__add_layerversiondependency__add_unique_layerversion
> orm:0014_auto__chg_field_package_summary__chg_field_layer_summary__chg_field_re
> orm:0015_auto__add_field_layer_vcs_web_url__add_field_layer_vcs_web_tree_base_u
> orm:0016_auto__add_field_release_helptext__chg_field_release_branch__add_index_
> orm:0017_auto__del_toastersettingdefaultlayer__add_releaselayersourcepriority__
> orm:0018_auto__add_field_layer_version_project
> orm:0019_auto__add_buildartifact
> orm:0020_auto__add_field_layer_version_local_path__add_field_recipe_pathflags__
> orm:0021_auto__chg_field_build_project__chg_field_project_bitbake_version__chg_
Traceback (most recent call last):
File "/home/ell/dev/toaster/poky/bitbake/bin/../lib/toaster/manage.py", line 10, in <module>
execute_from_command_line(sys.argv)
File "/home/ell/toaster-run/toaster-env/lib/python2.7/site-packages/django/core/management/__init__.py", line 399, in execute_from_command_line
utility.execute()
File "/home/ell/toaster-run/toaster-env/lib/python2.7/site-packages/django/core/management/__init__.py", line 392, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/home/ell/toaster-run/toaster-env/lib/python2.7/site-packages/django/core/management/base.py", line 242, in run_from_argv
self.execute(*args, **options.__dict__)
File "/home/ell/toaster-run/toaster-env/lib/python2.7/site-packages/django/core/management/base.py", line 285, in execute
output = self.handle(*args, **options)
File "/home/ell/toaster-run/toaster-env/lib/python2.7/site-packages/south/management/commands/migrate.py", line 111, in handle
ignore_ghosts = ignore_ghosts,
File "/home/ell/toaster-run/toaster-env/lib/python2.7/site-packages/south/migration/__init__.py", line 220, in migrate_app
success = migrator.migrate_many(target, workplan, database)
File "/home/ell/toaster-run/toaster-env/lib/python2.7/site-packages/south/migration/migrators.py", line 254, in migrate_many
result = migrator.__class__.migrate_many(migrator, target, migrations, database)
File "/home/ell/toaster-run/toaster-env/lib/python2.7/site-packages/south/migration/migrators.py", line 329, in migrate_many
result = self.migrate(migration, database)
File "/home/ell/toaster-run/toaster-env/lib/python2.7/site-packages/south/migration/migrators.py", line 133, in migrate
result = self.run(migration, database)
File "/home/ell/toaster-run/toaster-env/lib/python2.7/site-packages/south/migration/migrators.py", line 114, in run
return self.run_migration(migration, database)
File "/home/ell/toaster-run/toaster-env/lib/python2.7/site-packages/south/migration/migrators.py", line 93, in run_migration
print(self.run_migration_error(migration))
File "/home/ell/toaster-run/toaster-env/lib/python2.7/site-packages/south/migration/migrators.py", line 323, in run_migration_error
(self.format_backwards(migration), extra_info))
File "/home/ell/toaster-run/toaster-env/lib/python2.7/site-packages/south/migration/migrators.py", line 310, in format_backwards
self.backwards(migration)()
File "/home/ell/toaster-run/toaster-env/lib/python2.7/site-packages/south/migration/migrators.py", line 60, in <lambda>
return (lambda: direction(orm))
File "/home/ell/dev/toaster/poky/bitbake/lib/toaster/orm/migrations/0021_auto__chg_field_build_project__chg_field_project_bitbake_version__chg_.py", line 33, in backwards
raise RuntimeError("Cannot reverse this migration. 'Project.bitbake_version' and its values cannot be restored.")
RuntimeError: Cannot reverse this migration. 'Project.bitbake_version' and its values cannot be restored.
Error on migration, trying to recover...
Running migrations for orm:
- Migrating backwards to just after 0001_initial.
< orm:0020_auto__add_field_layer_version_local_path__add_field_recipe_pathflags__
(faked)
< orm:0019_auto__add_buildartifact
(faked)
< orm:0018_auto__add_field_layer_version_project
(faked)
< orm:0017_auto__del_toastersettingdefaultlayer__add_releaselayersourcepriority__
(faked)
< orm:0016_auto__add_field_release_helptext__chg_field_release_branch__add_index_
(faked)
< orm:0015_auto__add_field_layer_vcs_web_url__add_field_layer_vcs_web_tree_base_u
(faked)
< orm:0014_auto__chg_field_package_summary__chg_field_layer_summary__chg_field_re
(faked)
< orm:0013_auto__add_release__add_layerversiondependency__add_unique_layerversion
(faked)
< orm:0012_auto__add_field_projectlayer_optional__add_field_projecttarget_task
(faked)
< orm:0011_auto__add_field_projectlayer_dirpath
(faked)
< orm:0010_auto__add_field_project_branch__add_field_project_short_description__a
(faked)
< orm:0009_auto__add_projectvariable__add_projectlayer__add_projecttarget__add_pr
(faked)
< orm:0008_auto__chg_field_variablehistory_operation__chg_field_recipe_descriptio
(faked)
< orm:0007_auto__add_helptext
(faked)
< orm:0006_auto__add_field_target_image_size__add_field_target_license_manifest_p
(faked)
< orm:0005_auto__add_target_image_file__add_target_file__add_field_variablehistor
(faked)
< orm:0004_auto__add_field_package_installed_name
(faked)
< orm:0003_timespent
(faked)
< orm:0002_auto__add_field_build_timespent
(faked)
Running migrations for orm:
- Migrating forwards to 0024_auto__add_field_recipe_is_image.
> orm:0002_auto__add_field_build_timespent
FATAL ERROR - The following SQL query failed: ALTER TABLE `orm_build` ADD COLUMN `timespent` integer NOT NULL DEFAULT 0;
The error was: (1060, "Duplicate column name 'timespent'")
! Error found during real run of migration! Aborting.
! Since you have a database that does not support running
! schema-altering statements in transactions, we have had
! to leave it in an interim state between migrations.
! You *might* be able to recover with: - no dry run output for delete_foreign_key() due to dynamic DDL, sorry
= ALTER TABLE `orm_build` DROP COLUMN `timespent` CASCADE; []
! The South developers regret this has happened, and would
! like to gently persuade you to consider a slightly
! easier-to-deal-with DBMS (one that supports DDL transactions)
! NOTE: The error which caused the migration to fail is further up.
Error in migration: orm:0002_auto__add_field_build_timespent
Traceback (most recent call last):
File "/home/ell/dev/toaster/poky/bitbake/bin/../lib/toaster/manage.py", line 10, in <module>
execute_from_command_line(sys.argv)
File "/home/ell/toaster-run/toaster-env/lib/python2.7/site-packages/django/core/management/__init__.py", line 399, in execute_from_command_line
utility.execute()
File "/home/ell/toaster-run/toaster-env/lib/python2.7/site-packages/django/core/management/__init__.py", line 392, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/home/ell/toaster-run/toaster-env/lib/python2.7/site-packages/django/core/management/base.py", line 242, in run_from_argv
self.execute(*args, **options.__dict__)
File "/home/ell/toaster-run/toaster-env/lib/python2.7/site-packages/django/core/management/base.py", line 285, in execute
output = self.handle(*args, **options)
File "/home/ell/toaster-run/toaster-env/lib/python2.7/site-packages/south/management/commands/migrate.py", line 111, in handle
ignore_ghosts = ignore_ghosts,
File "/home/ell/toaster-run/toaster-env/lib/python2.7/site-packages/south/migration/__init__.py", line 220, in migrate_app
success = migrator.migrate_many(target, workplan, database)
File "/home/ell/toaster-run/toaster-env/lib/python2.7/site-packages/south/migration/migrators.py", line 254, in migrate_many
result = migrator.__class__.migrate_many(migrator, target, migrations, database)
File "/home/ell/toaster-run/toaster-env/lib/python2.7/site-packages/south/migration/migrators.py", line 329, in migrate_many
result = self.migrate(migration, database)
File "/home/ell/toaster-run/toaster-env/lib/python2.7/site-packages/south/migration/migrators.py", line 133, in migrate
result = self.run(migration, database)
File "/home/ell/toaster-run/toaster-env/lib/python2.7/site-packages/south/migration/migrators.py", line 114, in run
return self.run_migration(migration, database)
File "/home/ell/toaster-run/toaster-env/lib/python2.7/site-packages/south/migration/migrators.py", line 84, in run_migration
migration_function()
File "/home/ell/toaster-run/toaster-env/lib/python2.7/site-packages/south/migration/migrators.py", line 60, in <lambda>
return (lambda: direction(orm))
File "/home/ell/dev/toaster/poky/bitbake/lib/toaster/orm/migrations/0002_auto__add_field_build_timespent.py", line 14, in forwards
keep_default=False)
File "/home/ell/toaster-run/toaster-env/lib/python2.7/site-packages/south/db/generic.py", line 47, in _cache_clear
return func(self, table, *args, **opts)
File "/home/ell/toaster-run/toaster-env/lib/python2.7/site-packages/south/db/generic.py", line 418, in add_column
self.execute(sql)
File "/home/ell/toaster-run/toaster-env/lib/python2.7/site-packages/south/db/generic.py", line 282, in execute
cursor.execute(sql, params)
File "/home/ell/toaster-run/toaster-env/lib/python2.7/site-packages/django/db/backends/util.py", line 69, in execute
return super(CursorDebugWrapper, self).execute(sql, params)
File "/home/ell/toaster-run/toaster-env/lib/python2.7/site-packages/django/db/backends/util.py", line 53, in execute
return self.cursor.execute(sql, params)
File "/home/ell/toaster-run/toaster-env/lib/python2.7/site-packages/django/db/utils.py", line 99, in __exit__
six.reraise(dj_exc_type, dj_exc_value, traceback)
File "/home/ell/toaster-run/toaster-env/lib/python2.7/site-packages/django/db/backends/util.py", line 53, in execute
return self.cursor.execute(sql, params)
File "/home/ell/toaster-run/toaster-env/lib/python2.7/site-packages/django/db/backends/mysql/base.py", line 124, in execute
return self.cursor.execute(query, args)
File "/home/ell/toaster-run/toaster-env/lib/python2.7/site-packages/MySQLdb/cursors.py", line 205, in execute
self.errorhandler(self, exc, value)
File "/home/ell/toaster-run/toaster-env/lib/python2.7/site-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler
raise errorclass, errorvalue
django.db.utils.OperationalError: (1060, "Duplicate column name 'timespent'")
```
This is a blocker for me, as I need it fixed to be able to run Toaster (I can't get it to work with SQLite, as described in bug 8147). Which branch of poky should be referenced in the docs change? I think this is just an issue with MySQL performing a dry run of the SQL statements, which throws an error with some of the migrations as they can't be run backwards. (When using SQLite, migrations aren't dry run: see http://south.aeracode.org/wiki/Tutorial3.) Created attachment 2686 [details]
Design - Showing the default project storing the Knotty builds
Do we need to consider other mechanisms for triggering builds apart from the command line? Just wondering whether they should be called "Command line builds" or something like "External builds" instead. (I don't think "External builds" is correct either, but there could be a way to trigger a bitbake build, but not from the command line.) (In reply to comment #7) > Do we need to consider other mechanisms for triggering builds apart from the > command line? Just wondering whether they should be called "Command line > builds" or something like "External builds" instead. (I don't think > "External builds" is correct either, but there could be a way to trigger a > bitbake build, but not from the command line.) The only case I can think of is something like the Autobuilder (CI sending information about its builds to a Toaster instance). An easy workaround would be to allow people to change the name of the project. 'Command line builds' reflects what we believe is the most likely use case for this project. If it doesn't fit your use case, change the project name! :) Would that work? Yes, I think that would be fine. Should the UI therefore retain the option to edit the "default project"'s name? Note that I've mostly implemented a better default project. I have tried adding the default project to the db during initial setup by migration, rather than at runtime when the first command line build appears. However, this has the effect of putting a project into the database, which means the landing page is skipped (because there's a project). Should I keep the old behaviour for now (where the default project is created at runtime), even though it feels incorrect? Alternatively, I could modify the landing page (to only redirect if there is a project other than the default one) and modify the projects page to only show projects with a build count > 0. (In reply to comment #9) > Yes, I think that would be fine. Should the UI therefore retain the option > to edit the "default project"'s name? yes, please! > > Note that I've mostly implemented a better default project. > > I have tried adding the default project to the db during initial setup by > migration, rather than at runtime when the first command line build appears. > However, this has the effect of putting a project into the database, which > means the landing page is skipped (because there's a project). > > Should I keep the old behaviour for now (where the default project is > created at runtime), even though it feels incorrect? (In reply to comment #10) > Alternatively, I could modify the landing page (to only redirect if there is > a project other than the default one) This sounds better to me, particularly if creating the project at runtime is the wrong thing to do. and modify the projects page to only > show projects with a build count > 0. This will be strange for normal projects. It means that if I've just created a project, and then go to the all projects page, the project I just created won't be listed there, which will make me think that something has gone wrong. What's worse: there would be no way for me to access the project, since the only entry point for a project with no builds is the list of all projects. I'm afraid we'll need to do something like "if this is the default project and the number of builds in it is 0, do now show it". Created attachment 2693 [details]
Design - Showing the default project storing the Knotty builds
Adding some additional help text based on Tiago's feedback (pages 5 and 8)
Belen, could we move the new design for the command line builds out of this bug and into a fresh one? It would be good if we could just get the "default project" working close to how it works with SQLite right now, and think about a new design for that part of the application in future. GUI work is now covered by bug 8231 To test, set up Toaster with a MySQL database, as per instructions in http://www.yoctoproject.org/docs/1.8/toaster-manual/toaster-manual.html#setting-up-a-hosted-service-and-running-in-analysis-mode. Verified while installing MySQL instance for GDC. |