Django/migrate時の処理を読む（流れ）
をテンプレートにして作成
[
トップ
] [
新規
|
一覧
|
単語検索
|
最終更新
|
ヘルプ
]
開始行:
[[Djangoを読む]]
#contents
*はじめに [#y2679735]
やっとチュートリアルが[[その2>https://docs.djangoproject....
さて、チュートリアルその2の話題はモデルです。モデルの定義...
まずは、チュートリアルで初めに
$ python manage.py migrate
と打てと書いてあるのでmigrateコマンドの中身を見てDjangoが...
ちなみに、打ってみると以下のように出力されます。
Operations to perform:
Apply all migrations: admin, auth, contenttypes, sessi...
Running migrations:
Applying contenttypes.0001_initial... OK
Applying auth.0001_initial... OK
Applying admin.0001_initial... OK
Applying admin.0002_logentry_remove_auto_add... OK
Applying contenttypes.0002_remove_content_type_name......
Applying auth.0002_alter_permission_name_max_length......
Applying auth.0003_alter_user_email_max_length... OK
Applying auth.0004_alter_user_username_opts... OK
Applying auth.0005_alter_user_last_login_null... OK
Applying auth.0006_require_contenttypes_0002... OK
Applying auth.0007_alter_validators_add_error_messages...
Applying auth.0008_alter_user_username_max_length... OK
Applying sessions.0001_initial... OK
*django/core/management/commands/migrate.py [#t6a51bd4]
スタート地点はいつものようにCommandクラスが書かれているコ...
migrateコマンドのhandleは結構長いです。ざっと見た感じでは...
+データベースへの接続
+実行するマイグレーションの決定、順序付け
+マイグレーションの実行
ひとつずつ見ていきましょう。
*django/db [#y4ba13bd]
データベースの接続を行っていると思われる個所は以下のとこ...
#code(Python){{
# Get the database we're operating from
db = options['database']
connection = connections[db]
# Hook for backends needing any database preparat...
connection.prepare_database()
}}
connectionsとローカル変数の辞書のようにしれっと書いてあり...
#code(Python){{
from django.db import DEFAULT_DB_ALIAS, connections, rout...
}}
と、django.dbモジュールの属性です。
というわけで、視点をdjango/db/__init__.pyに向けると、
#code(Python){{
from django.db.utils import (
DEFAULT_DB_ALIAS, DJANGO_VERSION_PICKLE_KEY, Connecti...
ConnectionRouter, DatabaseError, DataError, Error, In...
InterfaceError, InternalError, NotSupportedError, Ope...
ProgrammingError,
)
connections = ConnectionHandler()
}}
utils.pyに移ってConnectionHandlerクラスの__getitem__メソ...
#code(Python){{
def __getitem__(self, alias):
if hasattr(self._connections, alias):
return getattr(self._connections, alias)
self.ensure_defaults(alias)
self.prepare_test_settings(alias)
db = self.databases[alias]
backend = load_backend(db['ENGINE'])
conn = backend.DatabaseWrapper(db, alias)
setattr(self._connections, alias, conn)
return conn
}}
load_backend関数を見るとENGINEとして設定したもののbase、...
なお、self.databasesはプロパティでsettings.pyのDATABASES...
で、prepare_databaseで接続を行っているのかと思ったらして...
*django/db/migrations/loader.py [#q3d4566a]
次は、実行するマイグレーションの決定、です。関係ありそう...
#code(Python){{
# Work out which apps have migrations and which d...
executor = MigrationExecutor(connection, self.mig...
# エラー処理と思われるもの省略
# If they supplied command line arguments, work o...
target_app_labels_only = True
if options['app_label'] and options['migration_na...
# 省略
elif options['app_label']:
# 省略
else:
targets = executor.loader.graph.leaf_nodes()
plan = executor.migration_plan(targets)
}}
キーになるのはMigrationExecutorのようです。上にあるimport...
また、executorの属性としてloaderとありますが、これはexecu...
#code(Python){{
class MigrationLoader(object):
"""
Loads migration files from disk, and their status fro...
Migration files are expected to live in the "migratio...
an app. Their names are entirely unimportant from a c...
but will probably follow the 1234_name.py convention.
On initialization, this class will scan those directo...
read the python files, looking for a class called Mig...
inherit from django.db.migrations.Migration. See
django.db.migrations.migration for what that looks li...
以下略
"""
def __init__(self, connection, load=True, ignore_no_m...
self.connection = connection
self.disk_migrations = None
self.applied_migrations = None
self.ignore_no_migrations = ignore_no_migrations
if load:
self.build_graph()
}}
クラスコメントを読むとこのクラスが何をしているのかがわか...
build_graphメソッドの先頭。
#code(Python){{
def build_graph(self):
"""
Builds a migration dependency graph using both th...
You'll need to rebuild the graph if you apply mig...
usually a problem as generally migration stuff ru...
"""
# Load disk data
self.load_disk()
# Load database data
if self.connection is None:
self.applied_migrations = set()
else:
recorder = MigrationRecorder(self.connection)
self.applied_migrations = recorder.applied_mi...
}}
load_diskメソッドでは先ほどクラスコメントにあったように、...
load_diskメソッドの実行が終わると、
{(アプリ名, マイグレーション名): Migrationインスタンス}
というような辞書オブジェクトdisk_migrationsが構築されます。
load_diskメソッドから返ってくると次はMigrationRecorderク...
さて、これでアプリのマイグレーション情報および、それがど...
#code(Python){{
# To start, populate the migration graph with nod...
# and their dependencies. Also make note of repla...
self.graph = MigrationGraph()
self.replacements = {}
for key, migration in self.disk_migrations.items():
self.graph.add_node(key, migration)
# Internal (aka same-app) dependencies.
self.add_internal_dependencies(key, migration)
# Replacing migrations.
if migration.replaces:
self.replacements[key] = migration
# Add external dependencies now that the internal...
for key, migration in self.disk_migrations.items():
self.add_external_dependencies(key, migration)
# Carry out replacements where possible.
for key, migration in self.replacements.items():
# 省略
}}
各Migrationにはdependenciesで依存（自分よりも前に実行して...
#code(Python){{
class Migration(migrations.Migration):
dependencies = [
('auth', '0001_initial'),
]
}}
グラフの構築ステップとしては、
+各マイグレーションをノードとして追加
+インターナル（自アプリ内）の依存を追加。これは通常、0008...
+エクスターナル（別アプリ）の依存を追加。これを別のループ...
この後、構築したグラフに誤りがないかのバリデーションがさ...
migrate.pyに戻って、今見てるところ再掲、
#code(Python){{
# Work out which apps have migrations and which d...
executor = MigrationExecutor(connection, self.mig...
# エラー処理と思われるもの省略
# If they supplied command line arguments, work o...
target_app_labels_only = True
if options['app_label'] and options['migration_na...
# 省略
elif options['app_label']:
# 省略
else:
targets = executor.loader.graph.leaf_nodes()
plan = executor.migration_plan(targets)
}}
graph.leaf_nodesはなんとなく想像がつくので省略します。
で、migration_planの方。これも全部載せていると長くなるの...
#code(Python){{
def migration_plan(self, targets, clean_start=False):
"""
Given a set of targets, returns a list of (Migrat...
"""
plan = []
if clean_start:
applied = set()
else:
applied = set(self.loader.applied_migrations)
for target in targets:
# If the target is (app_label, None), that me...
if target[1] is None:
# 省略
# If the migration is already applied, do bac...
# otherwise do forwards mode.
elif target in applied:
# 省略
else:
for migration in self.loader.graph.forwar...
if migration not in applied:
plan.append((self.loader.graph.no...
applied.add(migration)
return plan
}}
forwards_planの中まで追いかけるのはやめますが、コメントな...
結局、planには
(Migrationインスタンス, False)
というタプルのリストが格納されることになります。Falseとい...
*django/db/migrations/executor.py [#z06901ae]
実行するマイグレーションとその順序まで決定できたので残り...
migrate.pyに戻ってhandleメソッドの続きを見ると、重要そう...
#code(Python){{
pre_migrate_state = executor._create_project_stat...
# 省略
post_migrate_state = executor.migrate(
targets, plan=plan, state=pre_migrate_state.c...
fake_initial=fake_initial,
)
}}
executorのmigrateメソッド。いろいろ分岐していますが結局、...
#code(Python){{
def migrate(self, targets, plan=None, state=None, fak...
"""
Migrates the database up to the given targets.
Django first needs to create all project states b...
(un)applied and in a second step run all the data...
"""
if plan is None:
plan = self.migration_plan(targets)
# Create the forwards plan Django would follow on...
full_plan = self.migration_plan(self.loader.graph...
all_forwards = all(not backwards for mig, backwar...
all_backwards = all(backwards for mig, backwards ...
if not plan:
# 省略
elif all_forwards == all_backwards:
# 省略
elif all_forwards:
if state is None:
# The resulting state should still includ...
state = self._create_project_state(with_a...
state = self._migrate_all_forwards(state, pla...
else:
# No need to check for `elif all_backwards` h...
# would always evaluate to true.
state = self._migrate_all_backwards(plan, ful...
self.check_replacements()
return state
}}
_migrate_all_forwardsメソッド。
#code(Python){{
def _migrate_all_forwards(self, state, plan, full_pla...
"""
Take a list of 2-tuples of the form (migration in...
apply them in the order they occur in the full_pl...
"""
migrations_to_run = {m[0] for m in plan}
for migration, _ in full_plan:
if not migrations_to_run:
# We remove every migration that we appli...
# that we can bail out once the last migr...
# and don't always run until the very end...
# process.
break
if migration in migrations_to_run:
if 'apps' not in state.__dict__:
if self.progress_callback:
self.progress_callback("render_st...
state.apps # Render all -- performan...
if self.progress_callback:
self.progress_callback("render_su...
state = self.apply_migration(state, migra...
migrations_to_run.remove(migration)
return state
}}
各マイグレーションについてapply_migrationメソッドが実行さ...
#code(Python){{
def apply_migration(self, state, migration, fake=Fals...
"""
Runs a migration forwards.
"""
if self.progress_callback:
self.progress_callback("apply_start", migrati...
if not fake:
if fake_initial:
# Test to see if this is an already-appli...
applied, state = self.detect_soft_applied...
if applied:
fake = True
if not fake:
# Alright, do it normally
with self.connection.schema_editor(atomic...
state = migration.apply(state, schema...
# For replacement migrations, record individual s...
if migration.replaces:
for app_label, name in migration.replaces:
self.recorder.record_applied(app_label, n...
else:
self.recorder.record_applied(migration.app_la...
# Report progress
if self.progress_callback:
self.progress_callback("apply_success", migra...
return state
}}
マイグレーションの肝はここ。
#code(Python){{
with self.connection.schema_editor(atomic=migration.atomi...
state = migration.apply(state, schema_editor)
}}
ですが、今回はmigrateコマンドで各マイグレーションが実行さ...
*おわりに [#i2342cc5]
今回はマイグレーションの流れについて見てきました。各アプ...
そういえばデータベースへの接続、recorderはデータベースに...
終了行:
[[Djangoを読む]]
#contents
*はじめに [#y2679735]
やっとチュートリアルが[[その2>https://docs.djangoproject....
さて、チュートリアルその2の話題はモデルです。モデルの定義...
まずは、チュートリアルで初めに
$ python manage.py migrate
と打てと書いてあるのでmigrateコマンドの中身を見てDjangoが...
ちなみに、打ってみると以下のように出力されます。
Operations to perform:
Apply all migrations: admin, auth, contenttypes, sessi...
Running migrations:
Applying contenttypes.0001_initial... OK
Applying auth.0001_initial... OK
Applying admin.0001_initial... OK
Applying admin.0002_logentry_remove_auto_add... OK
Applying contenttypes.0002_remove_content_type_name......
Applying auth.0002_alter_permission_name_max_length......
Applying auth.0003_alter_user_email_max_length... OK
Applying auth.0004_alter_user_username_opts... OK
Applying auth.0005_alter_user_last_login_null... OK
Applying auth.0006_require_contenttypes_0002... OK
Applying auth.0007_alter_validators_add_error_messages...
Applying auth.0008_alter_user_username_max_length... OK
Applying sessions.0001_initial... OK
*django/core/management/commands/migrate.py [#t6a51bd4]
スタート地点はいつものようにCommandクラスが書かれているコ...
migrateコマンドのhandleは結構長いです。ざっと見た感じでは...
+データベースへの接続
+実行するマイグレーションの決定、順序付け
+マイグレーションの実行
ひとつずつ見ていきましょう。
*django/db [#y4ba13bd]
データベースの接続を行っていると思われる個所は以下のとこ...
#code(Python){{
# Get the database we're operating from
db = options['database']
connection = connections[db]
# Hook for backends needing any database preparat...
connection.prepare_database()
}}
connectionsとローカル変数の辞書のようにしれっと書いてあり...
#code(Python){{
from django.db import DEFAULT_DB_ALIAS, connections, rout...
}}
と、django.dbモジュールの属性です。
というわけで、視点をdjango/db/__init__.pyに向けると、
#code(Python){{
from django.db.utils import (
DEFAULT_DB_ALIAS, DJANGO_VERSION_PICKLE_KEY, Connecti...
ConnectionRouter, DatabaseError, DataError, Error, In...
InterfaceError, InternalError, NotSupportedError, Ope...
ProgrammingError,
)
connections = ConnectionHandler()
}}
utils.pyに移ってConnectionHandlerクラスの__getitem__メソ...
#code(Python){{
def __getitem__(self, alias):
if hasattr(self._connections, alias):
return getattr(self._connections, alias)
self.ensure_defaults(alias)
self.prepare_test_settings(alias)
db = self.databases[alias]
backend = load_backend(db['ENGINE'])
conn = backend.DatabaseWrapper(db, alias)
setattr(self._connections, alias, conn)
return conn
}}
load_backend関数を見るとENGINEとして設定したもののbase、...
なお、self.databasesはプロパティでsettings.pyのDATABASES...
で、prepare_databaseで接続を行っているのかと思ったらして...
*django/db/migrations/loader.py [#q3d4566a]
次は、実行するマイグレーションの決定、です。関係ありそう...
#code(Python){{
# Work out which apps have migrations and which d...
executor = MigrationExecutor(connection, self.mig...
# エラー処理と思われるもの省略
# If they supplied command line arguments, work o...
target_app_labels_only = True
if options['app_label'] and options['migration_na...
# 省略
elif options['app_label']:
# 省略
else:
targets = executor.loader.graph.leaf_nodes()
plan = executor.migration_plan(targets)
}}
キーになるのはMigrationExecutorのようです。上にあるimport...
また、executorの属性としてloaderとありますが、これはexecu...
#code(Python){{
class MigrationLoader(object):
"""
Loads migration files from disk, and their status fro...
Migration files are expected to live in the "migratio...
an app. Their names are entirely unimportant from a c...
but will probably follow the 1234_name.py convention.
On initialization, this class will scan those directo...
read the python files, looking for a class called Mig...
inherit from django.db.migrations.Migration. See
django.db.migrations.migration for what that looks li...
以下略
"""
def __init__(self, connection, load=True, ignore_no_m...
self.connection = connection
self.disk_migrations = None
self.applied_migrations = None
self.ignore_no_migrations = ignore_no_migrations
if load:
self.build_graph()
}}
クラスコメントを読むとこのクラスが何をしているのかがわか...
build_graphメソッドの先頭。
#code(Python){{
def build_graph(self):
"""
Builds a migration dependency graph using both th...
You'll need to rebuild the graph if you apply mig...
usually a problem as generally migration stuff ru...
"""
# Load disk data
self.load_disk()
# Load database data
if self.connection is None:
self.applied_migrations = set()
else:
recorder = MigrationRecorder(self.connection)
self.applied_migrations = recorder.applied_mi...
}}
load_diskメソッドでは先ほどクラスコメントにあったように、...
load_diskメソッドの実行が終わると、
{(アプリ名, マイグレーション名): Migrationインスタンス}
というような辞書オブジェクトdisk_migrationsが構築されます。
load_diskメソッドから返ってくると次はMigrationRecorderク...
さて、これでアプリのマイグレーション情報および、それがど...
#code(Python){{
# To start, populate the migration graph with nod...
# and their dependencies. Also make note of repla...
self.graph = MigrationGraph()
self.replacements = {}
for key, migration in self.disk_migrations.items():
self.graph.add_node(key, migration)
# Internal (aka same-app) dependencies.
self.add_internal_dependencies(key, migration)
# Replacing migrations.
if migration.replaces:
self.replacements[key] = migration
# Add external dependencies now that the internal...
for key, migration in self.disk_migrations.items():
self.add_external_dependencies(key, migration)
# Carry out replacements where possible.
for key, migration in self.replacements.items():
# 省略
}}
各Migrationにはdependenciesで依存（自分よりも前に実行して...
#code(Python){{
class Migration(migrations.Migration):
dependencies = [
('auth', '0001_initial'),
]
}}
グラフの構築ステップとしては、
+各マイグレーションをノードとして追加
+インターナル（自アプリ内）の依存を追加。これは通常、0008...
+エクスターナル（別アプリ）の依存を追加。これを別のループ...
この後、構築したグラフに誤りがないかのバリデーションがさ...
migrate.pyに戻って、今見てるところ再掲、
#code(Python){{
# Work out which apps have migrations and which d...
executor = MigrationExecutor(connection, self.mig...
# エラー処理と思われるもの省略
# If they supplied command line arguments, work o...
target_app_labels_only = True
if options['app_label'] and options['migration_na...
# 省略
elif options['app_label']:
# 省略
else:
targets = executor.loader.graph.leaf_nodes()
plan = executor.migration_plan(targets)
}}
graph.leaf_nodesはなんとなく想像がつくので省略します。
で、migration_planの方。これも全部載せていると長くなるの...
#code(Python){{
def migration_plan(self, targets, clean_start=False):
"""
Given a set of targets, returns a list of (Migrat...
"""
plan = []
if clean_start:
applied = set()
else:
applied = set(self.loader.applied_migrations)
for target in targets:
# If the target is (app_label, None), that me...
if target[1] is None:
# 省略
# If the migration is already applied, do bac...
# otherwise do forwards mode.
elif target in applied:
# 省略
else:
for migration in self.loader.graph.forwar...
if migration not in applied:
plan.append((self.loader.graph.no...
applied.add(migration)
return plan
}}
forwards_planの中まで追いかけるのはやめますが、コメントな...
結局、planには
(Migrationインスタンス, False)
というタプルのリストが格納されることになります。Falseとい...
*django/db/migrations/executor.py [#z06901ae]
実行するマイグレーションとその順序まで決定できたので残り...
migrate.pyに戻ってhandleメソッドの続きを見ると、重要そう...
#code(Python){{
pre_migrate_state = executor._create_project_stat...
# 省略
post_migrate_state = executor.migrate(
targets, plan=plan, state=pre_migrate_state.c...
fake_initial=fake_initial,
)
}}
executorのmigrateメソッド。いろいろ分岐していますが結局、...
#code(Python){{
def migrate(self, targets, plan=None, state=None, fak...
"""
Migrates the database up to the given targets.
Django first needs to create all project states b...
(un)applied and in a second step run all the data...
"""
if plan is None:
plan = self.migration_plan(targets)
# Create the forwards plan Django would follow on...
full_plan = self.migration_plan(self.loader.graph...
all_forwards = all(not backwards for mig, backwar...
all_backwards = all(backwards for mig, backwards ...
if not plan:
# 省略
elif all_forwards == all_backwards:
# 省略
elif all_forwards:
if state is None:
# The resulting state should still includ...
state = self._create_project_state(with_a...
state = self._migrate_all_forwards(state, pla...
else:
# No need to check for `elif all_backwards` h...
# would always evaluate to true.
state = self._migrate_all_backwards(plan, ful...
self.check_replacements()
return state
}}
_migrate_all_forwardsメソッド。
#code(Python){{
def _migrate_all_forwards(self, state, plan, full_pla...
"""
Take a list of 2-tuples of the form (migration in...
apply them in the order they occur in the full_pl...
"""
migrations_to_run = {m[0] for m in plan}
for migration, _ in full_plan:
if not migrations_to_run:
# We remove every migration that we appli...
# that we can bail out once the last migr...
# and don't always run until the very end...
# process.
break
if migration in migrations_to_run:
if 'apps' not in state.__dict__:
if self.progress_callback:
self.progress_callback("render_st...
state.apps # Render all -- performan...
if self.progress_callback:
self.progress_callback("render_su...
state = self.apply_migration(state, migra...
migrations_to_run.remove(migration)
return state
}}
各マイグレーションについてapply_migrationメソッドが実行さ...
#code(Python){{
def apply_migration(self, state, migration, fake=Fals...
"""
Runs a migration forwards.
"""
if self.progress_callback:
self.progress_callback("apply_start", migrati...
if not fake:
if fake_initial:
# Test to see if this is an already-appli...
applied, state = self.detect_soft_applied...
if applied:
fake = True
if not fake:
# Alright, do it normally
with self.connection.schema_editor(atomic...
state = migration.apply(state, schema...
# For replacement migrations, record individual s...
if migration.replaces:
for app_label, name in migration.replaces:
self.recorder.record_applied(app_label, n...
else:
self.recorder.record_applied(migration.app_la...
# Report progress
if self.progress_callback:
self.progress_callback("apply_success", migra...
return state
}}
マイグレーションの肝はここ。
#code(Python){{
with self.connection.schema_editor(atomic=migration.atomi...
state = migration.apply(state, schema_editor)
}}
ですが、今回はmigrateコマンドで各マイグレーションが実行さ...
*おわりに [#i2342cc5]
今回はマイグレーションの流れについて見てきました。各アプ...
そういえばデータベースへの接続、recorderはデータベースに...
ページ名:
![[PukiWiki]](image/pukiwiki.png "[PukiWiki]")
