This includes the code files in the docker image and creates a separate
data dir, which means we don't need to manually mount files into the
container anymore.
Removing the transaction will allow other (async) operations to succeed
while the import is running, otherwise the database will be locked.
It also actually speeds up the import process, and there's no good
reason to roll back if the import aborts mid run.
The two routes offer reloading the whole movie database from IMDb, and
reloading all users' IMDb ratings.
These functions (or very similar) were already available using the CLI,
but now they can be triggered remotely by an admin.
Any field marked as Relation is ignored by all model operations (like
converting to and from plain form). Fields marked as Relation are
meant to store the actual model data for foreign keys stored on the
model.
This includes the patches to get from the initial version of the
database to its current state - for the purpose of documentation.
Beware, if you already have an existing database, running this code
will likely result in an error and/or lead to some minor data loss.
You need to follow these steps to fix your DB:
1. delete all *.sql files from unwind/sql/
2. remove the .disabled suffix from unwind/sql/00000001-fix-db.sql.disabled
3. start the application once, this will fix up the database
4. revert all changes to unwind/sql/ – or leave it, it doesn't matter.
The vast majority of time was spent loading each movie individually.
We always need to load every movie to check if it needs updating, or
if it maybe doesn't exist yet. We can drastically increase performance
by loading the movies in large batches.
- Postpone setting an ID for a Movie until it's added to the database.
- Use larger chunks to count the lines of an input file.
- Skip creating temporary objects for the score-map.
A `RatingRow`'s `id` and `updated` are never used, but creating them
a million+ times is quite expensive, so initializing them with `None`
saves a lot of time.
`dataclasses`' `fields` function is also quite expensive; loading the
fields from a row directly saves a lot of CPU cycles.
We want to limit the number of movies, not ratings. By limiting the
number of movie IDs before loading all ratings with those movie IDs
we can achieve this.
