Archives
Contributors
Re: Removal of migration scripts on each new version
Re: Removal of migration scripts on each new version
Re: Removal of migration scripts on each new version
Although you can think at the beginning that there's no problem keeping the previous migration scripts on each version, there are tons of reasons for not doing it. First of all, it creates on people seeing that migration scripts the false sensation that they don't have to look in any other place for migrating between several versions except on the latest one, while there can happen a lot of things:
- At the moment you are rescuing the commit history to make the migration, the migration scripts for the previous version may still not be developed/merged, and they are incorporated later, so the latest version won't have these migration scripts while they are needed. And you can't demand "openupgraders" to do forward-ports of the migration scripts to each upper version.
- The initial migration scripts may be incorrect or incomplete, and there can be several patches later after upper versions were being fetched and migrated. This is even worse IMO, as you think you have the right migration scripts, while they are not the good ones. And again you can't demand the contributors to duplicate the work propagating the patch on the migration scripts (or well, being dreamer thinking that people are going to do it if you state that in the guidelines).
That's why the saner option is to purge these migration scripts from upper versions to only have one source of truth, and this is independent of the type of migration scripts. Stefan states "if they are safe for inclusion
in the later version", but that's something very difficult to be judged, specially for the migrators/reviewers which are not used to these migration scripts, so another reason for not having grey areas: you follow the same rule of not having them, as in the end, you have to check the same if they are outdated or missing, etc, etc.
If you do enterprise migrations jumping several versions, you should develop a tool to gather all the migration scripts from the intermediate versions and collect them together. Such tools may be shared and developed collaboratively for those interested. Anyway, take into account that there are a lot of reasons for these migration scripts (being gathered by the tool or being preserved through versions) to fail:
- They expect a specific data model state, based on the version. For example, you may have a migration script that is expecting a column that is declared in version 15, but later in 16, it disappears. If you migrate from 14 to 16, and you run that 15.0 script after having the enterprise migration done up to 16, and with the 16.0 OCA module, it will fail.
- They are also based on the stage of the update process (you have pre, post and end migration scripts). If you execute for example a pre-migration script renaming/creating a column in that phase, and it's already created due to regular update, it will crash. Some said in the past to put protections ("if" sentences checking if the column exists, and so on), but apart from over-complicating the code of migration scripts (and again, you can't demand openupgraders to do or review that job - and imagine if you have to spread it across all versions... -), it doesn't guarantee that the data will be consistent, as there can be a lot of combinations, as much as the source and target version combinations.
- Enterprise migration scripts "clean" the obsolete columns, tables, etc, making sometimes impossible to rebuild your connections for extra data, links, etc. OpenUpgrade/OCA migration scripts count with the obsolete columns/tables are still there for doing SQLs and other actions, so another possible reason to fail.
- A variant of the previous point: some scripts are developed counting with some specific columns created on the fly or renamed for doing the actions. Example: on the 13.0 migration, we created the columns old_invoice_id in account.move and old_invoice_line_id in account.move.line to point to the old account.invoice and account.invoice.line records, and most of the OCA modules adding fields on the invoice, require these fields for transferring data from one model to the other: https://github.com/OCA/bank-payment/blob/340e44234a165769115bc954120bfcde5dd7b74e/account_payment_partner/migrations/13.0.1.0.0/post-migration.py#L15
So, in summary, IMO, Odoo enterprise migration service should only be used if you don't have any extra modules in your DB. Once you are using OCA modules, go OpenUpgrade way, or you will spend more time reviewing all of these corner cases than doing it with OpenUpgrade. And if you are not happy with the availability date for OpenUpgrade new versions, help funding its development.
Regards.
by Pedro M. Baeza - 04:21 - 6 May 2025
Reference
-
Removal of migration scripts on each new version
Hi,
the migration guide mandates the following
> Remove any possible migration script from previous version (in a nutshell, removemigrationsfolder inside the module if exists).
(https://github.com/OCA/maintainer-tools/wiki/Migration-to-version-18.0#tasks-to-do-in-the-migration)
However, it is not uncommon to skip versions when migrating an Odoo instance. You would go from 15.0 or 16.0 to 18.0 rather than migrating every year. When using the Odoo enterprise migration, the migration scripts between the source and the target version are supposed to be present in the target version. So the migration guideline breaks this type of migration.
I had a disagreement with Pedro Baeza about this on one PR, but I keep coming across instances of this such as https://github.com/OCA/account-invoicing/pull/1874 today so I would like to discuss this in a wider audience.
My preference would be for the guideline to change to say that it is allowed to keep some of the scripts if they are safe for inclusion in the later version (such as the script from https://github.com/OCA/account-invoicing/pull/1874, which checks if a field already exists before trying to add it).
Can I have a temperature check from the community to see how you all feel about this?
Best regards,
Stefan
-- Opener B.V. - Business solutions driven by open source collaboration Stefan Rijnhart - Consultant/developer mail: stefan@opener.amsterdam tel: +31 (0) 6 1447 8606 web: https://opener.amsterdam
by Stefan Rijnhart - 01:00 - 6 May 2025