Skip to Content
  2. Mailing Lists
  3. Contributors
  4. requirements.txt: Repository Level vs Module Level

Archives

Contributors

Any module that verify numbers, i.e., employee's citizen ID

Supplier discount on all products supplied

requirements.txt: Repository Level vs Module Level

Hi all,

As i see, usually in OCA repositories "requirements.txt" file with pip dependencies is located in the root of the repository and contains a list of all python dependencies for all addons in the repository. This way it works fine, when we clone full repository, and install all dependencies for all modules there, thus when user will try to install new module on DB, 99% that all python dependencies will be satisfied.

But, let's take for example repository partner-contact repository for Odoo 12.0 and let's try to add the module 'partner_email_check' to odoo server. For this task, i will use [odoo-helper-scripts](https://github.com/katyukha/odoo-helper-scripts) that can automatically resolve repository dependencies (including those specified by oca_dependencies.txt). So, at first i would try to fetch (clone) repository partner-contact, and in this case system will automatically fetch 42 OCA repositories following 'oca_requirements.txt', and also it will try to install python dependencies mentioned in requirements.txt. It is good for development. But installation this way on prod, may lead to a lot of unneeded modules, that polutes system with strange dependencies that are in some cases may be not installable (for example because of system dependencies).

To solve this reason, we started to use 'assembly' approach, that assumes that we have to create separate git repository with only addons needed on server. But in this case, if assembly repo created automatically, there is no way to get python requirements for module, if it is not specified in module directory. Looking for python dependencies in manifest is also not good, because there are python packages exists, that has different name for package and python module inside package.

So, may be it have sense to place requirements.txt inside module directory? Thus module's requirements will be always delivered with module, that will make easier installation of module. Also, i think this way, it will be much easier to generate setup.py files for modules, that will contain info about module's python dependencies.

Possible drawbacks may be in case, when different versions of python dep will be specified in different modules. But same is applicable for repositories.

What do you think about this?

With regards,
Dmytro Katyukha
by dmytro.katyukha - 10:56 - 11 Feb 2021

Follow-Ups

  • Re: requirements.txt: Repository Level vs Module Level
    I think that the number of extra external dependencies is so low (or null, as most of the modules only require Odoo Framework), that managing them manually in an extra requirements.txt per project according needed modules is affordable instead of entering in the IMO hell pip method. That's what we do in Tecnativa and Doodba scaffolding has such option in /odoo/custom/dependencies/pip.txt. Other option in v13+ is to scrap the external dependencies from manifests for getting the proper pip packages.

    Regards.
    by Pedro M. Baeza - 11:40 - 11 Feb 2021
  • Re: requirements.txt: Repository Level vs Module Level
    Hello Dmytro,

    I feel that the best solution is to adopt pip-installed modules to your workflow.
    That may need a learning curve, but it already solves all problems you describe.
    I'm not the best person to get into the detail, but I'm sure there on people on this ML that can fill in.

    If you prefer to go the Git way, my opinion is that you should to deploy OCA modules from their Git repos.
    What I do is to have each OCA repo I need as a submodule, and then have a "link-addons" with symlinks for the modules used.
    Only "link-addons" is added to the server addons path.
    The requirements.txt is maintained manually - when a new "link-addons" symlink is added, is it has specific dependencies, the requirements.txt is also updated.

    For requirements.txt inside modules, this is not an issue in recent Odoo versions, and external_Dependencies now supports the package name.
    See https://github.com/odoo/odoo/issues/25541


    I hope this is helpful
    Daniel

    On 11/02/2021 09:56, Dmytro Katyukha wrote:
    Hi all,

    As i see, usually in OCA repositories "requirements.txt" file with pip dependencies is located in the root of the repository and contains a list of all python dependencies for all addons in the repository. This way it works fine, when we clone full repository, and install all dependencies for all modules there, thus when user will try to install new module on DB, 99% that all python dependencies will be satisfied.

    But, let's take for example repository partner-contact repository for Odoo 12.0 and let's try to add the module 'partner_email_check' to odoo server. For this task, i will use [odoo-helper-scripts](https://github.com/katyukha/odoo-helper-scripts) that can automatically resolve repository dependencies (including those specified by oca_dependencies.txt). So, at first i would try to fetch (clone) repository partner-contact, and in this case system will automatically fetch 42 OCA repositories following 'oca_requirements.txt', and also it will try to install python dependencies mentioned in requirements.txt. It is good for development. But installation this way on prod, may lead to a lot of unneeded modules, that polutes system with strange dependencies that are in some cases may be not installable (for example because of system dependencies).

    To solve this reason, we started to use 'assembly' approach, that assumes that we have to create separate git repository with only addons needed on server. But in this case, if assembly repo created automatically, there is no way to get python requirements for module, if it is not specified in module directory. Looking for python dependencies in manifest is also not good, because there are python packages exists, that has different name for package and python module inside package.

    So, may be it have sense to place requirements.txt inside module directory? Thus module's requirements will be always delivered with module, that will make easier installation of module. Also, i think this way, it will be much easier to generate setup.py files for modules, that will contain info about module's python dependencies.

    Possible drawbacks may be in case, when different versions of python dep will be specified in different modules. But same is applicable for repositories.

    What do you think about this?

    With regards,
    Dmytro Katyukha

    _______________________________________________
    Mailing-List: https://odoo-community.org/groups/contributors-15
    Post to: mailto:contributors@odoo-community.org
    Unsubscribe: https://odoo-community.org/groups?unsubscribe


    --
    Daniel Reis
    Managing Director
    M: +351 919991307
    E: DReis@OpenSourceIntegrators.com
    Av Doutor Desidério Cambournac 12 • 2710-553 Sintra, Portugal


    by Daniel Reis - 11:21 - 11 Feb 2021
  • Re: requirements.txt: Repository Level vs Module Level
    Hi Dmytro,

    You can achieve fine grained (per module) installation using pip.

    For OCA modules and your example: "pip install odoo12-addon-partner_email_check" should just work, as the OCA modules are available on PyPI.
    Dependencies on python libraries are installed automatically if they are declared in the manifest, as it is the case for that module.
    You can also install from git using the PEP 508 syntax for pip VCS urls: "pip install odoo12-addon-partner_email_check@git+https://github.com/OCA/partner-contact@12.0#subdirectory=setup/partner_email_check".

    You can find more information in this post from 2015 (time flies :) and how it is done in setuptools-odoo.

    > Looking for python dependencies in manifest is also not good, because there are python packages exists, that has different name for package and python module inside package.

    This particular problem is also resolved by setuptools-odoo (look for external dependencies override in the setuptools-odoo doc).
    And for Odoo >=13 you can and should declare the PyPI project name in external_dependencies in the manifest.

    Best regards,

    -sbi



    On Thu, Feb 11, 2021 at 10:56 AM Dmytro Katyukha <dmytro.katyukha@gmail.com> wrote:
    Hi all,

    As i see, usually in OCA repositories "requirements.txt" file with pip dependencies is located in the root of the repository and contains a list of all python dependencies for all addons in the repository. This way it works fine, when we clone full repository, and install all dependencies for all modules there, thus when user will try to install new module on DB, 99% that all python dependencies will be satisfied.

    But, let's take for example repository partner-contact repository for Odoo 12.0 and let's try to add the module 'partner_email_check' to odoo server. For this task, i will use [odoo-helper-scripts](https://github.com/katyukha/odoo-helper-scripts) that can automatically resolve repository dependencies (including those specified by oca_dependencies.txt). So, at first i would try to fetch (clone) repository partner-contact, and in this case system will automatically fetch 42 OCA repositories following 'oca_requirements.txt', and also it will try to install python dependencies mentioned in requirements.txt. It is good for development. But installation this way on prod, may lead to a lot of unneeded modules, that polutes system with strange dependencies that are in some cases may be not installable (for example because of system dependencies).

    To solve this reason, we started to use 'assembly' approach, that assumes that we have to create separate git repository with only addons needed on server. But in this case, if assembly repo created automatically, there is no way to get python requirements for module, if it is not specified in module directory. Looking for python dependencies in manifest is also not good, because there are python packages exists, that has different name for package and python module inside package.

    So, may be it have sense to place requirements.txt inside module directory? Thus module's requirements will be always delivered with module, that will make easier installation of module. Also, i think this way, it will be much easier to generate setup.py files for modules, that will contain info about module's python dependencies.

    Possible drawbacks may be in case, when different versions of python dep will be specified in different modules. But same is applicable for repositories.

    What do you think about this?

    With regards,
    Dmytro Katyukha

    _______________________________________________
    Mailing-List: https://odoo-community.org/groups/contributors-15
    Post to: mailto:contributors@odoo-community.org
    Unsubscribe: https://odoo-community.org/groups?unsubscribe

    by Stéphane Bidoul - 11:16 - 11 Feb 2021
  • Re: requirements.txt: Repository Level vs Module Level
    Hello,

    or... just use `pip install odoo${version}-addon-module-name` and you can forget about dependencies and about cloning over and over all the repos.

    Unless you want to use a local editable version, in which case you could simply do `pip install -e $repo/setup/module_name`.

    Extra goodie: you can pin the module version in your requirements.

    Hope this helps.
    Bests,
    S.

    On Thu, Feb 11, 2021 at 10:56 AM Dmytro Katyukha <dmytro.katyukha@gmail.com> wrote:
    Hi all,

    As i see, usually in OCA repositories "requirements.txt" file with pip dependencies is located in the root of the repository and contains a list of all python dependencies for all addons in the repository. This way it works fine, when we clone full repository, and install all dependencies for all modules there, thus when user will try to install new module on DB, 99% that all python dependencies will be satisfied.

    But, let's take for example repository partner-contact repository for Odoo 12.0 and let's try to add the module 'partner_email_check' to odoo server. For this task, i will use [odoo-helper-scripts](https://github.com/katyukha/odoo-helper-scripts) that can automatically resolve repository dependencies (including those specified by oca_dependencies.txt). So, at first i would try to fetch (clone) repository partner-contact, and in this case system will automatically fetch 42 OCA repositories following 'oca_requirements.txt', and also it will try to install python dependencies mentioned in requirements.txt. It is good for development. But installation this way on prod, may lead to a lot of unneeded modules, that polutes system with strange dependencies that are in some cases may be not installable (for example because of system dependencies).

    To solve this reason, we started to use 'assembly' approach, that assumes that we have to create separate git repository with only addons needed on server. But in this case, if assembly repo created automatically, there is no way to get python requirements for module, if it is not specified in module directory. Looking for python dependencies in manifest is also not good, because there are python packages exists, that has different name for package and python module inside package.

    So, may be it have sense to place requirements.txt inside module directory? Thus module's requirements will be always delivered with module, that will make easier installation of module. Also, i think this way, it will be much easier to generate setup.py files for modules, that will contain info about module's python dependencies.

    Possible drawbacks may be in case, when different versions of python dep will be specified in different modules. But same is applicable for repositories.

    What do you think about this?

    With regards,
    Dmytro Katyukha

    _______________________________________________
    Mailing-List: https://odoo-community.org/groups/contributors-15
    Post to: mailto:contributors@odoo-community.org
    Unsubscribe: https://odoo-community.org/groups?unsubscribe



    --
    Simone Orsi

    Full stack Python web developer, Odoo specialist, Odoo Community Board Member, Freelance in love with open source.
    by Simone Orsi - 11:16 - 11 Feb 2021