[16.0][IMP] Scan multiple branches on a specific repository, allowing the collect of migration data on specific modules by sebalix · Pull Request #95 · OCA/module-composition-analysis

sebalix · 2025-05-28T12:01:51Z

Address #67 (partially, the split of project / project version will be done in a separate PR later).

odoo.repository form:

sebalix · 2025-05-30T09:18:38Z

A constraint regarding the name is welcome to get a better error message (as to be an Odoo version like 18.0), currently _recompute_sequence is crashing with a wrong value.

We could even rename odoo.branch to odoo.version but it's a big change that brings no value for now (will do that later).

odoo.version is an improvement for code readability :)

oh boy ! it's so hard to read branch = version some time but not always :)

Yes I agree :/ I get lost sometimes myself. It's because I first wanted to represents branches that could be an Odoo version, then changed my mind.

hparfr · 2025-06-17T13:17:05Z

Specific here is not clear.

In my case I have repositories:

oca/sale-workflow

shopinvader/odoo-shopinvader -- is it specific ?

forgeFlow/stock-rma (not a fork of oca/rma)

akretion/ak-cloud-france-oi -- public modules but not a fork of OCA -- is it specific ?

akretion/private-repo-for-just-one-project -- a dedicated private repo for modules of one projet -- is it specific ?

And :

(gitlab)/the-project -- the project structure + some modules in ./local-src --> i don't plan scan it.

Specific is for repositories that are hosting modules specific to a project, and they could not follow the branch naming convention (14.0, 15.0, 16.0, ...).

In your case, the (gitlab)/the-project is the specific one. Maybe akretion/private-repo-for-just-one-project as well, I can't say.

But as soon as a module could be re-used by any project simply by including the repo as a submodule, it should be qualified as generic (specific = False). It doesn't matter if this repository is private or public.

In your example, oca/sale-workflow, shopinvader/odoo-shopinvader, forgeFlow/stock-rma and akretion/ak-cloud-france-oi are generic repositories containing generic modules, shared between several Odoo implementations, and following Odoo branches naming convention.

still unclear to me.

akretion/manufacture (fork of oca/manufacture) contains generic modules and follow only partially odoo branches namming convention.

"follow only partially odoo branches naming convention" => because you have dev branches there you mean. But such branches should not be scanned IMO, because the goal of such short-lifespan branches is to be merged in the main ones (16.0, 18.0...).

Specific repositories == Odoo projects repositories, they host the Odoo project, and could contain modules only tied to this project (not shared to other projects), like {customer}_sale. If you create a module that could be re-used by another project, you probably want to host it in a generic repository designed for that (OCA, or an internal one).

One thing we do not want is to configure every branches on every repositories, so with generic repositories (e.g. OCA, shopinvader...) we scan all odoo.branch available, while in specific repositories (often linked to a project, which exists in only 1 Odoo version, max 2 if you are currently migrating it), you probably use main/master as your production branch, and this one represents e.g. a 16.0 version (when specific = True, you can enter manually your odoo.repository.branch records to specify that).

TL;DR:

with specific=False (generic), branches scanned are the available odoo.branch records

with specific=True (i.e. projects), branches to scan are configured manually

It's a convention based on what exists, but of course if you have a generic repository where we name branches v16 / v17 / v18 this pattern doesn't work anymore and would need some updates.

Beside that, a specific module (so a module scanned in a specific repository) won't have the highest priority when it comes to find modules/dependencies here and there. E.g. you have a project A repo with specific module named m, and a project B repo with specific module also named m (it's possible because they are specific repos), if later a module is scanned elsewhere (in any other repo other than A or B), having a dependency against m, it won't find it on purpose, as a specific module can have reverse dependencies only in its own repository where it is hosted.

Maybe we could split these two notions in two separate fields:

a boolean field scan_default_branches to define if branches have to be scanned based on odoo.branch (Odoo naming convention) or based on user input with specific branch names (master etc)

a boolean field specific to ensure its modules will have its reverse dependencies in the same repo, so they cannot be used in other repo/project

Right now both are mixed 🤔

But such branches should not be scanned IMO, because the goal of such short-lifespan branches is to be merged in the main ones (16.0, 18.0...).

Ah ah ! if only !
I plan to scan them one way or another. We have so many PR waiting (or closed!) because of external factors or individuals.
We have also projects still in 14.0, we target publishing in OCA 18.0 and keep a back-port up-to-date.

I've made a proto few months ago, but it's far from working. Here is the big picture:

From the project page, instead of copy /pasting the module list, I copy paste our "spec.yaml", it's the gitaggretor file with shorter syntax and a module list.
The module list is then used to create symbolic links in a directory in the addon_path.

For instance:

l10n-france: modules: - l10n_fr_siret src: https://github.com/OCA/l10n-france 16.0 manufacture: modules: - mrp_bom_component_menu - mrp_restrict_lot - mrp_sale_info src: https://github.com/OCA/manufacture 16.0 manufacture-config: modules: - mrp_bom_configurable src: https://github.com/akretion/manufacture 16-bom-configurable

Will transform to:

./external-src/l10n-france: merges: - origin 18.0 remotes: origin: https://github.com/OCA/l10n-france target: origin 18.0 ./external-src/manufacture: merges: - origin 18.0 remotes: origin: https://github.com/OCA/manufacture target: origin 18.0 ./external-src/manufacture-config: merges: - origin 18.0-bom-configurable remotes: origin: https://github.com/akretion/manufacture target: origin 18.0-bom-configurable

and is piped into gitaggregator.

So with this input, I create "forked repos":

oca/manufacture Fork=False

akretion/manufacture Forked=oca/manufacture

And I create akretion/manufacture#18.0-bom-configurable branch.
This branch contains the module mrp_bom_configurable for 18.0

I don't oca-scan this branch. So I get only minimum information.
But it's good enough for

discovering which project is using this not-yet-merged module

follow migration avancement.

generate a spec.yaml for the project

Specific repositories
Thanks for this explaination.

It's a convention based on what exists, but of course if you have a generic repository where we name branches v16 / v17 / v18 this pattern doesn't work anymore and would need some updates.
v16 / v17 ... I think it's ok to configure it manually

Specifc module have low priority. -> yes ! We have so many "custom_reports" modules 😮‍💨

So, to recap the behaviors are:

low priority because there will be duplicate names

follows branch naming convention: 16.0 /17.0 / 18.0...

the scan is based on white list of branches / not on all existing branches

Ok, I understand better your workflow there. So splitting the current specific field in two with scan_default_branches + specific will make sense there, because even on your fork I guess you don't want to scan all the branches of this fork, but only a bunch of them containing strategic WIP module(s) like 18.0-bom-configurable.

@hparfr I added a new flag manual_branches (that is set to True during the upgrade if specific was set). You can now configure your own branches while keeping modules generic.
If you want to give the priority to modules hosted in such branch (long running development branches) when resolving dependencies you can set a higher priority/sequence to the repo. For now there is no possibility to choose which module version has to be included in a project if several are available.

hparfr · 2025-06-17T13:17:56Z

what does it mean ?

It is a technical field to list the available Odoo versions that have been scanned in this repository. It is used by odoo_project to filter available versions when selecting one.

If your project is not linked to a repository, all Odoo versions are legit, you can select the one you want.
But if your project is linked to a repository, you are forced to select an Odoo version that is available based on the branches scanned in this repository (could be a master branch representing the 14.0 version, and a 18.0-mig representing the 18.0 version, then you could select 14.0 or 18.0 when creating such project).

so if this project is linked to a repository containing branches: "master", "main", "staging", "feat/123", "feat/125"... "feat/9999"

What is the gain ?

To not make mistake, the user creating the project in your odoo-repository instance should not be able to select as Odoo version 16.0 if the underlying repository is hosting only versions 14.0 (for instance on master) and a 18.0 currently in migration.

And your branch feat/9999 won't be configured in the odoo.repository.branch record (<odoo.repository>.branch_ids field) to be scan I guess as it's a short-lifespan branch, that will land probably in master soon. The tool overall aims to scan only stable/production branches, and ongoing modules development should be find through PRs (this detection is a bit wobbly now and would need to be smarter BTW).

hparfr · 2025-06-17T13:18:32Z

is it in production version ?

No, I still didn't have time to separate the notion of "Odoo project" and "Odoo project instance". For now, odoo.project represents an instance of a project. There is no flag ATM to qualify an instance as the production/main one.

odoo.project data model should be renamed later to odoo.project.instance (or another name), and we should create the data model odoo.project that groups the instances (=> "Customer Project", having a 14.0 + 18.0 instances, the 14.0 being the production version).

With #98 , having one project is enough for me, at the moment.

hparfr · 2025-06-17T13:24:03Z

thanks for adressing this.

It's a lot of work ! I did a partial review; I takes hours to load all of this in my head :D

On repositories, the Odoo version and branch to clone can now be set manually in the `odoo.repository.branch` relation. This allows to set different Odoo versions/branches hosted in a repository instead of one.

…ories Now that multiple Odoo versions/branches can be set on a specific repository, we are able to collect the migration data of specific modules hosted in it.

…repositories

…) repository

sebalix · 2025-07-07T12:24:13Z

@hparfr if you are OK with the last change, I'll merge this one

hparfr · 2025-07-07T13:41:36Z

+from odoo.exceptions import ValidationError


 class OdooBranch(models.Model):


So in the end, you plan to rename this class to OdooVersion ?

In another PR yes

Or I can already rename the class if you prefer right now :)
But won't touch all M2O here and there for now

Ok. I don't have time to test today. Let's merge and see :)

sebalix added enhancement New feature or request work in progress labels May 28, 2025

sebalix force-pushed the 16-scan-multiple-branches-specific-repository branch 8 times, most recently from c89898c to 3091230 Compare May 30, 2025 08:54

sebalix commented May 30, 2025

View reviewed changes

sebalix force-pushed the 16-scan-multiple-branches-specific-repository branch 3 times, most recently from c11d52a to fd6bb8c Compare June 4, 2025 06:58

hparfr reviewed Jun 17, 2025

View reviewed changes

sebalix force-pushed the 16-scan-multiple-branches-specific-repository branch from 6dce834 to 7b7a9cd Compare June 30, 2025 07:44

sebalix added 6 commits July 7, 2025 08:47

odoo_repository: scan multiple branches of a repository

27fa57b

On repositories, the Odoo version and branch to clone can now be set manually in the `odoo.repository.branch` relation. This allows to set different Odoo versions/branches hosted in a repository instead of one.

odoo_repository_migration: collect migration data on specific reposit…

8f853e1

…ories Now that multiple Odoo versions/branches can be set on a specific repository, we are able to collect the migration data of specific modules hosted in it.

odoo_project: minor fixes/adaptations

bff345e

odoo_project_migration: fix recomputation of project modules state

1798200

fixup! odoo_repository_migration: collect migration data on specific …

7cb50d3

…repositories

fixup! odoo_repository: scan multiple branches of a specific (project…

a5af0a3

…) repository

sebalix force-pushed the 16-scan-multiple-branches-specific-repository branch from 7b7a9cd to a5af0a3 Compare July 7, 2025 06:50

sebalix marked this pull request as ready for review July 7, 2025 12:23

hparfr reviewed Jul 7, 2025

View reviewed changes

sebalix removed the work in progress label Jul 8, 2025

sebalix merged commit f494350 into OCA:16.0 Jul 8, 2025
4 checks passed

sebalix deleted the 16-scan-multiple-branches-specific-repository branch July 8, 2025 07:59

		from odoo.exceptions import ValidationError


		class OdooBranch(models.Model):

Uh oh!

Conversation

sebalix commented May 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

sebalix May 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

hparfr Jun 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sebalix Jun 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sebalix Jun 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sebalix Jun 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

hparfr commented Jun 17, 2025

Uh oh!

sebalix commented Jul 7, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

sebalix commented May 28, 2025 •

edited

Loading

sebalix May 30, 2025 •

edited

Loading

hparfr Jun 17, 2025 •

edited

Loading

sebalix Jun 27, 2025 •

edited

Loading

sebalix Jun 30, 2025 •

edited

Loading

sebalix Jun 30, 2025 •

edited

Loading