Skip to content

[16.0][IMP] Scan multiple branches on a specific repository, allowing the collect of migration data on specific modules #95

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
sebalix merged 6 commits into from
Jul 8, 2025
Merged

[16.0][IMP] Scan multiple branches on a specific repository, allowing the collect of migration data on specific modules #95

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
27fa57b
odoo_repository: scan multiple branches of a repository
sebalix May 28, 2025
8f853e1
odoo_repository_migration: collect migration data on specific reposit…
sebalix May 28, 2025
bff345e
odoo_project: minor fixes/adaptations
sebalix May 28, 2025
1798200
odoo_project_migration: fix recomputation of project modules state
sebalix May 30, 2025
7cb50d3
fixup! odoo_repository_migration: collect migration data on specific …
sebalix Jun 29, 2025
a5af0a3
fixup! odoo_repository: scan multiple branches of a specific (project…
sebalix Jul 7, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
54 changes: 36 additions & 18 deletions odoo_project/models/odoo_project.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,28 +16,34 @@ class OdooProject(models.Model):
active = fields.Boolean(default=True)
repository_id = fields.Many2one(
comodel_name="odoo.repository",
ondelete="restrict",
string="Repository",
domain=[
("clone_branch_id", "!=", False),
("specific", "=", True),
("odoo_version_id", "!=", False),
],
domain=[("specific", "=", True)],
Copy link
Copy Markdown
Contributor

@hparfr hparfr Jun 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Specific here is not clear.

In my case I have repositories:

  • oca/sale-workflow
  • shopinvader/odoo-shopinvader -- is it specific ?
  • forgeFlow/stock-rma (not a fork of oca/rma)
  • akretion/ak-cloud-france-oi -- public modules but not a fork of OCA -- is it specific ?
  • akretion/private-repo-for-just-one-project -- a dedicated private repo for modules of one projet -- is it specific ?

And :

  • (gitlab)/the-project -- the project structure + some modules in ./local-src --> i don't plan scan it.

Copy link
Copy Markdown
Collaborator Author

@sebalix sebalix Jun 27, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Specific is for repositories that are hosting modules specific to a project, and they could not follow the branch naming convention (14.0, 15.0, 16.0, ...).

In your case, the (gitlab)/the-project is the specific one. Maybe akretion/private-repo-for-just-one-project as well, I can't say.

But as soon as a module could be re-used by any project simply by including the repo as a submodule, it should be qualified as generic (specific = False). It doesn't matter if this repository is private or public.

In your example, oca/sale-workflow, shopinvader/odoo-shopinvader, forgeFlow/stock-rma and akretion/ak-cloud-france-oi are generic repositories containing generic modules, shared between several Odoo implementations, and following Odoo branches naming convention.

Copy link
Copy Markdown
Contributor

@hparfr hparfr Jun 27, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

still unclear to me.

akretion/manufacture (fork of oca/manufacture) contains generic modules and follow only partially odoo branches namming convention.

Copy link
Copy Markdown
Collaborator Author

@sebalix sebalix Jun 30, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"follow only partially odoo branches naming convention" => because you have dev branches there you mean. But such branches should not be scanned IMO, because the goal of such short-lifespan branches is to be merged in the main ones (16.0, 18.0...).

Specific repositories == Odoo projects repositories, they host the Odoo project, and could contain modules only tied to this project (not shared to other projects), like {customer}_sale. If you create a module that could be re-used by another project, you probably want to host it in a generic repository designed for that (OCA, or an internal one).

One thing we do not want is to configure every branches on every repositories, so with generic repositories (e.g. OCA, shopinvader...) we scan all odoo.branch available, while in specific repositories (often linked to a project, which exists in only 1 Odoo version, max 2 if you are currently migrating it), you probably use main/master as your production branch, and this one represents e.g. a 16.0 version (when specific = True, you can enter manually your odoo.repository.branch records to specify that).

TL;DR:

  • with specific=False (generic), branches scanned are the available odoo.branch records
  • with specific=True (i.e. projects), branches to scan are configured manually

It's a convention based on what exists, but of course if you have a generic repository where we name branches v16 / v17 / v18 this pattern doesn't work anymore and would need some updates.

Beside that, a specific module (so a module scanned in a specific repository) won't have the highest priority when it comes to find modules/dependencies here and there. E.g. you have a project A repo with specific module named m, and a project B repo with specific module also named m (it's possible because they are specific repos), if later a module is scanned elsewhere (in any other repo other than A or B), having a dependency against m, it won't find it on purpose, as a specific module can have reverse dependencies only in its own repository where it is hosted.

Copy link
Copy Markdown
Collaborator Author

@sebalix sebalix Jun 30, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we could split these two notions in two separate fields:

  • a boolean field scan_default_branches to define if branches have to be scanned based on odoo.branch (Odoo naming convention) or based on user input with specific branch names (master etc)
  • a boolean field specific to ensure its modules will have its reverse dependencies in the same repo, so they cannot be used in other repo/project

Right now both are mixed 🤔

Copy link
Copy Markdown
Contributor

@hparfr hparfr Jun 30, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But such branches should not be scanned IMO, because the goal of such short-lifespan branches is to be merged in the main ones (16.0, 18.0...).

Ah ah ! if only !
I plan to scan them one way or another. We have so many PR waiting (or closed!) because of external factors or individuals.
We have also projects still in 14.0, we target publishing in OCA 18.0 and keep a back-port up-to-date.

I've made a proto few months ago, but it's far from working. Here is the big picture:

From the project page, instead of copy /pasting the module list, I copy paste our "spec.yaml", it's the gitaggretor file with shorter syntax and a module list.
The module list is then used to create symbolic links in a directory in the addon_path.

For instance:

l10n-france:
    modules:
        - l10n_fr_siret
    src: https://github.com/OCA/l10n-france 16.0

manufacture:
    modules:
        - mrp_bom_component_menu
        - mrp_restrict_lot
        - mrp_sale_info
    src: https://github.com/OCA/manufacture 16.0

manufacture-config:
    modules:
        - mrp_bom_configurable
    src: https://github.com/akretion/manufacture 16-bom-configurable

Will transform to:

./external-src/l10n-france:
  merges:
  - origin 18.0
  remotes:
    origin: https://github.com/OCA/l10n-france
  target: origin 18.0
./external-src/manufacture:
  merges:
  - origin 18.0
  remotes:
    origin: https://github.com/OCA/manufacture
  target: origin 18.0
./external-src/manufacture-config:
  merges:
  - origin 18.0-bom-configurable
  remotes:
    origin: https://github.com/akretion/manufacture
  target: origin 18.0-bom-configurable

and is piped into gitaggregator.

So with this input, I create "forked repos":

  • oca/manufacture Fork=False
  • akretion/manufacture Forked=oca/manufacture

And I create akretion/manufacture#18.0-bom-configurable branch.
This branch contains the module mrp_bom_configurable for 18.0

I don't oca-scan this branch. So I get only minimum information.
But it's good enough for

  • discovering which project is using this not-yet-merged module
  • follow migration avancement.
  • generate a spec.yaml for the project

Specific repositories
Thanks for this explaination.

It's a convention based on what exists, but of course if you have a generic repository where we name branches v16 / v17 / v18 this pattern doesn't work anymore and would need some updates.
v16 / v17 ... I think it's ok to configure it manually

Specifc module have low priority. -> yes ! We have so many "custom_reports" modules 😮‍💨

So, to recap the behaviors are:

  • low priority because there will be duplicate names
  • follows branch naming convention: 16.0 /17.0 / 18.0...
  • the scan is based on white list of branches / not on all existing branches

Copy link
Copy Markdown
Collaborator Author

@sebalix sebalix Jun 30, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, I understand better your workflow there. So splitting the current specific field in two with scan_default_branches + specific will make sense there, because even on your fork I guess you don't want to scan all the branches of this fork, but only a bunch of them containing strategic WIP module(s) like 18.0-bom-configurable.

Copy link
Copy Markdown
Collaborator Author

@sebalix sebalix Jul 7, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@hparfr I added a new flag manual_branches (that is set to True during the upgrade if specific was set). You can now configure your own branches while keeping modules generic.
If you want to give the priority to modules hosted in such branch (long running development branches) when resolving dependencies you can set a higher priority/sequence to the repo. For now there is no possibility to choose which module version has to be included in a project if several are available.

store=True,
index=True,
help=(
"Repository is optional. "
"Repository this project is based on (optional). "
"You can start to build/simulate a project without repository "
"to get some figures."
),
)
available_odoo_version_ids = fields.One2many(
comodel_name="odoo.branch",
compute="_compute_available_odoo_version_ids",
string="Available Odoo Versions",
Copy link
Copy Markdown
Contributor

@hparfr hparfr Jun 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what does it mean ?

Copy link
Copy Markdown
Collaborator Author

@sebalix sebalix Jun 27, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is a technical field to list the available Odoo versions that have been scanned in this repository. It is used by odoo_project to filter available versions when selecting one.

Copy link
Copy Markdown
Collaborator Author

@sebalix sebalix Jun 27, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If your project is not linked to a repository, all Odoo versions are legit, you can select the one you want.
But if your project is linked to a repository, you are forced to select an Odoo version that is available based on the branches scanned in this repository (could be a master branch representing the 14.0 version, and a 18.0-mig representing the 18.0 version, then you could select 14.0 or 18.0 when creating such project).

Copy link
Copy Markdown
Contributor

@hparfr hparfr Jun 27, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so if this project is linked to a repository containing branches: "master", "main", "staging", "feat/123", "feat/125"... "feat/9999"

What is the gain ?

Copy link
Copy Markdown
Collaborator Author

@sebalix sebalix Jun 30, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To not make mistake, the user creating the project in your odoo-repository instance should not be able to select as Odoo version 16.0 if the underlying repository is hosting only versions 14.0 (for instance on master) and a 18.0 currently in migration.

Copy link
Copy Markdown
Collaborator Author

@sebalix sebalix Jun 30, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And your branch feat/9999 won't be configured in the odoo.repository.branch record (<odoo.repository>.branch_ids field) to be scan I guess as it's a short-lifespan branch, that will land probably in master soon. The tool overall aims to scan only stable/production branches, and ongoing modules development should be find through PRs (this detection is a bit wobbly now and would need to be smarter BTW).

)
odoo_version_id = fields.Many2one(
Copy link
Copy Markdown
Contributor

@hparfr hparfr Jun 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is it in production version ?

Copy link
Copy Markdown
Collaborator Author

@sebalix sebalix Jun 27, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, I still didn't have time to separate the notion of "Odoo project" and "Odoo project instance". For now, odoo.project represents an instance of a project. There is no flag ATM to qualify an instance as the production/main one.

odoo.project data model should be renamed later to odoo.project.instance (or another name), and we should create the data model odoo.project that groups the instances (=> "Customer Project", having a 14.0 + 18.0 instances, the 14.0 being the production version).

Copy link
Copy Markdown
Contributor

@hparfr hparfr Jun 27, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With #98 , having one project is enough for me, at the moment.

comodel_name="odoo.branch",
ondelete="restrict",
string="Odoo Version",
domain=[("odoo_version", "=", True)],
required=True,
compute="_compute_odoo_version_id",
store=True,
readonly=False,
index=True,
)
repository_branch_id = fields.Many2one(
comodel_name="odoo.repository.branch",
string="Repository / Branch",
compute="_compute_repository_branch_id",
store=True,
index=True,
)
project_module_ids = fields.One2many(
comodel_name="odoo.project.module",
Expand Down Expand Up @@ -74,10 +80,22 @@ class OdooProject(models.Model):
)

@api.depends("repository_id")
def _compute_odoo_version_id(self):
def _compute_available_odoo_version_ids(self):
all_versions = self.env["odoo.branch"]._get_all_odoo_versions()
for rec in self:
rec.available_odoo_version_ids = all_versions
if rec.repository_id:
rec.odoo_version_id = rec.repository_id.odoo_version_id
rec.available_odoo_version_ids = rec.repository_id.branch_ids.branch_id

@api.depends("repository_id", "odoo_version_id")
def _compute_repository_branch_id(self):
for rec in self:
rec.repository_branch_id = False
if not rec.repository_id or not rec.odoo_version_id:
continue
rec.repository_branch_id = rec.repository_id.branch_ids.filtered(
lambda rb: rb.branch_id == rec.odoo_version_id
)

@api.depends("project_module_ids.module_id")
def _compute_module_ids(self):
Expand All @@ -90,11 +108,11 @@ def _compute_modules_count(self):
rec.modules_count = len(rec.project_module_ids)

@api.depends(
"repository_id.branch_ids.module_ids", "project_module_ids.module_branch_id"
"repository_branch_id.module_ids", "project_module_ids.module_branch_id"
)
def _compute_module_not_installed_ids(self):
for rec in self:
all_module_ids = set(rec.repository_id.branch_ids.module_ids.ids)
all_module_ids = set(rec.repository_branch_id.module_ids.ids)
installed_module_ids = set(rec.project_module_ids.module_branch_id.ids)
rec.module_not_installed_ids = list(all_module_ids - installed_module_ids)

Expand Down Expand Up @@ -145,12 +163,12 @@ def action_find_unknown_modules(self):
module.action_find_pr_url()

def _get_repositories_to_scan(self):
"""Returnt the repositories to scan."""
"""Return the repositories to scan."""
domain = self.env["odoo.repository"]._cron_scanner_domain()
return self.project_module_ids.repository_id.filtered_domain(domain)

def _get_branches_to_scan(self):
"""Return the branches to scan."""
"""Return the branches/versions to scan."""
return self.project_module_ids.repository_branch_id.branch_id

def action_scan(self, force=False):
Expand All @@ -163,7 +181,7 @@ def action_scan(self, force=False):
branches = self._get_branches_to_scan()
if branches:
repositories.action_scan(
branches=branches.mapped("name"), force=force, raise_exc=False
branch_ids=branches.ids, force=force, raise_exc=False
)
# Scan the underlying project repository itself
self.repository_id.action_scan(force=force, raise_exc=True)
Expand Down
2 changes: 1 addition & 1 deletion odoo_project/tests/test_import_modules.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -134,8 +134,8 @@ def test_match_generic_module(self):

def test_match_project_repo_module(self):
# Assign a repository to the project
self.odoo_repository.odoo_version_id = self.branch
self.project.repository_id = self.odoo_repository
self.project.odoo_version_id = self.branch
mod1 = "test1"
mod2 = "test2"
mod1_in_repo = self.wiz_model._get_module(mod1)
Expand Down
8 changes: 6 additions & 2 deletions odoo_project/views/odoo_project.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -44,13 +44,17 @@
<h1><field name="name" placeholder="Project Name" /></h1>
</div>
<group>
<field name="available_odoo_version_ids" invisible="1" />
<field
name="repository_id"
attrs="{'readonly': [('project_module_ids', '!=', [])]}"
force_save="1"
attrs="{'readonly': [('repository_id', '!=', False), ('project_module_ids', '!=', [])]}"
/>
<field
name="odoo_version_id"
attrs="{'readonly': [('project_module_ids', '!=', [])]}"
force_save="1"
domain="[('id', 'in', available_odoo_version_ids)]"
attrs="{'readonly': [('odoo_version_id', '!=', False), ('project_module_ids', '!=', [])]}"
options="{'no_open': True}"
/>
</group>
Expand Down
8 changes: 6 additions & 2 deletions odoo_project_migration/models/odoo_project_module_migration.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,6 @@

class OdooProjectModuleMigration(models.Model):
_name = "odoo.project.module.migration"
# TODO test to inherit from 'module_migration_id' field too
_inherits = {"odoo.module.branch": "source_module_branch_id"}
_description = "Module migration line of an Odoo Project"
_order = (
Expand Down Expand Up @@ -144,7 +143,12 @@ def _compute_target_module_branch_id(self):
domain=[("installable", "=", True)],
)

@api.depends("migration_path_id", "source_module_branch_id")
# NOTE: 'migration_scan' is here to re-trigger the computation
# each time the source module has its state updated regarding migration.
# FIXME: this could trigger too much computations on irrelevant records
# (one not related to the updated migration path), we should switch to
# component events to handle such cases.
@api.depends("migration_path_id", "source_module_branch_id.migration_scan")
def _compute_module_migration_id(self):
migration_model = self.env["odoo.module.branch.migration"]
for rec in self:
Expand Down
2 changes: 1 addition & 1 deletion odoo_repository/__manifest__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
{
"name": "Odoo Repositories Data",
"summary": "Base module to host data collected from Odoo repositories.",
"version": "16.0.1.1.0",
"version": "16.0.1.3.0",
"category": "Tools",
"author": "Camptocamp, Odoo Community Association (OCA)",
"website": "https://github.com/camptocamp/odoo-repository",
Expand Down
15 changes: 5 additions & 10 deletions odoo_repository/data/odoo_branch.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,61 +14,56 @@

<record id="odoo_branch_8" model="odoo.branch">
<field name="name">8.0</field>
<field name="odoo_version" eval="True" />
<field name="active" eval="False" />
</record>

<record id="odoo_branch_9" model="odoo.branch">
<field name="name">9.0</field>
<field name="odoo_version" eval="True" />
<field name="active" eval="False" />
</record>

<record id="odoo_branch_10" model="odoo.branch">
<field name="name">10.0</field>
<field name="odoo_version" eval="True" />
<field name="active" eval="False" />
</record>

<record id="odoo_branch_11" model="odoo.branch">
<field name="name">11.0</field>
<field name="odoo_version" eval="True" />
<field name="active" eval="False" />
</record>

<record id="odoo_branch_12" model="odoo.branch">
<field name="name">12.0</field>
<field name="odoo_version" eval="True" />
<field name="active" eval="False" />
</record>

<record id="odoo_branch_13" model="odoo.branch">
<field name="name">13.0</field>
<field name="odoo_version" eval="True" />
<field name="active" eval="False" />
</record>

<record id="odoo_branch_14" model="odoo.branch">
<field name="name">14.0</field>
<field name="odoo_version" eval="True" />
<field name="active" eval="False" />
</record>

<record id="odoo_branch_15" model="odoo.branch">
<field name="name">15.0</field>
<field name="odoo_version" eval="True" />
<field name="active" eval="False" />
</record>

<record id="odoo_branch_16" model="odoo.branch">
<field name="name">16.0</field>
<field name="odoo_version" eval="True" />
<field name="active" eval="False" />
</record>

<record id="odoo_branch_17" model="odoo.branch">
<field name="name">17.0</field>
<field name="odoo_version" eval="True" />
<field name="active" eval="False" />
</record>

<record id="odoo_branch_18" model="odoo.branch">
<field name="name">18.0</field>
<field name="active" eval="False" />
</record>

Expand Down
Loading