-
Notifications
You must be signed in to change notification settings - Fork 28
Feature/org id code migration #1644
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
borkarsaish65
wants to merge
6
commits into
ELEVATE-Project:develop
Choose a base branch
from
borkarsaish65:feature/org_id_code_migration
base: develop
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
6 commits
Select commit
Hold shift + click to select a range
4c44ade
revert: restore docker-image.yml to master state
borkarsaish65 e589834
savepoint-1
borkarsaish65 6efdd38
Merge branch 'develop' of https://github.com/ELEVATE-Project/mentorin…
borkarsaish65 eab19b7
addressed CR comments
borkarsaish65 622937f
addressed CR comments
borkarsaish65 9a20f13
addressed CR comments
borkarsaish65 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
157 changes: 157 additions & 0 deletions
157
src/database/migrations/20260525000001-backfill-org-codes-from-ids.js
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,157 @@ | ||
| 'use strict' | ||
|
|
||
| /** | ||
| * PR 1 — Migrations gate for Issue 3 (organization_id → organization_code deprecation). | ||
| * | ||
| * Adds mentor_organization_code to sessions and backfills visible_to_organizations | ||
| * arrays in sessions + user_extensions from numeric IDs to org code strings. | ||
| * | ||
| * Strategy: load the full org lookup into JS, map in JS, write back in batches. | ||
| * No complex SQL joins — easy to read and debug. | ||
| * | ||
| * IMPORTANT: Deploy and verify this migration BEFORE deploying the PR 2 code changes. | ||
| */ | ||
| module.exports = { | ||
| async up(queryInterface, Sequelize) { | ||
| // ── 0. Add the new column (safe to run even if it already exists) ────── | ||
| const [existing] = await queryInterface.sequelize.query(` | ||
| SELECT 1 FROM information_schema.columns | ||
| WHERE table_name = 'sessions' AND column_name = 'mentor_organization_code' | ||
| `) | ||
| if (existing.length === 0) { | ||
| await queryInterface.addColumn('sessions', 'mentor_organization_code', { | ||
| type: Sequelize.STRING, | ||
| allowNull: true, | ||
| }) | ||
| console.log('Added column: sessions.mentor_organization_code') | ||
| } | ||
|
|
||
| // ── 1. Build org ID → code lookup (one query, shared by all steps) ──── | ||
| const [orgRows] = await queryInterface.sequelize.query(` | ||
| SELECT organization_id, organization_code, tenant_code | ||
| FROM organization_extension | ||
| `) | ||
|
|
||
| // Primary key: "tenantCode:orgId" → orgCode | ||
| const lookup = new Map(orgRows.map((r) => [`${r.tenant_code}:${r.organization_id}`, r.organization_code])) | ||
|
|
||
| // Fallback: orgId alone → orgCode, for rows where the session's tenant_code does not | ||
| // match the tenant_code on the organization_extension row for the same numeric org ID. | ||
| // Last-write wins when the same org_id appears under multiple tenants — acceptable | ||
| // for backfill because org codes are stable and unique per org across tenants. | ||
| const fallbackLookup = new Map(orgRows.map((r) => [String(r.organization_id), r.organization_code])) | ||
|
|
||
| const toCode = (id, tenantCode) => | ||
| lookup.get(`${tenantCode}:${String(id)}`) ?? fallbackLookup.get(String(id)) ?? null | ||
|
|
||
| // Convert each element of a visible_to_organizations array. | ||
| // Unknown IDs are left as-is so data is never silently lost. | ||
| const toCodeArray = (arr, tenantCode) => arr.map((id) => toCode(id, tenantCode) ?? id) | ||
|
|
||
| // Format a JS string array as a Postgres array literal: {val1,val2} | ||
| const pgArr = (arr) => `{${arr.map((v) => `"${v.replace(/"/g, '\\"')}"`).join(',')}}` | ||
|
|
||
| // ── Step 1: sessions.mentor_organization_code ───────────────────────── | ||
| console.log('\nStep 1: Backfilling sessions.mentor_organization_code ...') | ||
|
|
||
| const [sessions] = await queryInterface.sequelize.query(` | ||
| SELECT id, mentor_organization_id, tenant_code | ||
| FROM sessions | ||
| `) | ||
|
|
||
| const sessionCodeUpdates = sessions | ||
| .map((s) => ({ id: s.id, code: toCode(s.mentor_organization_id, s.tenant_code) })) | ||
| .filter((u) => u.code !== null) | ||
|
|
||
| if (sessionCodeUpdates.length) { | ||
| // Batch all updates in one VALUES-based UPDATE — no per-row round trips | ||
| const values = sessionCodeUpdates.map((u) => `(${u.id}, '${u.code.replace(/'/g, "''")}')`).join(', ') | ||
| await queryInterface.sequelize.query(` | ||
| UPDATE sessions | ||
| SET mentor_organization_code = v.code | ||
| FROM (VALUES ${values}) AS v(id, code) | ||
| WHERE sessions.id = v.id::int | ||
| `) | ||
| } | ||
|
|
||
| console.log(` Updated ${sessionCodeUpdates.length} / ${sessions.length} rows`) | ||
|
|
||
| // ── Step 2: sessions.visible_to_organization_codes (new column) ────── | ||
| console.log('\nStep 2: Adding and backfilling sessions.visible_to_organization_codes ...') | ||
|
|
||
| const [existingVtoSessions] = await queryInterface.sequelize.query(` | ||
| SELECT 1 FROM information_schema.columns | ||
| WHERE table_name = 'sessions' AND column_name = 'visible_to_organization_codes' | ||
| `) | ||
| if (existingVtoSessions.length === 0) { | ||
| await queryInterface.addColumn('sessions', 'visible_to_organization_codes', { | ||
| type: Sequelize.ARRAY(Sequelize.STRING), | ||
| allowNull: true, | ||
| }) | ||
| console.log(' Added column: sessions.visible_to_organization_codes') | ||
| } | ||
|
|
||
| const [sessionRows] = await queryInterface.sequelize.query(` | ||
| SELECT id, visible_to_organizations, tenant_code | ||
| FROM sessions | ||
| WHERE visible_to_organizations IS NOT NULL | ||
| `) | ||
|
|
||
| let sessionArrUpdated = 0 | ||
| for (const row of sessionRows) { | ||
| const newArr = toCodeArray(row.visible_to_organizations, row.tenant_code) | ||
| await queryInterface.sequelize.query( | ||
| 'UPDATE sessions SET visible_to_organization_codes = :arr WHERE id = :id', | ||
| { replacements: { arr: pgArr(newArr), id: row.id } } | ||
| ) | ||
| sessionArrUpdated++ | ||
| } | ||
|
borkarsaish65 marked this conversation as resolved.
|
||
|
|
||
| console.log(` Updated ${sessionArrUpdated} rows`) | ||
|
|
||
| // ── Step 3: user_extensions.visible_to_organization_codes ───────────── | ||
| console.log('\nStep 3: Adding and backfilling user_extensions.visible_to_organization_codes ...') | ||
|
|
||
| const [existingVtoUE] = await queryInterface.sequelize.query(` | ||
| SELECT 1 FROM information_schema.columns | ||
| WHERE table_name = 'user_extensions' AND column_name = 'visible_to_organization_codes' | ||
| `) | ||
| if (existingVtoUE.length === 0) { | ||
| await queryInterface.addColumn('user_extensions', 'visible_to_organization_codes', { | ||
| type: Sequelize.ARRAY(Sequelize.STRING), | ||
| allowNull: true, | ||
| }) | ||
| console.log(' Added column: user_extensions.visible_to_organization_codes') | ||
| } | ||
|
|
||
| const [ueRows] = await queryInterface.sequelize.query(` | ||
| SELECT user_id, visible_to_organizations, tenant_code | ||
| FROM user_extensions | ||
| WHERE visible_to_organizations IS NOT NULL | ||
| `) | ||
|
|
||
| let ueUpdated = 0 | ||
| for (const row of ueRows) { | ||
| const newArr = toCodeArray(row.visible_to_organizations, row.tenant_code) | ||
| await queryInterface.sequelize.query( | ||
| `UPDATE user_extensions | ||
| SET visible_to_organization_codes = :arr | ||
| WHERE user_id = :userId | ||
| AND tenant_code = :tenantCode`, | ||
| { replacements: { arr: pgArr(newArr), userId: row.user_id, tenantCode: row.tenant_code } } | ||
| ) | ||
| ueUpdated++ | ||
| } | ||
|
|
||
| console.log(` Updated ${ueUpdated} rows`) | ||
| }, | ||
|
|
||
| async down(queryInterface) { | ||
| // All new columns are dropped. Original columns (mentor_organization_id, | ||
| // visible_to_organizations) are untouched so data is fully preserved. | ||
| await queryInterface.removeColumn('sessions', 'mentor_organization_code') | ||
| await queryInterface.removeColumn('sessions', 'visible_to_organization_codes') | ||
| await queryInterface.removeColumn('user_extensions', 'visible_to_organization_codes') | ||
| console.log('Removed new columns. Original data in existing columns is untouched.') | ||
| }, | ||
| } | ||
61 changes: 61 additions & 0 deletions
61
src/database/migrations/20260525000002-rename-visible-to-org-codes.js
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,61 @@ | ||
| 'use strict' | ||
|
|
||
| /** | ||
| * PR 2 — Rename migration for Issue 3 (organization_id → organization_code deprecation). | ||
| * | ||
| * Renames the old visible_to_organizations columns (numeric IDs) to | ||
| * visible_to_organizations_numeric as a backup, then promotes visible_to_organization_codes | ||
| * (org code strings) to take the visible_to_organizations name. | ||
| * | ||
| * This keeps the original column name in the final schema (no code or model changes needed) | ||
| * and preserves the numeric backup so down() is fully reversible. | ||
| * | ||
| * IMPORTANT: Run AFTER verifying PR 1 migration data side-by-side. | ||
| * Deploy this migration together with the PR 2 code changes. | ||
| * The _numeric backup columns can be dropped in a follow-up cleanup migration once verified. | ||
| */ | ||
| module.exports = { | ||
| async up(queryInterface) { | ||
| // sessions: park old column as backup, promote new column to primary name | ||
| await queryInterface.renameColumn('sessions', 'visible_to_organizations', 'visible_to_organizations_numeric') | ||
| await queryInterface.renameColumn('sessions', 'visible_to_organization_codes', 'visible_to_organizations') | ||
| console.log( | ||
| 'sessions: visible_to_organizations (numeric) → visible_to_organizations_numeric (backup); visible_to_organization_codes → visible_to_organizations' | ||
| ) | ||
|
|
||
| // user_extensions: same swap | ||
| await queryInterface.renameColumn( | ||
| 'user_extensions', | ||
| 'visible_to_organizations', | ||
| 'visible_to_organizations_numeric' | ||
| ) | ||
| await queryInterface.renameColumn( | ||
| 'user_extensions', | ||
| 'visible_to_organization_codes', | ||
| 'visible_to_organizations' | ||
| ) | ||
| console.log( | ||
| 'user_extensions: visible_to_organizations (numeric) → visible_to_organizations_numeric (backup); visible_to_organization_codes → visible_to_organizations' | ||
| ) | ||
| }, | ||
|
|
||
| async down(queryInterface) { | ||
| // Fully reversible: rename both columns back to their pre-up names. | ||
| // Numeric ID data is preserved in visible_to_organizations_numeric throughout. | ||
| await queryInterface.renameColumn('sessions', 'visible_to_organizations', 'visible_to_organization_codes') | ||
| await queryInterface.renameColumn('sessions', 'visible_to_organizations_numeric', 'visible_to_organizations') | ||
|
|
||
| await queryInterface.renameColumn( | ||
| 'user_extensions', | ||
| 'visible_to_organizations', | ||
| 'visible_to_organization_codes' | ||
| ) | ||
| await queryInterface.renameColumn( | ||
| 'user_extensions', | ||
| 'visible_to_organizations_numeric', | ||
| 'visible_to_organizations' | ||
| ) | ||
|
|
||
| console.log('Reverted rename. Numeric ID data fully restored in visible_to_organizations.') | ||
| }, | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.