From fc17349d32d85278fc5fd0c4b3b4c6667d60cb2b Mon Sep 17 00:00:00 2001 From: Aleksandr Kislitsyn Date: Thu, 6 Aug 2026 14:43:13 +0200 Subject: [PATCH 1/3] Document AidboxMigration execution-type and improve 2607 release notes Adds documentation for running AidboxMigration SQL outside a transaction (HealthSamurai/sansara#8102), needed for statements PostgreSQL forbids inside a transaction block such as CREATE INDEX CONCURRENTLY. * migrations.md: new "Run SQL outside a transaction" section covering the execution-type parameter, which request contexts allow it (direct POST and batch bundles yes, transaction bundles rejected), a batch Init Bundle example, and the no-rollback caveat. * how-to-run-sql-via-init-bundle.md: use CREATE INDEX CONCURRENTLY as the main example, since a plain CREATE INDEX blocks writes for the duration of the build. * indexes/README.md: new "Applying an index" section listing the three routes for autocommit DDL. * sql-endpoints.md: cross-link from Aidbox-Sql-Autocommit to the migration route. * release-notes.md: rework the 2607 section with documentation links and fuller descriptions. Co-Authored-By: Claude Opus 5 (1M context) --- docs/api/rest-api/other/sql-endpoints.md | 2 +- docs/configuration/migrations.md | 95 +++++++++++++++++++ .../indexes/README.md | 12 +++ docs/overview/release-notes.md | 33 ++++++- .../how-to-run-sql-via-init-bundle.md | 34 ++++++- 5 files changed, 167 insertions(+), 9 deletions(-) diff --git a/docs/api/rest-api/other/sql-endpoints.md b/docs/api/rest-api/other/sql-endpoints.md index 2044ba964..ee3f72739 100644 --- a/docs/api/rest-api/other/sql-endpoints.md +++ b/docs/api/rest-api/other/sql-endpoints.md @@ -84,7 +84,7 @@ Every header below is optional. Defaults match a single-transaction read-write r | Header | Value | Effect | |---|---|---| -| `Aidbox-Sql-Autocommit` | `true` | Run outside a transaction. Required for `VACUUM`, `CREATE INDEX CONCURRENTLY`, `REINDEX CONCURRENTLY`. Rejected when [`db.pass-auth-vars`](../../../reference/all-settings.md#db.pass-auth-vars) is on **and** the request carries a resolvable identity — autocommit would drop the SQL identity injection that RLS relies on. | +| `Aidbox-Sql-Autocommit` | `true` | Run outside a transaction. Required for `VACUUM`, `CREATE INDEX CONCURRENTLY`, `REINDEX CONCURRENTLY`. To run such a statement as a one-time migration instead, see [`AidboxMigration` outside a transaction](../../../configuration/migrations.md#run-sql-outside-a-transaction). Rejected when [`db.pass-auth-vars`](../../../reference/all-settings.md#db.pass-auth-vars) is on **and** the request carries a resolvable identity — autocommit would drop the SQL identity injection that RLS relies on. | | `Aidbox-Sql-Timeout` | seconds, 1..86400 | Per-query `statement_timeout`. Empty / non-numeric / negative / out-of-range values are ignored. | | `Aidbox-Sql-Read-Only` | `true` | Run as read-only. Writes raise `ERROR: cannot execute … in a read-only transaction`. | | `Aidbox-Sql-Query-Id` | UUID | Tags the PG session via `application_name = aidbox-psql:`. The same UUID is used to cancel via `/$psql-cancel`. | diff --git a/docs/configuration/migrations.md b/docs/configuration/migrations.md index 90f8a8c56..19f5ccbea 100644 --- a/docs/configuration/migrations.md +++ b/docs/configuration/migrations.md @@ -123,6 +123,100 @@ After execution, the migration status changes to `done` and `result.valueBoolean Invalid SQL causes the migration to fail with a 422 error. In a transaction bundle, this rolls back the entire transaction and prevents Aidbox from starting. {% endhint %} +### Run SQL outside a transaction + +{% hint style="info" %} +Available since the 2607 release. +{% endhint %} + +By default Aidbox wraps the migration SQL in a transaction. PostgreSQL forbids some statements inside a transaction block, among them `CREATE INDEX CONCURRENTLY`, `DROP INDEX CONCURRENTLY`, `REINDEX CONCURRENTLY`, and `VACUUM`. Add the `execution-type` parameter with the value `not-in-transaction` to run the SQL with autocommit on: + +```json +{ + "resourceType": "AidboxMigration", + "id": "create-encounter-index-concurrently", + "action": "aidbox-migration-run-sql", + "status": "to-run", + "params": { + "resourceType": "Parameters", + "parameter": [ + { + "name": "sql", + "valueString": "CREATE INDEX CONCURRENTLY IF NOT EXISTS encounter_subject_id ON encounter ((resource #>> '{subject, id}'));" + }, + { + "name": "execution-type", + "valueCode": "not-in-transaction" + } + ] + } +} +``` + +| `execution-type` | Behavior | +|---|---| +| `in-transaction` | Default, also applied when the parameter is absent. Aidbox runs the SQL inside a transaction and rolls it back on failure. | +| `not-in-transaction` | Aidbox runs the SQL with autocommit on, outside any transaction. | + +A migration that runs outside a transaction cannot be part of a FHIR **transaction** bundle, because the bundle itself is one atomic transaction. Post it in one of these ways instead: + +| How you send it | `execution-type: not-in-transaction` | +|---|---| +| `POST /fhir/AidboxMigration` | Runs outside a transaction | +| `POST /AidboxMigration` (Aidbox format) | Runs outside a transaction | +| Entry in a `batch` bundle | Runs outside a transaction | +| Entry in a `transaction` bundle | Rejected with a 422 error | + +Init Bundle examples on this page use `"type": "transaction"`. To run a non-transactional migration on startup, set the bundle type to `batch`: + +```json +{ + "type": "batch", + "resourceType": "Bundle", + "entry": [ + { + "request": { + "method": "POST", + "url": "AidboxMigration", + "ifNoneExist": "id=create-encounter-index-concurrently" + }, + "resource": { + "resourceType": "AidboxMigration", + "id": "create-encounter-index-concurrently", + "action": "aidbox-migration-run-sql", + "status": "to-run", + "params": { + "resourceType": "Parameters", + "parameter": [ + { + "name": "sql", + "valueString": "CREATE INDEX CONCURRENTLY IF NOT EXISTS encounter_subject_id ON encounter ((resource #>> '{subject, id}'));" + }, + { + "name": "execution-type", + "valueCode": "not-in-transaction" + } + ] + } + } + } + ] +} +``` + +{% hint style="warning" %} +Aidbox does not roll back a migration that runs outside a transaction. A failed `CREATE INDEX CONCURRENTLY` leaves an invalid index behind, and PostgreSQL does not use it for queries. Write the statement with `IF NOT EXISTS`, drop the invalid index, and run the migration again. + +Find invalid indexes with: + +```sql +SELECT c.relname +FROM pg_index i +JOIN pg_class c ON c.oid = i.indexrelid +WHERE NOT i.indisvalid; +``` +{% endhint %} + ## Using with Init Bundle Set the `BOX_INIT_BUNDLE` environment variable to load migrations on startup: @@ -217,6 +311,7 @@ Aidbox also exposes a [`POST /db/migrations`](../api/rest-api/other/sql-endpoint | External client required | No | Yes (needs credentials and a healthy Aidbox) | | Idempotency | Built-in via `ifNoneExist` | Built-in via migration id tracking | | FHIR package installs | Yes | No | +| SQL outside a transaction | Yes, via [`execution-type`](#run-sql-outside-a-transaction) in a batch bundle or a direct POST | No, use [`$psql`](../api/rest-api/other/sql-endpoints.md#execution-headers) with `Aidbox-Sql-Autocommit: true` | Use `AidboxMigration` when you want zero-touch migrations on boot. Use `POST /db/migrations` when you need to apply migrations on demand from deployment scripts. diff --git a/docs/deployment-and-maintenance/indexes/README.md b/docs/deployment-and-maintenance/indexes/README.md index 0291eb183..2483ac5d1 100644 --- a/docs/deployment-and-maintenance/indexes/README.md +++ b/docs/deployment-and-maintenance/indexes/README.md @@ -166,6 +166,18 @@ params: [get-suggested-indexes.md](get-suggested-indexes.md) {% endcontent-ref %} +## Applying an index + +Index DDL usually uses `CREATE INDEX CONCURRENTLY`, which builds the index without blocking writes to the table. PostgreSQL refuses to run it inside a transaction block, so pick a route that runs it with autocommit on: + +| Route | How | +|---|---| +| [`POST /$psql`](../../api/rest-api/other/sql-endpoints.md#execution-headers) | Send `Aidbox-Sql-Autocommit: true`. Add `Aidbox-Sql-Async: true` to get a `202` back while PostgreSQL keeps building. | +| SQL Console in Aidbox UI | Switch transaction mode to `Autocommit`. | +| [`AidboxMigration`](../../configuration/migrations.md#run-sql-outside-a-transaction) | Set the `execution-type` parameter to `not-in-transaction`, in a batch bundle or a direct POST. Use this to apply the index on startup as a one-time migration. | + +A `CREATE INDEX CONCURRENTLY` that fails leaves an invalid index behind, and PostgreSQL ignores it when planning queries. Drop it and run the statement again. + ## Usage statistics Aidbox tracks how often each SearchParameter is queried and exposes the numbers via RPCs. Use them to rank "hot" parameters, decide which suggested indexes are worth creating, and confirm a created index is actually being used. Available since Aidbox 2605. diff --git a/docs/overview/release-notes.md b/docs/overview/release-notes.md index fcffad587..30b1f00a0 100644 --- a/docs/overview/release-notes.md +++ b/docs/overview/release-notes.md @@ -6,9 +6,36 @@ description: >- # Release Notes -## July 2026 _`edge`_ +## August 2026 _`edge`_ -## June 2026 _`latest, 2606`_ +## July 2026 _`latest, 2607`_ + +* Aidbox FHIR server + + **Features** + + * **[SMART Health Cards](../api/rest-api/other/smart-health-cards.md)** — the `$health-cards-issue` operation issues verifiable health credentials signed as a compact JWS, which a patient presents as a QR code or file. Aidbox publishes the verification public key at a JWKS endpoint, so any SMART Health Cards verifier can validate the cards. + * **[Reworked batch validation](../modules/profiling-and-validation/batch-resource-validation.md)** — the new resource-level `$batch-validate` operation validates resources already stored in the database against their schemas and FHIR profiles, synchronously or asynchronously, and stores offender-indexed results you can drill into. + * Performance improvements in [FHIR Bundle](../api/rest-api/bundle.md) processing. + * Performance improvements in the [`aidbox.bulk/load-from-bucket`](../api/bulk-api/bulk-import-from-an-s3-bucket.md) API. + + **Bug fixes and improvements** + + * Fixed patient filtering in [consent-aware bulk export](../api/bulk-api/export.md#consent-based-patient-filtering). + * Fixed validation of [FHIR Bundles](../api/rest-api/bundle.md). + * Fixed Resource Browser validating extensions inside `BackboneElement`s incorrectly. + * Fixed settings handling in Multibox. + * Fixed how FHIR [topic-based subscriptions](../modules/topic-based-subscriptions/fhir-topic-based-subscriptions.md) treat `backport-filter-criteria`. + * Improved the [AccessPolicy debug tool](../tutorials/security-access-control-tutorials/debug-access-control.md) in Aidbox UI. + * Improved support for the [`X-Original-Uri`](../api/rest-api/fhir-search/README.md) HTTP header. + * Added the [`execution-type`](../configuration/migrations.md#run-sql-outside-a-transaction) parameter to `AidboxMigration`, which runs the migration SQL outside a transaction. Use it for statements PostgreSQL forbids inside a transaction block, such as `CREATE INDEX CONCURRENTLY`. + * Bug fixes in the [Aidbox Metrics server](../modules/observability/metrics/monitoring/use-aidbox-metrics-server.md). + + **Changes and deprecations** + + * Removed the `/fhir/FHIRSchema` endpoint. Define FHIR profiles with [StructureDefinition](../modules/profiling-and-validation/fhir-schema-validator/README.md) resources instead. + +## June 2026 _`stable, 2606`_ * Aidbox FHIR server @@ -37,7 +64,7 @@ description: >- * **Legacy engine cleanup** — removed Zen validation and the Entity/Attribute framework, the legacy FHIR Terminology Repository (FTR), and the Zen-based repository, indexes, and search implementation. [FHIR Schema validation](../modules/profiling-and-validation/fhir-schema-validator/README.md) is now the only validation mode. The `BOX_FHIR_SCHEMA_VALIDATION` setting has been removed and has no effect — validation can no longer be disabled. -## May 2026 _`stable, 2605, LTS`_ +## May 2026 _`2605, LTS`_ * Aidbox FHIR server diff --git a/docs/tutorials/other-tutorials/how-to-run-sql-via-init-bundle.md b/docs/tutorials/other-tutorials/how-to-run-sql-via-init-bundle.md index a28ee6226..1d7161dc2 100644 --- a/docs/tutorials/other-tutorials/how-to-run-sql-via-init-bundle.md +++ b/docs/tutorials/other-tutorials/how-to-run-sql-via-init-bundle.md @@ -8,18 +8,20 @@ description: >- ## Objectives * Run SQL statements via [Init Bundle](../../configuration/init-bundle.md) -* Make it sage - don't run it on each Aidbox startup but instead do it **exactly once**. +* Make it safe: don't run it on each Aidbox startup but instead do it **exactly once**. ## Before you begin -* Make sure your Aidbox version is 2602 or newer +* Make sure your Aidbox version is 2607 or newer * Setup the local Aidbox instance using getting started [guide](../../getting-started/run-aidbox-locally.md) ## Using init bundle to run SQL statements Init bundle allows you to automatically execute a bundle of resources on Aidbox startup. -The following example shows how to use `AidboxMigration' resource to call an API for executing SQL statements **exactly once**. +The following example shows how to use `AidboxMigration` resource to call an API for executing SQL statements **exactly once**. + +The example creates an index with `CREATE INDEX CONCURRENTLY`. A plain `CREATE INDEX` takes a lock that blocks writes to the table until the build finishes, which on a large table means downtime. `CONCURRENTLY` builds the index while writes keep going. PostgreSQL forbids it inside a transaction block, so the bundle type is `batch` and the migration carries `execution-type: not-in-transaction`. See [Run SQL outside a transaction](../../configuration/migrations.md#run-sql-outside-a-transaction) for the details. 1. Create a new file for the Init Bundle. @@ -30,7 +32,7 @@ touch init-bundle.json paste the following content into the file: ```json { - "type": "transaction", + "type": "batch", "resourceType": "Bundle", "entry": [ { @@ -46,7 +48,11 @@ paste the following content into the file: "parameter": [ { "name": "sql", - "valueString": "CREATE INDEX IF NOT EXISTS encounter_subject_id ON encounter ((resource #>> '{ subject, id }'));" + "valueString": "CREATE INDEX CONCURRENTLY IF NOT EXISTS encounter_subject_id ON encounter ((resource #>> '{ subject, id }'));" + }, + { + "name": "execution-type", + "valueCode": "not-in-transaction" } ], "resourceType": "Parameters" @@ -85,3 +91,21 @@ SELECT FROM pg_indexes WHERE tablename = 'encounter'; ``` + +5. Check that the index is valid. + +```sql +SELECT c.relname, i.indisvalid +FROM pg_index i +JOIN pg_class c ON c.oid = i.indexrelid +WHERE c.relname = 'encounter_subject_id'; +``` + + +{% hint style="info" %} +A failed entry in a `batch` init bundle logs a warning and Aidbox continues starting, unlike a `transaction` init bundle, which blocks startup. Check the startup log and the index validity above to confirm the migration did what you expected. +{% endhint %} + +## Running SQL inside a transaction + +Aidbox wraps migration SQL in a transaction unless you set `execution-type` to `not-in-transaction`. Drop that parameter and use `"type": "transaction"` for the bundle when you want the statement rolled back on failure, which suits statements PostgreSQL allows inside a transaction block: `CREATE TABLE`, `INSERT` for seed data. From ca0721cec5e2546f31af4bd3640afe7c7f609906 Mon Sep 17 00:00:00 2001 From: Aleksandr Kislitsyn Date: Thu, 6 Aug 2026 17:05:20 +0200 Subject: [PATCH 2/3] Add Zen seed and C-CDA removals to 2607 changes and deprecations * Zen seed: sansara c95e50057a removes the seed/seed-v2 engines and migrates default OAuth2/SMART scopes to auth module resources; f9bcd85aa0 drops the SeedImport resource type. * C-CDA: sansara 32be24501 (merged in 085fb373e) removes the CCDA module, closing sansara#6558. The converter now ships separately. Co-Authored-By: Claude Opus 5 (1M context) --- docs/overview/release-notes.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/overview/release-notes.md b/docs/overview/release-notes.md index 30b1f00a0..b4302ee22 100644 --- a/docs/overview/release-notes.md +++ b/docs/overview/release-notes.md @@ -34,6 +34,8 @@ description: >- **Changes and deprecations** * Removed the `/fhir/FHIRSchema` endpoint. Define FHIR profiles with [StructureDefinition](../modules/profiling-and-validation/fhir-schema-validator/README.md) resources instead. + * **Zen seed removal** — removed the Zen `seed` and `seed-v2` engines along with the `SeedImport` resource type. Load configuration resources at startup with [Init Bundle](../configuration/init-bundle.md) instead. The default OAuth2 and SMART scopes now ship as built-in auth module resources, so Aidbox no longer seeds them. Existing `seedimport` tables are left in place, since they hold only seed bookkeeping. + * **C-CDA converter moved out of Aidbox** — the built-in C-CDA / FHIR converter module is removed, and the `/ccda/*` endpoints (`to-fhir`, `to-ccda`, `persist`, `validate`, `fhir-validate`) are no longer served by Aidbox. The converter now ships separately, with its own release cycle, and runs as a standalone container or an Aidbox app. ## June 2026 _`stable, 2606`_ From 103b6821d6d82492b0c37e5030282505af8b506c Mon Sep 17 00:00:00 2001 From: Aleksandr Kislitsyn Date: Thu, 6 Aug 2026 17:12:21 +0200 Subject: [PATCH 3/3] rn update --- docs/overview/release-notes.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/overview/release-notes.md b/docs/overview/release-notes.md index b4302ee22..7117601fc 100644 --- a/docs/overview/release-notes.md +++ b/docs/overview/release-notes.md @@ -34,8 +34,8 @@ description: >- **Changes and deprecations** * Removed the `/fhir/FHIRSchema` endpoint. Define FHIR profiles with [StructureDefinition](../modules/profiling-and-validation/fhir-schema-validator/README.md) resources instead. - * **Zen seed removal** — removed the Zen `seed` and `seed-v2` engines along with the `SeedImport` resource type. Load configuration resources at startup with [Init Bundle](../configuration/init-bundle.md) instead. The default OAuth2 and SMART scopes now ship as built-in auth module resources, so Aidbox no longer seeds them. Existing `seedimport` tables are left in place, since they hold only seed bookkeeping. - * **C-CDA converter moved out of Aidbox** — the built-in C-CDA / FHIR converter module is removed, and the `/ccda/*` endpoints (`to-fhir`, `to-ccda`, `persist`, `validate`, `fhir-validate`) are no longer served by Aidbox. The converter now ships separately, with its own release cycle, and runs as a standalone container or an Aidbox app. + * **Zen seed removal** — removed the Zen `seed` and `seed-v2` engines along with the `SeedImport` resource type. Load configuration resources at startup with [Init Bundle](../configuration/init-bundle.md) instead. + * **C-CDA converter moved out of Aidbox** — the built-in C-CDA / FHIR converter module is removed, and the `/ccda/*` endpoints (`to-fhir`, `to-ccda`, `persist`, `validate`, `fhir-validate`) are no longer served by Aidbox. Use [Interbox](https://www.health-samurai.io/docs/interbox) for running the converter. ## June 2026 _`stable, 2606`_ @@ -682,7 +682,7 @@ Minor updates: * [Added the SDC config resource for general settings](https://www.health-samurai.io/docs/formbox) * [Provided the ability to restrict the type of attached file](https://www.health-samurai.io/docs/formbox) * Supported cqf-expression to provide prefilling the value in the display field - * [Integrated the Termbox server with Aidbox Forms for using external terminologies](https://www.health-samurai.io/docs/formbox) + * [Integrated the Termbox server with Aidbox Forms for using external terminologies](https://www.health-samurai.io/docs/formbox) * [Added an ability to embed forms as web-component](https://www.health-samurai.io/docs/formbox) * [Provided the ability to display an attached image on the form](https://www.health-samurai.io/docs/formbox) * [C-CDA / FHIR converter](../modules/integration-toolkit/ccda-converter/)