diff --git a/docs/assets/source-datastores/add-datastores/connections/multi-schema/overview/overview.png b/docs/assets/source-datastores/add-datastores/connections/multi-schema/overview/overview.png new file mode 100644 index 0000000000..5da5b53ac0 Binary files /dev/null and b/docs/assets/source-datastores/add-datastores/connections/multi-schema/overview/overview.png differ diff --git a/docs/assets/source-datastores/external-scan/container-3.png b/docs/assets/source-datastores/external-scan/container-3.png deleted file mode 100644 index fd94a9fa64..0000000000 Binary files a/docs/assets/source-datastores/external-scan/container-3.png and /dev/null differ diff --git a/docs/assets/source-datastores/external-scan/datastore-1.png b/docs/assets/source-datastores/external-scan/datastore-1.png deleted file mode 100644 index 1f152f60aa..0000000000 Binary files a/docs/assets/source-datastores/external-scan/datastore-1.png and /dev/null differ diff --git a/docs/assets/source-datastores/external-scan/external-file-5.png b/docs/assets/source-datastores/external-scan/external-file-5.png deleted file mode 100644 index 864758efeb..0000000000 Binary files a/docs/assets/source-datastores/external-scan/external-file-5.png and /dev/null differ diff --git a/docs/assets/source-datastores/external-scan/external-scan-4.png b/docs/assets/source-datastores/external-scan/external-scan-4.png deleted file mode 100644 index 0bc3d04c15..0000000000 Binary files a/docs/assets/source-datastores/external-scan/external-scan-4.png and /dev/null differ diff --git a/docs/assets/source-datastores/external-scan/success-6.png b/docs/assets/source-datastores/external-scan/success-6.png deleted file mode 100644 index 4f56a27d3b..0000000000 Binary files a/docs/assets/source-datastores/external-scan/success-6.png and /dev/null differ diff --git a/docs/assets/source-datastores/external-scan/tables-2.png b/docs/assets/source-datastores/external-scan/tables-2.png deleted file mode 100644 index 5673508617..0000000000 Binary files a/docs/assets/source-datastores/external-scan/tables-2.png and /dev/null differ diff --git a/docs/assets/source-datastores/navigation-and-organization/assign-tags/action-4.png b/docs/assets/source-datastores/managing-datastores/assign-tags/action-4.png similarity index 100% rename from docs/assets/source-datastores/navigation-and-organization/assign-tags/action-4.png rename to docs/assets/source-datastores/managing-datastores/assign-tags/action-4.png diff --git a/docs/assets/source-datastores/navigation-and-organization/assign-tags/add-datastore-1.png b/docs/assets/source-datastores/managing-datastores/assign-tags/add-datastore-1.png similarity index 100% rename from docs/assets/source-datastores/navigation-and-organization/assign-tags/add-datastore-1.png rename to docs/assets/source-datastores/managing-datastores/assign-tags/add-datastore-1.png diff --git a/docs/assets/source-datastores/navigation-and-organization/assign-tags/assign-tag-2.png b/docs/assets/source-datastores/managing-datastores/assign-tags/assign-tag-2.png similarity index 100% rename from docs/assets/source-datastores/navigation-and-organization/assign-tags/assign-tag-2.png rename to docs/assets/source-datastores/managing-datastores/assign-tags/assign-tag-2.png diff --git a/docs/assets/source-datastores/navigation-and-organization/assign-tags/modal-window-5.png b/docs/assets/source-datastores/managing-datastores/assign-tags/modal-window-5.png similarity index 100% rename from docs/assets/source-datastores/navigation-and-organization/assign-tags/modal-window-5.png rename to docs/assets/source-datastores/managing-datastores/assign-tags/modal-window-5.png diff --git a/docs/assets/source-datastores/navigation-and-organization/assign-tags/select-tag-3.png b/docs/assets/source-datastores/managing-datastores/assign-tags/select-tag-3.png similarity index 100% rename from docs/assets/source-datastores/navigation-and-organization/assign-tags/select-tag-3.png rename to docs/assets/source-datastores/managing-datastores/assign-tags/select-tag-3.png diff --git a/docs/assets/source-datastores/navigation-and-organization/assign-tags/tag-snowflake-6.png b/docs/assets/source-datastores/managing-datastores/assign-tags/tag-snowflake-6.png similarity index 100% rename from docs/assets/source-datastores/navigation-and-organization/assign-tags/tag-snowflake-6.png rename to docs/assets/source-datastores/managing-datastores/assign-tags/tag-snowflake-6.png diff --git a/docs/assets/source-datastores/datastore-settings/delete-datastore/confirm-delete-24.png b/docs/assets/source-datastores/managing-datastores/delete-datastore/confirm-delete-24.png similarity index 100% rename from docs/assets/source-datastores/datastore-settings/delete-datastore/confirm-delete-24.png rename to docs/assets/source-datastores/managing-datastores/delete-datastore/confirm-delete-24.png diff --git a/docs/assets/source-datastores/datastore-settings/delete-datastore/delete-22.png b/docs/assets/source-datastores/managing-datastores/delete-datastore/delete-22.png similarity index 100% rename from docs/assets/source-datastores/datastore-settings/delete-datastore/delete-22.png rename to docs/assets/source-datastores/managing-datastores/delete-datastore/delete-22.png diff --git a/docs/assets/source-datastores/datastore-settings/delete-datastore/delete-datastore-23.png b/docs/assets/source-datastores/managing-datastores/delete-datastore/delete-datastore-23.png similarity index 100% rename from docs/assets/source-datastores/datastore-settings/delete-datastore/delete-datastore-23.png rename to docs/assets/source-datastores/managing-datastores/delete-datastore/delete-datastore-23.png diff --git a/docs/assets/source-datastores/datastore-settings/edit-datastore/connection-details-4.png b/docs/assets/source-datastores/managing-datastores/edit-datastore/connection-details-4.png similarity index 100% rename from docs/assets/source-datastores/datastore-settings/edit-datastore/connection-details-4.png rename to docs/assets/source-datastores/managing-datastores/edit-datastore/connection-details-4.png diff --git a/docs/assets/source-datastores/datastore-settings/edit-datastore/edit-datastore-3.png b/docs/assets/source-datastores/managing-datastores/edit-datastore/edit-datastore-3.png similarity index 100% rename from docs/assets/source-datastores/datastore-settings/edit-datastore/edit-datastore-3.png rename to docs/assets/source-datastores/managing-datastores/edit-datastore/edit-datastore-3.png diff --git a/docs/assets/source-datastores/datastore-settings/edit-datastore/save-datastore-5.png b/docs/assets/source-datastores/managing-datastores/edit-datastore/save-datastore-5.png similarity index 100% rename from docs/assets/source-datastores/datastore-settings/edit-datastore/save-datastore-5.png rename to docs/assets/source-datastores/managing-datastores/edit-datastore/save-datastore-5.png diff --git a/docs/assets/source-datastores/datastore-settings/edit-datastore/test.png b/docs/assets/source-datastores/managing-datastores/edit-datastore/test.png similarity index 100% rename from docs/assets/source-datastores/datastore-settings/edit-datastore/test.png rename to docs/assets/source-datastores/managing-datastores/edit-datastore/test.png diff --git a/docs/assets/source-datastores/navigation-and-organization/grouping/grouping-overview.png b/docs/assets/source-datastores/managing-datastores/grouping/grouping-overview.png similarity index 100% rename from docs/assets/source-datastores/navigation-and-organization/grouping/grouping-overview.png rename to docs/assets/source-datastores/managing-datastores/grouping/grouping-overview.png diff --git a/docs/assets/source-datastores/navigation-and-organization/grouping/managing-groups/assign-a-datastore/assign-dropdown.png b/docs/assets/source-datastores/managing-datastores/grouping/managing-groups/assign-a-datastore/assign-dropdown.png similarity index 100% rename from docs/assets/source-datastores/navigation-and-organization/grouping/managing-groups/assign-a-datastore/assign-dropdown.png rename to docs/assets/source-datastores/managing-datastores/grouping/managing-groups/assign-a-datastore/assign-dropdown.png diff --git a/docs/assets/source-datastores/navigation-and-organization/grouping/managing-groups/assign-a-datastore/assign-icon.png b/docs/assets/source-datastores/managing-datastores/grouping/managing-groups/assign-a-datastore/assign-icon.png similarity index 100% rename from docs/assets/source-datastores/navigation-and-organization/grouping/managing-groups/assign-a-datastore/assign-icon.png rename to docs/assets/source-datastores/managing-datastores/grouping/managing-groups/assign-a-datastore/assign-icon.png diff --git a/docs/assets/source-datastores/navigation-and-organization/grouping/managing-groups/assign-a-datastore/datastore-assigned.png b/docs/assets/source-datastores/managing-datastores/grouping/managing-groups/assign-a-datastore/datastore-assigned.png similarity index 100% rename from docs/assets/source-datastores/navigation-and-organization/grouping/managing-groups/assign-a-datastore/datastore-assigned.png rename to docs/assets/source-datastores/managing-datastores/grouping/managing-groups/assign-a-datastore/datastore-assigned.png diff --git a/docs/assets/source-datastores/navigation-and-organization/grouping/managing-groups/assign-a-datastore/datastore-in-group.png b/docs/assets/source-datastores/managing-datastores/grouping/managing-groups/assign-a-datastore/datastore-in-group.png similarity index 100% rename from docs/assets/source-datastores/navigation-and-organization/grouping/managing-groups/assign-a-datastore/datastore-in-group.png rename to docs/assets/source-datastores/managing-datastores/grouping/managing-groups/assign-a-datastore/datastore-in-group.png diff --git a/docs/assets/source-datastores/navigation-and-organization/grouping/managing-groups/create-a-group/create-button.png b/docs/assets/source-datastores/managing-datastores/grouping/managing-groups/create-a-group/create-button.png similarity index 100% rename from docs/assets/source-datastores/navigation-and-organization/grouping/managing-groups/create-a-group/create-button.png rename to docs/assets/source-datastores/managing-datastores/grouping/managing-groups/create-a-group/create-button.png diff --git a/docs/assets/source-datastores/navigation-and-organization/grouping/managing-groups/create-a-group/create-form.png b/docs/assets/source-datastores/managing-datastores/grouping/managing-groups/create-a-group/create-form.png similarity index 100% rename from docs/assets/source-datastores/navigation-and-organization/grouping/managing-groups/create-a-group/create-form.png rename to docs/assets/source-datastores/managing-datastores/grouping/managing-groups/create-a-group/create-form.png diff --git a/docs/assets/source-datastores/navigation-and-organization/grouping/managing-groups/create-a-group/group-created.png b/docs/assets/source-datastores/managing-datastores/grouping/managing-groups/create-a-group/group-created.png similarity index 100% rename from docs/assets/source-datastores/navigation-and-organization/grouping/managing-groups/create-a-group/group-created.png rename to docs/assets/source-datastores/managing-datastores/grouping/managing-groups/create-a-group/group-created.png diff --git a/docs/assets/source-datastores/navigation-and-organization/grouping/managing-groups/create-a-group/manage-groups-button.png b/docs/assets/source-datastores/managing-datastores/grouping/managing-groups/create-a-group/manage-groups-button.png similarity index 100% rename from docs/assets/source-datastores/navigation-and-organization/grouping/managing-groups/create-a-group/manage-groups-button.png rename to docs/assets/source-datastores/managing-datastores/grouping/managing-groups/create-a-group/manage-groups-button.png diff --git a/docs/assets/source-datastores/navigation-and-organization/grouping/managing-groups/delete-a-group/delete-button.png b/docs/assets/source-datastores/managing-datastores/grouping/managing-groups/delete-a-group/delete-button.png similarity index 100% rename from docs/assets/source-datastores/navigation-and-organization/grouping/managing-groups/delete-a-group/delete-button.png rename to docs/assets/source-datastores/managing-datastores/grouping/managing-groups/delete-a-group/delete-button.png diff --git a/docs/assets/source-datastores/navigation-and-organization/grouping/managing-groups/edit-a-group/edit-button.png b/docs/assets/source-datastores/managing-datastores/grouping/managing-groups/edit-a-group/edit-button.png similarity index 100% rename from docs/assets/source-datastores/navigation-and-organization/grouping/managing-groups/edit-a-group/edit-button.png rename to docs/assets/source-datastores/managing-datastores/grouping/managing-groups/edit-a-group/edit-button.png diff --git a/docs/assets/source-datastores/navigation-and-organization/grouping/managing-groups/edit-a-group/edit-form.png b/docs/assets/source-datastores/managing-datastores/grouping/managing-groups/edit-a-group/edit-form.png similarity index 100% rename from docs/assets/source-datastores/navigation-and-organization/grouping/managing-groups/edit-a-group/edit-form.png rename to docs/assets/source-datastores/managing-datastores/grouping/managing-groups/edit-a-group/edit-form.png diff --git a/docs/assets/source-datastores/navigation-and-organization/grouping/managing-groups/remove-a-datastore/assign-dropdown.png b/docs/assets/source-datastores/managing-datastores/grouping/managing-groups/remove-a-datastore/assign-dropdown.png similarity index 100% rename from docs/assets/source-datastores/navigation-and-organization/grouping/managing-groups/remove-a-datastore/assign-dropdown.png rename to docs/assets/source-datastores/managing-datastores/grouping/managing-groups/remove-a-datastore/assign-dropdown.png diff --git a/docs/assets/source-datastores/navigation-and-organization/grouping/managing-groups/remove-a-datastore/assign-icon.png b/docs/assets/source-datastores/managing-datastores/grouping/managing-groups/remove-a-datastore/assign-icon.png similarity index 100% rename from docs/assets/source-datastores/navigation-and-organization/grouping/managing-groups/remove-a-datastore/assign-icon.png rename to docs/assets/source-datastores/managing-datastores/grouping/managing-groups/remove-a-datastore/assign-icon.png diff --git a/docs/assets/source-datastores/navigation-and-organization/grouping/managing-groups/remove-a-datastore/datastore-ungrouped.png b/docs/assets/source-datastores/managing-datastores/grouping/managing-groups/remove-a-datastore/datastore-ungrouped.png similarity index 100% rename from docs/assets/source-datastores/navigation-and-organization/grouping/managing-groups/remove-a-datastore/datastore-ungrouped.png rename to docs/assets/source-datastores/managing-datastores/grouping/managing-groups/remove-a-datastore/datastore-ungrouped.png diff --git a/docs/assets/source-datastores/navigation-and-organization/grouping/managing-groups/remove-a-datastore/remove-group.png b/docs/assets/source-datastores/managing-datastores/grouping/managing-groups/remove-a-datastore/remove-group.png similarity index 100% rename from docs/assets/source-datastores/navigation-and-organization/grouping/managing-groups/remove-a-datastore/remove-group.png rename to docs/assets/source-datastores/managing-datastores/grouping/managing-groups/remove-a-datastore/remove-group.png diff --git a/docs/assets/source-datastores/datastore-settings/link-enrichment/caret-8.png b/docs/assets/source-datastores/managing-datastores/link-enrichment/caret-8.png similarity index 100% rename from docs/assets/source-datastores/datastore-settings/link-enrichment/caret-8.png rename to docs/assets/source-datastores/managing-datastores/link-enrichment/caret-8.png diff --git a/docs/assets/source-datastores/datastore-settings/link-enrichment/caret2.png b/docs/assets/source-datastores/managing-datastores/link-enrichment/caret2.png similarity index 100% rename from docs/assets/source-datastores/datastore-settings/link-enrichment/caret2.png rename to docs/assets/source-datastores/managing-datastores/link-enrichment/caret2.png diff --git a/docs/assets/source-datastores/datastore-settings/link-enrichment/connector-creds-10.png b/docs/assets/source-datastores/managing-datastores/link-enrichment/connector-creds-10.png similarity index 100% rename from docs/assets/source-datastores/datastore-settings/link-enrichment/connector-creds-10.png rename to docs/assets/source-datastores/managing-datastores/link-enrichment/connector-creds-10.png diff --git a/docs/assets/source-datastores/datastore-settings/link-enrichment/enrichment-6.png b/docs/assets/source-datastores/managing-datastores/link-enrichment/enrichment-6.png similarity index 100% rename from docs/assets/source-datastores/datastore-settings/link-enrichment/enrichment-6.png rename to docs/assets/source-datastores/managing-datastores/link-enrichment/enrichment-6.png diff --git a/docs/assets/source-datastores/datastore-settings/link-enrichment/enrichment-settings.png b/docs/assets/source-datastores/managing-datastores/link-enrichment/enrichment-settings.png similarity index 100% rename from docs/assets/source-datastores/datastore-settings/link-enrichment/enrichment-settings.png rename to docs/assets/source-datastores/managing-datastores/link-enrichment/enrichment-settings.png diff --git a/docs/assets/source-datastores/datastore-settings/link-enrichment/link-enrichment-7.png b/docs/assets/source-datastores/managing-datastores/link-enrichment/link-enrichment-7.png similarity index 100% rename from docs/assets/source-datastores/datastore-settings/link-enrichment/link-enrichment-7.png rename to docs/assets/source-datastores/managing-datastores/link-enrichment/link-enrichment-7.png diff --git a/docs/assets/source-datastores/datastore-settings/link-enrichment/modal.png b/docs/assets/source-datastores/managing-datastores/link-enrichment/modal.png similarity index 100% rename from docs/assets/source-datastores/datastore-settings/link-enrichment/modal.png rename to docs/assets/source-datastores/managing-datastores/link-enrichment/modal.png diff --git a/docs/assets/source-datastores/datastore-settings/link-enrichment/save-15.png b/docs/assets/source-datastores/managing-datastores/link-enrichment/save-15.png similarity index 100% rename from docs/assets/source-datastores/datastore-settings/link-enrichment/save-15.png rename to docs/assets/source-datastores/managing-datastores/link-enrichment/save-15.png diff --git a/docs/assets/source-datastores/datastore-settings/link-enrichment/save-enrichment-11.png b/docs/assets/source-datastores/managing-datastores/link-enrichment/save-enrichment-11.png similarity index 100% rename from docs/assets/source-datastores/datastore-settings/link-enrichment/save-enrichment-11.png rename to docs/assets/source-datastores/managing-datastores/link-enrichment/save-enrichment-11.png diff --git a/docs/assets/source-datastores/datastore-settings/link-enrichment/select-new-enrichment-9.png b/docs/assets/source-datastores/managing-datastores/link-enrichment/select-new-enrichment-9.png similarity index 100% rename from docs/assets/source-datastores/datastore-settings/link-enrichment/select-new-enrichment-9.png rename to docs/assets/source-datastores/managing-datastores/link-enrichment/select-new-enrichment-9.png diff --git a/docs/assets/source-datastores/datastore-settings/link-enrichment/window.png b/docs/assets/source-datastores/managing-datastores/link-enrichment/window.png similarity index 100% rename from docs/assets/source-datastores/datastore-settings/link-enrichment/window.png rename to docs/assets/source-datastores/managing-datastores/link-enrichment/window.png diff --git a/docs/assets/source-datastores/datastore-settings/quality-score-settings/decay-period-19.png b/docs/assets/source-datastores/managing-datastores/quality-score-settings/decay-period-19.png similarity index 100% rename from docs/assets/source-datastores/datastore-settings/quality-score-settings/decay-period-19.png rename to docs/assets/source-datastores/managing-datastores/quality-score-settings/decay-period-19.png diff --git a/docs/assets/source-datastores/datastore-settings/quality-score-settings/factor-weights-20.png b/docs/assets/source-datastores/managing-datastores/quality-score-settings/factor-weights-20.png similarity index 100% rename from docs/assets/source-datastores/datastore-settings/quality-score-settings/factor-weights-20.png rename to docs/assets/source-datastores/managing-datastores/quality-score-settings/factor-weights-20.png diff --git a/docs/assets/source-datastores/datastore-settings/quality-score-settings/score-17.png b/docs/assets/source-datastores/managing-datastores/quality-score-settings/score-17.png similarity index 100% rename from docs/assets/source-datastores/datastore-settings/quality-score-settings/score-17.png rename to docs/assets/source-datastores/managing-datastores/quality-score-settings/score-17.png diff --git a/docs/assets/source-datastores/datastore-settings/quality-score-settings/score-save-21.png b/docs/assets/source-datastores/managing-datastores/quality-score-settings/score-save-21.png similarity index 100% rename from docs/assets/source-datastores/datastore-settings/quality-score-settings/score-save-21.png rename to docs/assets/source-datastores/managing-datastores/quality-score-settings/score-save-21.png diff --git a/docs/assets/source-datastores/datastore-settings/quality-score-settings/score-settings-18.png b/docs/assets/source-datastores/managing-datastores/quality-score-settings/score-settings-18.png similarity index 100% rename from docs/assets/source-datastores/datastore-settings/quality-score-settings/score-settings-18.png rename to docs/assets/source-datastores/managing-datastores/quality-score-settings/score-settings-18.png diff --git a/docs/assets/source-datastores/navigation-and-organization/right-click-options/anomalies-10.png b/docs/assets/source-datastores/managing-datastores/right-click-options/anomalies-10.png similarity index 100% rename from docs/assets/source-datastores/navigation-and-organization/right-click-options/anomalies-10.png rename to docs/assets/source-datastores/managing-datastores/right-click-options/anomalies-10.png diff --git a/docs/assets/source-datastores/navigation-and-organization/right-click-options/anomalies-9.png b/docs/assets/source-datastores/managing-datastores/right-click-options/anomalies-9.png similarity index 100% rename from docs/assets/source-datastores/navigation-and-organization/right-click-options/anomalies-9.png rename to docs/assets/source-datastores/managing-datastores/right-click-options/anomalies-9.png diff --git a/docs/assets/source-datastores/navigation-and-organization/right-click-options/checks-7.png b/docs/assets/source-datastores/managing-datastores/right-click-options/checks-7.png similarity index 100% rename from docs/assets/source-datastores/navigation-and-organization/right-click-options/checks-7.png rename to docs/assets/source-datastores/managing-datastores/right-click-options/checks-7.png diff --git a/docs/assets/source-datastores/navigation-and-organization/right-click-options/checks-8.png b/docs/assets/source-datastores/managing-datastores/right-click-options/checks-8.png similarity index 100% rename from docs/assets/source-datastores/navigation-and-organization/right-click-options/checks-8.png rename to docs/assets/source-datastores/managing-datastores/right-click-options/checks-8.png diff --git a/docs/assets/source-datastores/navigation-and-organization/right-click-options/field-5.png b/docs/assets/source-datastores/managing-datastores/right-click-options/field-5.png similarity index 100% rename from docs/assets/source-datastores/navigation-and-organization/right-click-options/field-5.png rename to docs/assets/source-datastores/managing-datastores/right-click-options/field-5.png diff --git a/docs/assets/source-datastores/navigation-and-organization/right-click-options/field-6.png b/docs/assets/source-datastores/managing-datastores/right-click-options/field-6.png similarity index 100% rename from docs/assets/source-datastores/navigation-and-organization/right-click-options/field-6.png rename to docs/assets/source-datastores/managing-datastores/right-click-options/field-6.png diff --git a/docs/assets/source-datastores/navigation-and-organization/right-click-options/source-1.png b/docs/assets/source-datastores/managing-datastores/right-click-options/source-1.png similarity index 100% rename from docs/assets/source-datastores/navigation-and-organization/right-click-options/source-1.png rename to docs/assets/source-datastores/managing-datastores/right-click-options/source-1.png diff --git a/docs/assets/source-datastores/navigation-and-organization/right-click-options/source-2.png b/docs/assets/source-datastores/managing-datastores/right-click-options/source-2.png similarity index 100% rename from docs/assets/source-datastores/navigation-and-organization/right-click-options/source-2.png rename to docs/assets/source-datastores/managing-datastores/right-click-options/source-2.png diff --git a/docs/assets/source-datastores/navigation-and-organization/right-click-options/tables-files-3.png b/docs/assets/source-datastores/managing-datastores/right-click-options/tables-files-3.png similarity index 100% rename from docs/assets/source-datastores/navigation-and-organization/right-click-options/tables-files-3.png rename to docs/assets/source-datastores/managing-datastores/right-click-options/tables-files-3.png diff --git a/docs/assets/source-datastores/navigation-and-organization/right-click-options/tables-files-4.png b/docs/assets/source-datastores/managing-datastores/right-click-options/tables-files-4.png similarity index 100% rename from docs/assets/source-datastores/navigation-and-organization/right-click-options/tables-files-4.png rename to docs/assets/source-datastores/managing-datastores/right-click-options/tables-files-4.png diff --git a/docs/assets/source-datastores/datastore-settings/settings-overview/add-datastore-1.png b/docs/assets/source-datastores/managing-datastores/settings-overview/add-datastore-1.png similarity index 100% rename from docs/assets/source-datastores/datastore-settings/settings-overview/add-datastore-1.png rename to docs/assets/source-datastores/managing-datastores/settings-overview/add-datastore-1.png diff --git a/docs/assets/source-datastores/datastore-settings/settings-overview/settings-2.png b/docs/assets/source-datastores/managing-datastores/settings-overview/settings-2.png similarity index 100% rename from docs/assets/source-datastores/datastore-settings/settings-overview/settings-2.png rename to docs/assets/source-datastores/managing-datastores/settings-overview/settings-2.png diff --git a/docs/assets/source-datastores/datastore-settings/unlink-enrichment/enrichment-6.png b/docs/assets/source-datastores/managing-datastores/unlink-enrichment/enrichment-6.png similarity index 100% rename from docs/assets/source-datastores/datastore-settings/unlink-enrichment/enrichment-6.png rename to docs/assets/source-datastores/managing-datastores/unlink-enrichment/enrichment-6.png diff --git a/docs/assets/source-datastores/datastore-settings/unlink-enrichment/enrichment-settings.png b/docs/assets/source-datastores/managing-datastores/unlink-enrichment/enrichment-settings.png similarity index 100% rename from docs/assets/source-datastores/datastore-settings/unlink-enrichment/enrichment-settings.png rename to docs/assets/source-datastores/managing-datastores/unlink-enrichment/enrichment-settings.png diff --git a/docs/assets/source-datastores/datastore-settings/unlink-enrichment/unlink-option.png b/docs/assets/source-datastores/managing-datastores/unlink-enrichment/unlink-option.png similarity index 100% rename from docs/assets/source-datastores/datastore-settings/unlink-enrichment/unlink-option.png rename to docs/assets/source-datastores/managing-datastores/unlink-enrichment/unlink-option.png diff --git a/docs/assets/source-datastores/datastore-settings/unlink-enrichment/unlink-window.png b/docs/assets/source-datastores/managing-datastores/unlink-enrichment/unlink-window.png similarity index 100% rename from docs/assets/source-datastores/datastore-settings/unlink-enrichment/unlink-window.png rename to docs/assets/source-datastores/managing-datastores/unlink-enrichment/unlink-window.png diff --git a/docs/assets/source-datastores/datastore-settings/unlink-enrichment/unlink.png b/docs/assets/source-datastores/managing-datastores/unlink-enrichment/unlink.png similarity index 100% rename from docs/assets/source-datastores/datastore-settings/unlink-enrichment/unlink.png rename to docs/assets/source-datastores/managing-datastores/unlink-enrichment/unlink.png diff --git a/docs/assets/source-datastores/profile/activity-fields.png b/docs/assets/source-datastores/operations/profile/activity-fields.png similarity index 100% rename from docs/assets/source-datastores/profile/activity-fields.png rename to docs/assets/source-datastores/operations/profile/activity-fields.png diff --git a/docs/assets/source-datastore/profile/activity-heatmap.png b/docs/assets/source-datastores/operations/profile/activity-heatmap.png similarity index 100% rename from docs/assets/source-datastore/profile/activity-heatmap.png rename to docs/assets/source-datastores/operations/profile/activity-heatmap.png diff --git a/docs/assets/source-datastores/profile/activity-table.png b/docs/assets/source-datastores/operations/profile/activity-table.png similarity index 100% rename from docs/assets/source-datastores/profile/activity-table.png rename to docs/assets/source-datastores/operations/profile/activity-table.png diff --git a/docs/assets/source-datastore/profile/activity-top-panel.png b/docs/assets/source-datastores/operations/profile/activity-top-panel.png similarity index 100% rename from docs/assets/source-datastore/profile/activity-top-panel.png rename to docs/assets/source-datastores/operations/profile/activity-top-panel.png diff --git a/docs/assets/source-datastores/profile/adding-source-datastore.png b/docs/assets/source-datastores/operations/profile/adding-source-datastore.png similarity index 100% rename from docs/assets/source-datastores/profile/adding-source-datastore.png rename to docs/assets/source-datastores/operations/profile/adding-source-datastore.png diff --git a/docs/assets/source-datastores/profile/advanced.png b/docs/assets/source-datastores/operations/profile/advanced.png similarity index 100% rename from docs/assets/source-datastores/profile/advanced.png rename to docs/assets/source-datastores/operations/profile/advanced.png diff --git a/docs/assets/source-datastores/profile/daily.png b/docs/assets/source-datastores/operations/profile/daily.png similarity index 100% rename from docs/assets/source-datastores/profile/daily.png rename to docs/assets/source-datastores/operations/profile/daily.png diff --git a/docs/assets/source-datastores/profile/default-configuration.png b/docs/assets/source-datastores/operations/profile/default-configuration.png similarity index 100% rename from docs/assets/source-datastores/profile/default-configuration.png rename to docs/assets/source-datastores/operations/profile/default-configuration.png diff --git a/docs/assets/source-datastores/profile/histogram.png b/docs/assets/source-datastores/operations/profile/histogram.png similarity index 100% rename from docs/assets/source-datastores/profile/histogram.png rename to docs/assets/source-datastores/operations/profile/histogram.png diff --git a/docs/assets/source-datastores/profile/hourly.png b/docs/assets/source-datastores/operations/profile/hourly.png similarity index 100% rename from docs/assets/source-datastores/profile/hourly.png rename to docs/assets/source-datastores/operations/profile/hourly.png diff --git a/docs/assets/source-datastore/profile/infer-as-draft.png b/docs/assets/source-datastores/operations/profile/infer-as-draft.png similarity index 100% rename from docs/assets/source-datastore/profile/infer-as-draft.png rename to docs/assets/source-datastores/operations/profile/infer-as-draft.png diff --git a/docs/assets/source-datastores/profile/inference-state.png b/docs/assets/source-datastores/operations/profile/inference-state.png similarity index 100% rename from docs/assets/source-datastores/profile/inference-state.png rename to docs/assets/source-datastores/operations/profile/inference-state.png diff --git a/docs/assets/source-datastore/profile/inference-threshold-default.png b/docs/assets/source-datastores/operations/profile/inference-threshold-default.png similarity index 100% rename from docs/assets/source-datastore/profile/inference-threshold-default.png rename to docs/assets/source-datastores/operations/profile/inference-threshold-default.png diff --git a/docs/assets/source-datastore/profile/inference-threshold-level0.png b/docs/assets/source-datastores/operations/profile/inference-threshold-level0.png similarity index 100% rename from docs/assets/source-datastore/profile/inference-threshold-level0.png rename to docs/assets/source-datastores/operations/profile/inference-threshold-level0.png diff --git a/docs/assets/source-datastore/profile/inference-threshold-level1.png b/docs/assets/source-datastores/operations/profile/inference-threshold-level1.png similarity index 100% rename from docs/assets/source-datastore/profile/inference-threshold-level1.png rename to docs/assets/source-datastores/operations/profile/inference-threshold-level1.png diff --git a/docs/assets/source-datastore/profile/inference-threshold-level2.png b/docs/assets/source-datastores/operations/profile/inference-threshold-level2.png similarity index 100% rename from docs/assets/source-datastore/profile/inference-threshold-level2.png rename to docs/assets/source-datastores/operations/profile/inference-threshold-level2.png diff --git a/docs/assets/source-datastore/profile/inference-threshold-level3.png b/docs/assets/source-datastores/operations/profile/inference-threshold-level3.png similarity index 100% rename from docs/assets/source-datastore/profile/inference-threshold-level3.png rename to docs/assets/source-datastores/operations/profile/inference-threshold-level3.png diff --git a/docs/assets/source-datastore/profile/inference-threshold-level4.png b/docs/assets/source-datastores/operations/profile/inference-threshold-level4.png similarity index 100% rename from docs/assets/source-datastore/profile/inference-threshold-level4.png rename to docs/assets/source-datastores/operations/profile/inference-threshold-level4.png diff --git a/docs/assets/source-datastore/profile/inference-threshold-level5.png b/docs/assets/source-datastores/operations/profile/inference-threshold-level5.png similarity index 100% rename from docs/assets/source-datastore/profile/inference-threshold-level5.png rename to docs/assets/source-datastores/operations/profile/inference-threshold-level5.png diff --git a/docs/assets/source-datastore/profile/initiate-profile-button.png b/docs/assets/source-datastores/operations/profile/initiate-profile-button.png similarity index 100% rename from docs/assets/source-datastore/profile/initiate-profile-button.png rename to docs/assets/source-datastores/operations/profile/initiate-profile-button.png diff --git a/docs/assets/source-datastores/profile/monthly.png b/docs/assets/source-datastores/operations/profile/monthly.png similarity index 100% rename from docs/assets/source-datastores/profile/monthly.png rename to docs/assets/source-datastores/operations/profile/monthly.png diff --git a/docs/assets/source-datastore/profile/next-inference-settings.png b/docs/assets/source-datastores/operations/profile/next-inference-settings.png similarity index 100% rename from docs/assets/source-datastore/profile/next-inference-settings.png rename to docs/assets/source-datastores/operations/profile/next-inference-settings.png diff --git a/docs/assets/source-datastore/profile/next-operation-settings.png b/docs/assets/source-datastores/operations/profile/next-operation-settings.png similarity index 100% rename from docs/assets/source-datastore/profile/next-operation-settings.png rename to docs/assets/source-datastores/operations/profile/next-operation-settings.png diff --git a/docs/assets/source-datastores/profile/next.png b/docs/assets/source-datastores/operations/profile/next.png similarity index 100% rename from docs/assets/source-datastores/profile/next.png rename to docs/assets/source-datastores/operations/profile/next.png diff --git a/docs/assets/source-datastore/profile/operation-metrics-hover.png b/docs/assets/source-datastores/operations/profile/operation-metrics-hover.png similarity index 100% rename from docs/assets/source-datastore/profile/operation-metrics-hover.png rename to docs/assets/source-datastores/operations/profile/operation-metrics-hover.png diff --git a/docs/assets/source-datastore/profile/operation-status-aborted.png b/docs/assets/source-datastores/operations/profile/operation-status-aborted.png similarity index 100% rename from docs/assets/source-datastore/profile/operation-status-aborted.png rename to docs/assets/source-datastores/operations/profile/operation-status-aborted.png diff --git a/docs/assets/source-datastore/profile/operation-status-failure.png b/docs/assets/source-datastores/operations/profile/operation-status-failure.png similarity index 100% rename from docs/assets/source-datastore/profile/operation-status-failure.png rename to docs/assets/source-datastores/operations/profile/operation-status-failure.png diff --git a/docs/assets/source-datastore/profile/operation-status-running.png b/docs/assets/source-datastores/operations/profile/operation-status-running.png similarity index 100% rename from docs/assets/source-datastore/profile/operation-status-running.png rename to docs/assets/source-datastores/operations/profile/operation-status-running.png diff --git a/docs/assets/source-datastore/profile/operation-status-success-warning.png b/docs/assets/source-datastores/operations/profile/operation-status-success-warning.png similarity index 100% rename from docs/assets/source-datastore/profile/operation-status-success-warning.png rename to docs/assets/source-datastores/operations/profile/operation-status-success-warning.png diff --git a/docs/assets/source-datastore/profile/operation-status-success.png b/docs/assets/source-datastores/operations/profile/operation-status-success.png similarity index 100% rename from docs/assets/source-datastore/profile/operation-status-success.png rename to docs/assets/source-datastores/operations/profile/operation-status-success.png diff --git a/docs/assets/source-datastores/profile/profile-aborted.png b/docs/assets/source-datastores/operations/profile/profile-aborted.png similarity index 100% rename from docs/assets/source-datastores/profile/profile-aborted.png rename to docs/assets/source-datastores/operations/profile/profile-aborted.png diff --git a/docs/assets/source-datastores/profile/profile-operation-all.png b/docs/assets/source-datastores/operations/profile/profile-operation-all.png similarity index 100% rename from docs/assets/source-datastores/profile/profile-operation-all.png rename to docs/assets/source-datastores/operations/profile/profile-operation-all.png diff --git a/docs/assets/source-datastores/profile/profile-operation-next.png b/docs/assets/source-datastores/operations/profile/profile-operation-next.png similarity index 100% rename from docs/assets/source-datastores/profile/profile-operation-next.png rename to docs/assets/source-datastores/operations/profile/profile-operation-next.png diff --git a/docs/assets/source-datastores/profile/profile-operation-schedule.png b/docs/assets/source-datastores/operations/profile/profile-operation-schedule.png similarity index 100% rename from docs/assets/source-datastores/profile/profile-operation-schedule.png rename to docs/assets/source-datastores/operations/profile/profile-operation-schedule.png diff --git a/docs/assets/source-datastores/profile/profile-operation-specific.png b/docs/assets/source-datastores/operations/profile/profile-operation-specific.png similarity index 100% rename from docs/assets/source-datastores/profile/profile-operation-specific.png rename to docs/assets/source-datastores/operations/profile/profile-operation-specific.png diff --git a/docs/assets/source-datastores/profile/profile-operation-tag.png b/docs/assets/source-datastores/operations/profile/profile-operation-tag.png similarity index 100% rename from docs/assets/source-datastores/profile/profile-operation-tag.png rename to docs/assets/source-datastores/operations/profile/profile-operation-tag.png diff --git a/docs/assets/source-datastores/profile/profile-result.png b/docs/assets/source-datastores/operations/profile/profile-result.png similarity index 100% rename from docs/assets/source-datastores/profile/profile-result.png rename to docs/assets/source-datastores/operations/profile/profile-result.png diff --git a/docs/assets/source-datastores/profile/profile-running.png b/docs/assets/source-datastores/operations/profile/profile-running.png similarity index 100% rename from docs/assets/source-datastores/profile/profile-running.png rename to docs/assets/source-datastores/operations/profile/profile-running.png diff --git a/docs/assets/source-datastores/profile/profile-scan-operation.png b/docs/assets/source-datastores/operations/profile/profile-scan-operation.png similarity index 100% rename from docs/assets/source-datastores/profile/profile-scan-operation.png rename to docs/assets/source-datastores/operations/profile/profile-scan-operation.png diff --git a/docs/assets/source-datastores/profile/profile-success.png b/docs/assets/source-datastores/operations/profile/profile-success.png similarity index 100% rename from docs/assets/source-datastores/profile/profile-success.png rename to docs/assets/source-datastores/operations/profile/profile-success.png diff --git a/docs/assets/source-datastores/profile/profile-warning.png b/docs/assets/source-datastores/operations/profile/profile-warning.png similarity index 100% rename from docs/assets/source-datastores/profile/profile-warning.png rename to docs/assets/source-datastores/operations/profile/profile-warning.png diff --git a/docs/assets/source-datastores/profile/profile.png b/docs/assets/source-datastores/operations/profile/profile.png similarity index 100% rename from docs/assets/source-datastores/profile/profile.png rename to docs/assets/source-datastores/operations/profile/profile.png diff --git a/docs/assets/source-datastores/profile/profiled.png b/docs/assets/source-datastores/operations/profile/profiled.png similarity index 100% rename from docs/assets/source-datastores/profile/profiled.png rename to docs/assets/source-datastores/operations/profile/profiled.png diff --git a/docs/assets/source-datastores/profile/record-dropdown.png b/docs/assets/source-datastores/operations/profile/record-dropdown.png similarity index 100% rename from docs/assets/source-datastores/profile/record-dropdown.png rename to docs/assets/source-datastores/operations/profile/record-dropdown.png diff --git a/docs/assets/source-datastore/profile/record-limit-dropdown.png b/docs/assets/source-datastores/operations/profile/record-limit-dropdown.png similarity index 100% rename from docs/assets/source-datastore/profile/record-limit-dropdown.png rename to docs/assets/source-datastores/operations/profile/record-limit-dropdown.png diff --git a/docs/assets/source-datastore/profile/record-limit-input.png b/docs/assets/source-datastores/operations/profile/record-limit-input.png similarity index 100% rename from docs/assets/source-datastore/profile/record-limit-input.png rename to docs/assets/source-datastores/operations/profile/record-limit-input.png diff --git a/docs/assets/source-datastores/profile/record-limit.png b/docs/assets/source-datastores/operations/profile/record-limit.png similarity index 100% rename from docs/assets/source-datastores/profile/record-limit.png rename to docs/assets/source-datastores/operations/profile/record-limit.png diff --git a/docs/assets/source-datastore/profile/results-button.png b/docs/assets/source-datastores/operations/profile/results-button.png similarity index 100% rename from docs/assets/source-datastore/profile/results-button.png rename to docs/assets/source-datastores/operations/profile/results-button.png diff --git a/docs/assets/source-datastore/profile/results-container-list.png b/docs/assets/source-datastores/operations/profile/results-container-list.png similarity index 100% rename from docs/assets/source-datastore/profile/results-container-list.png rename to docs/assets/source-datastores/operations/profile/results-container-list.png diff --git a/docs/assets/source-datastore/profile/results-expand-container.png b/docs/assets/source-datastores/operations/profile/results-expand-container.png similarity index 100% rename from docs/assets/source-datastore/profile/results-expand-container.png rename to docs/assets/source-datastores/operations/profile/results-expand-container.png diff --git a/docs/assets/source-datastore/profile/run-button-overview.png b/docs/assets/source-datastores/operations/profile/run-button-overview.png similarity index 100% rename from docs/assets/source-datastore/profile/run-button-overview.png rename to docs/assets/source-datastores/operations/profile/run-button-overview.png diff --git a/docs/assets/source-datastore/profile/run-now-button.png b/docs/assets/source-datastores/operations/profile/run-now-button.png similarity index 100% rename from docs/assets/source-datastore/profile/run-now-button.png rename to docs/assets/source-datastores/operations/profile/run-now-button.png diff --git a/docs/assets/source-datastores/profile/run-profile.png b/docs/assets/source-datastores/operations/profile/run-profile.png similarity index 100% rename from docs/assets/source-datastores/profile/run-profile.png rename to docs/assets/source-datastores/operations/profile/run-profile.png diff --git a/docs/assets/source-datastores/profile/run.png b/docs/assets/source-datastores/operations/profile/run.png similarity index 100% rename from docs/assets/source-datastores/profile/run.png rename to docs/assets/source-datastores/operations/profile/run.png diff --git a/docs/assets/source-datastores/profile/sandbox-tpch.png b/docs/assets/source-datastores/operations/profile/sandbox-tpch.png similarity index 100% rename from docs/assets/source-datastores/profile/sandbox-tpch.png rename to docs/assets/source-datastores/operations/profile/sandbox-tpch.png diff --git a/docs/assets/source-datastore/profile/schedule-advanced.png b/docs/assets/source-datastores/operations/profile/schedule-advanced.png similarity index 100% rename from docs/assets/source-datastore/profile/schedule-advanced.png rename to docs/assets/source-datastores/operations/profile/schedule-advanced.png diff --git a/docs/assets/source-datastore/profile/schedule-button.png b/docs/assets/source-datastores/operations/profile/schedule-button.png similarity index 100% rename from docs/assets/source-datastore/profile/schedule-button.png rename to docs/assets/source-datastores/operations/profile/schedule-button.png diff --git a/docs/assets/source-datastore/profile/schedule-confirm-button.png b/docs/assets/source-datastores/operations/profile/schedule-confirm-button.png similarity index 100% rename from docs/assets/source-datastore/profile/schedule-confirm-button.png rename to docs/assets/source-datastores/operations/profile/schedule-confirm-button.png diff --git a/docs/assets/source-datastore/profile/schedule-daily.png b/docs/assets/source-datastores/operations/profile/schedule-daily.png similarity index 100% rename from docs/assets/source-datastore/profile/schedule-daily.png rename to docs/assets/source-datastores/operations/profile/schedule-daily.png diff --git a/docs/assets/source-datastore/profile/schedule-hourly.png b/docs/assets/source-datastores/operations/profile/schedule-hourly.png similarity index 100% rename from docs/assets/source-datastore/profile/schedule-hourly.png rename to docs/assets/source-datastores/operations/profile/schedule-hourly.png diff --git a/docs/assets/source-datastore/profile/schedule-monthly.png b/docs/assets/source-datastores/operations/profile/schedule-monthly.png similarity index 100% rename from docs/assets/source-datastore/profile/schedule-monthly.png rename to docs/assets/source-datastores/operations/profile/schedule-monthly.png diff --git a/docs/assets/source-datastore/profile/schedule-name.png b/docs/assets/source-datastores/operations/profile/schedule-name.png similarity index 100% rename from docs/assets/source-datastore/profile/schedule-name.png rename to docs/assets/source-datastores/operations/profile/schedule-name.png diff --git a/docs/assets/source-datastore/profile/schedule-weekly.png b/docs/assets/source-datastores/operations/profile/schedule-weekly.png similarity index 100% rename from docs/assets/source-datastore/profile/schedule-weekly.png rename to docs/assets/source-datastores/operations/profile/schedule-weekly.png diff --git a/docs/assets/source-datastores/profile/schedule.png b/docs/assets/source-datastores/operations/profile/schedule.png similarity index 100% rename from docs/assets/source-datastores/profile/schedule.png rename to docs/assets/source-datastores/operations/profile/schedule.png diff --git a/docs/assets/source-datastore/profile/search-operation.png b/docs/assets/source-datastores/operations/profile/search-operation.png similarity index 100% rename from docs/assets/source-datastore/profile/search-operation.png rename to docs/assets/source-datastores/operations/profile/search-operation.png diff --git a/docs/assets/source-datastores/profile/search.png b/docs/assets/source-datastores/operations/profile/search.png similarity index 100% rename from docs/assets/source-datastores/profile/search.png rename to docs/assets/source-datastores/operations/profile/search.png diff --git a/docs/assets/source-datastore/profile/select-all-tables.png b/docs/assets/source-datastores/operations/profile/select-all-tables.png similarity index 100% rename from docs/assets/source-datastore/profile/select-all-tables.png rename to docs/assets/source-datastores/operations/profile/select-all-tables.png diff --git a/docs/assets/source-datastore/profile/select-by-tag.png b/docs/assets/source-datastores/operations/profile/select-by-tag.png similarity index 100% rename from docs/assets/source-datastore/profile/select-by-tag.png rename to docs/assets/source-datastores/operations/profile/select-by-tag.png diff --git a/docs/assets/source-datastore/profile/select-specific-tables.png b/docs/assets/source-datastores/operations/profile/select-specific-tables.png similarity index 100% rename from docs/assets/source-datastore/profile/select-specific-tables.png rename to docs/assets/source-datastores/operations/profile/select-specific-tables.png diff --git a/docs/assets/source-datastore/profile/starting-threshold.png b/docs/assets/source-datastores/operations/profile/starting-threshold.png similarity index 100% rename from docs/assets/source-datastore/profile/starting-threshold.png rename to docs/assets/source-datastores/operations/profile/starting-threshold.png diff --git a/docs/assets/source-datastores/profile/total-anomalies.png b/docs/assets/source-datastores/operations/profile/total-anomalies.png similarity index 100% rename from docs/assets/source-datastores/profile/total-anomalies.png rename to docs/assets/source-datastores/operations/profile/total-anomalies.png diff --git a/docs/assets/source-datastores/profile/totals-profile.png b/docs/assets/source-datastores/operations/profile/totals-profile.png similarity index 100% rename from docs/assets/source-datastores/profile/totals-profile.png rename to docs/assets/source-datastores/operations/profile/totals-profile.png diff --git a/docs/assets/source-datastores/profile/view-table.png b/docs/assets/source-datastores/operations/profile/view-table.png similarity index 100% rename from docs/assets/source-datastores/profile/view-table.png rename to docs/assets/source-datastores/operations/profile/view-table.png diff --git a/docs/assets/source-datastores/profile/weekly.png b/docs/assets/source-datastores/operations/profile/weekly.png similarity index 100% rename from docs/assets/source-datastores/profile/weekly.png rename to docs/assets/source-datastores/operations/profile/weekly.png diff --git a/docs/assets/source-datastore/sync/activity-heatmap.png b/docs/assets/source-datastores/operations/sync/activity-heatmap.png similarity index 100% rename from docs/assets/source-datastore/sync/activity-heatmap.png rename to docs/assets/source-datastores/operations/sync/activity-heatmap.png diff --git a/docs/assets/source-datastore/sync/activity-top-panel.png b/docs/assets/source-datastores/operations/sync/activity-top-panel.png similarity index 100% rename from docs/assets/source-datastore/sync/activity-top-panel.png rename to docs/assets/source-datastores/operations/sync/activity-top-panel.png diff --git a/docs/assets/source-datastore/sync/initiate-sync-checkbox.png b/docs/assets/source-datastores/operations/sync/initiate-sync-checkbox.png similarity index 100% rename from docs/assets/source-datastore/sync/initiate-sync-checkbox.png rename to docs/assets/source-datastores/operations/sync/initiate-sync-checkbox.png diff --git a/docs/assets/source-datastore/sync/operation-scheduled.png b/docs/assets/source-datastores/operations/sync/operation-scheduled.png similarity index 100% rename from docs/assets/source-datastore/sync/operation-scheduled.png rename to docs/assets/source-datastores/operations/sync/operation-scheduled.png diff --git a/docs/assets/source-datastore/sync/operation-status-aborted.png b/docs/assets/source-datastores/operations/sync/operation-status-aborted.png similarity index 100% rename from docs/assets/source-datastore/sync/operation-status-aborted.png rename to docs/assets/source-datastores/operations/sync/operation-status-aborted.png diff --git a/docs/assets/source-datastore/sync/operation-status-failure.png b/docs/assets/source-datastores/operations/sync/operation-status-failure.png similarity index 100% rename from docs/assets/source-datastore/sync/operation-status-failure.png rename to docs/assets/source-datastores/operations/sync/operation-status-failure.png diff --git a/docs/assets/source-datastore/sync/operation-status-running.png b/docs/assets/source-datastores/operations/sync/operation-status-running.png similarity index 100% rename from docs/assets/source-datastore/sync/operation-status-running.png rename to docs/assets/source-datastores/operations/sync/operation-status-running.png diff --git a/docs/assets/source-datastore/sync/operation-status-success.png b/docs/assets/source-datastores/operations/sync/operation-status-success.png similarity index 100% rename from docs/assets/source-datastore/sync/operation-status-success.png rename to docs/assets/source-datastores/operations/sync/operation-status-success.png diff --git a/docs/assets/source-datastore/sync/operation-status-warning.png b/docs/assets/source-datastores/operations/sync/operation-status-warning.png similarity index 100% rename from docs/assets/source-datastore/sync/operation-status-warning.png rename to docs/assets/source-datastores/operations/sync/operation-status-warning.png diff --git a/docs/assets/source-datastore/sync/operation-triggered.png b/docs/assets/source-datastores/operations/sync/operation-triggered.png similarity index 100% rename from docs/assets/source-datastore/sync/operation-triggered.png rename to docs/assets/source-datastores/operations/sync/operation-triggered.png diff --git a/docs/assets/source-datastore/sync/post-sync-container-fields.png b/docs/assets/source-datastores/operations/sync/post-sync-container-fields.png similarity index 100% rename from docs/assets/source-datastore/sync/post-sync-container-fields.png rename to docs/assets/source-datastores/operations/sync/post-sync-container-fields.png diff --git a/docs/assets/source-datastore/sync/post-sync-container-names.png b/docs/assets/source-datastores/operations/sync/post-sync-container-names.png similarity index 100% rename from docs/assets/source-datastore/sync/post-sync-container-names.png rename to docs/assets/source-datastores/operations/sync/post-sync-container-names.png diff --git a/docs/assets/source-datastore/sync/post-sync-table-settings.png b/docs/assets/source-datastores/operations/sync/post-sync-table-settings.png similarity index 100% rename from docs/assets/source-datastore/sync/post-sync-table-settings.png rename to docs/assets/source-datastores/operations/sync/post-sync-table-settings.png diff --git a/docs/assets/source-datastore/sync/run-dropdown-sync.png b/docs/assets/source-datastores/operations/sync/run-dropdown-sync.png similarity index 100% rename from docs/assets/source-datastore/sync/run-dropdown-sync.png rename to docs/assets/source-datastores/operations/sync/run-dropdown-sync.png diff --git a/docs/assets/source-datastore/sync/run-now-button.png b/docs/assets/source-datastores/operations/sync/run-now-button.png similarity index 100% rename from docs/assets/source-datastore/sync/run-now-button.png rename to docs/assets/source-datastores/operations/sync/run-now-button.png diff --git a/docs/assets/source-datastore/sync/schedule-advanced.png b/docs/assets/source-datastores/operations/sync/schedule-advanced.png similarity index 100% rename from docs/assets/source-datastore/sync/schedule-advanced.png rename to docs/assets/source-datastores/operations/sync/schedule-advanced.png diff --git a/docs/assets/source-datastore/sync/schedule-button.png b/docs/assets/source-datastores/operations/sync/schedule-button.png similarity index 100% rename from docs/assets/source-datastore/sync/schedule-button.png rename to docs/assets/source-datastores/operations/sync/schedule-button.png diff --git a/docs/assets/source-datastore/sync/schedule-confirm-button.png b/docs/assets/source-datastores/operations/sync/schedule-confirm-button.png similarity index 100% rename from docs/assets/source-datastore/sync/schedule-confirm-button.png rename to docs/assets/source-datastores/operations/sync/schedule-confirm-button.png diff --git a/docs/assets/source-datastore/sync/schedule-daily.png b/docs/assets/source-datastores/operations/sync/schedule-daily.png similarity index 100% rename from docs/assets/source-datastore/sync/schedule-daily.png rename to docs/assets/source-datastores/operations/sync/schedule-daily.png diff --git a/docs/assets/source-datastore/sync/schedule-hourly.png b/docs/assets/source-datastores/operations/sync/schedule-hourly.png similarity index 100% rename from docs/assets/source-datastore/sync/schedule-hourly.png rename to docs/assets/source-datastores/operations/sync/schedule-hourly.png diff --git a/docs/assets/source-datastore/sync/schedule-monthly.png b/docs/assets/source-datastores/operations/sync/schedule-monthly.png similarity index 100% rename from docs/assets/source-datastore/sync/schedule-monthly.png rename to docs/assets/source-datastores/operations/sync/schedule-monthly.png diff --git a/docs/assets/source-datastore/sync/schedule-name.png b/docs/assets/source-datastores/operations/sync/schedule-name.png similarity index 100% rename from docs/assets/source-datastore/sync/schedule-name.png rename to docs/assets/source-datastores/operations/sync/schedule-name.png diff --git a/docs/assets/source-datastore/sync/schedule-weekly.png b/docs/assets/source-datastores/operations/sync/schedule-weekly.png similarity index 100% rename from docs/assets/source-datastore/sync/schedule-weekly.png rename to docs/assets/source-datastores/operations/sync/schedule-weekly.png diff --git a/docs/assets/source-datastore/sync/select-source-datastore.png b/docs/assets/source-datastores/operations/sync/select-source-datastore.png similarity index 100% rename from docs/assets/source-datastore/sync/select-source-datastore.png rename to docs/assets/source-datastores/operations/sync/select-source-datastore.png diff --git a/docs/assets/source-datastore/sync/sync-button-overview.png b/docs/assets/source-datastores/operations/sync/sync-button-overview.png similarity index 100% rename from docs/assets/source-datastore/sync/sync-button-overview.png rename to docs/assets/source-datastores/operations/sync/sync-button-overview.png diff --git a/docs/assets/source-datastore/sync/sync-completed-activity.png b/docs/assets/source-datastores/operations/sync/sync-completed-activity.png similarity index 100% rename from docs/assets/source-datastore/sync/sync-completed-activity.png rename to docs/assets/source-datastores/operations/sync/sync-completed-activity.png diff --git a/docs/assets/source-datastore/sync/sync-completed-containers.png b/docs/assets/source-datastores/operations/sync/sync-completed-containers.png similarity index 100% rename from docs/assets/source-datastore/sync/sync-completed-containers.png rename to docs/assets/source-datastores/operations/sync/sync-completed-containers.png diff --git a/docs/assets/source-datastore/sync/sync-error-aborted.png b/docs/assets/source-datastores/operations/sync/sync-error-aborted.png similarity index 100% rename from docs/assets/source-datastore/sync/sync-error-aborted.png rename to docs/assets/source-datastores/operations/sync/sync-error-aborted.png diff --git a/docs/assets/source-datastore/sync/sync-running-automatically.png b/docs/assets/source-datastores/operations/sync/sync-running-automatically.png similarity index 100% rename from docs/assets/source-datastore/sync/sync-running-automatically.png rename to docs/assets/source-datastores/operations/sync/sync-running-automatically.png diff --git a/docs/assets/source-datastore/sync/sync-settings-include.png b/docs/assets/source-datastores/operations/sync/sync-settings-include.png similarity index 100% rename from docs/assets/source-datastore/sync/sync-settings-include.png rename to docs/assets/source-datastores/operations/sync/sync-settings-include.png diff --git a/docs/assets/source-datastore/sync/sync-settings-prune-recreate.png b/docs/assets/source-datastores/operations/sync/sync-settings-prune-recreate.png similarity index 100% rename from docs/assets/source-datastore/sync/sync-settings-prune-recreate.png rename to docs/assets/source-datastores/operations/sync/sync-settings-prune-recreate.png diff --git a/docs/assets/source-datastores/profile/inference-threshold-level0.png b/docs/assets/source-datastores/profile/inference-threshold-level0.png deleted file mode 100644 index bc7fd6e8dd..0000000000 Binary files a/docs/assets/source-datastores/profile/inference-threshold-level0.png and /dev/null differ diff --git a/docs/assets/source-datastores/profile/inference-threshold-level1.png b/docs/assets/source-datastores/profile/inference-threshold-level1.png deleted file mode 100644 index cc5f6bfe9b..0000000000 Binary files a/docs/assets/source-datastores/profile/inference-threshold-level1.png and /dev/null differ diff --git a/docs/assets/source-datastores/profile/inference-threshold-level2.png b/docs/assets/source-datastores/profile/inference-threshold-level2.png deleted file mode 100644 index 55b2cc7466..0000000000 Binary files a/docs/assets/source-datastores/profile/inference-threshold-level2.png and /dev/null differ diff --git a/docs/assets/source-datastores/profile/inference-threshold-level3.png b/docs/assets/source-datastores/profile/inference-threshold-level3.png deleted file mode 100644 index 504f604e2c..0000000000 Binary files a/docs/assets/source-datastores/profile/inference-threshold-level3.png and /dev/null differ diff --git a/docs/assets/source-datastores/profile/inference-threshold-level4.png b/docs/assets/source-datastores/profile/inference-threshold-level4.png deleted file mode 100644 index 724208e57f..0000000000 Binary files a/docs/assets/source-datastores/profile/inference-threshold-level4.png and /dev/null differ diff --git a/docs/assets/source-datastores/profile/inference-threshold-level5.png b/docs/assets/source-datastores/profile/inference-threshold-level5.png deleted file mode 100644 index 80a28ab917..0000000000 Binary files a/docs/assets/source-datastores/profile/inference-threshold-level5.png and /dev/null differ diff --git a/docs/assets/source-datastores/profile/schedule-name.png b/docs/assets/source-datastores/profile/schedule-name.png deleted file mode 100644 index 0d0aabb954..0000000000 Binary files a/docs/assets/source-datastores/profile/schedule-name.png and /dev/null differ diff --git a/docs/assets/source-datastores/profile/starting-threshold.png b/docs/assets/source-datastores/profile/starting-threshold.png deleted file mode 100644 index 02a58e07f2..0000000000 Binary files a/docs/assets/source-datastores/profile/starting-threshold.png and /dev/null differ diff --git a/docs/assets/source-datastores/scan/aborted-operation.png b/docs/assets/source-datastores/scan/aborted-operation.png deleted file mode 100644 index 0d221ff54e..0000000000 Binary files a/docs/assets/source-datastores/scan/aborted-operation.png and /dev/null differ diff --git a/docs/assets/source-datastores/scan/activity-operation.png b/docs/assets/source-datastores/scan/activity-operation.png deleted file mode 100644 index 77ce1f6782..0000000000 Binary files a/docs/assets/source-datastores/scan/activity-operation.png and /dev/null differ diff --git a/docs/assets/source-datastores/scan/activity.png b/docs/assets/source-datastores/scan/activity.png deleted file mode 100644 index 8f5e1840c3..0000000000 Binary files a/docs/assets/source-datastores/scan/activity.png and /dev/null differ diff --git a/docs/assets/source-datastores/scan/advanced.png b/docs/assets/source-datastores/scan/advanced.png deleted file mode 100644 index d9dc7381e7..0000000000 Binary files a/docs/assets/source-datastores/scan/advanced.png and /dev/null differ diff --git a/docs/assets/source-datastores/scan/all-operation.png b/docs/assets/source-datastores/scan/all-operation.png deleted file mode 100644 index dceb1f6238..0000000000 Binary files a/docs/assets/source-datastores/scan/all-operation.png and /dev/null differ diff --git a/docs/assets/source-datastores/scan/anomaly-option.png b/docs/assets/source-datastores/scan/anomaly-option.png deleted file mode 100644 index e178d0154f..0000000000 Binary files a/docs/assets/source-datastores/scan/anomaly-option.png and /dev/null differ diff --git a/docs/assets/source-datastores/scan/anomalyy.png b/docs/assets/source-datastores/scan/anomalyy.png deleted file mode 100644 index 3ad1eb1504..0000000000 Binary files a/docs/assets/source-datastores/scan/anomalyy.png and /dev/null differ diff --git a/docs/assets/source-datastores/scan/click-next.png b/docs/assets/source-datastores/scan/click-next.png deleted file mode 100644 index 908786e9e4..0000000000 Binary files a/docs/assets/source-datastores/scan/click-next.png and /dev/null differ diff --git a/docs/assets/source-datastores/scan/click-schedule.png b/docs/assets/source-datastores/scan/click-schedule.png deleted file mode 100644 index f0a2023fa2..0000000000 Binary files a/docs/assets/source-datastores/scan/click-schedule.png and /dev/null differ diff --git a/docs/assets/source-datastores/scan/computed.png b/docs/assets/source-datastores/scan/computed.png deleted file mode 100644 index baaf397289..0000000000 Binary files a/docs/assets/source-datastores/scan/computed.png and /dev/null differ diff --git a/docs/assets/source-datastores/scan/daily.png b/docs/assets/source-datastores/scan/daily.png deleted file mode 100644 index f31651e0bc..0000000000 Binary files a/docs/assets/source-datastores/scan/daily.png and /dev/null differ diff --git a/docs/assets/source-datastores/scan/details-page.png b/docs/assets/source-datastores/scan/details-page.png deleted file mode 100644 index 901b59c052..0000000000 Binary files a/docs/assets/source-datastores/scan/details-page.png and /dev/null differ diff --git a/docs/assets/source-datastores/scan/drop-down.png b/docs/assets/source-datastores/scan/drop-down.png deleted file mode 100644 index 55c5842011..0000000000 Binary files a/docs/assets/source-datastores/scan/drop-down.png and /dev/null differ diff --git a/docs/assets/source-datastores/scan/hourly.png b/docs/assets/source-datastores/scan/hourly.png deleted file mode 100644 index 6c07a7fabb..0000000000 Binary files a/docs/assets/source-datastores/scan/hourly.png and /dev/null differ diff --git a/docs/assets/source-datastores/scan/incremental.png b/docs/assets/source-datastores/scan/incremental.png deleted file mode 100644 index d0e44ef8bc..0000000000 Binary files a/docs/assets/source-datastores/scan/incremental.png and /dev/null differ diff --git a/docs/assets/source-datastores/scan/monthly.png b/docs/assets/source-datastores/scan/monthly.png deleted file mode 100644 index 298a326c1d..0000000000 Binary files a/docs/assets/source-datastores/scan/monthly.png and /dev/null differ diff --git a/docs/assets/source-datastores/scan/next-button.png b/docs/assets/source-datastores/scan/next-button.png deleted file mode 100644 index 19dff994f8..0000000000 Binary files a/docs/assets/source-datastores/scan/next-button.png and /dev/null differ diff --git a/docs/assets/source-datastores/scan/next-buttonn.png b/docs/assets/source-datastores/scan/next-buttonn.png deleted file mode 100644 index 3b78db87db..0000000000 Binary files a/docs/assets/source-datastores/scan/next-buttonn.png and /dev/null differ diff --git a/docs/assets/source-datastores/scan/next.png b/docs/assets/source-datastores/scan/next.png deleted file mode 100644 index b21d599deb..0000000000 Binary files a/docs/assets/source-datastores/scan/next.png and /dev/null differ diff --git a/docs/assets/source-datastores/scan/nextt.png b/docs/assets/source-datastores/scan/nextt.png deleted file mode 100644 index 906da0e830..0000000000 Binary files a/docs/assets/source-datastores/scan/nextt.png and /dev/null differ diff --git a/docs/assets/source-datastores/scan/record-limit-line.png b/docs/assets/source-datastores/scan/record-limit-line.png deleted file mode 100644 index d267ea022f..0000000000 Binary files a/docs/assets/source-datastores/scan/record-limit-line.png and /dev/null differ diff --git a/docs/assets/source-datastores/scan/record-limit-options.png b/docs/assets/source-datastores/scan/record-limit-options.png deleted file mode 100644 index b8ce34cb2f..0000000000 Binary files a/docs/assets/source-datastores/scan/record-limit-options.png and /dev/null differ diff --git a/docs/assets/source-datastores/scan/record-limit.png b/docs/assets/source-datastores/scan/record-limit.png deleted file mode 100644 index e234031c15..0000000000 Binary files a/docs/assets/source-datastores/scan/record-limit.png and /dev/null differ diff --git a/docs/assets/source-datastores/scan/records-scan-operation.png b/docs/assets/source-datastores/scan/records-scan-operation.png deleted file mode 100644 index d944dfe949..0000000000 Binary files a/docs/assets/source-datastores/scan/records-scan-operation.png and /dev/null differ diff --git a/docs/assets/source-datastores/scan/result-scan-operation.png b/docs/assets/source-datastores/scan/result-scan-operation.png deleted file mode 100644 index a295cb757b..0000000000 Binary files a/docs/assets/source-datastores/scan/result-scan-operation.png and /dev/null differ diff --git a/docs/assets/source-datastores/scan/result.png b/docs/assets/source-datastores/scan/result.png deleted file mode 100644 index ccdc0d837e..0000000000 Binary files a/docs/assets/source-datastores/scan/result.png and /dev/null differ diff --git a/docs/assets/source-datastores/scan/run-now.png b/docs/assets/source-datastores/scan/run-now.png deleted file mode 100644 index 9ff257827b..0000000000 Binary files a/docs/assets/source-datastores/scan/run-now.png and /dev/null differ diff --git a/docs/assets/source-datastores/scan/run.png b/docs/assets/source-datastores/scan/run.png deleted file mode 100644 index 1811982e47..0000000000 Binary files a/docs/assets/source-datastores/scan/run.png and /dev/null differ diff --git a/docs/assets/source-datastores/scan/running.png b/docs/assets/source-datastores/scan/running.png deleted file mode 100644 index cfd2a1cb28..0000000000 Binary files a/docs/assets/source-datastores/scan/running.png and /dev/null differ diff --git a/docs/assets/source-datastores/scan/sacn-operation.png b/docs/assets/source-datastores/scan/sacn-operation.png deleted file mode 100644 index 0e7d1a1bed..0000000000 Binary files a/docs/assets/source-datastores/scan/sacn-operation.png and /dev/null differ diff --git a/docs/assets/source-datastores/scan/sacn.png b/docs/assets/source-datastores/scan/sacn.png deleted file mode 100644 index 35465d57b9..0000000000 Binary files a/docs/assets/source-datastores/scan/sacn.png and /dev/null differ diff --git a/docs/assets/source-datastores/scan/scan-operation.png b/docs/assets/source-datastores/scan/scan-operation.png deleted file mode 100644 index c68ad9eb9e..0000000000 Binary files a/docs/assets/source-datastores/scan/scan-operation.png and /dev/null differ diff --git a/docs/assets/source-datastores/scan/scan-settings.png b/docs/assets/source-datastores/scan/scan-settings.png deleted file mode 100644 index 4c7dacf4e2..0000000000 Binary files a/docs/assets/source-datastores/scan/scan-settings.png and /dev/null differ diff --git a/docs/assets/source-datastores/scan/schedule-name.png b/docs/assets/source-datastores/scan/schedule-name.png deleted file mode 100644 index 123e5a1522..0000000000 Binary files a/docs/assets/source-datastores/scan/schedule-name.png and /dev/null differ diff --git a/docs/assets/source-datastores/scan/schedule.png b/docs/assets/source-datastores/scan/schedule.png deleted file mode 100644 index 1779e7ace9..0000000000 Binary files a/docs/assets/source-datastores/scan/schedule.png and /dev/null differ diff --git a/docs/assets/source-datastores/scan/select-check.png b/docs/assets/source-datastores/scan/select-check.png deleted file mode 100644 index 61ad2e8c9e..0000000000 Binary files a/docs/assets/source-datastores/scan/select-check.png and /dev/null differ diff --git a/docs/assets/source-datastores/scan/side-menu.png b/docs/assets/source-datastores/scan/side-menu.png deleted file mode 100644 index 07b6580635..0000000000 Binary files a/docs/assets/source-datastores/scan/side-menu.png and /dev/null differ diff --git a/docs/assets/source-datastores/scan/source-record-limit.png b/docs/assets/source-datastores/scan/source-record-limit.png deleted file mode 100644 index e41fb3c3ab..0000000000 Binary files a/docs/assets/source-datastores/scan/source-record-limit.png and /dev/null differ diff --git a/docs/assets/source-datastores/scan/specific.png b/docs/assets/source-datastores/scan/specific.png deleted file mode 100644 index 8fc50e6429..0000000000 Binary files a/docs/assets/source-datastores/scan/specific.png and /dev/null differ diff --git a/docs/assets/source-datastores/scan/starting-threshold.png b/docs/assets/source-datastores/scan/starting-threshold.png deleted file mode 100644 index ab1531458a..0000000000 Binary files a/docs/assets/source-datastores/scan/starting-threshold.png and /dev/null differ diff --git a/docs/assets/source-datastores/scan/success-2.png b/docs/assets/source-datastores/scan/success-2.png deleted file mode 100644 index 97d7b05c32..0000000000 Binary files a/docs/assets/source-datastores/scan/success-2.png and /dev/null differ diff --git a/docs/assets/source-datastores/scan/sucess.png b/docs/assets/source-datastores/scan/sucess.png deleted file mode 100644 index 030a0976db..0000000000 Binary files a/docs/assets/source-datastores/scan/sucess.png and /dev/null differ diff --git a/docs/assets/source-datastores/scan/tag.png b/docs/assets/source-datastores/scan/tag.png deleted file mode 100644 index 0c1d9a48cd..0000000000 Binary files a/docs/assets/source-datastores/scan/tag.png and /dev/null differ diff --git a/docs/assets/source-datastores/scan/warning.png b/docs/assets/source-datastores/scan/warning.png deleted file mode 100644 index 55705f7d77..0000000000 Binary files a/docs/assets/source-datastores/scan/warning.png and /dev/null differ diff --git a/docs/assets/source-datastores/scan/weekly.png b/docs/assets/source-datastores/scan/weekly.png deleted file mode 100644 index 8a4e690b7f..0000000000 Binary files a/docs/assets/source-datastores/scan/weekly.png and /dev/null differ diff --git a/docs/assets/source-datastores/shared/manage-groups-button.png b/docs/assets/source-datastores/shared/manage-groups-button.png new file mode 100644 index 0000000000..ff7e55ef51 Binary files /dev/null and b/docs/assets/source-datastores/shared/manage-groups-button.png differ diff --git a/docs/assets/source-datastores/shared/select-source-datastore.png b/docs/assets/source-datastores/shared/select-source-datastore.png new file mode 100644 index 0000000000..741773e144 Binary files /dev/null and b/docs/assets/source-datastores/shared/select-source-datastore.png differ diff --git a/docs/enrichment-support/supported-enrichment-datastores.md b/docs/enrichment-support/supported-enrichment-datastores.md index 719d73b11c..0d0d8efb9d 100644 --- a/docs/enrichment-support/supported-enrichment-datastores.md +++ b/docs/enrichment-support/supported-enrichment-datastores.md @@ -1,150 +1,55 @@ # Supported Enrichment Datastores -Qualytics supports enrichment datastore connectors that help enhance data discovery, profiling, and quality checks. Some connectors include enrichment capabilities, while others provide only standard connectivity. +An enrichment datastore is a dedicated datastore linked to a source datastore that persists scan results, anomalies, remediation data, and source record examples. Not all connectors support enrichment — the tables below show which connectors can be used as enrichment datastores. -In this guide, we will cover: +Enrichment support requires the connector to have **write capabilities** so that Qualytics can create and manage enrichment tables (source records, anomaly records, remediation tables, and metadata tables) in the target datastore. -- **[JDBC Connectors](#jdbc-connectors)** -- **[DFS Connectors](#dfs-connectors)** +!!! info "Available Datastore Connectors" + For the full list of all supported source datastore connectors (including those without enrichment support), see the [Available Datastore Connectors](../source-datastore/add-datastores/available-datastore-connectors.md){:target="_blank"} page. -## JDBC Connectors +!!! info "Enrichment Table Types" + To understand the different table types created in an enrichment datastore (source records, anomaly records, remediation, metadata), see the [Enrichment Table Types](../enrichment/table-types.md){:target="_blank"} documentation. -The table below shows the list of JDBC connectors and whether they support enrichment or not: +## JDBC Connectors - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
No.ConnectorEnrichment Support
01.Athena❌
02.Big Query✅
03.Databricks✅
04.DB2✅
05.Dremio❌
06.Fabric Analytics❌
07.Hive❌
08.MariaDB✅
09.Microsoft SQL Server✅
10.MySQL✅
11.Oracle❌
12.PostgreSQL✅
13.Presto❌
14.Redshift✅
15.Snowflake✅
16.Synapse✅
17.Teradata❌
18.TimescaleDB❌
19.Trino✅
+JDBC connectors that support enrichment can store scan results and anomaly data directly in the relational database. Out of 19 JDBC connectors, **11 support enrichment**. -## DFS Connectors +
-The table below shows the list of DFS connectors and whether they support enrichment or not: +| No. | Connector | Logo | Enrichment Support | +| :--- | :--- | :---: | :---: | +| 1. | [Athena](../source-datastore/add-datastores/athena.md) | ![Athena](../assets/source-datastores/add-datastores/connector-logos/logo-athena.svg){ width="24" } | :material-close-circle-outline:{ .lg title="Not supported" } | +| 2. | [BigQuery](../source-datastore/add-datastores/bigquery.md#add-enrichment-datastore) | ![BigQuery](../assets/source-datastores/add-datastores/connector-logos/logo-big-query.png){ width="24" } | :material-check-circle:{ .lg title="Supported" } | +| 3. | [Databricks](../source-datastore/add-datastores/databricks.md#add-enrichment-datastore) | ![Databricks](../assets/source-datastores/add-datastores/connector-logos/logo-databricks.png){ width="24" } | :material-check-circle:{ .lg title="Supported" } | +| 4. | [DB2](../source-datastore/add-datastores/db2.md#add-enrichment-datastore) | ![DB2](../assets/source-datastores/add-datastores/connector-logos/logo-ibm-db2.png){ width="24" } | :material-check-circle:{ .lg title="Supported" } | +| 5. | [Dremio](../source-datastore/add-datastores/dremio.md) | ![Dremio](../assets/source-datastores/add-datastores/connector-logos/logo-dremio.svg){ width="24" } | :material-close-circle-outline:{ .lg title="Not supported" } | +| 6. | [Fabric Analytics](../source-datastore/add-datastores/fabric-analytics.md) | ![Fabric](../assets/source-datastores/add-datastores/connector-logos/logo-microsoft-fabric.svg){ width="24" } | :material-close-circle-outline:{ .lg title="Not supported" } | +| 7. | [Hive](../source-datastore/add-datastores/hive.md) | ![Hive](../assets/source-datastores/add-datastores/connector-logos/logo-hive.png){ width="24" } | :material-close-circle-outline:{ .lg title="Not supported" } | +| 8. | [MariaDB](../source-datastore/add-datastores/maria-db.md#add-enrichment-datastore) | ![MariaDB](../assets/source-datastores/add-datastores/connector-logos/logo-mariadb.svg){ width="24" } | :material-check-circle:{ .lg title="Supported" } | +| 9. | [Microsoft SQL Server](../source-datastore/add-datastores/microsoft-sql-server.md#add-enrichment-datastore) | ![SQL Server](../assets/source-datastores/add-datastores/connector-logos/logo-ms-sql.svg){ width="24" } | :material-check-circle:{ .lg title="Supported" } | +| 10. | [MySQL](../source-datastore/add-datastores/mysql.md#add-enrichment-datastore) | ![MySQL](../assets/source-datastores/add-datastores/connector-logos/mysql-icon.svg){ width="24" } | :material-check-circle:{ .lg title="Supported" } | +| 11. | [Oracle](../source-datastore/add-datastores/oracle.md) | ![Oracle](../assets/source-datastores/add-datastores/connector-logos/oracle-icon.svg){ width="24" } | :material-close-circle-outline:{ .lg title="Not supported" } | +| 12. | [PostgreSQL](../source-datastore/add-datastores/postgresql.md#add-enrichment-datastore) | ![PostgreSQL](../assets/source-datastores/add-datastores/connector-logos/logo-postgres.svg){ width="24" } | :material-check-circle:{ .lg title="Supported" } | +| 13. | [Presto](../source-datastore/add-datastores/presto.md) | ![Presto](../assets/source-datastores/add-datastores/connector-logos/presto.png){ width="24" } | :material-close-circle-outline:{ .lg title="Not supported" } | +| 14. | [Redshift](../source-datastore/add-datastores/redshift.md#add-enrichment-datastore) | ![Redshift](../assets/source-datastores/add-datastores/connector-logos/logo-redshift.svg){ width="24" } | :material-check-circle:{ .lg title="Supported" } | +| 15. | [Snowflake](../source-datastore/add-datastores/snowflake.md#add-enrichment-datastore-connection) | ![Snowflake](../assets/source-datastores/add-datastores/connector-logos/snowflake.svg){ width="24" } | :material-check-circle:{ .lg title="Supported" } | +| 16. | [Synapse](../source-datastore/add-datastores/synapse.md#add-enrichment-datastore) | ![Synapse](../assets/source-datastores/add-datastores/connector-logos/logo-synapses.png){ width="24" } | :material-check-circle:{ .lg title="Supported" } | +| 17. | [Teradata](../source-datastore/add-datastores/teradata.md) | ![Teradata](../assets/source-datastores/add-datastores/connector-logos/logo-teradata.png){ width="24" } | :material-close-circle-outline:{ .lg title="Not supported" } | +| 18. | [TimescaleDB](../source-datastore/add-datastores/timescale-db.md) | ![Timescale](../assets/source-datastores/add-datastores/connector-logos/logo-timescaledb.png){ width="24" } | :material-close-circle-outline:{ .lg title="Not supported" } | +| 19. | [Trino](../source-datastore/add-datastores/trino.md) | ![Trino](../assets/source-datastores/add-datastores/connector-logos/trino.svg){ width="24" } | :material-check-circle:{ .lg title="Supported" } | - - - - - - - - - - - - - - - - - - - - - - - - - -
No.ConnectorEnrichment Support
01.Amazon S3✅
02.Azure Datalake Storage (ABFS)✅
03.Google Cloud Storage (GCS)✅
\ No newline at end of file +
+ +## DFS Connectors + +DFS connectors that support enrichment store scan results and anomaly data as files (Parquet/JSON) in cloud object storage. **All 3 DFS connectors support enrichment**. + +
+ +| No. | Connector | Logo | Enrichment Support | +| :--- | :--- | :---: | :---: | +| 1. | [Amazon S3](../source-datastore/add-datastores/amazon-s3.md#add-enrichment-datastore) | ![S3](../assets/source-datastores/add-datastores/connector-logos/logo-s3.svg){ width="24" } | :material-check-circle:{ .lg title="Supported" } | +| 2. | [Azure Datalake Storage (ABFS)](../source-datastore/add-datastores/azure-datalake-storage.md#add-enrichment-datastore) | ![ABFS](../assets/source-datastores/add-datastores/connector-logos/logo-abfs.svg){ width="24" } | :material-check-circle:{ .lg title="Supported" } | +| 3. | [Google Cloud Storage (GCS)](../source-datastore/add-datastores/google-cloud-storage.md#add-enrichment-datastore) | ![GCS](../assets/source-datastores/add-datastores/connector-logos/logo-gcs.svg){ width="24" } | :material-check-circle:{ .lg title="Supported" } | + +
diff --git a/docs/settings/security/team-permissions.md b/docs/settings/security/team-permissions.md index 434301774b..ef3b0453d4 100644 --- a/docs/settings/security/team-permissions.md +++ b/docs/settings/security/team-permissions.md @@ -48,6 +48,7 @@ Legend: | Assign Tags in Flows | ❌ | ❌ | ✅ | ✅ | ✅ | | View Datastore Groups | ✅ | ✅ | ✅ | ✅ | ✅ | | Assign Datastore to Group | ❌ | ❌ | ❌ | ❌ | ✅ | +| Multi-Schema Datastore Creation | ❌ | ❌ | ❌ | ❌ | ❌ | | View Field Status | ✅ | ✅ | ✅ | ✅ | ✅ | | Mask / Unmask Field | ❌ | ❌ | ❌ | ❌ | ✅ | | Exclude Field | ❌ | ❌ | ❌ | ❌ | ✅ | @@ -88,6 +89,7 @@ Editor role allows users to manage datastore functions comprehensively. They can | Feature | Operation | Can View/Can Run | Can Manage | |:-----------------------|:----------------------------|:------------------|:-------------| | **Datastores** | Add Datastore | ❌ | ✅ | +| | Multi-Schema Datastore Creation | ❌ | ✅ | | | Edit Settings | ❌ | ✅ | | **Enrichment** | Add Enrichment | ❌ | ✅ | | | Edit Enrichment | ❌ | ✅ | diff --git a/docs/source-datastore/add-datastores/multi-schema/concepts/how-it-works.md b/docs/source-datastore/add-datastores/multi-schema/concepts/how-it-works.md new file mode 100644 index 0000000000..83ffab7cac --- /dev/null +++ b/docs/source-datastore/add-datastores/multi-schema/concepts/how-it-works.md @@ -0,0 +1,103 @@ +# How Multi-Schema Creation Works + +## Overview + +The multi-schema creation flow is a two-step wizard that guides you through discovering schemas from a connection and creating multiple source datastores at once. + +
+ +```mermaid +graph TD + A["Select Connection"] --> B["Discover Catalogs/Schemas"] + B --> C["Select Schemas"] + C --> D["Configure Properties"] + D --> E["Validate & Create"] + E --> F["Link Enrichment (Optional)"] +``` + +
+ +## Step 1: Source Datastore Configuration + +In the first step, you configure the connection and select which schemas to onboard. + +### Connection Selection + +You can either: + +- **Create a new connection**: Provide all connection details (host, port, credentials, etc.) from scratch. +- **Use an existing connection**: Select a previously saved connection to reuse its credentials. + +For more information about connections, refer to the [Connection Overview](../../connections/overview-of-a-connection.md) documentation. + +### Catalog Discovery + +For connectors that support a catalog hierarchy (e.g., databases in PostgreSQL, databases in Snowflake, projects in BigQuery), the first discovery step retrieves the list of available catalogs from the connection. You select which catalog to browse for schemas. + +!!! tip + Click the **refresh** button next to the catalog dropdown to reload the available catalogs from your connection. + +Not all connectors have a catalog level. For connectors like Oracle or DB2, schemas are discovered directly without a catalog selection step. + +### Schema Discovery and Selection + +Once a catalog is selected (or for connectors without a catalog level), the system discovers all available schemas and presents them in a multi-select dropdown. + +Key behaviors: + +- **Warning icons**: Schemas that already have an existing datastore display a warning icon with a tooltip showing which datastores are associated. +- **Search filtering**: You can filter schemas by name using the search field in the dropdown. +- **Selection count**: The dropdown shows how many schemas are selected (e.g., "3 selected"). + +!!! warning + You can select schemas that already have datastores, but this will create duplicate datastores for those schemas. + +### Name Template + +The **Name Template** field lets you define a naming pattern for the datastores that will be created. Use the `{{schema}}` placeholder, which will be replaced with each schema name. + +For example, with the template `production_{{schema}}` and schemas `public`, `staging`, and `analytics`: + +| Schema | Generated Datastore Name | +| :--- | :--- | +| `public` | `production_public` | +| `staging` | `production_staging` | +| `analytics` | `production_analytics` | + +!!! tip + A preview of the generated names is displayed below the name template field, showing how the first few datastores will be named. + +### Additional Properties + +| Field | Required | Description | +| :--- | :--- | :--- | +| **Teams** | Yes | One or more teams to associate with all newly created datastores. | +| **Group** | No | Assign all datastores to a [datastore group](../../../managing-datastores/grouping/overview.md). | +| **Initiate Sync** | No | Automatically run a sync operation on each newly created datastore after creation. | + +### Validation + +Before creating the datastores, click **Test Connection** to validate connectivity for all selected schemas. The validation runs per-schema and reports individual results, so you can identify which schemas have issues before proceeding. + +## Step 2: Enrichment Datastore Linking (Optional) + +After configuring the source datastores, an optional second step lets you link all newly created datastores to a single enrichment datastore. The enrichment datastore stores analyzed results, anomalies, and additional metadata. + +You can: + +- **Use an existing enrichment datastore**: Select a previously created enrichment datastore. +- **Create a new enrichment datastore**: Configure a new enrichment datastore during the same flow. + +!!! info + All source datastores created in the batch will be linked to the same enrichment datastore. The enrichment prefix is shared across all of them. + +For step-by-step instructions on adding a datastore, refer to the [Add Source Datastore](../../overview-of-a-jdbc-datastore.md) documentation. + +## What Happens After Creation + +Once the creation process completes, Qualytics returns a summary showing: + +- **Created datastores**: The list of successfully created datastore IDs. +- **Errors**: Any schemas that failed to create, with the corresponding error message. + +If **Initiate Sync** was enabled, a sync operation will automatically start on each successfully created datastore, detecting containers and fields within each schema. diff --git a/docs/source-datastore/add-datastores/multi-schema/concepts/multi-schema-api.md b/docs/source-datastore/add-datastores/multi-schema/concepts/multi-schema-api.md new file mode 100644 index 0000000000..eabdbbd202 --- /dev/null +++ b/docs/source-datastore/add-datastores/multi-schema/concepts/multi-schema-api.md @@ -0,0 +1,847 @@ +# Multi-Schema Creation API + +This page documents the API endpoints related to multi-schema source datastore creation. Use these endpoints to programmatically discover catalogs and schemas, validate connectivity, and bulk-create datastores. + +All endpoints use the base URL of your Qualytics deployment (e.g., `https://your-instance.qualytics.io/api`). + +## Discover Catalogs + +Retrieves the list of available catalogs (databases/projects) from a connection. + +=== "New Connection" + **Endpoint**: `POST /api/connections/catalogs` + + **Permission**: Manager + + Send the full connection details in the request body. + + **Response**: `list[string]` — a list of catalog names. + + ??? example "Example request and response" + + **Request**: + + ```bash + curl -X POST "https://your-instance.qualytics.io/api/connections/catalogs" \ + -H "Authorization: Bearer YOUR_TOKEN" \ + -H "Content-Type: application/json" \ + -d '{ + "name": "snowflake_production", + "type": "snowflake", + "host": "acme.snowflakecomputing.com", + "username": "qualytics_user", + "password": "your_password", + "parameters": { + "role": "qualytics_read_role", + "warehouse": "qualytics_wh" + } + }' + ``` + + **Response**: + + ```json + ["production_db", "staging_db", "analytics_db"] + ``` + +=== "Existing Connection" + **Endpoint**: `GET /api/connections/{connection_id}/catalogs` + + **Permission**: Manager + + **Response**: `list[DiscoveredCatalog]` — catalogs annotated with existing datastores. + + ??? example "Example request and response" + + **Request**: + + ```bash + curl -X GET "https://your-instance.qualytics.io/api/connections/7/catalogs" \ + -H "Authorization: Bearer YOUR_TOKEN" + ``` + + **Response**: + + ```json + [ + { + "name": "production_db", + "existing_datastores": [ + { "id": 50, "name": "prod_public" } + ] + }, + { + "name": "staging_db", + "existing_datastores": [] + } + ] + ``` + +!!! note + Only JDBC and Native connectors support catalog discovery. Calling this endpoint for a DFS connection returns `409 Conflict`. + +--- + +## Discover Schemas + +Retrieves the list of available schemas within a catalog. + +**Query Parameters**: + +| Parameter | Type | Required | Description | +| :--- | :--- | :--- | :--- | +| `catalog` | string | No | Catalog/database name to filter schemas by | + +=== "New Connection" + **Endpoint**: `POST /api/connections/schemas` + + **Permission**: Manager + + Send the full connection details in the request body. + + **Response**: `list[string]` — a list of schema names. + + ??? example "Example request and response" + + **Request**: + + ```bash + curl -X POST "https://your-instance.qualytics.io/api/connections/schemas?catalog=production_db" \ + -H "Authorization: Bearer YOUR_TOKEN" \ + -H "Content-Type: application/json" \ + -d '{ + "name": "snowflake_production", + "type": "snowflake", + "host": "acme.snowflakecomputing.com", + "username": "qualytics_user", + "password": "your_password", + "parameters": { + "role": "qualytics_read_role", + "warehouse": "qualytics_wh" + } + }' + ``` + + **Response**: + + ```json + ["public", "sales", "finance", "hr", "marketing"] + ``` + +=== "Existing Connection" + **Endpoint**: `GET /api/connections/{connection_id}/schemas` + + **Permission**: Manager + + **Response**: `list[DiscoveredSchema]` — schemas annotated with existing datastores. + + ??? example "Example request and response" + + **Request**: + + ```bash + curl -X GET "https://your-instance.qualytics.io/api/connections/7/schemas?catalog=production_db" \ + -H "Authorization: Bearer YOUR_TOKEN" + ``` + + **Response**: + + ```json + [ + { + "name": "public", + "existing_datastores": [] + }, + { + "name": "sales", + "existing_datastores": [ + { "id": 60, "name": "prod_sales" } + ] + }, + { + "name": "finance", + "existing_datastores": [] + } + ] + ``` + +!!! note + Only JDBC and Native connectors support schema discovery. Calling this endpoint for a DFS connection returns `409 Conflict`. + +--- + +## Validate Schemas + +Validates connectivity for one or more schemas before creating datastores. The validation runs per-schema and reports individual results. + +=== "New Connection" + **Endpoint**: `POST /api/connections/datastores/validate` + + **Permission**: Manager + + Send the full connection details along with the schemas to validate. + + ??? example "Example request and response" + + **Request**: + + ```bash + curl -X POST "https://your-instance.qualytics.io/api/connections/datastores/validate" \ + -H "Authorization: Bearer YOUR_TOKEN" \ + -H "Content-Type: application/json" \ + -d '{ + "connection": { + "name": "snowflake_production", + "type": "snowflake", + "host": "acme.snowflakecomputing.com", + "username": "qualytics_user", + "password": "your_password", + "parameters": { + "role": "qualytics_read_role", + "warehouse": "qualytics_wh" + } + }, + "database": "production_db", + "schemas": ["public", "sales", "finance"] + }' + ``` + + **Response**: + + ```json + { + "results": [ + { "schema_name": "public", "status": "success" }, + { "schema_name": "sales", "status": "success" }, + { "schema_name": "finance", "status": "failure", "message": "Permission denied for schema finance" } + ] + } + ``` + +=== "Existing Connection" + **Endpoint**: `POST /api/connections/{connection_id}/datastores/validate` + + **Permission**: Manager + + ??? example "Example request and response" + + **Request**: + + ```bash + curl -X POST "https://your-instance.qualytics.io/api/connections/7/datastores/validate" \ + -H "Authorization: Bearer YOUR_TOKEN" \ + -H "Content-Type: application/json" \ + -d '{ + "database": "production_db", + "schemas": ["public", "sales", "finance"] + }' + ``` + + **Response**: + + ```json + { + "results": [ + { "schema_name": "public", "status": "success" }, + { "schema_name": "sales", "status": "success" }, + { "schema_name": "finance", "status": "failure", "message": "Permission denied for schema finance" } + ] + } + ``` + +**Request Body**: + +| Property | Type | Required | Default | Description | +| :--- | :--- | :--- | :--- | :--- | +| `schemas` | list[string] | Yes | — | List of schema names to validate (max 50) | +| `database` | string | No | `null` | Database/catalog name | +| `enrichment_only` | boolean | No | `false` | When `true`, validates write permissions (DDL) in addition to read | + +--- + +## Bulk Create Datastores + +Creates multiple source datastores from selected schemas in a single operation. + +=== "New Connection" + **Endpoint**: `POST /api/connections/datastores/bulk` + + **Permission**: Manager + + Creates the connection and bulk-creates datastores in a single request. + + ??? example "Example request and response" + + **Request**: + + ```bash + curl -X POST "https://your-instance.qualytics.io/api/connections/datastores/bulk" \ + -H "Authorization: Bearer YOUR_TOKEN" \ + -H "Content-Type: application/json" \ + -d '{ + "connection": { + "name": "snowflake_production", + "type": "snowflake", + "host": "acme.snowflakecomputing.com", + "username": "qualytics_user", + "password": "your_password", + "parameters": { + "role": "qualytics_read_role", + "warehouse": "qualytics_wh" + } + }, + "database": "production_db", + "schemas": ["public", "sales", "finance"], + "name_template": "prod_{{schema}}", + "description": "Production Snowflake datastores", + "teams": ["Data Platform"], + "trigger_catalog": true, + "enrichment_datastore_id": 42, + "enrichment_prefix": "_prod", + "enrichment_source_record_limit": 100, + "enrichment_remediation_strategy": "append", + "high_count_rollup_threshold": 10, + "group_id": 5, + "tags": ["production", "snowflake"] + }' + ``` + + **Response**: + + ```json + { + "created": [101, 102, 103], + "errors": [] + } + ``` + +=== "Existing Connection" + **Endpoint**: `POST /api/connections/{connection_id}/datastores/bulk` + + **Permission**: Manager + + ??? example "Example request and response" + + **Request**: + + ```bash + curl -X POST "https://your-instance.qualytics.io/api/connections/7/datastores/bulk" \ + -H "Authorization: Bearer YOUR_TOKEN" \ + -H "Content-Type: application/json" \ + -d '{ + "database": "production_db", + "schemas": ["public", "sales"], + "name_template": "prod_{{schema}}", + "teams": ["Data Platform", "Analytics"], + "trigger_catalog": true + }' + ``` + + **Response**: + + ```json + { + "created": [110, 111], + "errors": [] + } + ``` + +??? example "Example response with errors" + + When some schemas fail, the response includes both successes and errors: + + ```json + { + "created": [101, 102], + "errors": [ + { + "schema_name": "finance", + "error": "Permission denied for schema finance" + } + ] + } + ``` + +!!! note + The bulk creation is non-atomic. Schemas that succeed are created even if other schemas fail. The response includes both the list of created datastore IDs and any errors encountered. + +--- + +## Link Enrichment Datastore + +If you did not include `enrichment_datastore_id` during bulk creation, you can link an enrichment datastore to each source datastore individually after creation. + +### Link Enrichment to a Source Datastore + +**Endpoint**: `PATCH /api/datastores/{datastore_id}/enrichment/{enrichment_id}` + +**Permission**: Member + +??? example "Example request" + + Link enrichment datastore ID 42 to source datastore ID 110: + + **Request**: + + ```bash + curl -X PATCH "https://your-instance.qualytics.io/api/datastores/110/enrichment/42" \ + -H "Authorization: Bearer YOUR_TOKEN" + ``` + + **Response**: Returns the updated datastore object with enrichment linked. + +!!! tip + To automate enrichment linking across multiple datastores created in bulk, loop through the `created` IDs from the bulk creation response: + + ```bash + for DATASTORE_ID in 110 111 112; do + curl -X PATCH "https://your-instance.qualytics.io/api/datastores/$DATASTORE_ID/enrichment/42" \ + -H "Authorization: Bearer YOUR_TOKEN" + done + ``` + +### Unlink Enrichment from a Source Datastore + +**Endpoint**: `DELETE /api/datastores/{datastore_id}/enrichment` + +**Permission**: Admin + +??? example "Example request" + + ```bash + curl -X DELETE "https://your-instance.qualytics.io/api/datastores/110/enrichment" \ + -H "Authorization: Bearer YOUR_TOKEN" + ``` + + **Response**: `204 No Content` + +!!! warning + Unlinking enrichment requires the **Admin** role, which is a higher permission than linking. + +--- + +## Request Fields Reference + +### Bulk Create Request + +| Property | Type | Required | Default | Description | +| :--- | :--- | :--- | :--- | :--- | +| `connection` | object | Yes (new only) | — | Connection details to create before bulk creation | +| `schemas` | list[string] | Yes | — | List of schema names to create datastores for | +| `database` | string | No | `null` | Database/catalog name (for connectors with catalog hierarchy) | +| `name_template` | string | No | `{connection_name}_{{schema}}` | Naming pattern with `{{schema}}` placeholder | +| `description` | string | No | `null` | Description applied to all created datastores | +| `enrichment_only` | boolean | No | `false` | Whether created datastores are enrichment-only | +| `enrichment_datastore_id` | integer | No | `null` | Existing enrichment datastore ID to link to each created datastore | +| `enrichment_prefix` | string | No | `null` | Enrichment container prefix name | +| `enrichment_source_record_limit` | integer | No | `10` | Max source records written to enrichment per anomaly (1–1,000,000,000) | +| `enrichment_remediation_strategy` | string | No | `"none"` | Replication strategy: `"none"`, `"append"`, or `"overwrite"` | +| `high_count_rollup_threshold` | integer | No | `10` | Max anomalies per check before rolling up (1–1,000) | +| `group_id` | integer | No | `null` | Existing datastore group ID to assign to all datastores | +| `group` | object | No | `null` | Inline group to create and assign (mutually exclusive with `group_id`) | +| `tags` | list[string] | No | `null` | Tags to apply to all created datastores | +| `teams` | list[string] | No | `null` | Teams to assign to all created datastores | +| `trigger_catalog` | boolean | No | `null` | Whether to trigger a sync operation after each datastore creation | + +!!! warning + `group_id` and `group` are mutually exclusive — provide only one. + +### Validate Request + +| Property | Type | Required | Default | Description | +| :--- | :--- | :--- | :--- | :--- | +| `schemas` | list[string] | Yes | — | List of schema names to validate (max 50) | +| `database` | string | No | `null` | Database/catalog name | +| `enrichment_only` | boolean | No | `false` | When `true`, validates write permissions (DDL) in addition to read | + +--- + +## Practical Guides + +### End-to-End: Onboard All Schemas from a Snowflake Database + +A common scenario for data engineers: you have a Snowflake database with multiple schemas and want to onboard them all into Qualytics with enrichment and auto-sync. + +??? example "Complete workflow — Snowflake with new connection" + + **1. Discover what databases are available:** + + ```bash + curl -X POST "https://your-instance.qualytics.io/api/connections/catalogs" \ + -H "Authorization: Bearer YOUR_TOKEN" \ + -H "Content-Type: application/json" \ + -d '{ + "name": "snowflake_prod", + "type": "snowflake", + "host": "acme.snowflakecomputing.com", + "username": "qualytics_user", + "password": "your_password", + "parameters": { + "role": "qualytics_read_role", + "warehouse": "qualytics_wh" + } + }' + ``` + + Response: `["production_db", "staging_db", "analytics_db"]` + + **2. Discover schemas in your target database:** + + ```bash + curl -X POST "https://your-instance.qualytics.io/api/connections/schemas?catalog=production_db" \ + -H "Authorization: Bearer YOUR_TOKEN" \ + -H "Content-Type: application/json" \ + -d '{ ... same connection payload ... }' + ``` + + Response: `["public", "sales", "finance", "hr", "marketing"]` + + **3. Validate the schemas you want to onboard:** + + ```bash + curl -X POST "https://your-instance.qualytics.io/api/connections/datastores/validate" \ + -H "Authorization: Bearer YOUR_TOKEN" \ + -H "Content-Type: application/json" \ + -d '{ + "connection": { ... same connection payload ... }, + "database": "production_db", + "schemas": ["public", "sales", "finance", "hr", "marketing"] + }' + ``` + + Response — check that all schemas passed: + + ```json + { + "results": [ + { "schema_name": "public", "status": "success" }, + { "schema_name": "sales", "status": "success" }, + { "schema_name": "finance", "status": "success" }, + { "schema_name": "hr", "status": "success" }, + { "schema_name": "marketing", "status": "success" } + ] + } + ``` + + **4. Bulk create all datastores:** + + ```bash + curl -X POST "https://your-instance.qualytics.io/api/connections/datastores/bulk" \ + -H "Authorization: Bearer YOUR_TOKEN" \ + -H "Content-Type: application/json" \ + -d '{ + "connection": { ... same connection payload ... }, + "database": "production_db", + "schemas": ["public", "sales", "finance", "hr", "marketing"], + "name_template": "prod_snowflake_{{schema}}", + "description": "Snowflake production schemas", + "teams": ["Data Platform"], + "trigger_catalog": true, + "enrichment_datastore_id": 42, + "enrichment_source_record_limit": 100, + "enrichment_remediation_strategy": "append", + "tags": ["production", "snowflake"] + }' + ``` + + Response: + + ```json + { + "created": [101, 102, 103, 104, 105], + "errors": [] + } + ``` + + Done — 5 datastores created, all linked to enrichment, sync running on each. + +--- + +### End-to-End: Onboard PostgreSQL Schemas Using an Existing Connection + +A common scenario for analysts: your DBA already configured a connection in Qualytics and you want to add new schemas from it. + +??? example "Complete workflow — PostgreSQL with existing connection" + + **1. Find available databases (catalogs):** + + ```bash + curl -X GET "https://your-instance.qualytics.io/api/connections/7/catalogs" \ + -H "Authorization: Bearer YOUR_TOKEN" + ``` + + Response — shows which databases already have datastores: + + ```json + [ + { + "name": "app_db", + "existing_datastores": [ + { "id": 50, "name": "app_db_public" } + ] + }, + { + "name": "analytics_db", + "existing_datastores": [] + } + ] + ``` + + **2. Discover schemas in the `analytics_db` database:** + + ```bash + curl -X GET "https://your-instance.qualytics.io/api/connections/7/schemas?catalog=analytics_db" \ + -H "Authorization: Bearer YOUR_TOKEN" + ``` + + Response — `existing_datastores` helps you avoid duplicates: + + ```json + [ + { "name": "raw", "existing_datastores": [] }, + { "name": "staging", "existing_datastores": [] }, + { "name": "curated", "existing_datastores": [] } + ] + ``` + + **3. Validate and create:** + + ```bash + curl -X POST "https://your-instance.qualytics.io/api/connections/7/datastores/bulk" \ + -H "Authorization: Bearer YOUR_TOKEN" \ + -H "Content-Type: application/json" \ + -d '{ + "database": "analytics_db", + "schemas": ["raw", "staging", "curated"], + "name_template": "analytics_{{schema}}", + "teams": ["Analytics"], + "trigger_catalog": true + }' + ``` + + Response: + + ```json + { + "created": [201, 202, 203], + "errors": [] + } + ``` + + **4. Link enrichment after creation:** + + ```bash + for ID in 201 202 203; do + curl -X PATCH "https://your-instance.qualytics.io/api/datastores/$ID/enrichment/42" \ + -H "Authorization: Bearer YOUR_TOKEN" + done + ``` + +--- + +### End-to-End: Onboard BigQuery Datasets + +BigQuery uses "project" as catalog and "dataset" as schema. The flow is the same but the terminology differs. + +??? example "Complete workflow — BigQuery with new connection" + + **1. Discover available projects:** + + ```bash + curl -X POST "https://your-instance.qualytics.io/api/connections/catalogs" \ + -H "Authorization: Bearer YOUR_TOKEN" \ + -H "Content-Type: application/json" \ + -d '{ + "name": "bigquery_prod", + "type": "bigquery", + "host": "https://www.googleapis.com/bigquery/v2", + "username": "project-id", + "password": "service-account-key-json", + "parameters": {} + }' + ``` + + Response: `["my-project-prod", "my-project-staging"]` + + **2. Discover datasets in your project:** + + ```bash + curl -X POST "https://your-instance.qualytics.io/api/connections/schemas?catalog=my-project-prod" \ + -H "Authorization: Bearer YOUR_TOKEN" \ + -H "Content-Type: application/json" \ + -d '{ ... same connection payload ... }' + ``` + + Response: `["raw_events", "user_analytics", "financial_data"]` + + **3. Bulk create:** + + ```bash + curl -X POST "https://your-instance.qualytics.io/api/connections/datastores/bulk" \ + -H "Authorization: Bearer YOUR_TOKEN" \ + -H "Content-Type: application/json" \ + -d '{ + "connection": { ... same connection payload ... }, + "database": "my-project-prod", + "schemas": ["raw_events", "user_analytics", "financial_data"], + "name_template": "bq_{{schema}}", + "teams": ["Data Engineering"], + "trigger_catalog": true + }' + ``` + + Response: + + ```json + { + "created": [301, 302, 303], + "errors": [] + } + ``` + +--- + +### Handling Errors and Retrying Failed Schemas + +When bulk creation partially fails, some datastores are created and some are not. Here is how to handle this. + +??? example "Handling partial failures" + + **Scenario**: You tried to create 5 datastores but 2 failed due to permission issues. + + **Response from bulk create:** + + ```json + { + "created": [101, 102, 103], + "errors": [ + { "schema_name": "restricted_schema", "error": "Permission denied" }, + { "schema_name": "locked_schema", "error": "Schema is locked for maintenance" } + ] + } + ``` + + **What to do:** + + 1. The 3 successful datastores (`101`, `102`, `103`) are fully operational — no action needed. + 2. Fix the underlying issues for the failed schemas (e.g., grant `USAGE` permission, wait for maintenance to complete). + 3. Retry only the failed schemas: + + ```bash + curl -X POST "https://your-instance.qualytics.io/api/connections/7/datastores/bulk" \ + -H "Authorization: Bearer YOUR_TOKEN" \ + -H "Content-Type: application/json" \ + -d '{ + "database": "production_db", + "schemas": ["restricted_schema", "locked_schema"], + "name_template": "prod_{{schema}}", + "teams": ["Data Platform"], + "trigger_catalog": true, + "enrichment_datastore_id": 42 + }' + ``` + +--- + +### Automation Script + +A ready-to-use Python script that automates the full multi-schema onboarding flow: discover schemas, validate, create, and link enrichment. + +??? example "Python automation script" + + ```python + import requests + + BASE_URL = "https://your-instance.qualytics.io/api" + TOKEN = "YOUR_TOKEN" + HEADERS = { + "Authorization": f"Bearer {TOKEN}", + "Content-Type": "application/json", + } + + CONNECTION_ID = 7 + DATABASE = "production_db" + NAME_TEMPLATE = "prod_{{schema}}" + TEAMS = ["Data Platform"] + ENRICHMENT_ID = 42 + + # Step 1: Discover available schemas + resp = requests.get( + f"{BASE_URL}/connections/{CONNECTION_ID}/schemas", + headers=HEADERS, + params={"catalog": DATABASE}, + ) + resp.raise_for_status() + discovered = resp.json() + + # Filter out schemas that already have datastores + new_schemas = [ + s["name"] for s in discovered + if len(s["existing_datastores"]) == 0 + ] + print(f"Found {len(new_schemas)} new schemas: {new_schemas}") + + if not new_schemas: + print("No new schemas to onboard.") + exit() + + # Step 2: Validate connectivity + resp = requests.post( + f"{BASE_URL}/connections/{CONNECTION_ID}/datastores/validate", + headers=HEADERS, + json={ + "database": DATABASE, + "schemas": new_schemas, + }, + ) + resp.raise_for_status() + validation = resp.json() + + failed = [r for r in validation["results"] if r["status"] == "failure"] + if failed: + print(f"Warning: {len(failed)} schemas failed validation:") + for f in failed: + print(f" - {f['schema_name']}: {f.get('message', 'Unknown error')}") + + valid_schemas = [ + r["schema_name"] for r in validation["results"] + if r["status"] == "success" + ] + print(f"Proceeding with {len(valid_schemas)} valid schemas.") + + # Step 3: Bulk create datastores + resp = requests.post( + f"{BASE_URL}/connections/{CONNECTION_ID}/datastores/bulk", + headers=HEADERS, + json={ + "database": DATABASE, + "schemas": valid_schemas, + "name_template": NAME_TEMPLATE, + "teams": TEAMS, + "trigger_catalog": True, + "enrichment_datastore_id": ENRICHMENT_ID, + }, + ) + resp.raise_for_status() + result = resp.json() + + print(f"Created {len(result['created'])} datastores: {result['created']}") + if result["errors"]: + print(f"Errors: {result['errors']}") + ``` + +--- + +## Permission Summary + +| Operation | Minimum Permission | +| :--- | :--- | +| Discover catalogs | Manager | +| Discover schemas | Manager | +| Validate schemas | Manager | +| Bulk create datastores | Manager | +| Link enrichment to source datastore | Member | +| Unlink enrichment from source datastore | Admin | + +!!! tip + For complete API documentation, including full request/response schemas, visit the [API docs](https://demo.qualytics.io/api/docs). diff --git a/docs/source-datastore/add-datastores/multi-schema/concepts/multi-schema-faq.md b/docs/source-datastore/add-datastores/multi-schema/concepts/multi-schema-faq.md new file mode 100644 index 0000000000..f433c851d8 --- /dev/null +++ b/docs/source-datastore/add-datastores/multi-schema/concepts/multi-schema-faq.md @@ -0,0 +1,210 @@ +# Multi-Schema Creation FAQ + +## General + +### What is multi-schema source datastore creation? + +It is a feature that allows you to discover and select multiple schemas from a single connection and create all corresponding source datastores in one step, instead of adding each schema individually. + +### Which connectors support multi-schema creation? + +All JDBC-based connectors support multi-schema creation. DFS-based datastores (Amazon S3, Azure Datalake Storage, Google Cloud Storage) do not support it because they do not use a catalog/schema hierarchy. See the full list in [Supported Connectors](supported-connectors.md). + +### Can I use multi-schema creation with an existing connection? + +Yes. You can either create a new connection or reuse an existing one. When using an existing connection, you can only configure the datastore-specific properties (catalog, schemas, name template, teams, group, and initiate sync). + +### Is there a limit on how many schemas I can select at once? + +There is no hard limit on the number of schemas you can select for creation. However, the validation endpoint accepts a maximum of **50 schemas per request**. If you have more than 50 schemas, validate them in batches. The creation endpoint itself has no such limit. + +### Can I edit the datastores created in bulk after creation? + +Yes. Each datastore created through the multi-schema flow is a regular source datastore. You can edit, delete, or reconfigure them individually through the [Datastore Settings](../../../managing-datastores/settings-overview.md) just like any other datastore. + +### Can I delete all datastores created in bulk at once? + +No. There is no bulk delete option tied to the multi-schema creation. Each datastore must be deleted individually through the [Delete Datastore](../../../managing-datastores/delete-datastore.md) settings. + +### Can I use multi-schema creation for enrichment datastores? + +Yes. You can set `enrichment_only: true` in the bulk creation request to create enrichment datastores instead of source datastores. This is useful when you need to set up enrichment across multiple schemas from the same connection. + +### A new schema was added to my database. How do I add it to Qualytics? + +Run the multi-schema flow again using the same connection. The schema discovery will show all available schemas, including the new one. Schemas that already have a datastore will display a warning icon, so you can easily identify and select only the new schema. + +Alternatively, you can use the [API](multi-schema-api.md) to discover and create the datastore for the new schema programmatically. + +### How do I know which datastores were created by multi-schema vs. individually? + +There is no visual indicator that distinguishes them. Once created, a datastore from multi-schema is identical to one created individually. If you need to track this, use a consistent naming convention through the name template (e.g., `batch_{{schema}}`) or apply a specific tag during creation. + +## Permissions + +### Who can use multi-schema creation? + +Users with the **Manager** or **Admin** role can use multi-schema creation. This is the same permission required to create a single datastore. + +### Are there different permissions for discovery, validation, and creation? + +No. All operations in the multi-schema flow — catalog discovery, schema discovery, validation, and bulk creation — require the same **Manager** role. + +!!! info "Full Permission Matrix" + For a complete overview of all permissions across all features, see the [Team Permissions](../../../../settings/security/team-permissions.md) page. + +### Can I assign different teams to different schemas? + +No. In the multi-schema flow, all source datastores created in the batch are assigned to the same teams. If you need different team assignments, you can change them individually after creation through the [Edit Datastore](../../../managing-datastores/edit-datastore.md) settings. + +## Schema Discovery + +### How are schemas discovered? + +When you provide connection credentials and select a catalog (if applicable), Qualytics queries the database's information schema to retrieve the list of available schemas. This happens in real-time during the creation flow. + +### What does the warning icon next to a schema mean? + +The warning icon indicates that the schema already has one or more existing datastores in Qualytics. A tooltip shows which datastores are associated. You can still select it, but this will create a duplicate datastore for that schema. + +### Can I refresh the list of available schemas? + +Yes. Click the **refresh** button next to the catalog or schema dropdown to re-query the connection and reload the available options. + +### Why are some schemas missing from the list? + +The schema discovery relies on the permissions of the database user configured in the connection. If a schema does not appear, the user may not have the necessary privileges (e.g., `USAGE` on the schema) to see it. Verify your database user's permissions with your DBA. + +### Does discovery show system schemas (e.g., information_schema, pg_catalog)? + +Yes — discovery returns all schemas visible to the database user, including system schemas. You can simply skip selecting them. Only the schemas you explicitly select will have datastores created. + +## Name Templates + +### What is a name template? + +A name template is a naming pattern that uses the `{{schema}}` placeholder. When datastores are created, `{{schema}}` is replaced with the actual schema name. For example, `production_{{schema}}` with schema `public` becomes `production_public`. + +### Is `{{schema}}` the only placeholder available? + +Yes. Currently, `{{schema}}` is the only supported placeholder in the name template. + +### Can I use the name template without the `{{schema}}` placeholder? + +No. The `{{schema}}` placeholder is required to ensure each datastore gets a unique name. Without it, all datastores would have the same name. + +### Can I preview the generated names before creating? + +Yes. A preview is displayed below the name template field showing how the first few datastores will be named based on your selected schemas. + +### What happens if the generated name conflicts with an existing datastore? + +The creation will proceed and a new datastore with that name will be created. Qualytics does not enforce unique datastore names, but using distinct name templates helps avoid confusion. + +### Can I rename a datastore after it was created by multi-schema? + +Yes. You can rename any datastore through the [Edit Datastore](../../../managing-datastores/edit-datastore.md) settings, regardless of how it was created. + +## Enrichment + +### Is linking an enrichment datastore required? + +No. The enrichment linking step is optional. You can click **Finish** after the source datastore configuration to skip it. However, linking an enrichment datastore is recommended for full data quality visibility. + +### Can I link different enrichment datastores to different schemas? + +No. In the multi-schema flow, all source datastores created in the batch are linked to the same enrichment datastore. If you need different enrichment datastores for different schemas, you can change the enrichment linking individually after creation through the [Link Enrichment](../../../managing-datastores/link-enrichment.md) settings. + +### Can I change the enrichment datastore after creation? + +Yes. You can unlink and relink a different enrichment datastore for any individual source datastore through the [Link Enrichment](../../../managing-datastores/link-enrichment.md) and [Unlink Enrichment](../../../managing-datastores/unlink-enrichment.md) settings. + +### What is the enrichment prefix? + +The prefix is a string prepended to table names when Qualytics writes metadata to the enrichment datastore. It helps uniquely identify which source datastore the enrichment data belongs to. It is auto-generated from the name template but can be customized. + +### Is the enrichment prefix the same for all datastores created in the batch? + +The prefix is auto-generated based on each datastore's name, so each datastore in the batch will have its own unique prefix derived from the name template and the schema name. + +## Integration with Other Features + +### Can I use multi-schema creation together with datastore grouping? + +Yes. You can assign all newly created datastores to a [datastore group](../../../managing-datastores/grouping/overview.md) during the creation flow by selecting a group in the form. All datastores in the batch will be assigned to the same group. + +### Can I add tags during multi-schema creation? + +Yes, through the API. The bulk creation request accepts a `tags` field that applies the specified tags to all created datastores. The UI form does not currently include a tags field during creation, but you can add tags to individual datastores after creation. + +### Do the created datastores automatically appear in my team's view? + +Yes. The datastores are assigned to the teams you select during creation and will be visible to all users in those teams according to their permission level. + +### Does multi-schema creation work with Secrets Management (HashiCorp Vault)? + +Yes. If your connection uses Secrets Management, you can use `${key}` references in connection properties. The secrets are resolved dynamically each time the connection is used, including during schema discovery and datastore creation. + +## Operations + +### What happens when I enable "Initiate Sync"? + +When enabled, a sync operation is automatically triggered on each newly created source datastore after creation. The sync operations run independently and detect containers and fields within each schema. + +### Do all sync operations run at the same time? + +The sync operations are triggered sequentially as each datastore is created, but they run independently and may execute in parallel depending on system capacity. + +### Can I schedule operations in bulk for the datastores created? + +There is no bulk scheduling option tied to multi-schema creation. Operations (sync, profile, scan) must be scheduled individually for each datastore. However, you can use the [API](multi-schema-api.md) to automate scheduling across multiple datastores programmatically. + +## Performance + +### Does multi-schema creation take longer with many schemas? + +Yes. Each schema is validated and created individually within the bulk operation. More schemas means more validation and creation steps. For very large numbers of schemas (50+), consider breaking the operation into smaller batches. + +### Is there a validation limit? + +Yes. The validation endpoint accepts a maximum of **50 schemas per request**. If you need to validate more, split them into multiple validation requests. The creation endpoint itself has no such limit. + +### Can I run multi-schema creation while other operations are running? + +Yes. Multi-schema creation does not block other operations. However, if you enable "Initiate Sync", the sync operations that start after creation will compete for system resources with any other running operations. + +## Errors and Troubleshooting + +### What happens if some schemas fail validation? + +The validation runs per-schema and reports individual results. You can review which schemas failed, fix the underlying issues (e.g., permissions), and retry. Only schemas that pass validation will be created. + +### What happens if some schemas fail during creation? + +The bulk creation operation is non-atomic — schemas that succeed are created, and schemas that fail are reported with error messages. You can retry the failed schemas individually or through a new multi-schema creation. + +### How do I retry only the schemas that failed? + +Run the multi-schema flow again (or use the API) with only the failed schema names. The previously created datastores will not be affected. See the [Handling Errors](multi-schema-api.md#handling-errors-and-retrying-failed-schemas) section in the API documentation for a practical example. + +### I don't see the catalog/schema dropdowns. Why? + +Make sure you have: + +1. Selected a JDBC connector (DFS connectors do not support multi-schema). +2. Provided valid connection credentials. +3. Clicked **Test Connection** or waited for the catalog/schema discovery to complete. + +If the connection credentials are invalid, the catalog and schema discovery will fail silently. + +### The schema list is empty even though my database has schemas. What should I do? + +This usually indicates a permissions issue. Verify that: + +1. The database user has `USAGE` or equivalent privileges on the schemas. +2. The catalog (database) was correctly selected (for two-step connectors). +3. The connection credentials are valid — try clicking the **refresh** button to retry discovery. + +### What happens to the datastores if the connection is deleted? + +Deleting a connection does not delete the datastores that were created from it. The datastores will lose their connection reference and you will need to reconfigure a connection for them to remain operational. diff --git a/docs/source-datastore/add-datastores/multi-schema/concepts/supported-connectors.md b/docs/source-datastore/add-datastores/multi-schema/concepts/supported-connectors.md new file mode 100644 index 0000000000..72d260b2e2 --- /dev/null +++ b/docs/source-datastore/add-datastores/multi-schema/concepts/supported-connectors.md @@ -0,0 +1,55 @@ +# Supported Connectors + +Multi-schema discovery is available for JDBC-based connectors that support schema-level separation. DFS-based datastores (Amazon S3, Azure Datalake Storage, Google Cloud Storage) do not support multi-schema creation because they do not use a catalog/schema hierarchy. + +## Connector Reference + +The table below shows each supported connector, the field used for catalog selection (if any), and the field used as the schema target for multi-select. + +| Connector | Catalog Field | Schema Field | +|-----------|--------------|--------------| +| [Athena](../../athena.md) | Catalog | Schema | +| [BigQuery](../../bigquery.md) | Project | Dataset | +| [Databricks](../../databricks.md) | Catalog | Schema | +| [DB2](../../db2.md) | — | Schema | +| [Dremio](../../dremio.md) | — | Schema | +| [Fabric Analytics](../../fabric-analytics.md) | Database | Schema | +| [Hive](../../hive.md) | — | Database | +| [MariaDB](../../maria-db.md) | — | Database | +| [Microsoft SQL Server](../../microsoft-sql-server.md) | Database | Schema | +| [MySQL](../../mysql.md) | — | Database | +| [Oracle](../../oracle.md) | — | Schema | +| [PostgreSQL](../../postgresql.md) | Database | Schema | +| [Presto](../../presto.md) | Catalog | Schema | +| [Redshift](../../redshift.md) | Database | Schema | +| [Snowflake](../../snowflake.md) | Database | Schema | +| [Synapse](../../synapse.md) | Database | Schema | +| [Teradata](../../teradata.md) | — | Database | +| [Timescale DB](../../timescale-db.md) | — | Schema | +| [Trino](../../trino.md) | Catalog | Schema | + +## Understanding the Columns + +### Catalog Field + +The **Catalog Field** is the first-level hierarchy used to group schemas. For connectors that show a "—", no catalog selection is needed — schemas are discovered directly from the connection. + +Examples: + +- **PostgreSQL**: The catalog is the `Database`. You first select a database, then discover schemas within it. +- **BigQuery**: The catalog is the `Project`. You first select a project, then discover datasets within it. +- **Oracle**: There is no catalog level. Schemas are discovered directly. + +### Schema Field + +The **Schema Field** is the target for multi-select. This is the level at which individual source datastores are created. + +!!! note + For connectors like **MySQL**, **MariaDB**, **Hive**, and **Teradata**, the "database" field acts as the schema target since these systems do not have a separate schema concept. Each "database" in these systems is treated as a schema for the purpose of multi-schema creation. + +## Two-Step vs. Single-Step Discovery + +| Flow Type | Connectors | Description | +|-----------|-----------|-------------| +| **Two-step** (Catalog → Schema) | PostgreSQL, Snowflake, BigQuery, SQL Server, Synapse, Databricks, Redshift, Fabric Analytics, Presto, Trino, Athena | First select a catalog, then discover and select schemas within it. | +| **Single-step** (Schema only) | Oracle, DB2, MySQL, MariaDB, Hive, Teradata, Dremio, Timescale DB | Schemas are discovered directly without a catalog selection. | diff --git a/docs/source-datastore/add-datastores/multi-schema/overview.md b/docs/source-datastore/add-datastores/multi-schema/overview.md new file mode 100644 index 0000000000..bdd7b3ebf8 --- /dev/null +++ b/docs/source-datastore/add-datastores/multi-schema/overview.md @@ -0,0 +1,85 @@ +# Multi-Schema Overview + +Multi-Schema Creation allows you to discover and select multiple schemas from a single connection and create all corresponding source datastores in one step. Instead of adding each schema individually, you can onboard multiple schemas at once, streamlining the setup process for organizations with many schemas across their data platforms. + +![multi-schema-overview](../../../assets/source-datastores/add-datastores/connections/multi-schema/overview/overview.png) + +## Why Multi-Schema Creation Matters + +As organizations scale their data infrastructure, databases often contain dozens or even hundreds of schemas. Onboarding each schema individually is time-consuming and error-prone. Multi-Schema Creation solves this by letting you: + +- **Batch onboarding**: Create multiple source datastores from a single connection in one operation. +- **Schema discovery**: Automatically discover available catalogs and schemas from your connection. +- **Optional enrichment linking**: Link all newly created source datastores to a single enrichment datastore during the same flow. +- **Name templates**: Use a naming pattern with a `{{schema}}` placeholder to generate consistent datastore names. + +!!! note "Permissions" + Multi-schema creation requires the **Manager** or **Admin** role. This is the same permission required to create a single datastore. For more details, see the [Team Permissions](../../../settings/security/team-permissions.md) page. + +## Getting Started + +To create multi-schema datastores, use the **Add Source Datastore** flow. The multi-schema options will appear automatically when you select a supported JDBC connector. + +
+ +- :material-plus-circle:{ .lg .middle } **Add Datastore with a New Connection** + + --- + + Create multiple source datastores by setting up a new connection from scratch. + + [:octicons-arrow-right-24: Setup a Connection](../connections/new-connection.md) + +- :material-link-variant:{ .lg .middle } **Add Datastore with an Existing Connection** + + --- + + Create multiple source datastores by reusing credentials from a connection that already exists. + + [:octicons-arrow-right-24: Reuse a Connection](../connections/existing-connection.md) + +
+ +## Deep Dive + +
+ +- :material-cog-outline:{ .lg .middle } **How It Works** + + --- + + Understand the multi-schema creation flow, schema discovery, and name templates. + + [:octicons-arrow-right-24: How It Works](concepts/how-it-works.md) + +- :material-database-check:{ .lg .middle } **Supported Connectors** + + --- + + See which connectors support multi-schema discovery and their catalog/schema mappings. + + [:octicons-arrow-right-24: Supported Connectors](concepts/supported-connectors.md) + +
+ +## Resources + +
+ +- :material-api:{ .lg .middle } **API** + + --- + + API endpoints for bulk datastore creation, schema discovery, and validation. + + [:octicons-arrow-right-24: API](concepts/multi-schema-api.md) + +- :material-frequently-asked-questions:{ .lg .middle } **FAQ** + + --- + + Answers to common questions about multi-schema source datastore creation. + + [:octicons-arrow-right-24: FAQ](concepts/multi-schema-faq.md) + +
diff --git a/docs/source-datastore/add-datastores/overview-of-a-datastore.md b/docs/source-datastore/add-datastores/overview-of-a-datastore.md index bb84b53497..ab0395d89f 100644 --- a/docs/source-datastore/add-datastores/overview-of-a-datastore.md +++ b/docs/source-datastore/add-datastores/overview-of-a-datastore.md @@ -81,6 +81,18 @@ Qualytics provides verified connectors for the following source datastores: | 21. | [Timescale DB](../add-datastores/timescale-db.md)| | 22. | [Trino](../add-datastores/trino.md)| +## Multi-Schema Source Datastore Creation + +For JDBC-based connectors, Qualytics supports creating multiple source datastores from a single connection in one step. This is useful when your database has multiple schemas that need to be onboarded simultaneously. + +With multi-schema creation, you can: + +* Automatically discover available catalogs and schemas from a connection. +* Select multiple schemas and create all corresponding datastores in a single operation. +* Optionally link all newly created datastores to an enrichment datastore. + +For detailed instructions, refer to the [**Multi-Schema Source Datastore Creation**](../add-datastores/multi-schema/overview.md) documentation. + ## Connection Management To connect to a datastore, users must provide the required connection details, such as Host/Port or URI. These fields may vary depending on the datastore and are essential for establishing a secure and reliable connection to the target database. diff --git a/docs/source-datastore/add-datastores/overview-of-a-jdbc-datastore.md b/docs/source-datastore/add-datastores/overview-of-a-jdbc-datastore.md index 1e0c4883fe..86dd9bf82f 100644 --- a/docs/source-datastore/add-datastores/overview-of-a-jdbc-datastore.md +++ b/docs/source-datastore/add-datastores/overview-of-a-jdbc-datastore.md @@ -31,6 +31,12 @@ Qualytics provides verified connectors for the following relational databases: * [Timescale DB](../add-datastores/timescale-db.md) * [Trino](../add-datastores/trino.md) +## Multi-Schema Source Datastore Creation + +JDBC datastores support multi-schema creation, allowing you to discover and select multiple schemas from a single connection and create all corresponding source datastores in one step. This eliminates the need to add each schema individually. + +For detailed instructions, refer to the [**Multi-Schema Source Datastore Creation**](multi-schema/overview.md) documentation. + ## Connection Details To connect to a JDBC datastore, users must provide the required connection details, such as Host/Port or URI. These fields may vary depending on the datastore and are essential for establishing a secure and reliable connection to the target database. diff --git a/docs/source-datastore/navigation-and-organization/assign-tags.md b/docs/source-datastore/managing-datastores/assign-tags.md similarity index 71% rename from docs/source-datastore/navigation-and-organization/assign-tags.md rename to docs/source-datastore/managing-datastores/assign-tags.md index 35b78d4be6..7a37c012e1 100644 --- a/docs/source-datastore/navigation-and-organization/assign-tags.md +++ b/docs/source-datastore/managing-datastores/assign-tags.md @@ -6,23 +6,23 @@ In this documentation, we will explore the steps to assign a tag to the datastor **Step 1**: Login in to your Qualytics account and select the **datastore** from the left menu on which you want to assign a tag. -![add-datastore](../../assets/source-datastores/navigation-and-organization/assign-tags/add-datastore-1.png) +![add-datastore](../../assets/source-datastores/managing-datastores/assign-tags/add-datastore-1.png) **Step 2**: Click on **Assign Tag to this Datastore** located at the bottom-left corner of the interface. -![assign-tag](../../assets/source-datastores/navigation-and-organization/assign-tags/assign-tag-2.png) +![assign-tag](../../assets/source-datastores/managing-datastores/assign-tags/assign-tag-2.png) **Step 3**: A drop-up menu will appear, providing you with a list of tags. Assign an appropriate **tag** to your datastore to simplify sorting, accessing, and managing data. -![select-tag](../../assets/source-datastores/navigation-and-organization/assign-tags/select-tag-3.png) +![select-tag](../../assets/source-datastores/managing-datastores/assign-tags/select-tag-3.png) You can also create a new tag by clicking on the call to action (➕) button. -![call-to-action](../../assets/source-datastores/navigation-and-organization/assign-tags/action-4.png) +![call-to-action](../../assets/source-datastores/managing-datastores/assign-tags/action-4.png) A modal window will appear, providing the options to create the tag. Enter the required values to get started. -![modal-window](../../assets/source-datastores/navigation-and-organization/assign-tags/modal-window-5.png) +![modal-window](../../assets/source-datastores/managing-datastores/assign-tags/modal-window-5.png) For more information on creating tags, refer to the [Add Tag section](../../tags/add-tag.md). @@ -30,4 +30,4 @@ In this documentation, we will explore the steps to assign a tag to the datastor For demonstration, we have assigned the **High** tag for the Snowflake source datastore **Covid-19 Data**, so it will automatically be applied to all related tables and checks within the datastore. -![tag-snowflake](../../assets/source-datastores/navigation-and-organization/assign-tags/tag-snowflake-6.png) +![tag-snowflake](../../assets/source-datastores/managing-datastores/assign-tags/tag-snowflake-6.png) diff --git a/docs/source-datastore/datastore-settings/delete-datastore.md b/docs/source-datastore/managing-datastores/delete-datastore.md similarity index 59% rename from docs/source-datastore/datastore-settings/delete-datastore.md rename to docs/source-datastore/managing-datastores/delete-datastore.md index 6f1c70ed05..7ae6f5fc26 100644 --- a/docs/source-datastore/datastore-settings/delete-datastore.md +++ b/docs/source-datastore/managing-datastores/delete-datastore.md @@ -2,14 +2,14 @@ **Step 1**: Click on the **Delete** option in the settings icon. -![delete](../../assets/source-datastores/datastore-settings/delete-datastore/delete-22.png) +![delete](../../assets/source-datastores/managing-datastores/delete-datastore/delete-22.png) **Step 2**: A modal window titled **Delete Datastore** will appear. -![delete-datastore](../../assets/source-datastores/datastore-settings/delete-datastore/delete-datastore-23.png) +![delete-datastore](../../assets/source-datastores/managing-datastores/delete-datastore/delete-datastore-23.png) **Step 3**: Enter the name of the datastore in the given field (confirmation check) and then click on the **I’M SURE, DELETE THIS DATASTORE** button to delete the datastore. -![confirm-delete](../../assets/source-datastores/datastore-settings/delete-datastore/confirm-delete-24.png) +![confirm-delete](../../assets/source-datastores/managing-datastores/delete-datastore/confirm-delete-24.png) After clicking the **I’M SURE, DELETE THIS DATASTORE** button, a success notification appears confirming the deletion. \ No newline at end of file diff --git a/docs/source-datastore/datastore-settings/edit-datastore.md b/docs/source-datastore/managing-datastores/edit-datastore.md similarity index 62% rename from docs/source-datastore/datastore-settings/edit-datastore.md rename to docs/source-datastore/managing-datastores/edit-datastore.md index f3adb27b51..1dad5103a9 100644 --- a/docs/source-datastore/datastore-settings/edit-datastore.md +++ b/docs/source-datastore/managing-datastores/edit-datastore.md @@ -2,20 +2,20 @@ **Step 1:** Click on the **Edit** option. -![edit-datastore](../../assets/source-datastores/datastore-settings/edit-datastore/edit-datastore-3.png) +![edit-datastore](../../assets/source-datastores/managing-datastores/edit-datastore/edit-datastore-3.png) **Step 2:** After selecting the **Edit** option, a modal window will appear, displaying the connection details. This window allows you to modify any specific connection details. -![connection-details](../../assets/source-datastores/datastore-settings/edit-datastore/connection-details-4.png) +![connection-details](../../assets/source-datastores/managing-datastores/edit-datastore/connection-details-4.png) **Step 3:** After editing the connection details, click on the **Test Connection** button to check and verify its connection. -![test](../../assets/source-datastores/datastore-settings/edit-datastore/test.png) +![test](../../assets/source-datastores/managing-datastores/edit-datastore/test.png) If the credentials and provided connection details are verified, a success message will be displayed indicating that the connection has been verified. **Step 4:** Click on the **Save** button. -![save-datastore](../../assets/source-datastores/datastore-settings/edit-datastore/save-datastore-5.png) +![save-datastore](../../assets/source-datastores/managing-datastores/edit-datastore/save-datastore-5.png) After clicking on the **Save** button, a success notification appears on the screen showing the action was completed successfully. \ No newline at end of file diff --git a/docs/source-datastore/navigation-and-organization/grouping/concepts/grouping-api.md b/docs/source-datastore/managing-datastores/grouping/concepts/grouping-api.md similarity index 100% rename from docs/source-datastore/navigation-and-organization/grouping/concepts/grouping-api.md rename to docs/source-datastore/managing-datastores/grouping/concepts/grouping-api.md diff --git a/docs/source-datastore/navigation-and-organization/grouping/concepts/grouping-faq.md b/docs/source-datastore/managing-datastores/grouping/concepts/grouping-faq.md similarity index 100% rename from docs/source-datastore/navigation-and-organization/grouping/concepts/grouping-faq.md rename to docs/source-datastore/managing-datastores/grouping/concepts/grouping-faq.md diff --git a/docs/source-datastore/navigation-and-organization/grouping/concepts/permissions.md b/docs/source-datastore/managing-datastores/grouping/concepts/permissions.md similarity index 100% rename from docs/source-datastore/navigation-and-organization/grouping/concepts/permissions.md rename to docs/source-datastore/managing-datastores/grouping/concepts/permissions.md diff --git a/docs/source-datastore/navigation-and-organization/grouping/concepts/understanding-grouping.md b/docs/source-datastore/managing-datastores/grouping/concepts/understanding-grouping.md similarity index 100% rename from docs/source-datastore/navigation-and-organization/grouping/concepts/understanding-grouping.md rename to docs/source-datastore/managing-datastores/grouping/concepts/understanding-grouping.md diff --git a/docs/source-datastore/navigation-and-organization/grouping/managing-groups/assign-a-datastore.md b/docs/source-datastore/managing-datastores/grouping/managing-groups/assign-a-datastore.md similarity index 57% rename from docs/source-datastore/navigation-and-organization/grouping/managing-groups/assign-a-datastore.md rename to docs/source-datastore/managing-datastores/grouping/managing-groups/assign-a-datastore.md index 3c791efb15..21a5386d25 100644 --- a/docs/source-datastore/navigation-and-organization/grouping/managing-groups/assign-a-datastore.md +++ b/docs/source-datastore/managing-datastores/grouping/managing-groups/assign-a-datastore.md @@ -9,19 +9,19 @@ This guide walks you through the steps to assign a datastore to an existing grou **Step 1**: In the tree view, hover over the datastore you want to assign. An **Assign to group** icon will appear. -![assign-icon](../../../../assets/source-datastores/navigation-and-organization/grouping/managing-groups/assign-a-datastore/assign-icon.png) +![assign-icon](../../../../assets/source-datastores/managing-datastores/grouping/managing-groups/assign-a-datastore/assign-icon.png) **Step 2**: Click the **Assign to group** button. A dropdown will appear with the list of available groups. -![assign-dropdown](../../../../assets/source-datastores/navigation-and-organization/grouping/managing-groups/assign-a-datastore/assign-dropdown.png) +![assign-dropdown](../../../../assets/source-datastores/managing-datastores/grouping/managing-groups/assign-a-datastore/assign-dropdown.png) **Step 3**: Select the group you want to assign the datastore to. -![datastore-assigned](../../../../assets/source-datastores/navigation-and-organization/grouping/managing-groups/assign-a-datastore/datastore-assigned.png) +![datastore-assigned](../../../../assets/source-datastores/managing-datastores/grouping/managing-groups/assign-a-datastore/datastore-assigned.png) **Step 4**: The datastore will immediately move under the selected group in the tree view, and the group icon will be displayed next to the datastore. -![datastore-in-group](../../../../assets/source-datastores/navigation-and-organization/grouping/managing-groups/assign-a-datastore/datastore-in-group.png) +![datastore-in-group](../../../../assets/source-datastores/managing-datastores/grouping/managing-groups/assign-a-datastore/datastore-in-group.png) !!! tip You can also **create a new group** directly from the assign menu by clicking the **Create** button, if you have the Manager role. diff --git a/docs/source-datastore/navigation-and-organization/grouping/managing-groups/create-a-group.md b/docs/source-datastore/managing-datastores/grouping/managing-groups/create-a-group.md similarity index 64% rename from docs/source-datastore/navigation-and-organization/grouping/managing-groups/create-a-group.md rename to docs/source-datastore/managing-datastores/grouping/managing-groups/create-a-group.md index 5462abac60..6a6f3abde4 100644 --- a/docs/source-datastore/navigation-and-organization/grouping/managing-groups/create-a-group.md +++ b/docs/source-datastore/managing-datastores/grouping/managing-groups/create-a-group.md @@ -9,11 +9,11 @@ This guide walks you through the steps to create a new datastore group in Qualyt **Step 1**: Log in to your Qualytics account and click on the **Manage groups** button (bookmark icon) in the tree view header. -![manage-groups-button](../../../../assets/source-datastores/navigation-and-organization/grouping/managing-groups/create-a-group/manage-groups-button.png) +![manage-groups-button](../../../../assets/source-datastores/shared/manage-groups-button.png) **Step 2**: In the Manage Groups panel, click the **Create group** button. -![create-button](../../../../assets/source-datastores/navigation-and-organization/grouping/managing-groups/create-a-group/create-button.png) +![create-button](../../../../assets/source-datastores/managing-datastores/grouping/managing-groups/create-a-group/create-button.png) **Step 3**: A dialog will appear with the following fields. Fill in the details and click **Create** to create the group. @@ -22,11 +22,11 @@ This guide walks you through the steps to create a new datastore group in Qualyt | **Name** | Enter a unique name for the group (required, max 100 characters). | | **Icon** | Select an icon to visually identify the group. Available options: Bookmark, Folder, Shape, Chart, Flask, Star, Texture, Bronze, Silver, Gold. | -![create-form](../../../../assets/source-datastores/navigation-and-organization/grouping/managing-groups/create-a-group/create-form.png) +![create-form](../../../../assets/source-datastores/managing-datastores/grouping/managing-groups/create-a-group/create-form.png) **Step 4**: The new group will appear in the Manage Groups panel. -![group-created](../../../../assets/source-datastores/navigation-and-organization/grouping/managing-groups/create-a-group/group-created.png) +![group-created](../../../../assets/source-datastores/managing-datastores/grouping/managing-groups/create-a-group/group-created.png) !!! info The newly created group will only appear in the tree view once a datastore is assigned to it. Until then, it is only visible in the Manage Groups panel. diff --git a/docs/source-datastore/navigation-and-organization/grouping/managing-groups/delete-a-group.md b/docs/source-datastore/managing-datastores/grouping/managing-groups/delete-a-group.md similarity index 73% rename from docs/source-datastore/navigation-and-organization/grouping/managing-groups/delete-a-group.md rename to docs/source-datastore/managing-datastores/grouping/managing-groups/delete-a-group.md index 0375428ca2..29a1797a89 100644 --- a/docs/source-datastore/navigation-and-organization/grouping/managing-groups/delete-a-group.md +++ b/docs/source-datastore/managing-datastores/grouping/managing-groups/delete-a-group.md @@ -12,11 +12,11 @@ This guide walks you through the steps to delete a datastore group. **Step 1**: Click on the **Manage groups** button (bookmark icon) in the tree view header. -![manage-groups-button](../../../../assets/source-datastores/navigation-and-organization/grouping/managing-groups/create-a-group/manage-groups-button.png) +![manage-groups-button](../../../../assets/source-datastores/shared/manage-groups-button.png) **Step 2**: In the Manage Groups panel, find the group you want to delete and click the **Delete group** button (trash icon). The group will be deleted immediately. -![delete-button](../../../../assets/source-datastores/navigation-and-organization/grouping/managing-groups/delete-a-group/delete-button.png) +![delete-button](../../../../assets/source-datastores/managing-datastores/grouping/managing-groups/delete-a-group/delete-button.png) !!! warning The deletion happens immediately — there is no confirmation dialog. Make sure you are deleting the correct group. diff --git a/docs/source-datastore/navigation-and-organization/grouping/managing-groups/edit-a-group.md b/docs/source-datastore/managing-datastores/grouping/managing-groups/edit-a-group.md similarity index 66% rename from docs/source-datastore/navigation-and-organization/grouping/managing-groups/edit-a-group.md rename to docs/source-datastore/managing-datastores/grouping/managing-groups/edit-a-group.md index 379232d35b..e90afc853d 100644 --- a/docs/source-datastore/navigation-and-organization/grouping/managing-groups/edit-a-group.md +++ b/docs/source-datastore/managing-datastores/grouping/managing-groups/edit-a-group.md @@ -9,11 +9,11 @@ This guide walks you through the steps to edit an existing datastore group — i **Step 1**: Click on the **Manage groups** button (bookmark icon) in the tree view header. -![manage-groups-button](../../../../assets/source-datastores/navigation-and-organization/grouping/managing-groups/create-a-group/manage-groups-button.png) +![manage-groups-button](../../../../assets/source-datastores/shared/manage-groups-button.png) **Step 2**: In the Manage Groups panel, find the group you want to edit and click the **Edit group** button (pencil icon). -![edit-button](../../../../assets/source-datastores/navigation-and-organization/grouping/managing-groups/edit-a-group/edit-button.png) +![edit-button](../../../../assets/source-datastores/managing-datastores/grouping/managing-groups/edit-a-group/edit-button.png) **Step 3**: A dialog will appear with the group's current name and icon. Update the fields as needed and click **Save** to apply the changes. @@ -22,6 +22,6 @@ This guide walks you through the steps to edit an existing datastore group — i | **Name** | Change the group name (must remain unique, max 100 characters). | | **Icon** | Select a different icon for the group. | -![edit-form](../../../../assets/source-datastores/navigation-and-organization/grouping/managing-groups/edit-a-group/edit-form.png) +![edit-form](../../../../assets/source-datastores/managing-datastores/grouping/managing-groups/edit-a-group/edit-form.png) **Step 4**: The updated name and icon will be reflected in the tree view. diff --git a/docs/source-datastore/navigation-and-organization/grouping/managing-groups/remove-a-datastore.md b/docs/source-datastore/managing-datastores/grouping/managing-groups/remove-a-datastore.md similarity index 50% rename from docs/source-datastore/navigation-and-organization/grouping/managing-groups/remove-a-datastore.md rename to docs/source-datastore/managing-datastores/grouping/managing-groups/remove-a-datastore.md index cf87ab9bbe..629e8e00be 100644 --- a/docs/source-datastore/navigation-and-organization/grouping/managing-groups/remove-a-datastore.md +++ b/docs/source-datastore/managing-datastores/grouping/managing-groups/remove-a-datastore.md @@ -9,16 +9,16 @@ This guide walks you through the steps to remove a datastore from its current gr **Step 1**: In the tree view, hover over the datastore you want to remove from its group. An **assign menu** icon will appear. -![assign-icon](../../../../assets/source-datastores/navigation-and-organization/grouping/managing-groups/remove-a-datastore/assign-icon.png) +![assign-icon](../../../../assets/source-datastores/managing-datastores/grouping/managing-groups/remove-a-datastore/assign-icon.png) **Step 2**: Click the assign menu icon. A dropdown will appear showing the currently assigned group. -![assign-dropdown](../../../../assets/source-datastores/navigation-and-organization/grouping/managing-groups/remove-a-datastore/assign-dropdown.png) +![assign-dropdown](../../../../assets/source-datastores/managing-datastores/grouping/managing-groups/remove-a-datastore/assign-dropdown.png) **Step 3**: Click the **close icon** (x) next to the currently selected group. -![remove-group](../../../../assets/source-datastores/navigation-and-organization/grouping/managing-groups/remove-a-datastore/remove-group.png) +![remove-group](../../../../assets/source-datastores/managing-datastores/grouping/managing-groups/remove-a-datastore/remove-group.png) **Step 4**: The datastore will move to the **Ungrouped** section of the tree view. -![datastore-ungrouped](../../../../assets/source-datastores/navigation-and-organization/grouping/managing-groups/remove-a-datastore/datastore-ungrouped.png) +![datastore-ungrouped](../../../../assets/source-datastores/managing-datastores/grouping/managing-groups/remove-a-datastore/datastore-ungrouped.png) diff --git a/docs/source-datastore/navigation-and-organization/grouping/overview.md b/docs/source-datastore/managing-datastores/grouping/overview.md similarity index 94% rename from docs/source-datastore/navigation-and-organization/grouping/overview.md rename to docs/source-datastore/managing-datastores/grouping/overview.md index 4f649c4aae..f093a581ed 100644 --- a/docs/source-datastore/navigation-and-organization/grouping/overview.md +++ b/docs/source-datastore/managing-datastores/grouping/overview.md @@ -2,7 +2,7 @@ Datastore Grouping is a feature in Qualytics that allows you to organize your datastores into custom groups within the tree view. Instead of browsing a flat list of datastores, you can create meaningful categories — such as by environment, team, domain, or priority — making it easier to locate and manage your data sources. -![grouping-overview](../../../assets/source-datastores/navigation-and-organization/grouping/grouping-overview.png) +![grouping-overview](../../../assets/source-datastores/managing-datastores/grouping/grouping-overview.png) ## Why Datastore Grouping Matters diff --git a/docs/source-datastore/datastore-settings/link-enrichment.md b/docs/source-datastore/managing-datastores/link-enrichment.md similarity index 84% rename from docs/source-datastore/datastore-settings/link-enrichment.md rename to docs/source-datastore/managing-datastores/link-enrichment.md index 49cc42efd9..f56154fcc3 100644 --- a/docs/source-datastore/datastore-settings/link-enrichment.md +++ b/docs/source-datastore/managing-datastores/link-enrichment.md @@ -2,11 +2,11 @@ **Step 1:** Click on the **Enrichment** from the dropdown list. -![enrichment](../../assets/source-datastores/datastore-settings/link-enrichment/enrichment-6.png) +![enrichment](../../assets/source-datastores/managing-datastores/link-enrichment/enrichment-6.png) A modal window **Link Enrichment Datastore** will appear, providing you with two options to link an **enrichment datastore**. -![link-enrichment](../../assets/source-datastores/datastore-settings/link-enrichment/link-enrichment-7.png) +![link-enrichment](../../assets/source-datastores/managing-datastores/link-enrichment/link-enrichment-7.png) | REF. | FIELDS | ACTIONS | | :---- | :---- | :---- | @@ -23,11 +23,11 @@ If the toggle for **Add new connection** is turned on, then this will prompt you **Step 1:** Click on the caret button and select **Add Enrichment Datastore**. -![caret](../../assets/source-datastores/datastore-settings/link-enrichment/caret-8.png) +![caret](../../assets/source-datastores/managing-datastores/link-enrichment/caret-8.png) A modal window **Link Enrichment Datastore** will appear. Enter the following details to create an enrichment datastore with a new connection. -![window](../../assets/source-datastores/datastore-settings/link-enrichment/window.png) +![window](../../assets/source-datastores/managing-datastores/link-enrichment/window.png) | REF. | FIELDS | ACTIONS | | :---- | :---- | :---- | @@ -44,17 +44,17 @@ A modal window **Link Enrichment Datastore** will appear. Enter the following de !!! note Connection details can vary from datastore to datastore. For illustration, we have demonstrated linking BigQuery as a new enrichment datastore. -![select-new-enrichment](../../assets/source-datastores/datastore-settings/link-enrichment/select-new-enrichment-9.png) +![select-new-enrichment](../../assets/source-datastores/managing-datastores/link-enrichment/select-new-enrichment-9.png) **Step 3:** After adding the source datastore details, click on the **Test Connection** button to check and verify its connection. -![connector-creds](../../assets/source-datastores/datastore-settings/link-enrichment/connector-creds-10.png) +![connector-creds](../../assets/source-datastores/managing-datastores/link-enrichment/connector-creds-10.png) If the credentials and provided details are verified, a success message will be displayed indicating that the connection has been verified. **Step 4:** Click on the **Save** button. -![save-enrichment](../../assets/source-datastores/datastore-settings/link-enrichment/save-enrichment-11.png) +![save-enrichment](../../assets/source-datastores/managing-datastores/link-enrichment/save-enrichment-11.png) After clicking on the **Save** button, a success notification appears on the screen showing the action was completed successfully. @@ -64,11 +64,11 @@ If the **Use an existing enrichment datastore** option is selected from the drop **Step 1:** Click on the caret button and select **Use Enrichment Datastore**. -![caret](../../assets/source-datastores/datastore-settings/link-enrichment/caret2.png) +![caret](../../assets/source-datastores/managing-datastores/link-enrichment/caret2.png) **Step 2:** A modal window **Link Enrichment Datastore** will appear. Add a prefix name and select an existing enrichment datastore from the dropdown list. -![modal](../../assets/source-datastores/datastore-settings/link-enrichment/modal.png) +![modal](../../assets/source-datastores/managing-datastores/link-enrichment/modal.png) | REF. | FIELDS | ACTIONS | | :---- | :---- | :---- | @@ -80,7 +80,7 @@ If the **Use an existing enrichment datastore** option is selected from the drop **Step 3:** View and check the connection details of the enrichment datastore and click on the **Save** button. -![save](../../assets/source-datastores/datastore-settings/link-enrichment/save-15.png) +![save](../../assets/source-datastores/managing-datastores/link-enrichment/save-15.png) After clicking on the **Save** button, a success notification appears on the screen showing the action was completed successfully. diff --git a/docs/source-datastore/datastore-settings/quality-score-settings.md b/docs/source-datastore/managing-datastores/quality-score-settings.md similarity index 81% rename from docs/source-datastore/datastore-settings/quality-score-settings.md rename to docs/source-datastore/managing-datastores/quality-score-settings.md index 45bf2947d3..cfaffe4da4 100644 --- a/docs/source-datastore/datastore-settings/quality-score-settings.md +++ b/docs/source-datastore/managing-datastores/quality-score-settings.md @@ -17,22 +17,22 @@ The **Quality Score Settings** allow users to tailor the impact of each quality **Step 1**: Click on the **Score** option in the settings icon. -![score](../../assets/source-datastores/datastore-settings/quality-score-settings/score-17.png) +![score](../../assets/source-datastores/managing-datastores/quality-score-settings/score-17.png) **Step 2**: A modal window "**Quality Score Settings**" will appear. -![score-settings](../../assets/source-datastores/datastore-settings/quality-score-settings/score-settings-18.png) +![score-settings](../../assets/source-datastores/managing-datastores/quality-score-settings/score-settings-18.png) **Step 3**: The **Decay Period** slider sets the time frame over which the system evaluates historical data to determine the quality score. The decay period for considering past data events defaults to 180 days, but can be customized to fit your operational needs, ensuring the scores reflect the most relevant data quality insights. -![decay-period](../../assets/source-datastores/datastore-settings/quality-score-settings/decay-period-19.png) +![decay-period](../../assets/source-datastores/managing-datastores/quality-score-settings/decay-period-19.png) **Step 4**: Adjust the **Factor Weights** using the sliding bar. The factor weights determine the importance of different data quality aspects. -![factor-weights](../../assets/source-datastores/datastore-settings/quality-score-settings/factor-weights-20.png) +![factor-weights](../../assets/source-datastores/managing-datastores/quality-score-settings/factor-weights-20.png) **Step 5**: Click on the **Save** button to save the quality score settings. -![score-save](../../assets/source-datastores/datastore-settings/quality-score-settings/score-save-21.png) +![score-save](../../assets/source-datastores/managing-datastores/quality-score-settings/score-save-21.png) After clicking the **Save** button, a success notification appears on the screen showing the action was completed successfully. \ No newline at end of file diff --git a/docs/source-datastore/navigation-and-organization/right-click-options.md b/docs/source-datastore/managing-datastores/right-click-options.md similarity index 81% rename from docs/source-datastore/navigation-and-organization/right-click-options.md rename to docs/source-datastore/managing-datastores/right-click-options.md index 2231b7ab47..cf5b0842b5 100644 --- a/docs/source-datastore/navigation-and-organization/right-click-options.md +++ b/docs/source-datastore/managing-datastores/right-click-options.md @@ -22,7 +22,7 @@ Log in to your Qualytics account and right-click on the source datastore whether - Copy Name. -![add-datastore](../../assets/source-datastores/navigation-and-organization/right-click-options/source-1.png) +![add-datastore](../../assets/source-datastores/managing-datastores/right-click-options/source-1.png) | No | Field | Description | | :---- | :---- | :---- | @@ -33,7 +33,7 @@ Log in to your Qualytics account and right-click on the source datastore whether Alternatively, you can access these right-click options by performing the direct right-click operation on a source datastore from the list. -![add-datastore](../../assets/source-datastores/navigation-and-organization/right-click-options/source-2.png) +![add-datastore](../../assets/source-datastores/managing-datastores/right-click-options/source-2.png) ### Right Click Tables & Files @@ -49,7 +49,7 @@ A dropdown list of options will appear: - Copy Name. -![tables-files](../../assets/source-datastores/navigation-and-organization/right-click-options/tables-files-3.png) +![tables-files](../../assets/source-datastores/managing-datastores/right-click-options/tables-files-3.png) | No | Field | Description | | :---- | :---- | :---- | @@ -60,7 +60,7 @@ A dropdown list of options will appear: Alternatively, you can access these right-click options by opening the dedicated page of the source datastore, navigating to its **Tables** or files section, and performing the right-click operation on any table or file from the list. -![tables-files](../../assets/source-datastores/navigation-and-organization/right-click-options/tables-files-4.png) +![tables-files](../../assets/source-datastores/managing-datastores/right-click-options/tables-files-4.png) ### Right Click Fields @@ -76,7 +76,7 @@ A dropdown list of options will appear: - Copy Name. -![field](../../assets/source-datastores/navigation-and-organization/right-click-options/field-5.png) +![field](../../assets/source-datastores/managing-datastores/right-click-options/field-5.png) | No | Field | Description | | :---- | :---- | :---- | @@ -87,7 +87,7 @@ A dropdown list of options will appear: Alternatively, you can access these right-click options by opening the dedicated page of the table, navigating to its **Fields** section, and performing the right-click operation on any field from the list. -![field](../../assets/source-datastores/navigation-and-organization/right-click-options/field-6.png) +![field](../../assets/source-datastores/managing-datastores/right-click-options/field-6.png) ### Right Click Checks @@ -103,7 +103,7 @@ A dropdown list of options will appear: - Copy Name. -![checks](../../assets/source-datastores/navigation-and-organization/right-click-options/checks-7.png) +![checks](../../assets/source-datastores/managing-datastores/right-click-options/checks-7.png) | No | Field | Description | | :---- | :---- | :---- | @@ -114,7 +114,7 @@ A dropdown list of options will appear: Alternatively, you can access these right-click options by navigating to the Checks from the Explore section. -![checks](../../assets/source-datastores/navigation-and-organization/right-click-options/checks-8.png) +![checks](../../assets/source-datastores/managing-datastores/right-click-options/checks-8.png) ### Right Click Anomalies @@ -130,7 +130,7 @@ A dropdown list of options will appear: - Copy Name. -![anomalies](../../assets/source-datastores/navigation-and-organization/right-click-options/anomalies-9.png) +![anomalies](../../assets/source-datastores/managing-datastores/right-click-options/anomalies-9.png) | No | Field | Description | | :---- | :---- | :---- | @@ -141,4 +141,4 @@ A dropdown list of options will appear: Alternatively, you can access these right-click options by navigating to the Anomalies from the Explore section. -![anomalies](../../assets/source-datastores/navigation-and-organization/right-click-options/anomalies-10.png) +![anomalies](../../assets/source-datastores/managing-datastores/right-click-options/anomalies-10.png) diff --git a/docs/source-datastore/datastore-settings/settings-overview.md b/docs/source-datastore/managing-datastores/settings-overview.md similarity index 92% rename from docs/source-datastore/datastore-settings/settings-overview.md rename to docs/source-datastore/managing-datastores/settings-overview.md index 7c36f05c06..67905cb4a0 100644 --- a/docs/source-datastore/datastore-settings/settings-overview.md +++ b/docs/source-datastore/managing-datastores/settings-overview.md @@ -8,7 +8,7 @@ Let's get started 🚀 **Step 1**: Select a source **datastore** from the side menu for which you would like to manage the settings. -![add-datastore](../../assets/source-datastores/datastore-settings/settings-overview/add-datastore-1.png) +![add-datastore](../../assets/source-datastores/managing-datastores/settings-overview/add-datastore-1.png) **Step 2**: Click on the **Settings** icon from the top right window. A drop-down menu will appear with the following options: @@ -17,7 +17,7 @@ Let's get started 🚀 3. Score 4. Delete -![settings](../../assets/source-datastores/datastore-settings/settings-overview/settings-2.png) +![settings](../../assets/source-datastores/managing-datastores/settings-overview/settings-2.png) ## Edit Datastore diff --git a/docs/source-datastore/datastore-settings/unlink-enrichment.md b/docs/source-datastore/managing-datastores/unlink-enrichment.md similarity index 57% rename from docs/source-datastore/datastore-settings/unlink-enrichment.md rename to docs/source-datastore/managing-datastores/unlink-enrichment.md index f9f900ec93..6ab73e57c9 100644 --- a/docs/source-datastore/datastore-settings/unlink-enrichment.md +++ b/docs/source-datastore/managing-datastores/unlink-enrichment.md @@ -2,23 +2,23 @@ **Step 1:** Click on the **Enrichment** from the drop-down list. -![enrichment-option](../../assets/source-datastores/datastore-settings/unlink-enrichment/enrichment-6.png) +![enrichment-option](../../assets/source-datastores/managing-datastores/unlink-enrichment/enrichment-6.png) A modal window titled **Enrichment Datastore Settings** will appear, displaying configuration options for the linked enrichment datastore. -![enrichment-settings](../../assets/source-datastores/datastore-settings/unlink-enrichment/enrichment-settings.png) +![enrichment-settings](../../assets/source-datastores/managing-datastores/unlink-enrichment/enrichment-settings.png) **Step 2:** Click the **Unlink Enrichment Datastore** option (represented by **unlink** icon) located on the right side of the Details section to remove the linked enrichment datastore. -![unlink-option](../../assets/source-datastores/datastore-settings/unlink-enrichment/unlink-option.png) +![unlink-option](../../assets/source-datastores/managing-datastores/unlink-enrichment/unlink-option.png) A modal window titled **Unlink Enrichment Datastore** will appear. -![unlink-window](../../assets/source-datastores/datastore-settings/unlink-enrichment/unlink-window.png) +![unlink-window](../../assets/source-datastores/managing-datastores/unlink-enrichment/unlink-window.png) **Step 3:** Click the **Unlink** button to remove the enrichment datastore connection. -![unlink](../../assets/source-datastores/datastore-settings/unlink-enrichment/unlink.png) +![unlink](../../assets/source-datastores/managing-datastores/unlink-enrichment/unlink.png) After clicking the **Unlink** button, a success message confirms that the datastore has been updated successfully. diff --git a/docs/source-datastore/operations/profile.md b/docs/source-datastore/operations/profile.md index 2cab5b5755..37914b4b42 100644 --- a/docs/source-datastore/operations/profile.md +++ b/docs/source-datastore/operations/profile.md @@ -30,17 +30,17 @@ The inferred data quality checks are rigorously tested against the actual source **Step 1**: Select a source datastore from the side menu on which you would like to perform the profile operation. -![Select a source datastore from the side menu](../../assets/source-datastore/sync/select-source-datastore.png) +![Select a source datastore from the side menu](../../assets/source-datastores/shared/select-source-datastore.png) **Step 2**: Clicking on your preferred datastore will navigate you to the datastore details page. Within the overview tab (default view), click on the **Run** button under **Profile** to initiate the profile operation. -![Run button under Profile in the datastore overview](../../assets/source-datastore/profile/run-button-overview.png) +![Run button under Profile in the datastore overview](../../assets/source-datastores/operations/profile/run-button-overview.png) ## Configuration **Step 1**: Click on the **Run** button to initiate the profile operation. -![Initiate profile operation button](../../assets/source-datastore/profile/initiate-profile-button.png) +![Initiate profile operation button](../../assets/source-datastores/operations/profile/initiate-profile-button.png) !!! note You can run Profile Operation anytime to update the inferred data quality checks, automatically based on new data in the datastore. It is recommended to schedule the profile operations periodically to update inferred rules. More details are discussed in the **Schedule** section below. @@ -51,23 +51,23 @@ The inferred data quality checks are rigorously tested against the actual source This option includes all tables or files currently available in the datastore for profiling. Selecting this will profile every table within the source datastore without the need for further selection. -![Select all tables or file patterns for profiling](../../assets/source-datastore/profile/select-all-tables.png) +![Select all tables or file patterns for profiling](../../assets/source-datastores/operations/profile/select-all-tables.png) **2. Specific** This option allows users to manually select individual tables or files for profiling. It provides the flexibility to focus on particular tables of interest, which can be useful if the user is only interested in a subset of the available data. -![Select specific tables for profiling](../../assets/source-datastore/profile/select-specific-tables.png) +![Select specific tables for profiling](../../assets/source-datastores/operations/profile/select-specific-tables.png) **3. Tag** This option automatically profiles tables associated with selected tags. Tags are used to categorize tables, and by selecting a specific tag, all tables associated with that tag will be profiled. This option helps in managing and profiling grouped data efficiently. -![Select tables by tag for profiling](../../assets/source-datastore/profile/select-by-tag.png) +![Select tables by tag for profiling](../../assets/source-datastores/operations/profile/select-by-tag.png) **Step 3**: After making the relevant selections, click on the **Next** button to configure the **Operation Settings**. -![Next button to configure operation settings](../../assets/source-datastore/profile/next-operation-settings.png) +![Next button to configure operation settings](../../assets/source-datastores/operations/profile/next-operation-settings.png) **Step 4**: Configure the following two **Read Settings**: @@ -85,24 +85,24 @@ This setting allows users to specify a minimum incremental identifier value to s !!! note The starting threshold i.e. **Greater Than Time** and **Greater Than Batch** are applicable only to the tables or files with an incremental timestamp strategy. -![Starting threshold configuration with greater than time and batch options](../../assets/source-datastore/profile/starting-threshold.png) +![Starting threshold configuration with greater than time and batch options](../../assets/source-datastores/operations/profile/starting-threshold.png) ### Record Limit Define the number of records to be profiled per table: This feature allows users to manually enter a custom record limit value using a text field in the profile operation. This setting helps in controlling the scope of the profiling operation, particularly for large datasets, by capping the number of records to analyze. -![Record limit input field](../../assets/source-datastore/profile/record-limit-input.png) +![Record limit input field](../../assets/source-datastores/operations/profile/record-limit-input.png) You can also use a drop-down menu to quickly select from commonly used limits such as 1M, 10M, 100M, and All. -![Record limit dropdown with preset values](../../assets/source-datastore/profile/record-limit-dropdown.png) +![Record limit dropdown with preset values](../../assets/source-datastores/operations/profile/record-limit-dropdown.png) !!! note The number of records must be between **1** and **1,000,000,000**. **Step 5:** After making the relevant selections, click on the **Next** button to configure the **Inference Settings**. -![Next button to configure inference settings](../../assets/source-datastore/profile/next-inference-settings.png) +![Next button to configure inference settings](../../assets/source-datastores/operations/profile/next-inference-settings.png) **Step 6:** Configure the following two **Inference Settings**: @@ -117,7 +117,7 @@ The Inference Threshold allows you to customize the data quality checks that are By default, the Inference Threshold is set to **2**, which provides a comprehensive range of checks designed to ensure data integrity across different scenarios. Users have the flexibility to adjust this threshold based on their specific needs, allowing for either basic or advanced checks as required. -![Default inference threshold configuration set to level 2](../../assets/source-datastore/profile/inference-threshold-default.png) +![Default inference threshold configuration set to level 2](../../assets/source-datastores/operations/profile/inference-threshold-default.png) #### Levels of Check Inference @@ -130,7 +130,7 @@ The Inference Threshold ranges from **0** to **5**, with each level including pr At this level, no checks are automatically inferred. This is suitable when users want complete control over which checks are applied, or if no checks are needed. Ideal for scenarios where profiling should not infer any constraints, and all checks will be manually defined. -![Inference threshold set to level 0 with no automatic checks](../../assets/source-datastore/profile/inference-threshold-level0.png) +![Inference threshold set to level 0 with no automatic checks](../../assets/source-datastores/operations/profile/inference-threshold-level0.png) ##### Level 1: Basic Data Integrity and Simple Value Threshold Checks @@ -156,7 +156,7 @@ The following table shows the inferred checks that the Analytics Engine can gene | Not Negative | [See more.](https://userguide.qualytics.io/checks/not-negative-check/) | | Not Future | [See more.](https://userguide.qualytics.io/checks/not-future-check/) | -![Inference threshold set to level 1 with basic integrity checks](../../assets/source-datastore/profile/inference-threshold-level1.png) +![Inference threshold set to level 1 with basic integrity checks](../../assets/source-datastores/operations/profile/inference-threshold-level1.png) ##### Level 2: Value Range and Pattern Checks @@ -181,7 +181,7 @@ The following table shows the inferred checks that the Analytics Engine can gene | Matches Pattern | [See more.](https://userguide.qualytics.io/checks/matches-pattern-check/) | | Unique | [See more.](https://userguide.qualytics.io/checks/unique-check/) | -![Inference threshold set to level 2 with value range and pattern checks](../../assets/source-datastore/profile/inference-threshold-level2.png) +![Inference threshold set to level 2 with value range and pattern checks](../../assets/source-datastores/operations/profile/inference-threshold-level2.png) ##### Level 3: Time Series and Comparative Relationship Checks @@ -206,7 +206,7 @@ The following table shows the inferred checks that the Analytics Engine can gene | Less Than Field | [See more.](https://userguide.qualytics.io/checks/less-than-field-check/) | | Equal To Field | [See more.](https://userguide.qualytics.io/checks/equal-to-field-check/) | -![Inference threshold set to level 3 with time series and comparative checks](../../assets/source-datastore/profile/inference-threshold-level3.png) +![Inference threshold set to level 3 with time series and comparative checks](../../assets/source-datastores/operations/profile/inference-threshold-level3.png) ##### Level 4: Linear Regression and Cross-Datastore Relationship Checks @@ -228,7 +228,7 @@ The following table shows the inferred checks that the Analytics Engine can gene | Predicted By | [See more.](https://userguide.qualytics.io/checks/predicted-by-check/) | | Is Replica Of (_is sunsetting_) | [See more.](https://userguide.qualytics.io/checks/is-replica-of-check/) | -![Inference threshold set to level 4 with regression and cross-datastore checks](../../assets/source-datastore/profile/inference-threshold-level4.png) +![Inference threshold set to level 4 with regression and cross-datastore checks](../../assets/source-datastores/operations/profile/inference-threshold-level4.png) ##### Level 5: Shape Checks @@ -246,7 +246,7 @@ This table shows the inferred checks that the Analytics Engine can generate base | Matches Pattern (Shape) | [See more.](https://userguide.qualytics.io/checks/matches-pattern-check/) | | Not Null (Shape) | [See more.](https://userguide.qualytics.io/checks/not-null-check/) | -![Inference threshold set to level 5 with shape checks](../../assets/source-datastore/profile/inference-threshold-level5.png) +![Inference threshold set to level 5 with shape checks](../../assets/source-datastores/operations/profile/inference-threshold-level5.png) !!! warning If the checks inferred during a profile operation do not detect any anomalies, and the check inference level decreases in the next profile operation, the checks that did not generate anomalies will be archived or discarded. However, if the checks detect any anomalies, they will be retained to continue monitoring the data and addressing potential issues. @@ -255,37 +255,37 @@ This table shows the inferred checks that the Analytics Engine can generate base Check the box labeled **"Infer As Draft"** to ensure that all inferred checks will be generated in a draft state. This allows for greater flexibility as you can review and refine these checks before they are finalized. -![Infer as Draft checkbox option](../../assets/source-datastore/profile/infer-as-draft.png) +![Infer as Draft checkbox option](../../assets/source-datastores/operations/profile/infer-as-draft.png) ### Run Instantly Click on the **Run Now** button, and perform the profile operation immediately. -![Run Now button to execute profile operation immediately](../../assets/source-datastore/profile/run-now-button.png) +![Run Now button to execute profile operation immediately](../../assets/source-datastores/operations/profile/run-now-button.png) ### Schedule **Step 1**: Click on the **Schedule** button to configure the available schedule options in the profile operation. -![Schedule button to configure profile operation scheduling](../../assets/source-datastore/profile/schedule-button.png) +![Schedule button to configure profile operation scheduling](../../assets/source-datastores/operations/profile/schedule-button.png) **Step 2**: Set the scheduling preferences for the profile operation. **1. Hourly**: This option allows you to schedule the profile operation to run every hour at a specified minute. You can define the frequency in hours and the exact minute within the hour the profiling should start. Example: If set to "Every 1 hour(s) on minute 0," the profile operation will run every hour at the top of the hour (e.g., 1:00, 2:00, 3:00). -![Hourly schedule configuration](../../assets/source-datastore/profile/schedule-hourly.png) +![Hourly schedule configuration](../../assets/source-datastores/operations/profile/schedule-hourly.png) **2. Daily**: This option schedules the profile operation to run once every day at a specific time. You specify the number of days between scans and the exact time of day in UTC. Example: If set to "Every 1 day(s) at 00:00 UTC," the scan will run every day at midnight UTC. -![Daily schedule configuration](../../assets/source-datastore/profile/schedule-daily.png) +![Daily schedule configuration](../../assets/source-datastores/operations/profile/schedule-daily.png) **3. Weekly**: This option schedules the profile operation to run on specific days of the week at a set time. You select the days of the week and the exact time of day in UTC for the profile operation to run. Example: If configured to run on "Sunday" and "Friday" at 00:00 UTC, the scan will execute at midnight UTC on these days. -![Weekly schedule configuration](../../assets/source-datastore/profile/schedule-weekly.png) +![Weekly schedule configuration](../../assets/source-datastores/operations/profile/schedule-weekly.png) **4. Monthly**: This option schedules the profile operation to run once a month on a specific day at a set time. You specify the day of the month and the time of day in UTC. If set to "On the 1st day of every 1 month(s), at 00:00 UTC," the profile operation will run on the first day of each month at midnight UTC. -![Monthly schedule configuration](../../assets/source-datastore/profile/schedule-monthly.png) +![Monthly schedule configuration](../../assets/source-datastores/operations/profile/schedule-monthly.png) **5. Advanced**: The advanced section for scheduling operations allows users to set up more complex and custom scheduling using Cron expressions. This option is particularly useful for defining specific times and intervals for profile operations with precision. @@ -315,15 +315,15 @@ Users can define other specific schedules by adjusting the Cron expression. For To define a custom schedule, enter the appropriate Cron expression in the **Custom Cron Schedule (UTC)** field before specifying the schedule name. This will allow for precise control over the timing of the profile operation, ensuring it runs exactly when needed according to your specific requirements. -![Advanced schedule configuration with cron expression](../../assets/source-datastore/profile/schedule-advanced.png) +![Advanced schedule configuration with cron expression](../../assets/source-datastores/operations/profile/schedule-advanced.png) **Step 3**: Define the **Schedule Name** to identify the scheduled operation at the running time. -![Schedule name input field](../../assets/source-datastore/profile/schedule-name.png) +![Schedule name input field](../../assets/source-datastores/operations/profile/schedule-name.png) **Step 4**: Click on the **Schedule** button to activate your profile operation schedule. -![Schedule confirm button to activate the profile operation schedule](../../assets/source-datastore/profile/schedule-confirm-button.png) +![Schedule confirm button to activate the profile operation schedule](../../assets/source-datastores/operations/profile/schedule-confirm-button.png) !!! note You will receive a notification when the profile operation is completed. @@ -348,7 +348,7 @@ When the profile operation is completed, you will receive the notification and c - Operation Status - Table -![Activity tab top panel with runs, schedule, search, sort, and filter options](../../assets/source-datastore/profile/activity-top-panel.png) +![Activity tab top panel with runs, schedule, search, sort, and filter options](../../assets/source-datastores/operations/profile/activity-top-panel.png) ### Activity Heatmap @@ -357,7 +357,7 @@ The activity heatmap shown in the snippet below represents activity levels over !!! tip You can click on any of the squares from the Activity Heatmap to filter operations -![Activity heatmap showing operation frequency over time](../../assets/source-datastore/profile/activity-heatmap.png) +![Activity heatmap showing operation frequency over time](../../assets/source-datastores/operations/profile/activity-heatmap.png) ### Operation Detail @@ -380,7 +380,7 @@ This status indicates that the profile operation is still running at the moment | 11. | Abort | The "Abort" button enables you to stop the ongoing profile operation. | | 12. |Summary | The "Summary" section provides a real-time overview of the profile operation's progress. It includes key metrics such as: