Releases · datacoves/snowcap

20 May 15:27

github-actions

v1.0.15

4952365

v1.0.15 Latest

Latest

Merge pull request #16 from usbrandon/feat/iceberg-rest-catalog-integration

feat(catalog_integration): add IcebergRestCatalogIntegration

Merge pull request #18 from usbrandon/cortex-support

Support Snowflake Cortex permissions: DRG fetch fix + USE AI FUNCTIONS + CORTEX SEARCH SERVICE grants

docs: add Cortex Search Service grant doc + USE AI FUNCTIONS example

Address documentation needs for the Cortex permission support added in
this PR. Follows the pattern set by PR #16 (Iceberg REST Catalog
Integration docs) per @noel's previous feedback — a resource-specific
docs page with YAML + Python examples and a "Minimal example" section
covering real-world gotchas.

docs/resources/cortex_search_service.md (new): grant patterns for
CORTEX SEARCH SERVICE (USAGE, MONITOR), plus a complete worked
example wiring up Cortex Code + Cortex AI SQL + Cortex Search access
for a single role. Documents the PUBLIC-default behavior of
CORTEX_USER and the explicit-grant requirement for COPILOT_USER, the
database-role + account-priv layering, and the multi-grantee fetch
prerequisite (this PR).
docs/resources/grant.md: add USE AI FUNCTIONS on ACCOUNT and
USAGE/MONITOR on CORTEX SEARCH SERVICE to the Object Grants YAML
example.
mkdocs.yml: register CortexSearchService under a new "AI" nav group
alongside Access Control. Sets a place for future AI/Cortex resource
pages (Cortex Search Service body modeling, semantic views, etc.)
without bloating "Integrations" or "Access Control".

Co-Authored-By: Claude Opus 4.7 (1M context) noreply@anthropic.com

feat(cortex_search): support CORTEX SEARCH SERVICE grants

Adds CORTEX SEARCH SERVICE as a schema-scoped grantable resource type so
users can write:

grants:
  - priv: USAGE
    on: cortex search service <db>.<schema>.<svc>
    to: <role>
  - priv: MONITOR
    on: cortex search service <db>.<schema>.<svc>
    to: <observability_role>

Snowflake's Cortex Search Service supports four privileges (USAGE,
MONITOR, OWNERSHIP, ALL); USAGE is required to run
SNOWFLAKE.CORTEX.SEARCH_PREVIEW and MONITOR is required for the new
get_ai_observability_events() log access. The schema-scope priv
CREATE CORTEX SEARCH SERVICE was already in SchemaPriv.

Changes:

ResourceType.CORTEX_SEARCH_SERVICE added to the enum (alphabetical).
CortexSearchServicePriv(Priv) class with ALL/USAGE/MONITOR/OWNERSHIP.
PRIVS_FOR_RESOURCE_TYPE entry mapping the type to the priv class.
RESOURCE_SCOPES entry registering SchemaScope() (following the
INTEGRATION precedent: no concrete resource class yet, just enough to
let priv: ... on: cortex search service ... parse and render).
data_provider.list_grants: drop CREATE CORTEX SEARCH SERVICE from the
"Skip undocumented privs" filter — it is documented (in SchemaPriv)
and should round-trip via list_grants like any other schema priv.
CANCEL QUERY remains in the skip list (genuinely unmodeled).
Unit tests covering the priv enum values and end-to-end Grant SQL
rendering for both USAGE and MONITOR.

Full Cortex Search Service resource modeling (CREATE ... AS ,
embedding model, target_lag, attributes) is left for a future PR — this
change just covers grants.

Co-Authored-By: Claude Opus 4.7 (1M context) noreply@anthropic.com

fix(database_role_grant): return filtered row + add USE AI FUNCTIONS priv

Two related fixes that together unlock modeling Snowflake Cortex
permissions in snowcap.

fetch_database_role_grant returned show_result[0] (unfiltered first row
of SHOW GRANTS) instead of role_grants[0] (the row matching the URN's
requested grantee). For any database role granted to multiple
account/database roles, every per-grantee fetch returned the first
grantee's data — causing the diff engine to emit a spurious UpdateResource
(e.g. "to_role: ACCOUNTADMIN -> DBT_DEVELOPER") whenever a second grantee
was declared in YAML. There is no update_database_role_grant lifecycle
hook so the apply would also fail with invalid SQL.

Adds an integration regression test
(test_fetch_grant_of_database_role_multiple_grantees) that grants one
database role to two account roles and fetches each FQN separately;
previously both fetches returned the same first-grantee row.
Adds USE_AI_FUNCTIONS = "USE AI FUNCTIONS" to AccountPriv. This account-
level privilege gates Snowflake Cortex AI SQL functions (AI_COMPLETE,
AI_FILTER, SUMMARIZE, COMPLETE, embed/search REST APIs, etc.). Without
it in the enum, priv: USE AI FUNCTIONS / on: account in YAML raises a
parse error.

Together these enable native modeling of Cortex Code, Cortex AI SQL, and
Cortex Agents access in YAML via database_role_grants (CORTEX_USER,
CORTEX_AGENT_USER, COPILOT_USER, AI_FUNCTIONS_USER) and grants (USE AI
FUNCTIONS on account).

Co-Authored-By: Claude Opus 4.7 (1M context) noreply@anthropic.com

docs(catalog_integration): add minimal AWS S3 Tables + Lake Formation example

Adds an end-to-end "Minimal example: AWS S3 Tables behind Lake Formation"
section to the IcebergRestCatalogIntegration doc, distilled from the BCS
production onboarding for our Epicor P21 → S3 Tables → Snowflake pipeline.

Covers the four AWS-side pieces (S3 Tables bucket, Lake Formation
integration, LF grants, cross-account IAM role) and the two Snowcap
resources that bind them to Snowflake (catalog_integrations +
storage_integrations together), plus the post-deploy CREATE ICEBERG TABLE

SYSTEM$VERIFY_CATALOG_INTEGRATION verification step.

Includes the gotchas we hit during onboarding:

GlueCatalogIntegration/CATALOG_SOURCE=GLUE rejects the federated
<account>:s3tablescatalog/<bucket> form for GLUE_CATALOG_ID with
SQL compilation error 22023/1008 — this is the failure mode that
motivated adding IcebergRestCatalogIntegration in the first place.
Lake Formation DESCRIBE/SELECT grants are easy to forget; DESC
CATALOG INTEGRATION succeeds without them but CREATE ICEBERG TABLE
fails with a vague 403.
access_delegation_mode=VENDED_CREDENTIALS is required when the
writer (e.g., pyiceberg-rest on EC2) relies on Lake Formation to
vend temporary S3 credentials.
sigv4_signing_region must match the S3 Tables bucket region.

Test suite (tests/test_polymorphic_resources.py) still passes (28/28).

Co-Authored-By: Claude Opus 4.7 (1M context) noreply@anthropic.com

docs(catalog_integration): IcebergRestCatalogIntegration docs + fixture fix

Address @noel's second-round PR #16 feedback:

tests/fixtures/json/iceberg_rest_catalog_integration.json:
add "refresh_interval_seconds": null to match resource serialization
pattern used by other fixtures. Full unit suite (1480 tests) now passes.
docs/resources/iceberg_rest_catalog_integration.md:
new page following glue_catalog_integration.md template. Covers YAML +
Python examples for the AWS S3 Tables (SIGV4 + AWS_GLUE) production path
and lists rest_authentication fields per auth type (SIGV4/OAUTH/BEARER).
mkdocs.yml:
register IcebergRestCatalogIntegration under Integrations > Catalog
between Glue and ObjectStore.

Co-Authored-By: Claude Opus 4.7 (1M context) noreply@anthropic.com

fix: add type annotation to _EXECUTION_CACHE for mypy

Co-Authored-By: Claude Opus 4.5 noreply@anthropic.com

Merge pull request #17 from usbrandon/feat/git-repository

feat(git_repository): add GitRepository resource + grant support

feat(api_integration): support non-AWS provider subtypes (Azure/Google/GitHTTPS)

fetch_api_integration crashed with KeyError: 'api_aws_role_arn' when DESC'ing
API integrations whose api_provider is anything other than AWS Gateway flavors
— e.g., GIT_HTTPS_API (used by Snowflake's git repos), AZURE_API_MANAGEMENT,
GOOGLE_API_GATEWAY. The _APIIntegration dataclass also required
api_aws_role_arn, blocking instantiation for non-AWS providers.

Changes:

ApiProvider enum expanded with AZURE_API_MANAGEMENT, GOOGLE_API_GATEWAY,
and GIT_HTTPS_API alongside the existing AWS variants.
_APIIntegration now has optional azure_tenant_id / azure_ad_application_id
(Azure), google_audience (GCP), and api_aws_role_arn is no longer required.
APIIntegration.__init__ keeps api_aws_role_arn in its original positional
slot (just optional with default None) so existing positional callers
continue to work; new fields appended after the original positional set.
New Props entries for the Azure/Google fields so they round-trip through
CREATE/ALTER API INTEGRATION SQL.
fetch_api_integration uses properties.get() for the provider-specific
fields so missing keys return None instead of crashing.
tests/fixtures/json/api_integration.json extended with null entries for the
new optional fields so the identity test still passes.
3 new TestAPIIntegration cases covering GIT_HTTPS_API, AZURE_API_MANAGEMENT,
and GOOGLE_API_GATEWAY shapes.
Docs updated with a provider-vs-required-fields table + a YAML example for
the GitHub case used by GitRepository.

All 1492 non-integration tests pass.

feat(integration): support generic INTEGRATION grants (umbrella ResourceType)

Snowflake's GRANT USAGE ON INTEGRATION <name> SQL is valid for any subtype
(API/CATALOG/EXTERNAL_ACCESS/NOTIFICATION/SECURITY/STORAGE). Users naturally
write on: integration <fqn> in YAML, but the umbrella ResourceType.INTEGRATION
had no RESOURCE_SCOPES entry, no fetcher, and no lister — so any such grant
crashed with:

KeyError: <ResourceType.INTEGRATION: 'INTEGRATION'>

Fix:

Register RESOURCE_SCOPES[ResourceType.INTEGRATION] = AccountScope()
(snowcap/resources/resource.py).
Add fetch_integration (walks SHOW INTEGRATIONS) and list_integrations
to snowcap/data_provider.py so the existing fetch/list dispatch resolves.

No concrete Integration resource class. Declarative management still goes
through the specific subtypes (APIIntegration, `Cat...

Contributors

noel

Assets 2

04 May 14:07

github-actions

v1.0.14

3b7131b

v1.0.14

Improve error msgs (#15)

improve error msgs during apply
grant roles before apply

Assets 2

02 May 22:58

github-actions

v1.0.13

0b7785b

v1.0.13

Fix some issues (#14)

Fix deadlock issue
Fix comparisons None vs empty string and lower vs uppercase
Fix issue on new snowflake account where role is needed before it can be used e.g. as an owner of a new db

Assets 2