Skip to content

Add ADF outbound access control via Azure Policy to prevent data exfiltration#229

Draft
SvenAelterman with Copilot wants to merge 2 commits into
mainfrom
copilot/add-outbound-access-control
Draft

Add ADF outbound access control via Azure Policy to prevent data exfiltration#229
SvenAelterman with Copilot wants to merge 2 commits into
mainfrom
copilot/add-outbound-access-control

Conversation

Copilot AI commented Mar 27, 2026

Copy link
Copy Markdown

Data Factory has no restriction on which external domains its pipelines can communicate with, enabling potential data exfiltration. This adds enforcement via the built-in Azure Policy 3d02a511-74e5-4dab-a5fd-878704d4a61a ("[Preview]: Azure Data Factory pipelines should only communicate with allowed domains") with Deny effect.

adf.bicep

  • New allowedStorageAccountNames parameter (array, default []) for storage accounts beyond the private storage account
  • Computes the full FQDN allowlist: blob, dfs, and file endpoints for every storage account in scope, plus the Key Vault used by the linked service (using environment().suffixes.keyvaultDns)
  • Creates a Microsoft.Authorization/policyAssignments resource scoped to the resource group
var allStorageAccountNames = union([privateStorageAcctName], allowedStorageAccountNames)
var storageAllowedDomains = flatten(map(allStorageAccountNames, name => [
  '${name}.blob.${storageSuffix}'
  '${name}.dfs.${storageSuffix}'
  '${name}.file.${storageSuffix}'
]))
var allowedDomains = concat(storageAllowedDomains, ['${keyVaultName}${keyVaultSuffix}'])

main.bicep

  • Passes the public and airlock storage account names to adfModule via allowedStorageAccountNames
  • Adds airlockStorageAccountNameForPolicy to resolve the airlock account name without creating a circular dependency through centralizedModule (which itself depends on adfModule.outputs.principalId). For centralized review, the name is extracted from centralAirlockResources.storageAccountId; for spoke-local review, the module output is used directly.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • aka.ms
    • Triggering command: /usr/local/bin/bicep bicep lint research-spoke/spoke-modules/airlock/adf.bicep (dns block)
    • Triggering command: /usr/local/bin/bicep bicep lint research-spoke/spoke-modules/airlock/main.bicep (dns block)
  • learn.microsoft.com
    • Triggering command: /home/REDACTED/work/_temp/ghcca-node/node/bin/node /home/REDACTED/work/_temp/ghcca-node/node/bin/node --enable-source-maps /home/REDACTED/work/_temp/copilot-developer-action-main/dist/index.js (dns block)

If you need me to access, download, or install something from one of these locations, you can either:


💬 Send tasks to Copilot coding agent from Slack and Teams to turn conversations into code. Copilot posts an update in your thread when it's finished.

Assigns the built-in Azure Policy '3d02a511-74e5-4dab-a5fd-878704d4a61a'
('[Preview]: Azure Data Factory pipelines should only communicate with
allowed domains') to prevent data exfiltration by restricting ADF
pipeline outbound traffic to the storage accounts referenced in pipelines
(private, public, and airlock) plus the Key Vault used for linked service
connection strings.

- adf.bicep: Add allowedStorageAccountNames parameter, compute allowed
  domain names (blob/dfs/file endpoints per account + KV FQDN), create
  policy assignment resource with Deny effect
- main.bicep: Pass public and airlock storage account names to adfModule;
  add airlockStorageAccountNameForPolicy variable to avoid circular dependency
  with centralizedModule

Agent-Logs-Url: https://github.com/Azure/HubAndSpokeResearchEnclave/sessions/58fbf88c-7835-43e4-9cc1-37f48d77fd6c

Co-authored-by: SvenAelterman <17446043+SvenAelterman@users.noreply.github.com>
Copilot AI changed the title [WIP] Add support for outbound access control with Data Factory Add ADF outbound access control via Azure Policy to prevent data exfiltration Mar 27, 2026
Copilot AI requested a review from SvenAelterman March 27, 2026 00:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants