diff --git a/.github/FUNDING.yml b/.github/FUNDING.yml new file mode 100644 index 0000000..cfa292c --- /dev/null +++ b/.github/FUNDING.yml @@ -0,0 +1,3 @@ +# These are supported funding model platforms + +github:, [Nano-Core] diff --git a/.github/workflows/build-and-deploy.yml b/.github/workflows/build-and-deploy.yml new file mode 100644 index 0000000..130b021 --- /dev/null +++ b/.github/workflows/build-and-deploy.yml @@ -0,0 +1,34 @@ +name: Build and Deploy +on: + pull_request: + branches: + - master + push: + branches: + - master +env: + VERSION: 10.0.0-rc1 +jobs: + build-and-deploy: + runs-on: ubuntu-latest + permissions: + contents: write + id-token: write + concurrency: + group: ${{ github.repository }} + cancel-in-progress: false + steps: + - uses: actions/checkout@v6 + + - name: GitHub Release + if: github.ref == 'refs/heads/master' + uses: ncipollo/release-action@v1 + with: + tag: v${{ env.VERSION }}.${{ github.run_number }} + name: "Release ${{ env.VERSION }}" + body: | + Version: ${{ env.VERSION }} + Commit: ${{ github.sha }} + token: ${{ secrets.GITHUB_TOKEN }} + draft: false + prerelease: ${{ contains(env.VERSION, '-') }} \ No newline at end of file diff --git a/.gitignore b/.gitignore new file mode 100644 index 0000000..9921fc0 --- /dev/null +++ b/.gitignore @@ -0,0 +1,11 @@ +.vs +*.user +*.userprefs +*.suo +_ReSharper* +**/bin +**/obj +*.DotSettings.User +packages +.env +**/Properties/launchSettings.json \ No newline at end of file diff --git a/LICENSE b/LICENSE index ed88279..a7f8871 100644 --- a/LICENSE +++ b/LICENSE @@ -1,6 +1,6 @@ MIT License -Copyright (c) 2026 Nano-Core +Copyright (c) 2024 Nano Core Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal diff --git a/Nano.Azure.Account/README.md b/Nano.Azure.Account/README.md new file mode 100644 index 0000000..a797f02 --- /dev/null +++ b/Nano.Azure.Account/README.md @@ -0,0 +1,82 @@ +# Nano.Azure.Account + +> _Azure account and environments, including tenant access, subscriptions, and deployment identities_ + +*** + +## Table of Contents +* **[Summary](#summary)** +* **[Registration](#registration)** + * **[Account And Subscription](#account-and-subscription)** + * **[Service Principal](#service-principal)** +* **[Dependencies](#dependencies)** + +## Summary +This is the foundation of the Nano Azure infrastructure. It prepares the Azure environment required to host Nano applications, including. + +- Azure tenant access +- Subscription setup for environments +- Deployment identity (service principal) +- Secure configuration of CI/CD secrets + +This component must be deployed first, as all other **[Nano.Azure](https://github.com/Nano-Core/Nano.Azure/tree/master/README.md#nanoazure)** components depend on it. + +> 📖 Learn more about **[Microsoft Azure](https://azure.microsoft.com)**. + +## Registration +There are no prerequisites for registering Azure providers when setting up the Azure account. + +> ⚠️ Ensure all required variables are specified in the PowerShell script before execution. + +### Account +First, create an **[Azure account](https://azure.microsoft.com/en-us/pricing/purchase-options/azure-account)** if you do not already have one. + +### Subscription +Next, create a _Pay-As-You-Go_ (or another suitable offer) subscription for each required environment, such as **Staging** and **Production**, as well as any additional +environments you may need. This must be done directly in the Azure Portal using the +**[Create Subscription](https://portal.azure.com/#view/Microsoft_Azure_Billing/CatalogBlade/appId/AddSubscriptionButton)** page. + +> ⚠️ Subscription creation is not supported via Azure CLI and must be completed in the Azure Portal. + +Next, to enable automated deployments, store the tenant id of the Azure account and the subscription ids, as GitHub organization secrets. + +| Secret | Type | Description | +| ----------------------------------------- | -------- |---------------------------------------------------------- | +| `AZURE_TENANT_ID` | Secrets | The account identifier of the Azure account. | +| `{{environment}}_AZURE_SUBSCRIPTION_ID` | Secrets | The identifier of the subscription of the environment. | + +Open PowerShell and sign in to your Azure account. Select the appropriate subscription depending on the environment you are setting up. + +```powershell +az login; +``` + +You can also set the subscription using. + +```powershell +az account set -s {{subscription-id}}; +``` + +To verify that you are signed in and connected to the correct subscription, use the following command. It should then show you the tenant and subscription id. + +```powershell +az account show; +``` + +### Service Principal +Last, execute `deploy.ps1` to create the `nano-deploy-service-principal`. This service principal is used for deploying additional resources to the subscriptions. From the output, +note down the `appId` and `password`. + +Then create the GitHub secrets as shown below. + +To ge the values to set for the variables use this command. + +| Secret | Type | Description | +| ---------------------- | -------- | -------------------------------------------------------------------------- | +| `AZURE_CLIENT_ID` | Secrets | The app id of the service principal for use with deployments. | +| `AZURE_CLIENT_SECRET` | Secrets | The secret (password) used for authentication for the service-principal. | + +> 💡 If you later need to retrieve information about the created service principal, you can find it in **[Azure App Registrations](https://portal.azure.com/#view/Microsoft_AAD_RegisteredApps/ApplicationsListBlade)**. + +## Dependencies +The repository has no dependencies and must be the first component created when setting up the Nano infrastructure. All other deployments depend on it. diff --git a/Nano.Azure.Account/deploy.ps1 b/Nano.Azure.Account/deploy.ps1 new file mode 100644 index 0000000..4e8ce83 --- /dev/null +++ b/Nano.Azure.Account/deploy.ps1 @@ -0,0 +1,12 @@ +$env:AZURE_SUBSCRIPTION_ID_STAGING = ""; +$env:AZURE_SUBSCRIPTION_ID_PRODUCTION = ""; +$env:SERVICE_PRINCIPAL_NAME = "nano-deploy-service-principal"; + +## Service Principal +az ad sp create-for-rbac ` + --name $env:SERVICE_PRINCIPAL_NAME ` + --role Contributor ` + --scope ` + "/subscriptions/$env:AZURE_SUBSCRIPTION_ID_STAGING" ` + "/subscriptions/$env:AZURE_SUBSCRIPTION_ID_PRODUCTION" ` + --years 2; diff --git a/Nano.Azure.Backup/.diagnostic-settings/logs.json b/Nano.Azure.Backup/.diagnostic-settings/logs.json new file mode 100644 index 0000000..79593d8 --- /dev/null +++ b/Nano.Azure.Backup/.diagnostic-settings/logs.json @@ -0,0 +1,38 @@ +[ + { + "category": "AzureBackupReport", + "enabled": true + }, + { + "category": "CoreAzureBackup", + "enabled": true + }, + { + "category": "AddonAzureBackupJobs", + "enabled": true + }, + { + "category": "AddonAzureBackupAlerts", + "enabled": true + }, + { + "category": "AddonAzureBackupPolicy", + "enabled": true + }, + { + "category": "AddonAzureBackupStorage", + "enabled": true + }, + { + "category": "AddonAzureBackupProtectedInstance", + "enabled": true + }, + { + "category": "ASRReplicatedItems", + "enabled": true + }, + { + "category": "AzureBackupOperations", + "enabled": true + } +] diff --git a/Nano.Azure.Backup/.diagnostic-settings/metrics.json b/Nano.Azure.Backup/.diagnostic-settings/metrics.json new file mode 100644 index 0000000..551d2f1 --- /dev/null +++ b/Nano.Azure.Backup/.diagnostic-settings/metrics.json @@ -0,0 +1,7 @@ +[ + { + "category": "AllMetrics", + "enabled": true, + "timeGrain": "PT1M" + } +] \ No newline at end of file diff --git a/Nano.Azure.Backup/README.md b/Nano.Azure.Backup/README.md new file mode 100644 index 0000000..497e996 --- /dev/null +++ b/Nano.Azure.Backup/README.md @@ -0,0 +1,76 @@ +# Nano.Azure.Backup + +> _Centralized data backup and recovery for Nano._ + +*** + +## Table of Contents +* **[Summary](#summary)** +* **[Registration](#registration)** + * **[Backup Vault](#backup-vault)** + * **[Data Redundancy](#data-redundancy)** + * **[Immutability State](#immutability-state)** + * **[Diagnostics Settings](#diagnostics-settings)** + * **[Alerts](#alerts)** +* **[Dependencies](#dependencies)** + +## Summary +Azure Backup Vault is a cloud-based service that securely stores backup data for Azure resources like virtual machines, storage accounts, databases, and more. It provides +centralized management, automated backup scheduling, and encryption for data protection. With flexible retention policies, it supports efficient disaster recovery and ensures +your data is protected and recoverable in the event of a failure or loss. + +Create a Backup Vault used for file share and database backups, as well as other backup-enabled resources. + +> 📖 Learn more about **[Azure Backup](https://learn.microsoft.com/azure/backup)**. + +## Registration +Start by registering the required Azure providers and creating the resource group, by executing the top part of the `deploy.ps1`. + +> ⚠️ Ensure all required variables are specified in the PowerShell script before execution. + +Add the resource group name as GitHub organization variables. + +| Secret | Type | Description | +| ------------------------------- | ------- |----------------------------------------------- | +| `AZURE_RESOURCE_GROUP_BACKUP` | vars | The Azure resource group of the backup vault. | + +### Backup Vault +Execute the next part of the `deploy.ps1` to create the backup vault on Azure. + +By default, the Backup Vault is configured with `--soft-delete-feature-state` enabled and `--soft-delete-duration` set to 14 days. Use `az backup vault backup-properties set` to +customize these settings. This also means that backup items created from protected Azure resources can prevent the vault from being deleted until the configured soft-delete +retention period has expired or the source has been unregistered. + +### Data Redundancy +The `--backup-storage-redundancy` parameter defines how backup data is replicated for durability. In this case, it is set to `ZoneRedundant`, meaning data is replicated across +multiple availability zones within the same region to protect against zonal failures. Other options are `LocallyRedundant`, which stores copies within a single region, and +`GeoRedundant`, which replicates data to a secondary region for broader disaster recovery protection. + +### Immutability State +By default, immutability is set to `Unlocked`, meaning it is enabled but can still be modified or disabled at a later stage. It is recommended to set the value to `Locked`. In +`Locked` mode, immutability is permanently enforced and cannot be reversed, ensuring that backup data is protected against deletion or modification for the configured +retention period. + +### Diagnostics Settings +Next, execute the script section that configures diagnostic settings for the backup vault. + +The diagnostic settings for backup metrics are configured to `AllMetrics` with a 1-minute aggregation interval, which can be adjusted if required. This ensures high-resolution +monitoring of backup performance data. The logs configuration includes all Backup-related logs, while Site Recovery logs are excluded as they are not used by Nano. The full list +of supported metric categories for the backup resource can be retrieved using the following command. + +```powershell +az monitor diagnostic-settings categories list --resource $env:BACKUP_ID; +``` + +### Alerts +The alerts used are the default Azure Backup alerts. + +An Alert Processing Rule is configured to route these alert notifications through the Action Group created as part of +**[Nano.Azure.Monitoring](https://github.com/Nano-Core/Nano.Azure/tree/master/Nano.Azure.Monitoring/README.md#nanoazuremonitoring)**. + +## Dependencies +Backup has the following dependencies that must be deployed or otherwise satisfied prior to setup. + +| Dependency | Description | +| ------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------- | +| **[Nano.Azure.Monitoring](https://github.com/Nano-Core/Nano.Azure/tree/master/Nano.Azure.Monitoring/README.md#nanoazuremonitoring)** | Components for centralized monitoring and logging | diff --git a/Nano.Azure.Backup/deploy.ps1 b/Nano.Azure.Backup/deploy.ps1 new file mode 100644 index 0000000..312b47e --- /dev/null +++ b/Nano.Azure.Backup/deploy.ps1 @@ -0,0 +1,62 @@ +$env:ENVIRONMENT = ""; +$env:AZURE_LOCATION = "Sweden Central"; +$env:AZURE_RESOURCE_GROUP = "Nano-Backup"; +$env:AZURE_RESOURCE_GROUP_LOGS = "Nano-Logs"; +$env:APP_NAME = "nano-backup-vault-" + $env:ENVIRONMENT.ToLower(); + +# Register Providers +az provider register --namespace Microsoft.RecoveryServices; + +# Resource Group +az group create ` + -n $env:AZURE_RESOURCE_GROUP ` + -l $env:AZURE_LOCATION; + +# Backup Vault +az backup vault create ` + -n $env:APP_NAME ` + -g $env:AZURE_RESOURCE_GROUP ` + -l $env:AZURE_LOCATION ` + --immutability-state Unlocked ` + --job-failure-alerts Enable ` + --cross-subscription-restore-state Enable ` + --classic-alerts Disable ` + --public-network-access Disable; + +# Data Redundancy +az backup vault backup-properties set ` + -n $env:APP_NAME ` + -g $env:AZURE_RESOURCE_GROUP ` + --backup-storage-redundancy ZoneRedundant; + +# Immutability State (Optional) +az backup vault update ` + -n $env:APP_NAME ` + -g $env:AZURE_RESOURCE_GROUP ` + --immutability-state Locked; + +# Diagnostics Settings +$env:BACKUP_ID = az backup vault show -g $env:AZURE_RESOURCE_GROUP -n $env:APP_NAME --query id -o tsv; +$env:DIAGNOSTIC_SETTINGS_NAME = "diagnostics-" + $env:APP_NAME; +$env:WORKSPACE_ID = az monitor log-analytics workspace list -g $env:AZURE_RESOURCE_GROUP_LOGS --query [0].[id] -o tsv; + +az monitor diagnostic-settings create ` + --name $env:DIAGNOSTIC_SETTINGS_NAME ` + --workspace $env:WORKSPACE_ID ` + --resource $env:BACKUP_ID ` + --logs '@.diagnostic-settings/logs.json' ` + --metrics '@.diagnostic-settings/metrics.json'; + +# Alerts +$env:BACKUP_ID = az backup vault show -g $env:AZURE_RESOURCE_GROUP -n $env:APP_NAME --query id -o tsv; +$env:AZURE_RESOURCE_GROUP_ID = az group show -n $env:AZURE_RESOURCE_GROUP --query id -o tsv; +$env:ACTION_GROUP = az monitor action-group list -g $env:AZURE_RESOURCE_GROUP_LOGS --query [0].[id] -o tsv; +$env:WORKSPACE_ID = az monitor log-analytics workspace list -g $env:AZURE_RESOURCE_GROUP_LOGS --query [0].[id] -o tsv; + +az monitor alert-processing-rule create ` + --name 'Backup Alert Processing Rule ' ` + --resource-group $env:AZURE_RESOURCE_GROUP ` + --scope $env:BACKUP_ID ` + --rule-type AddActionGroups ` + --action-groups $env:ACTION_GROUP ` + --enabled true; diff --git a/Nano.Azure.DNS/README.md b/Nano.Azure.DNS/README.md new file mode 100644 index 0000000..03e21bc --- /dev/null +++ b/Nano.Azure.DNS/README.md @@ -0,0 +1,66 @@ +# Nano.Azure.Dns + +> _DNS zone used for Kubernetes (AKS) domain management and service routing._ + +*** + +## Table of Contents +* **[Summary](#summary)** +* **[Registration](#registration)** + * **[DNS Zone](#dns-zone)** + * **[Managed Identity (Kubernetes)](#managed-identity-kubernetes)** + * **[External DNS Registrar Delegation](#external-dns-registrar-delegation)** +* **[Multiple DNS Zone](#multiple-dns-zone)** +* **[Dependencies](#dependencies)** + +## Summary +Azure DNS is a cloud-based Domain Name System service that provides reliable and secure DNS hosting and management for domain records. It enables the creation and control of DNS zones and +records, supporting services such as domain resolution, subdomain delegation, and automated DNS record management. With built-in high availability, global redundancy, and integration with +Azure identity and security services, Azure DNS ensures fast, resilient, and secure name resolution for applications and infrastructure. + +> 📖 Learn more about **[Azure DNS](https://learn.microsoft.com/en-us/azure/dns/dns-overview)**. + +## Registration +Start by registering the required Azure providers and creating the resource group, by executing the top part of the `deploy.ps1`. + +> ⚠️ Ensure all required variables are specified in the PowerShell script before execution. + +Add the resource group name as GitHub organization variables. + +| Secret | Type | Description | +| --------------------------- | ------- |------------------------------------------- | +| `AZURE_RESOURCE_GROUP_DNS` | vars | The Azure resource group of the DNS Zone. | + +### DNS Zone +Creates the Azure DNS zone for the application domain, which will host all DNS records for the delegated subdomain and enable cert-manager DNS-01 validation. + +### Managed Identity (Kubernetes) +Creates a Managed Identity in Azure and assigns it the DNS Zone Contributor role on the DNS zone, allowing it to manage DNS records required for automated DNS-based operations such as domain +validation and certificate issuance. + +### External DNS Registrar Delegation +At your external DNS provider (e.g., GoDaddy, Namecheap, or any domain registrar), you must delegate the subdomain to Azure DNS by creating NS (Name Server) records. This ensures that all DNS +resolution for the subdomain is handled by Azure DNS instead of the external provider. + +Run the following command to get the Azure name servers. + +```powershell +az network dns zone show -g $env:AZURE_RESOURCE_GROUP -n $env:APP_DOMAIN_NAME --query nameServers -o tsv; +``` + +For each returned name server, add an NS record for the delegated subdomain. + +| Type | Host / Name | Value | +| ---- |---------------------- | ----------------- | +| NS | $env:APP_DOMAIN_NAME | {{name-server}} | + +## Multiple DNS Zone +Creating multiple DNS zones for different domain names is fully supported. Create each additional DNS zone as needed, but reuse the existing Managed Identity rather than creating a +new one. Instead, add a new role assignment for the existing identity that grants access to the additional DNS zone. + +## Dependencies +DNS has the following dependencies that must be deployed or otherwise satisfied prior to setup. + +| Dependency | Description | +| ---------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------- | +| **[Nano.Azure.Account](https://github.com/Nano-Core/Nano.Azure/tree/master/Nano.Azure.Account/README.md#nanoazureaccount)** | The is the foundation or prerequites of the Nano Azure infrastructure. | diff --git a/Nano.Azure.DNS/deploy.ps1 b/Nano.Azure.DNS/deploy.ps1 new file mode 100644 index 0000000..1a62780 --- /dev/null +++ b/Nano.Azure.DNS/deploy.ps1 @@ -0,0 +1,34 @@ +$env:ENVIRONMENT = ""; +$env:AZURE_LOCATION = "North Europe"; +$env:AZURE_RESOURCE_GROUP = "Nano-Dns"; +$env:APP_NAME = "dns-zone"; +$env:DOMAIN_NAME = ""; + +# Register Providers +az provider register --namespace Microsoft.Network; + +# Resurce Group +az group create ` + -n $env:AZURE_RESOURCE_GROUP ` + -l $env:AZURE_LOCATION; + +# Dns Zone +az network dns zone create ` + -g $env:AZURE_RESOURCE_GROUP ` + -n $env:DOMAIN_NAME; + +# Managed Identity (Kubernetes) +$env:APP_NAME_IDENTITY = $env:APP_NAME + "-identity"; + +az identity create ` + -g $env:AZURE_RESOURCE_GROUP ` + -n $env:APP_NAME_IDENTITY; + +$env:DNS_ZONE_ID = az network dns zone show -g $env:AZURE_RESOURCE_GROUP -n $env:DOMAIN_NAME --query id -o tsv; +$env:PRINCIPAL_ID = az identity show -g $env:AZURE_RESOURCE_GROUP -n $env:APP_NAME_IDENTITY --query principalId -o tsv; + +az role assignment create ` + --assignee-object-id $env:PRINCIPAL_ID ` + --assignee-principal-type ServicePrincipal ` + --role "DNS Zone Contributor" ` + --scope $env:DNS_ZONE_ID; diff --git a/Nano.Azure.Delivery/README.md b/Nano.Azure.Delivery/README.md new file mode 100644 index 0000000..be1f32b --- /dev/null +++ b/Nano.Azure.Delivery/README.md @@ -0,0 +1,52 @@ +# Nano.Azure.Delivery + +> _Delivery infrastructure for GitHub Self-Hosted Runners._ + +*** + +## Table of Contents +* **[Summary](#summary)** +* **[Registration](#registration)** + * **[Kubernete VNet Integration](#kubernete-vnet-integration)** + * **[Log Analytics](#log-analytics)** +* **[Dependencies](#dependencies)** + +## Summary +This script provisions the foundational Azure infrastructure required to run GitHub Actions self-hosted runners in a secure, isolated, and scalable environment. It sets up the required +Azure resource groups, networking components, and monitoring services, including a dedicated virtual network subnet for Azure Container Apps and integration with Log Analytics for +centralized observability. + +Finally, it creates an Azure Container Apps Environment configured for internal-only access and zone redundancy, providing a hardened and reliable execution layer for GitHub-based +CI/CD workloads. + +## Registration +Start by registering the required Azure providers and creating the resource group, by executing the top part of the `deploy.ps1`. + +> ⚠️ Ensure all required variables are specified in the PowerShell script before execution. + +Add the resource group name as GitHub organization variables. + +| Secret | Type | Description | +| -------------------------------- | ------- |------------------------------------ | +| `AZURE_RESOURCE_GROUP_DELIVERY` | vars | The Delivery Azure resource group. | + +### Kubernete VNet Integration +The Azure Container Apps Environment is deployed into a dedicated subnet within an existing Virtual Network. This is required when the underlying Kubernetes cluster (AKS) is private +and not publicly accessible. By delegating a dedicated subnet to Container Apps, we ensure secure network-level isolation and enable controlled communication between the runner +infrastructure and the private AKS cluster. + +This design avoids public exposure while maintaining full internal connectivity within the delivery platform. + +### Log Analytics +The Container Apps Environment is configured to use Log Analytics as its logging destination. By providing the workspace ID and shared key, all runtime logs, platform events, and +diagnostics are streamed to a centralized Log Analytics workspace. + +This enables consistent observability across the delivery platform and simplifies monitoring, troubleshooting, and auditability of the self-hosted runner infrastructure. + +## Dependencies +Delivery has the following dependencies that must be deployed or otherwise satisfied prior to setup. + +| Dependency | Description | +| ------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------- | +| **[Nano.Azure.Monitoring](https://github.com/Nano-Core/Nano.Azure/tree/master/Nano.Azure.Monitoring/README.md#nanoazuremonitoring)** | Components for centralized monitoring and logging. | +| **[Nano.Azure.Kubernetes](https://github.com/Nano-Core/Nano.Azure/tree/master/Nano.Azure.Kubernetes/README.md#nanoazurekubernetes)** | The Azure Kubernetes Service (AKS). | diff --git a/Nano.Azure.Delivery/deploy.ps1 b/Nano.Azure.Delivery/deploy.ps1 new file mode 100644 index 0000000..775fd3c --- /dev/null +++ b/Nano.Azure.Delivery/deploy.ps1 @@ -0,0 +1,46 @@ +$env:ENVIRONMENT = ""; +$env:AZURE_LOCATION = "North Europe"; +$env:AZURE_RESOURCE_GROUP = "Nano-Delivery"; +$env:AZURE_RESOURCE_GROUP_ASSETS = "Nano-Delivery-Assets"; +$env:AZURE_RESOURCE_GROUP_LOGS = "Nano-Logs"; +$env:AZURE_RESOURCE_GROUP_KUBERNETES_ASSETS = "Nano-Kubernetes-Assets"; +$env:APP_NAME = $env:ENVIRONMENT.ToLower() + "-delivery-platform"; + +# Register Providers +az provider register --namespace Microsoft.App +az provider register --namespace Microsoft.Network +az provider register --namespace Microsoft.OperationalInsights + +# Resurce Group +az group create ` + -n $env:AZURE_RESOURCE_GROUP ` + -l $env:AZURE_LOCATION; + +# Container App Environment +$env:SUBNET_NAME = "aks-subnet-ci"; +$env:SUBNET_ADDRESS_PREFIXES = "10.232.0.0/24"; +$env:VNET_NAME = az network vnet list -g $env:AZURE_RESOURCE_GROUP_KUBERNETES_ASSETS --query [0].name -o tsv; + +az network vnet subnet create ` + -g $env:AZURE_RESOURCE_GROUP_KUBERNETES_ASSETS ` + -n $env:SUBNET_NAME ` + --vnet-name $env:VNET_NAME ` + --address-prefixes $env:SUBNET_ADDRESS_PREFIXES ` + --delegations "Microsoft.App/environments"; + +$env:SUBNET_ID = az network vnet subnet show -g $env:AZURE_RESOURCE_GROUP_KUBERNETES_ASSETS -n $env:SUBNET_NAME --vnet-name $env:VNET_NAME --query id --output tsv; +$env:LOG_ANALYTICS_WORKSPACE_CUSTOMER_ID = az monitor log-analytics workspace list -g $env:AZURE_RESOURCE_GROUP_LOGS --query [0].customerId -o tsv; +$env:LOG_ANALYTICS_WORKSPACE_NAME = az monitor log-analytics workspace list -g $env:AZURE_RESOURCE_GROUP_LOGS --query [0].name -o tsv; +$env:LOG_ANALYTICS_WORKSPACE_KEY = az monitor log-analytics workspace get-shareD-keys -g $env:AZURE_RESOURCE_GROUP_LOGS -n $env:LOG_ANALYTICS_WORKSPACE_NAME --query primarySharedKey -o tsv + +az containerapp env create ` + -g $env:AZURE_RESOURCE_GROUP ` + -n $env:APP_NAME ` + -l $env:AZURE_LOCATION ` + --infrastructure-resource-group $env:AZURE_RESOURCE_GROUP_ASSETS ` + --infrastructure-subnet-resource-id $env:SUBNET_ID ` + --logs-destination log-analytics ` + --logs-workspace-id $env:LOG_ANALYTICS_WORKSPACE_CUSTOMER_ID ` + --logs-workspace-key $env:LOG_ANALYTICS_WORKSPACE_KEY ` + --internal-only true ` + --zone-redundant; diff --git a/Nano.Azure.Kubernetes/.alerts/node-health.json b/Nano.Azure.Kubernetes/.alerts/node-health.json new file mode 100644 index 0000000..c7c2ba7 --- /dev/null +++ b/Nano.Azure.Kubernetes/.alerts/node-health.json @@ -0,0 +1,37 @@ +[ + { + "alert": "KubeNodeNotReady", + "expression": "kube_node_status_condition{condition=\"Ready\", status=\"false\"} == 1", + "severity": 1, + "for": "PT5M", + "enabled": true + }, + { + "alert": "KubeNodeUnreachable", + "expression": "kube_node_spec_taint{key=\"node.kubernetes.io/unreachable\"} == 1", + "severity": 2, + "for": "PT10M", + "enabled": true + }, + { + "alert": "KubeNodeReadinessFlapping", + "expression": "changes(kube_node_status_condition{condition=\"Ready\"}[15m]) > 2", + "severity": 3, + "for": "PT10M", + "enabled": true + }, + { + "alert": "KubeNodeMemoryPressure", + "expression": "kube_node_status_condition{condition=\"MemoryPressure\", status=\"true\"} == 1", + "severity": 2, + "for": "PT5M", + "enabled": true + }, + { + "alert": "KubeNodeCPUPressure", + "expression": "1 - avg by (node) (rate(node_cpu_seconds_total{mode=\"idle\"}[5m])) > 0.85", + "severity": 2, + "for": "PT5M", + "enabled": true + } +] \ No newline at end of file diff --git a/Nano.Azure.Kubernetes/.alerts/resource-saturation.json b/Nano.Azure.Kubernetes/.alerts/resource-saturation.json new file mode 100644 index 0000000..f71e911 --- /dev/null +++ b/Nano.Azure.Kubernetes/.alerts/resource-saturation.json @@ -0,0 +1,53 @@ +[ + { + "alert": "KubeContainerCPURising", + "expression": "sum(rate(container_cpu_usage_seconds_total{image!=\"\", container!=\"POD\"}[5m])) by (pod, namespace) / sum(kube_pod_container_resource_limits{resource=\"cpu\"}) by (pod, namespace) > 0.6", + "severity": 3, + "for": "PT10M", + "enabled": true + }, + { + "alert": "KubeContainerCPUHigh", + "expression": "sum(rate(container_cpu_usage_seconds_total{image!=\"\", container!=\"POD\"}[5m])) by (pod, namespace) / sum(kube_pod_container_resource_limits{resource=\"cpu\"}) by (pod, namespace) > 0.8", + "severity": 2, + "for": "PT10M", + "enabled": true + }, + { + "alert": "KubeContainerCPUCritical", + "expression": "sum(rate(container_cpu_usage_seconds_total{image!=\"\", container!=\"POD\"}[5m])) by (pod, namespace) / sum(kube_pod_container_resource_limits{resource=\"cpu\"}) by (pod, namespace) > 0.95", + "severity": 1, + "for": "PT5M", + "enabled": true + }, + + { + "alert": "KubeContainerMemoryRising", + "expression": "sum(container_memory_working_set_bytes{container!=\"POD\", image!=\"\"}) by (pod, namespace) / sum(kube_pod_container_resource_limits{resource=\"memory\"}) by (pod, namespace) > 0.6", + "severity": 3, + "for": "PT10M", + "enabled": true + }, + { + "alert": "KubeContainerMemoryHigh", + "expression": "sum(container_memory_working_set_bytes{container!=\"POD\", image!=\"\"}) by (pod, namespace) / sum(kube_pod_container_resource_limits{resource=\"memory\"}) by (pod, namespace) > 0.8", + "severity": 2, + "for": "PT10M", + "enabled": true + }, + { + "alert": "KubeContainerMemoryCritical", + "expression": "sum(container_memory_working_set_bytes{container!=\"POD\", image!=\"\"}) by (pod, namespace) / sum(kube_pod_container_resource_limits{resource=\"memory\"}) by (pod, namespace) > 0.95", + "severity": 1, + "for": "PT5M", + "enabled": true + }, + + { + "alert": "KubePVUsageHigh", + "expression": "max(kubelet_volume_stats_used_bytes / kubelet_volume_stats_capacity_bytes) by (persistentvolumeclaim, namespace) > 0.8", + "severity": 2, + "for": "PT30M", + "enabled": true + } +] \ No newline at end of file diff --git a/Nano.Azure.Kubernetes/.alerts/scheduling-scaling.json b/Nano.Azure.Kubernetes/.alerts/scheduling-scaling.json new file mode 100644 index 0000000..6154022 --- /dev/null +++ b/Nano.Azure.Kubernetes/.alerts/scheduling-scaling.json @@ -0,0 +1,30 @@ +[ + { + "alert": "KubeHpaMaxedOut", + "expression": "(kube_horizontalpodautoscaler_status_current_replicas == kube_horizontalpodautoscaler_spec_max_replicas) and (kube_horizontalpodautoscaler_status_desired_replicas >= kube_horizontalpodautoscaler_spec_max_replicas)", + "severity": 3, + "for": "PT10M", + "enabled": true + }, + { + "alert": "KubeHpaReplicasMismatch", + "expression": "kube_horizontalpodautoscaler_status_desired_replicas != kube_horizontalpodautoscaler_status_current_replicas", + "severity": 3, + "for": "PT10M", + "enabled": true + }, + { + "alert": "KubeDaemonSetNotScheduled", + "expression": "sum by (namespace, daemonset) (kube_daemonset_status_desired_number_scheduled - kube_daemonset_status_current_number_scheduled) > 0", + "severity": 3, + "for": "PT10M", + "enabled": true + }, + { + "alert": "KubePodSchedulingPressure", + "expression": "sum by (namespace, pod) (kube_pod_status_phase{phase=\"Pending\"}) > 0", + "severity": 3, + "for": "PT10M", + "enabled": true + } +] \ No newline at end of file diff --git a/Nano.Azure.Kubernetes/.alerts/workload-stability.json b/Nano.Azure.Kubernetes/.alerts/workload-stability.json new file mode 100644 index 0000000..53923d7 --- /dev/null +++ b/Nano.Azure.Kubernetes/.alerts/workload-stability.json @@ -0,0 +1,44 @@ +[ + { + "alert": "KubePodNotReady", + "expression": "sum(kube_pod_status_phase{phase=~\"Pending|Unknown|Failed\"}) by (pod) > 0", + "severity": 3, + "for": "PT10M", + "enabled": true + }, + { + "alert": "KubePodCrashLooping", + "expression": "max_over_time(kube_pod_container_status_waiting_reason{reason=\"CrashLoopBackOff\"}[5m]) > 0", + "severity": 4, + "for": "PT5M", + "enabled": true + }, + { + "alert": "KubePodImagePullFailure", + "expression": "sum(kube_pod_container_status_waiting_reason{reason=~\"ImagePullBackOff|ErrImagePull\"}) > 0", + "severity": 3, + "for": "PT5M", + "enabled": true + }, + { + "alert": "KubeContainerOOMKilled", + "expression": "sum(kube_pod_container_status_last_terminated_reason{reason=\"OOMKilled\"}) > 0", + "severity": 4, + "for": "PT5M", + "enabled": true + }, + { + "alert": "KubePodRestartRateHigh", + "expression": "increase(kube_pod_container_status_restarts_total[5m]) > 3", + "severity": 3, + "for": "PT5M", + "enabled": true + }, + { + "alert": "KubeDeploymentReplicasMismatch", + "expression": "kube_deployment_spec_replicas != kube_deployment_status_replicas_available", + "severity": 4, + "for": "PT10M", + "enabled": true + } +] \ No newline at end of file diff --git a/Nano.Azure.Kubernetes/.data-collection-settings/data-collection-settings.json b/Nano.Azure.Kubernetes/.data-collection-settings/data-collection-settings.json new file mode 100644 index 0000000..7ab587c --- /dev/null +++ b/Nano.Azure.Kubernetes/.data-collection-settings/data-collection-settings.json @@ -0,0 +1,14 @@ +{ + "interval": "1m", + "namespaceFilteringMode": "Exclude", + "namespaces": [ + ], + "enableContainerLogV2": true, + "streams": [ + "Microsoft-ContainerLogV2", + "Microsoft-KubeEvents", + "Microsoft-KubePodInventory", + "Microsoft-KubeNodeInventory", + "Microsoft-KubePVInventory" + ] +} \ No newline at end of file diff --git a/Nano.Azure.Kubernetes/.diagnostic-settings/alb/logs.json b/Nano.Azure.Kubernetes/.diagnostic-settings/alb/logs.json new file mode 100644 index 0000000..553a579 --- /dev/null +++ b/Nano.Azure.Kubernetes/.diagnostic-settings/alb/logs.json @@ -0,0 +1,6 @@ +[ + { + "category": "TrafficControllerAccessLog", + "enabled": true + } +] \ No newline at end of file diff --git a/Nano.Azure.Kubernetes/.diagnostic-settings/load-balancer/logs.json b/Nano.Azure.Kubernetes/.diagnostic-settings/load-balancer/logs.json new file mode 100644 index 0000000..b7f978a --- /dev/null +++ b/Nano.Azure.Kubernetes/.diagnostic-settings/load-balancer/logs.json @@ -0,0 +1,6 @@ +[ + { + "category": "LoadBalancerHealthEvent", + "enabled": true + } +] \ No newline at end of file diff --git a/Nano.Azure.Kubernetes/.diagnostic-settings/load-balancer/metrics.json b/Nano.Azure.Kubernetes/.diagnostic-settings/load-balancer/metrics.json new file mode 100644 index 0000000..551d2f1 --- /dev/null +++ b/Nano.Azure.Kubernetes/.diagnostic-settings/load-balancer/metrics.json @@ -0,0 +1,7 @@ +[ + { + "category": "AllMetrics", + "enabled": true, + "timeGrain": "PT1M" + } +] \ No newline at end of file diff --git a/Nano.Azure.Kubernetes/.diagnostic-settings/logs.json b/Nano.Azure.Kubernetes/.diagnostic-settings/logs.json new file mode 100644 index 0000000..8d12367 --- /dev/null +++ b/Nano.Azure.Kubernetes/.diagnostic-settings/logs.json @@ -0,0 +1,30 @@ +[ + { + "category": "kube-apiserver", + "enabled": true + }, + { + "category": "kube-audit", + "enabled": true + }, + { + "category": "kube-audit-admin", + "enabled": true + }, + { + "category": "kube-controller-manager", + "enabled": true + }, + { + "category": "kube-scheduler", + "enabled": true + }, + { + "category": "cluster-autoscaler", + "enabled": true + }, + { + "category": "cloud-controller-manager", + "enabled": true + } +] \ No newline at end of file diff --git a/Nano.Azure.Kubernetes/.kubernetes/application-load-balancer.yaml b/Nano.Azure.Kubernetes/.kubernetes/application-load-balancer.yaml new file mode 100644 index 0000000..10306f0 --- /dev/null +++ b/Nano.Azure.Kubernetes/.kubernetes/application-load-balancer.yaml @@ -0,0 +1,8 @@ +apiVersion: alb.networking.azure.io/v1 +kind: ApplicationLoadBalancer +metadata: + name: %APP_NAME%-alb + namespace: %KUBERNETES_NAMESPACE% +spec: + associations: + - "%ALB_SUBNET_ID%" \ No newline at end of file diff --git a/Nano.Azure.Kubernetes/README.md b/Nano.Azure.Kubernetes/README.md new file mode 100644 index 0000000..174c72d --- /dev/null +++ b/Nano.Azure.Kubernetes/README.md @@ -0,0 +1,488 @@ +# Nano.Azure.Kubernetes + +> _Azure Kubernetes Service (AKS) deployment for Nano applications._ + +*** + +## Table of Contents +* **[Summary](#summary)** +* **[Registration](#registration)** + * **[Kubernetes Cluster](#kubernetes-cluster)** + * **[Apps Namespace](#apps-namespace)** + * **[Gateway Load Balancer](#gateway-load-balancer)** + * **[Automatic Upgrades](#automatic-upgrades)** + * **[Node Scaling](#node-scaling)** + * **[Network Policy](#network-policy)** + * **[Private API Server](#private-api-server)** + * **[DNS Resolver](#dns-resolver)** + * **[VPN Gateway](#vpn-gateway)** + * **[System Nodepool](#system-nodepool)** + * **[GPU Nodepool](#gpu-Nodepool)** + * **[Maintenance Window](#maintenance-window)** + * **[Monitoring](#monitoring)** + * **[Alerts](#alerts)** + * **[Site Recovery](#site-recovery)** + * **[Image Cleaner](#image-cleaner)** + * **[Diagnostic Settings](#diagnostic-settings)** + * **[Network Rules](#network-rules)** + * **[Microsoft Defender](#m,icrosoft-defender)** + * **[Policy](#policy)** +* **[Image Pull Secret](#image-pull-secret)** +* **[Configure kubectl Access](#configure-kubectl-access)** +* **[Common `kubectl` Commands](#common-kubectl-commands)** +* **[Dependencies](#dependencies)** + +## Summary +Azure Kubernetes Service (AKS) is a managed Kubernetes service that simplifies the deployment and operation of Kubernetes clusters. AKS automates tasks such as provisioning, +scaling, and upgrading clusters, reducing the operational burden on teams. It supports both Linux and Windows containers and provides built-in monitoring and security features. +AKS allows for easy scaling of applications and ensures high availability, making it ideal for running containerized workloads in production environments. + +Create an Azure Kubernetes Service (AKS) cluster to orchestrate Nano infrastructure components and deploy applications. + +> 📖 Learn more about **[Azure Kubernetes (AKS)](https://learn.microsoft.com/en-us/azure/aks)**. + +## Registration +Start by registering the required Azure providers and creating the resource group, by executing the top part of the `deploy.ps1`. + +> ⚠️ Ensure all required variables are specified in the PowerShell script before execution. + +Add the following GitHub organization variables. + +| Secret | Type | Description | +| ----------------------------------------- | ------- |------------------------------------------------------------------- | +| `AZURE_RESOURCE_GROUP_KUBERNETES` | vars | The Azure resource group of the Kubernetes cluster (AKS). | +| `AZURE_RESOURCE_GROUP_KUBERNETES_ASSETS` | vars | The Azure resource group of the Kubernetes cluster (AKS) Asserts. | + +### Kubernetes Cluster +Execute the next part of the `deploy.ps1` to create a managed Kubernetes cluster (AKS) on Azure. + +The SLA tier is set to `standard`, enabling Azure's standard managed cluster service with a financially backed SLA. Other values are: `free` and `premium`. + +The available Kubernetes versions in a specific region can also be queried, by this command. + +```powershell +az aks get-versions -l $env:AZURE_LOCATION --output table; +``` + +Other node pool SKUs are available and can be selected based on workload requirements and regional availability in Azure. To view all supported SKUs for a specific region, use the +following command. + +```powershell +az vm list-skus -l $env:AZURE_LOCATION --query "[?resourceType=='virtualMachines'].[name, tier]" -o table; +``` + +> ⚠️ For production-grade Azure Kubernetes Service (AKS) clusters, a minimum of three nodes is recommended, each with at least four vCPUs. + +Create the required secrets in GitHub for the Kubernetes cluster. + +| Secret | Type | Description | +| ---------------------------------------- | -------- | ----------------------------------------------------- | +| `AZURE_RESOURCE_GROUP_KUBERNETES` | vars | The Azure resource group of the Kubernetes cluster. | +| `AZURE_RESOURCE_GROUP_KUBERNETES_ASSETS` | vars | The Azure resource group of the Kubernetes assets. | + +### Apps Namespace +This step creates the default Kubernetes namespace used by all Nano applications and components. It provides a consistent isolation boundary and ensures Gateway API resources, +services, and supporting objects such as secrets and config maps are grouped together, reducing permission complexity and configuration overhead. + +It is created during cluster bootstrap and is safe to reapply in CI/CD pipelines. All application resources should target this namespace unless a specific multi-namespace design +is explicitly required. + +Execute the `namespace.ps1` script to create the Kubernetes namespace. + +### Gateway Load Balancer +Application Gateway API support is enabled in the AKS cluster through `--enable-gateway-api`, adding the Kubernetes Gateway API custom resource definitions (CRDs) required for +defining `Gateway`, `HTTPRoute`, and related networking resources. This introduces a modern Kubernetes-native traffic management model that replaces traditional Ingress-based +routing with a more flexible and extensible API. + +Together with `--enable-application-load-balancer`, AKS deploys and configures the Azure Application Load Balancer (ALB) Controller within the cluster. The controller extends the +cluster with the custom resource definitions (CRDs) required for managing `ApplicationLoadBalancer` resources and integrates Kubernetes Gateway API resources with Azure Application +Gateway for Containers. + +> ⚠️ Make sure to choose a region that supports Azure Application Gateway for Containers. See **[Supported Regions](https://learn.microsoft.com/en-us/azure/application-gateway/for-containers/overview#supported-regions)**. + +A separate subnet is created for the Azure Application Load Balancer, and the proper role assignments created for the load balancer to have the proper network permissions. + +> 📖 Learn more about how to **[Deploy Application Gateway for Containers ALB Controller using AKS Add-on](https://learn.microsoft.com/en-us/azure/application-gateway/for-containers/quickstart-deploy-application-gateway-for-containers-alb-controller-addon)**. +> 📖 Learn more about how to **[Create Application Gateway for Containers managed by ALB Controller](https://learn.microsoft.com/en-us/azure/application-gateway/for-containers/quickstart-create-application-gateway-for-containers-managed-by-alb-controller)**. + +Finally, the Kubernetes resource is deployed in the cluster, which triggers the creation of the Azure Application Load Balancer resource. The Application Load Balancer provides the +Azure-managed traffic distribution layer for Kubernetes ingress, routing external traffic into the cluster through the configured subnet and integrating with the Gateway API for +application-level routing. + +This resource is managed by Azure and forms the foundation for inbound connectivity. + +> ⚠️ Only a single `ApplicationLoadBalancer` should be deployed. + +The deployed Application Load Balancer can be retrieved using the following command. + +```powershell +kubectl get ApplicationLoadBalancer -n {{namespace}}; +``` + +### Automatic Upgrades +Automatic upgrades ensure that the AKS cluster remains up to date with the latest security patches and node image updates. The `--auto-upgrade-channel patch` setting enables automatic +Kubernetes patch version updates, helping keep the cluster secure with minimal manual intervention. In addition, `--node-os-upgrade-channel NodeImage` ensures that the underlying node +operating system images are automatically updated, providing the latest fixes and improvements for the cluster nodes. + +### Node Scaling +Cluster autoscaling in AKS automatically adjusts the number of nodes in a node pool based on workload demand. When enabled with `--enable-cluster-autoscaler`, AKS scales up when pods +cannot be scheduled due to insufficient requested resources or scheduling constraints, and scales down when existing nodes are underutilized and workloads can be safely rescheduled. The +`--min-count` and `--max-count` parameters define the scaling boundaries to ensure the cluster operates within a controlled range of nodes. + +When configuring the Horizontal Pod Autoscaler (HPA), Kubernetes uses resource requests as the baseline for scaling decisions. However, scaling is often intended to react closer to +resource limits to better reflect actual capacity pressure. To achieve this, the target utilization can be derived using the formula below, which converts a desired scale percentage into +the HPA average utilization value: + +Formula: `Pod Resource Limit(L) x Desired Scale percentage(P) / Pod Resource Request(R) * 100 = HPA Average Utilization` +Example: `2.100 * 80% / 700 * 100 = 240` + +> 💡 For more advanced and event-driven scaling scenarios, KEDA can be used to provide more fine-grained and configurable scaling options at the application level. + +### Network Policy +The AKS cluster is configured with the Azure CNI network plugin in overlay mode to enable scalable pod networking while conserving virtual network IP space. By using `--network-plugin azure` +with `--network-plugin-mode=overlay`, pods receive IPs from an overlay network instead of the underlying VNet. The `--network-policy azure` setting enforces Azure Network Policies, +allowing fine-grained control over traffic flow between pods and services to improve cluster security and isolation. + +### Private API Server +The AKS cluster is configured to use a private API server by integrating the control plane with the existing virtual network. By enabling private cluster mode and API server VNet integration, +access to the Kubernetes control plane is restricted to the private network, eliminating any public endpoint exposure. The API server is hosted within the same virtual network infrastructure +without requiring a separate dedicated subnet, ensuring that all communication with the control plane remains private and accessible only through connected network resources. + +This also means that interacting with the Kubernetes cluster using `kubectl` is restricted and requires a VPN connection to the Kubernetes virtual network. See [VPN Gateway](#vpn-gateway). + +### DNS Resolver +The DNS Resolver setup provisions an Azure DNS Resolver within the virtual network and configures an inbound endpoint that is reachable from both VPN clients and resources running +inside the VNet. This enables consistent resolution of private DNS zones, such as those used by Azure Private Endpoints for services like Azure Databases and Storage Accounts, without +relying on manual DNS configuration or host-based overrides. + +Once configured, the VPN gateway distributes the resolver’s IP address to connected clients as their DNS server. This ensures that any `privatelink` hostname is automatically resolved +to its correct private IP address within the virtual network, allowing seamless connectivity to private Azure resources from local machines as well as from workloads running inside +Kubernetes or other compute environments in the same network. + +> ⚠️ The DNS Resolver must be set before generating the VPN client profile, otherwise private DNS zones won’t be included and private endpoints won’t resolve from VPN-connected clients. + +### VPN Gateway +The VPN Gateway provides secure remote access to the Kubernetes virtual network, enabling private connectivity to cluster resources such as the AKS API server and internal services. It is +deployed using a route-based VPN gateway with a zone-redundant SKU to ensure high availability. + +A dedicated `GatewaySubnet` is created within the virtual network, and a public IP is assigned to the gateway for client connectivity. Azure automatically uses this subnet during gateway +deployment. When selecting the `--address-prefix` for the subnet, ensure it does not overlap with existing subnet ranges and is fully contained within the VNet address space. + +To retrieve the address range of the VNet, run the following command. + +```powershell +az network vnet show -g $env:AZURE_RESOURCE_GROUP_ASSETS -n $env:VNET_NAME --query addressSpace.addressPrefixes -o tsv; +``` + +To identify the IP ranges already in use by existing subnets in the Kubernetes virtual network, use the following commands. + +```powershell +az network vnet subnet list -g $env:AZURE_RESOURCE_GROUP_ASSETS --vnet-name $env:VNET_NAME --query "[].{Name:name,Prefix:addressPrefix}" -o table +``` + +Point-to-site VPN configuration is enabled using Azure AD authentication, allowing users to securely connect using OpenVPN. A dedicated client address pool is allocated for +VPN clients, ensuring isolated IP assignment within the network. + +This setup ensures secure, identity-based access to the private Kubernetes environment without exposing internal resources publicly. + +Download and install the Azure VPN Client for your operating system. + +| OS | Link | +| -------- | ------------------------------------------------------------------------------------ | +| Linux | https://apps.microsoft.com/detail/9np355qt2sqb | +| Mac | https://apps.apple.com/app/azure-vpn-client/id1557555267 | +| Windows | https://learn.microsoft.com/azure/vpn-gateway/point-to-site-vpn-client-cert-linux | + +Finally, run the following command to retrieve the VPN client configuration download link. Use this file to import the configuration into the Azure VPN Client. + +```powershell +az network vnet-gateway vpn-client generate ` + -g $env:AZURE_RESOURCE_GROUP_ASSETS ` + -n $env:VNET_GATEWAY_NAME; +``` + +> ⚠️ Creating the Gateway can take 20–30 minutes to complete. + +### System Nodepool +The system nodepool is used to host critical Kubernetes system components such as DNS, metrics, and core add-ons. It is deployed in `System` mode and configured with dedicated compute +resources to ensure stable and reliable cluster operations. To isolate system workloads from application workloads, the nodepool is tainted with `CriticalAddonsOnly=true:NoSchedule`, +ensuring only system pods are scheduled on these nodes. + +Auto-scaling is enabled with a fixed range (min/max), allowing controlled scaling behavior while maintaining a minimum number of nodes for cluster stability. Host-level encryption is +also enabled to improve security of data at rest on the nodes. + +After provisioning, the default nodepool is updated to run in `User` mode, ensuring a clear separation between system and application workloads. + +The system node pool is deployed across three availability zones to improve resiliency, fault tolerance, and workload availability within the cluster. + +### GPU Nodepool +The GPU nodepool is an optional dedicated nodepool designed to run workloads that require GPU acceleration, such as machine learning, AI inference, or compute-intensive processing tasks. It +is configured in `User` mode and isolated from system workloads using a custom label (`nodepool.compute=gpu`) to allow targeted scheduling of GPU-enabled pods. + +Cluster autoscaling is enabled to dynamically adjust capacity based on demand, with a defined minimum and maximum node range to balance availability and cost. Host-level encryption is +enabled to secure data at rest on GPU nodes, and GPU instance profiling is configured to support optimized GPU resource allocation. + +The GPU node pool is deployed across three availability zones to improve resiliency, fault tolerance, and workload availability within the cluster. + +### Maintenance Window +The maintenance window is configured to run on Sunday at 04:00 UTC, but can be adjusted to any time during the week based on operational requirements. This is done by modifying the +`--weekday` and `--start-hour` parameters in the `deploy.ps1` script. The `--duration` parameter should be set to a minimum of 4 hours. + +> ⚠️ Note that the Azure Portal does not display the default maintenance schedule unless custom update schedules are explicitly configured for the different maintenance controls. + +### Monitoring +Monitoring is distributed across Container Insights, Azure Managed Prometheus, and Azure Managed Grafana, with each component responsible for a distinct layer of observability +in the system. + +Container Insights is used for centralized logging, Kubernetes events, and cluster inventory. It collects container stdout/stderr logs, Kubernetes events, and pod/node inventory from +across the cluster, providing visibility into node and cluster availability and operational state through Log Analytics. Metrics collection through Container Insights is intentionally +minimized because metrics are handled by Prometheus. The data collection rule is a production-oriented configuration and can be customized or omitted in development or test environments +to reduce cost, with Azure defaults available as an alternative. The `--data-collection-settings` parameter controls this configuration during AKS enablement and can be adjusted later +if requirements change. + +Azure Managed Prometheus is used for collecting Kubernetes and application metrics, which form the basis for dashboards, analysis, and alerting. It is enabled through an Azure Monitor +managed Prometheus workspace, which automatically provisions scraping and collection rules for Kubernetes metrics. Azure Managed Grafana is connected to the Prometheus workspace for +visualization and deeper analysis of the collected metrics. Prometheus Rule Groups are used for metric-based alerting using PromQL expressions, with support for extending scraping +configuration to include additional metrics beyond the default setup when required. + +> ⚠️ Azure Prometheus uses different CRDs: `azmonitoring.coreos.com/v1` instead of `monitoring.coreos.com/v1`. + +The following table provides an overview of how observability responsibilities are distributed across the different Azure monitoring components in the AKS architecture. + +| Layer | Azure Service | Responsibility | +| ------------------- | ----------------------------------------------- | ----------------------------------------------------------------- | +| Application Metrics | Azure Managed Prometheus | Application, pod, node, and Kubernetes metrics. | +| Logging & Events | Container Insights | Container logs, Kubernetes events, pod/node inventory. | +| Dashboards | Azure Managed Grafana | Metrics visualization and dashboards. | +| Alerting | Prometheus Rule Groups | Metric-based alerting using PromQL. | + +> 📖 Learn more about **[Azure Kubernetes Monitoring](https://learn.microsoft.com/en-us/azure/azure-monitor/containers/kubernetes-monitoring-tutorial)**. + +### Alerts +Alerts are handled through Azure Managed Prometheus using Prometheus Rule Groups with PromQL-based expressions. Prometheus-based alerts provide higher accuracy and more detailed +insights that the legacy Container Insight Alerts. The full set of Prometheus alerts is listed below. + +| Name | Description | Severity | Window Size | +| ------------------------------ | ------------------------------------------------------------- | -------- | ----------- | +| Node Not Ready | Node Ready condition is false | Critical | PT5M | +| Node Unreachable | Node is marked with unreachable taint | Severe | PT10M | +| Node Readiness Flapping | Frequent changes in Node Ready status | Warn | PT10M | +| Node Memory Pressure | Node is under MemoryPressure condition | Severe | PT5M | +| Node CPU Pressure | Node CPU usage is above 85% | Severe | PT5M | +| Container CPU Rising | Container CPU usage exceeds 60% of resource limits | Warn | PT10M | +| Container CPU High | Container CPU usage exceeds 80% of resource limits | Severe | PT10M | +| Container CPU Critical | Container CPU usage exceeds 95% of resource limits | Critical | PT5M | +| Container Memory Rising | Container memory usage exceeds 60% of resource limits | Warn | PT10M | +| Container Memory High | Container memory usage exceeds 80% of resource limits | Severe | PT10M | +| Container Memory Critical | Container memory usage exceeds 95% of resource limits | Critical | PT5M | +| PV Usage High | PersistentVolume usage exceeds 80% capacity | Severe | PT30M | +| Hpa Maxed Out | HPA is at maximum replicas and cannot scale further | Warn | PT10M | +| Hpa Replicas Mismatch | Desired replicas differ from current replicas | Warn | PT10M | +| Daemon Set Not Scheduled | Desired DaemonSet pods are not fully scheduled | Warn | PT10M | +| Pod Scheduling Pressure | Pods are stuck in Pending state | Warn | PT10M | +| Pod No tReady | Pods are in Pending, Unknown, or Failed state | Warn | PT10M | +| Pod Crash Looping | Pods are in CrashLoopBackOff state | Info | PT5M | +| Pod Image Pull Failure | Pods failing due to ImagePullBackOff or ErrImagePull | Warn | PT5M | +| Container OOM Killed | Containers terminated due to OOMKilled | Info | PT5M | +| PodRestart Rate High | Pods with more than 3 restarts in 5 minutes | Warn | PT5M | +| Deployment Replicas Mismatch | Available replicas do not match desired deployment replicas | Info | PT10M | + +For Container Insights, the following alerts are supplied in the `deploy.ps1` script. This is provided as a fallback in cases where Prometheus is not installed in the Kubernetes cluster. + +| Name | Description | Severity | Window Size | +| ---------------------------------------- | --------------------------------------------------- | -------- | ----------- | +| Node CPU Rising (60) | Node CPU sustained pressure above 60% | Warn | PT5M | +| Node CPU High (80) | Node CPU usage above 80% | Severe | PT5M | +| Node CPU Critical (95) | Node CPU usage above 95% | Critical | PT5M | +| Node Memory Rising (60) | Node memory sustained pressure above 60% | Warn | PT5M | +| Node Memory High (80) | Node memory usage above 80% | Severe | PT5M | +| Node Memory Critical (95) | Node memory usage above 95% | Critical | PT5M | +| Node Disk High (80) | Node disk usage above 80% | Severe | PT5M | +| Container CPU High (Absolute NanoCores) | Container CPU high usage (requires limits set) | Severe | PT5M | +| Container Memory High (Absolute Bytes) | Container memory high usage (absolute threshold) | Severe | PT5M | +| Node Not Ready | Node is not in Ready state | Critical | PT5M | +| Pod CrashLoopBackOff | Pods in CrashLoopBackOff state | Critical | PT5M | +| Image Pull Failures | Image pull failures detected | Severe | PT5M | +| OOM Killed Containers | Containers terminated due to OOM | Critical | PT5M | +| Pod Restart Rate High | High restart rate in last 5 minutes | Severe | PT5M | +| Pods Pending | Pods stuck in Pending state | Warn | PT5M | + +> ⚠️ Legacy alerts are less precise and rely on fixed container thresholds (e.g. 800 mCPU) rather than percentage-based utilization of resource limits. + +### Site Recovery +Backup and site recovery are not configured for the AKS cluster. The deployment follows an ephemeral infrastructure model, where workloads can be recreated at any time, and all critical +data is stored in external managed services rather than within the cluster. + +Enabling Kubernetes backup requires Private Endpoints between the AKS virtual network and the Backup Vault to ensure secure and private connectivity. + +### Image Cleaner +The Image Cleaner feature is optional and helps reduce disk usage on cluster nodes by automatically removing unused container images. When enabled, it periodically cleans up images that +are no longer referenced by running workloads. + +The cleanup interval is configured using `--image-cleaner-interval-hours`, which defines how frequently the process runs (e.g. every 72 hours). The value can be adjusted based on +operational requirements. + +### Diagnostic Settings +Next, create the diagnostic settings for the Kubernetes (AKS) cluster. + +AKS Diagnostic Settings are used for Kubernetes control plane and platform-level logs such as API server, audit, scheduler, autoscaler, and controller manager logs. Diagnostic metrics +(`AllMetrics`) are intentionally disabled because Prometheus already handles metrics collection and alerting. + +> ⚠️ If Prometheus is not installed, enable `Microsoft-InsightsMetrics` and `AllMetrics`. + +You can retrieve the full list of supported metric and log categories for the Azure Kuberentes Cluster (AKS) resource using the following command. + +```powershell +az monitor diagnostic-settings categories list --resource $env:KUBERNETES_ID; +``` + +Azure Load Balancer Diagnostic Settings are configured to collect `LoadBalancerHealthEvent` logs for backend probe failures and health status, with optional `AllMetrics` enabled using +a 1-minute aggregation interval for metric collection. + +You can retrieve the full list of supported metric and log categories for the Azure Load Balancer resource using the following command. + +```powershell +az monitor diagnostic-settings categories list --resource $env:LOAD_BALANCER_ID; +``` + +Application Gateway for Containers (ALB) Diagnostic Settings are enabled to collect ingress access logs, providing visibility into HTTP requests, routing behavior, and response +outcomes at the cluster edge. + +You can retrieve the full list of supported metric and log categories for the Azure Application Load Balancer resource using the following command. + +```powershell +az monitor diagnostic-settings categories list --resource $env:ALB_ID; +``` + +The following table provides an overview of log and metric coverage in Diagnostic Settings, outlining how observability responsibilities are distributed within the AKS architecture. + +| Layer | Azure Service | Responsibility | +| ------------------- | ----------------------------------------------- | ----------------------------------------------------------------- | +| AKS Control Plane | AKS Diagnostic Settings | API server, audit, scheduler, autoscaler, and controller logs. | +| Azure Networking | Azure Load Balancer Diagnostic Settings | Load balancer health and probe events. | +| Edge / Ingress | ALB Diagnostic Settings | Application Gateway for Containers access and firewall logs. | + +### Network Rules +The Kubernetes Cluster has no public access by default. + +Optionally, IP address whitelisting can be configured to allow access to the storage file shares. By default, access is fully restricted, and no external connections are permitted. +The Nano system does not depend on IP whitelisting for connectivity, and using it is generally discouraged as it can negatively impact the overall cloud security score. If IP +whitelisting is required, it can be configured using the following command. + +```powershell +$env:NETWORK_RULE_WHITE_LISTED_IP_ADDRESS = ""; + +az aks updaate ` + -g $env:AZURE_RESOURCE_GROUP ` + -n $env:APP_NAME ` + -l $env:AZURE_LOCATION ` + --api-server-authorized-ip-ranges $env:NETWORK_RULE_WHITE_LISTED_IP_ADDRESS; +``` + +> ⚠️ Whitelisting IP addresses can reduce the overall security posture and negatively impact the security score. + +### Policy +Adds the default Azure Policy assignment to the cluster, enabling built-in compliance monitoring and governance enforcement. This provides visibility into configuration drift, +security posture, and best-practice adherence, while continuously evaluating the cluster against Azure Policy definitions and initiatives. + +> 📖 Learn more about **[Azure Policy](https://portal.azure.com/#view/Microsoft_Azure_Policy/PolicyMenuBlade.MenuView/~/Overview)**. + +### Microsoft Defender +Microsoft Defender for Containers is used to enhance security monitoring and threat detection for the AKS cluster. It integrates with the Log Analytics workspace created via +**[Nano.Azure.Monitoring](https://github.com/Nano-Core/Nano.Azure/tree/master/Nano.Azure.Monitoring/README.md#nanoazuremonitoring)** to centralize security telemetry and enable +vulnerability and runtime threat detection for container workloads. + +The Defender configuration is minimal and primarily used to link the cluster to the Log Analytics workspace through a generated configuration file. This file contains the workspace +resource ID and is required during setup. + +However, Microsoft Defender for AKS must still be explicitly enabled in the Azure Portal after deployment to fully activate security protections for the cluster. + +## Image Pull Secret +This step creates a Kubernetes image pull secret that allows the cluster to authenticate against the container registry and pull private images. + +Execute the `image-pull-secret.ps1` script to create the required Docker registry secret in the cluster. The script provisions a `docker-registry` type secret that Kubernetes uses +when pulling container images. This secret is referenced by workloads that require access to images stored in the container registry. + +📖 Learn how to configure access to **[GitHub Container Registry](https://github.com/Nano-Core/Nano.GitHub/tree/master/Nano.GitHub.ContainerRegistry)** to obtain the credentials needed to +create a Kubernetes image-pull secret for pulling private images during GitHub Actions deployments. + +## Configure kubectl Access +Once the cluster has been registered, retrieve the credentials using the following command. + +```powershell +az aks get-credentials -g $env:AZURE_RESOURCE_GROUP -n $env:APP_NAME --public-fqdn; +``` + +You can now use `kubectl` to manage the cluster. + +> ⚠️ Make sure you are connected the the VPn Gateway, if the Kubernetes cluster is set up with private access. + +## Common `kubectl` Commands +The following section contains commonly used Kubernetes (kubectl) commands for managing and troubleshooting resources in the cluster. These commands can be used to inspect workloads, view +logs, monitor deployments, validate configuration, and diagnose issues related to networking, storage, and application health. + +List all Kubernetes nodes to view cluster infrastructure, status, and readiness of compute resources. + +```powershell +kubectl get nodes +``` + +Describe the `{{node-name}}` to inspect its capacity, conditions, system information, and scheduling details. + +```powershell +kubectl describe node {{node-name}}; +``` + +Describe a pod with the `{{pod-name}}` in the `{{namespace}}` to inspect its full configuration, runtime status, and recent events for troubleshooting and validation. + +```powershell +kubectl describe pod {{pod-name}} -n {{namespace}}; +``` + +List all pods in the `{{namespace}}` to view running workloads, status, and readiness information. + +```powershell +kubectl get pods -n {{namespace}}; +``` + +View the highest resource-consuming pods in the `{{namespace}}` to monitor CPU and memory usage. + +```powershell +kubectl top pods -n {{namespace}} +``` + +Retrieve events for the `{{pod-name}}` in the `{{namespace}}` to analyze scheduling, warnings, and runtime issues. + +```powershell +kubectl events -n {{namespace}} --field-selector InvolvedObject.Name {{pod-name}}; +``` + +Fetch logs from all pods labeled `{{app}}` and filter results using `{{search}}` to quickly locate relevant runtime output or errors. + +```powershell +kubectl logs -l -n {{namespace}} app={{app}} --tail -1 | findstr -i '{{search}}'; +``` + +Manually trigger a new Kubernetes Job from an existing CronJob for immediate execution. + +```powershell +kubectl create job --from cronjob/{{cronjob-name}} {{job-name}} +``` + +Temporarily suspend a CronJob to prevent scheduled executions while maintaining its configuration in the cluster. + +```powershell +kubectl patch cronjob {{cronjob-name}} -p '{"spec": {"suspend": true}}' +``` + + +```powershell +kubectl port-forward pod/ 8080:80 -n {{namespace}} +``` + +## Dependencies +Kubernetes has the following dependencies that must be deployed or otherwise satisfied prior to setup. + +| Dependency | Description | +| ------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------- | +| **[Nano.Azure.Monitoring](https://github.com/Nano-Core/Nano.Azure/tree/master/Nano.Azure.Monitoring/README.md#nanoazuremonitoring)** | Components for centralized monitoring and logging. | diff --git a/Nano.Azure.Kubernetes/deploy.ps1 b/Nano.Azure.Kubernetes/deploy.ps1 new file mode 100644 index 0000000..cae646d --- /dev/null +++ b/Nano.Azure.Kubernetes/deploy.ps1 @@ -0,0 +1,570 @@ +$env:ENVIRONMENT = ""; +$env:AZURE_TENANT_ID = ""; +$env:AZURE_LOCATION = "North Europe"; +$env:AZURE_RESOURCE_GROUP = "Nano-Kubernetes"; +$env:AZURE_RESOURCE_GROUP_LOGS = "Nano-Logs"; +$env:AZURE_RESOURCE_GROUP_ASSETS = "Nano-Kubernetes-Assets"; +$env:KUBERNETES_VERSION = "1.35.0"; +$env:KUBERNETES_TIER = "standard"; +$env:KUBERNETES_NODEPOOL_NAME = "default"; +$env:KUBERNETES_NODEPOOL_LABEL_COMPUTE = "cpu" +$env:KUBERNETES_NODE_SIZE = "standard_d2as_v6"; +$env:KUBERNETES_NODE_COUNT = 3; +$env:KUBERNETES_NODES_MIN = 3; +$env:KUBERNETES_NODES_MAX = 6; +$env:APP_NAME = $env:ENVIRONMENT.ToLower() + "-cluster"; + +# Register Providers +az provider register --namespace Microsoft.PolicyInsights; +az provider register --namespace Microsoft.Network; +az provider register --namespace Microsoft.NetworkFunction; +az provider register --namespace Microsoft.ServiceNetworking; + +az feature register --namespace "Microsoft.ContainerService" --name "ManagedGatewayAPIPreview"; +az feature register --namespace "Microsoft.ContainerService" --name "ApplicationLoadBalancerPreview"; + +az extension add --name alb; +az extension add --name aks-preview; + +# Resource Group +az group create ` + -n $env:AZURE_RESOURCE_GROUP ` + -l $env:AZURE_LOCATION; + +# Kubernetes Cluster +az aks create ` + -g $env:AZURE_RESOURCE_GROUP ` + -n $env:APP_NAME ` + -l $env:AZURE_LOCATION ` + --tier $env:KUBERNETES_TIER ` + --kubernetes-version $env:KUBERNETES_VERSION ` + --node-count $env:KUBERNETES_NODE_COUNT ` + --node-resource-group $env:AZURE_RESOURCE_GROUP_ASSETS ` + --node-vm-size $env:KUBERNETES_NODE_SIZE ` + --nodepool-name $env:KUBERNETES_NODEPOOL_NAME ` + --nodepool-labels nodepool.compute=$env:KUBERNETES_NODEPOOL_LABEL_COMPUTE ` + --node-os-upgrade-channel NodeImage ` + --auto-upgrade-channel patch ` + --enable-cluster-autoscaler ` + --max-count $env:KUBERNETES_NODES_MAX ` + --min-count $env:KUBERNETES_NODES_MIN ` + --network-plugin azure ` + --network-plugin-mode overlay ` + --network-policy azure ` + --enable-encryption-at-host ` + --enable-private-cluster ` + --enable-managed-identity ` + --enable-workload-identity ` + --enable-oidc-issuer ` + --enable-gateway-api ` + --enable-application-load-balancer ` + --ssh-access Disabled ` + --zones 1 2 3; + +# Load Balancer +$env:SUBNET_ALB_NAME = "aks-subnet-alb"; +$env:SUBNET_ADDRESS_PREFIXES = "10.230.0.0/24"; +$env:VNET_NAME = az network vnet list -g $env:AZURE_RESOURCE_GROUP_ASSETS --query [0].name -o tsv; + +az network vnet subnet create ` + -g $env:AZURE_RESOURCE_GROUP_ASSETS ` + -n $env:SUBNET_ALB_NAME ` + --vnet-name $env:VNET_NAME ` + --address-prefixes $env:SUBNET_ADDRESS_PREFIXES ` + --delegations 'Microsoft.ServiceNetworking/trafficControllers'; + +$env:ALB_IDENTITY_NAME = az identity list -g $env:AZURE_RESOURCE_GROUP_ASSETS --query "[?contains(name, 'applicationloadbalancer')].name" -o tsv +$env:PRINCIPAL_ID = az identity show -g $env:AZURE_RESOURCE_GROUP_ASSETS -n $env:ALB_IDENTITY_NAME --query principalId -o tsv; +$env:ALB_SUBNET_ID = az network vnet subnet show -n $env:SUBNET_ALB_NAME -g $env:AZURE_RESOURCE_GROUP_ASSETS --vnet-name $env:VNET_NAME --query id --output tsv; +$env:AZURE_RESOURCE_GROUP_ASSETS_ID = az group show -n $env:AZURE_RESOURCE_GROUP_ASSETS --query id; + +az role assignment create ` + --assignee-object-id $env:PRINCIPAL_ID ` + --assignee-principal-type ServicePrincipal ` + --scope $env:AZURE_RESOURCE_GROUP_ASSETS_ID ` + --role "Contributor"; + +az role assignment create ` + --assignee-object-id $env:PRINCIPAL_ID ` + --assignee-principal-type ServicePrincipal ` + --scope $env:ALB_SUBNET_ID ` + --role "Network Contributor"; + +$env:APPLICATION_LOAD_BALANCER_PATH = Join-Path $env:USERPROFILE "application-load-balancer.yaml"; +$env:KUBERNETES_NAMESPACE = "apps"; +$env:VNET_NAME = az network vnet list -g $env:AZURE_RESOURCE_GROUP_ASSETS --query [0].name -o tsv; +$env:ALB_SUBNET_ID = az network vnet subnet show -n $env:SUBNET_ALB_NAME -g $env:AZURE_RESOURCE_GROUP_ASSETS --vnet-name $env:VNET_NAME --query id -o tsv; + +Get-Content .kubernetes/application-load-balancer.yaml | foreach { [Environment]::ExpandEnvironmentVariables($_) } | Set-Content $env:APPLICATION_LOAD_BALANCER_PATH; + +az aks command invoke ` + -g $env:AZURE_RESOURCE_GROUP ` + -n $env:APP_NAME ` + --file $env:APPLICATION_LOAD_BALANCER_PATH ` + --command "kubectl apply -f application-load-balancer.yaml" ` + --output json | ConvertFrom-Json; + +# DNS Resolver +$env:DNS_RESOLVER_NAME = $env:APP_NAME + "-dns-resolver"; +$env:SUBNET_DNS_RESOLVER_NAME = "aks-dns-resolver-subnet"; +$env:VNET_ID = az network vnet list -g $env:AZURE_RESOURCE_GROUP_ASSETS --query [0].id -o tsv; + +az extension add --name dns-resolver; + +az dns-resolver create ` + -g $env:AZURE_RESOURCE_GROUP_ASSETS ` + -n $env:DNS_RESOLVER_NAME ` + -l $env:AZURE_LOCATION ` + --id $env:VNET_ID; + +$env:VNET_NAME = az network vnet list -g $env:AZURE_RESOURCE_GROUP_ASSETS --query [0].name -o tsv; +$env:SUBNET_ADDRESS_PREFIXES = "10.233.0.0/28"; + +az network vnet subnet create ` + -g $env:AZURE_RESOURCE_GROUP_ASSETS ` + -n $env:SUBNET_DNS_RESOLVER_NAME ` + --vnet-name $env:VNET_NAME ` + --address-prefixes $env:SUBNET_ADDRESS_PREFIXES ` + --delegations "Microsoft.Network/dnsResolvers"; + +$env:DNS_RESOLVER_SUBNET_ID = az network vnet subnet show -g $env:AZURE_RESOURCE_GROUP_ASSETS -n $env:SUBNET_DNS_RESOLVER_NAME --vnet-name $env:VNET_NAME --query id -o tsv; + +az dns-resolver inbound-endpoint create ` + -g $env:AZURE_RESOURCE_GROUP_ASSETS ` + -l $env:AZURE_LOCATION ` + --name inbound ` + --dns-resolver-name $env:DNS_RESOLVER_NAME ` + --ip-configurations "[{private-ip-address:'',private-ip-allocation-method:'Dynamic',id:'$($env:DNS_RESOLVER_SUBNET_ID)'}]"; + +$env:DNS_RESOLVER_PRIVATE_ID = az dns-resolver inbound-endpoint show -g $env:AZURE_RESOURCE_GROUP_ASSETS --dns-resolver-name $env:DNS_RESOLVER_NAME --name inbound --query ipConfigurations[0].privateIpAddress -o tsv; + +az network vnet update ` + -g $env:AZURE_RESOURCE_GROUP_ASSETS ` + -n $env:VNET_NAME ` + --dns-servers $env:DNS_RESOLVER_PRIVATE_ID; + +# VPN Gateway +$env:VNET_ID = az network vnet list -g $env:AZURE_RESOURCE_GROUP_ASSETS --query [0].id -o tsv; +$env:VNET_NAME = az network vnet list -g $env:AZURE_RESOURCE_GROUP_ASSETS --query [0].name -o tsv; +$env:VNET_GATEWAY_NAME = $env:APP_NAME + "-vnet-vpn-gateway"; +$env:VNET_GATEWAY_IP_NAME = $env:APP_NAME + "-vnet-vpn-gateway-ip"; +$env:VNET_GATEWAY_VPN_CLIENT_ADDRESS_POOL = "['172.16.201.0/24']"; +$env:VNET_GATEWAY_ADD_TENANT = "https://login.microsoftonline.com/" + $env:AZURE_TENANT_ID; +$env:VNET_GATEWAY_ADD_AUDIENCE = "41b23e61-6c1e-4545-b367-cd054e0ed4b4"; +$env:VNET_GATEWAY_ADD_ISSUER = "https://sts.windows.net/" + $env:AZURE_TENANT_ID + "/"; +$env:SUBNET_ADDRESS_PREFIXES = "10.231.0.0/24"; + +az network public-ip create ` + -g $env:AZURE_RESOURCE_GROUP_ASSETS ` + -n $env:VNET_GATEWAY_IP_NAME ` + -z 1 2 3 ` + --sku Standard ` + --allocation-method Static; + +az network vnet subnet create ` + -g $env:AZURE_RESOURCE_GROUP_ASSETS ` + -n GatewaySubnet ` + --vnet-name $env:VNET_NAME ` + --address-prefix $env:SUBNET_ADDRESS_PREFIXES; + +az network vnet-gateway create ` + -g $env:AZURE_RESOURCE_GROUP_ASSETS ` + -n $env:VNET_GATEWAY_NAME ` + --public-ip-address $env:VNET_GATEWAY_IP_NAME ` + --vnet $env:VNET_ID ` + --gateway-type Vpn ` + --vpn-type RouteBased ` + --sku VpnGw1AZ ` + --vpn-gateway-generation Generation2; + +az network vnet-gateway update ` + -g $env:AZURE_RESOURCE_GROUP_ASSETS ` + -n $env:VNET_GATEWAY_NAME ` + --set "vpnClientConfiguration.vpnClientAddressPool.addressPrefixes=$env:VNET_GATEWAY_VPN_CLIENT_ADDRESS_POOL" ` + --set "vpnClientConfiguration.vpnClientProtocols=['OpenVPN']" ` + --set "vpnClientConfiguration.vpnAuthenticationTypes=['Aad']" ` + --set vpnClientConfiguration.aadTenant=$env:VNET_GATEWAY_ADD_TENANT ` + --set vpnClientConfiguration.aadAudience=$env:VNET_GATEWAY_ADD_AUDIENCE ` + --set vpnClientConfiguration.aadIssuer=$env:VNET_GATEWAY_ADD_ISSUER; + +az network vnet-gateway vpn-client generate ` + -g $env:AZURE_RESOURCE_GROUP_ASSETS ` + -n $env:VNET_GATEWAY_NAME; + +# System Nodepool +$env:KUBERNETES_NODEPOOL_SYSTEM_NAME = "system"; +$env:KUBERNETES_SYSTEM_NODE_SIZE = "standard_d2as_v6"; +$env:KUBERNETES_SYSTEM_NODE_COUNT=3; +$env:KUBERNETES_SYSTEM_NODES_MIN=3; +$env:KUBERNETES_SYSTEM_NODES_MAX=3; + +az aks nodepool add ` + -g $env:AZURE_RESOURCE_GROUP ` + -n $env:KUBERNETES_NODEPOOL_SYSTEM_NAME ` + --cluster-name $env:APP_NAME ` + --kubernetes-version $env:KUBERNETES_VERSION ` + --mode System ` + --os-type Linux ` + --node-taints CriticalAddonsOnly=true:NoSchedule ` + --node-vm-size $env:KUBERNETES_SYSTEM_NODE_SIZE ` + --node-count $env:KUBERNETES_SYSTEM_NODE_COUNT ` + --min-count $env:KUBERNETES_SYSTEM_NODES_MIN ` + --max-count $env:KUBERNETES_SYSTEM_NODES_MAX ` + --enable-encryption-at-host ` + --enable-cluster-autoscaler ` + --ssh-access Disabled ` + --zones 1 2 3; + +az aks nodepool update ` + -g $env:AZURE_RESOURCE_GROUP ` + --cluster-name $env:APP_NAME ` + --name $env:KUBERNETES_NODEPOOL_NAME ` + --mode user; + +# GPU Nodepool (Optional) +$env:KUBERNETES_NODEPOOL_GPU_NAME = "gpu"; +$env:KUBERNETES_GPU_NODE_SIZE = ""; +$env:KUBERNETES_GPU_NODE_COUNT=1; +$env:KUBERNETES_GPU_NODES_MIN=1; +$env:KUBERNETES_GPU_NODES_MAX=3; + +az aks nodepool add ` + -g $env:AZURE_RESOURCE_GROUP ` + -n $env:KUBERNETES_NODEPOOL_GPU_NAME ` + --cluster-name $env:APP_NAME ` + --kubernetes-version $env:KUBERNETES_VERSION ` + --mode User ` + --os-type Linux ` + --node-vm-size $env:KUBERNETES_GPU_NODE_SIZE ` + --node-count $env:KUBERNETES_GPU_NODE_COUNT ` + --min-count $env:KUBERNETES_GPU_NODES_MIN ` + --max-count $env:KUBERNETES_GPU_NODES_MAX ` + --labels nodepool.compute=gpu ` + --enable-cluster-autoscaler ` + --enable-encryption-at-host ` + --gpu-driver Install ` + --gpu-instance-profile MIG1g ` + --ssh-access Disabled ` + --zones 1 2 3; + +# Maintenance +az aks maintenanceconfiguration add ` + -g $env:AZURE_RESOURCE_GROUP ` + --cluster-name $env:APP_NAME ` + --name default ` + --weekday Sunday ` + --start-hour 4 ` + --duration 4; + +# Monitoring (Container Insights) +$env:LOG_ANALYTICS_WORKSPACE_ID = az monitor log-analytics workspace list -g $env:AZURE_RESOURCE_GROUP_LOGS --query [0].[id] -o tsv; + +az aks enable-addons ` + -g $env:AZURE_RESOURCE_GROUP ` + -n $env:APP_NAME ` + --addon monitoring ` + --workspace-resource-id $env:LOG_ANALYTICS_WORKSPACE_ID ` + --data-collection-settings '.data-collection-settings/data-collection-settings.json'; + +# Monitoring (Prometheus) +az extension add -n amg; +az config set extension.dynamic_install_allow_preview=true; + +$env:APP_NAME_GRAFANA = "grafana-" + (Get-Random -Maximum 999999).ToString("D6"); +$env:MONITOR_WORKSPACE_ID = az monitor account list -g $env:AZURE_RESOURCE_GROUP_LOGS --query [0].[id] -o tsv; + +$env:GRAFANA_ID = az grafana create ` + -g $env:AZURE_RESOURCE_GROUP ` + -n $env:APP_NAME_GRAFANA ` + -l $env:AZURE_LOCATION ` + --query id -o tsv; + +az aks update ` + -g $env:AZURE_RESOURCE_GROUP ` + -n $env:APP_NAME ` + --enable-azure-monitor-metrics ` + --azure-monitor-workspace-resource-id $env:MONITOR_WORKSPACE_ID ` + --grafana-resource-id $env:GRAFANA_ID; + +# Alerts (Prometheus) +az extension add -n alertsmanagement; + +$env:ACTION_GROUP = az monitor action-group list -g $env:AZURE_RESOURCE_GROUP_LOGS --query [0].[id] -o tsv; + +az alerts-management prometheus-rule-group create ` + --name 'Prometheus Alerts - Resource Saturation' ` + --resource-group $env:AZURE_RESOURCE_GROUP ` + --location $env:AZURE_LOCATION ` + --cluster-name $env:APP_NAME ` + --enabled true ` + --description "Resource saturation alerts" ` + --interval PT5M ` + --scopes $env:MONITOR_WORKSPACE_ID ` + --rules '.alerts/resource-saturation.json'; + +az alerts-management prometheus-rule-group create ` + --name 'Prometheus Alerts - Workload Stability' ` + --resource-group $env:AZURE_RESOURCE_GROUP ` + --location $env:AZURE_LOCATION ` + --cluster-name $env:APP_NAME ` + --enabled true ` + --description "Workload stability alerts" ` + --interval PT5M ` + --scopes $env:MONITOR_WORKSPACE_ID ` + --rules '.alerts/workload-stability.json'; + +az alerts-management prometheus-rule-group create ` + --name 'Prometheus Alerts - Scheduling Scaling' ` + --resource-group $env:AZURE_RESOURCE_GROUP ` + --location $env:AZURE_LOCATION ` + --cluster-name $env:APP_NAME ` + --enabled true ` + --description "Scheduling scaling alerts" ` + --interval PT5M ` + --scopes $env:MONITOR_WORKSPACE_ID ` + --rules '.alerts/scheduling-scaling.json'; + +az alerts-management prometheus-rule-group create ` + --name 'Prometheus Alerts - Node Health' ` + --resource-group $env:AZURE_RESOURCE_GROUP ` + --location $env:AZURE_LOCATION ` + --cluster-name $env:APP_NAME ` + --enabled true ` + --description "Node health alerts" ` + --interval PT5M ` + --scopes $env:MONITOR_WORKSPACE_ID ` + --rules '.alerts/node-health.json'; + +# Alerts (Legacy) +az extension add --name scheduled-query; + +$env:KUBERNETES_ID = az aks show -g $env:AZURE_RESOURCE_GROUP -n $env:APP_NAME --query id -o tsv; +$env:ACTION_GROUP = az monitor action-group list -g $env:AZURE_RESOURCE_GROUP_LOGS --query [0].[id] -o tsv; + +az monitor metrics alert create ` + --name "Node CPU Rising (60)" ` + --description "Node CPU sustained pressure above 60%." ` + --resource-group $env:AZURE_RESOURCE_GROUP ` + --scopes $env:KUBERNETES_ID ` + --condition "avg node_cpu_usage_percentage > 60" ` + --window-size PT5M ` + --evaluation-frequency PT1M ` + --action $env:ACTION_GROUP ` + --severity 3; + +az monitor metrics alert create ` + --name "Node CPU High (80)" ` + --description "Node CPU usage above 80%." ` + --resource-group $env:AZURE_RESOURCE_GROUP ` + --scopes $env:KUBERNETES_ID ` + --condition "avg node_cpu_usage_percentage > 80" ` + --window-size PT5M ` + --evaluation-frequency PT1M ` + --action $env:ACTION_GROUP ` + --severity 2; + +az monitor metrics alert create ` + --name "Node CPU Critical (95)" ` + --description "Node CPU usage above 95%." ` + --resource-group $env:AZURE_RESOURCE_GROUP ` + --scopes $env:KUBERNETES_ID ` + --condition "avg node_cpu_usage_percentage > 95" ` + --window-size PT5M ` + --evaluation-frequency PT1M ` + --action $env:ACTION_GROUP ` + --severity 1; + +az monitor metrics alert create ` + --name "Node Memory Rising (60)" ` + --description "Node memory sustained pressure above 60%." ` + --resource-group $env:AZURE_RESOURCE_GROUP ` + --scopes $env:KUBERNETES_ID ` + --condition "avg node_memory_working_set_percentage > 60" ` + --window-size PT5M ` + --evaluation-frequency PT1M ` + --action $env:ACTION_GROUP ` + --severity 3; + +az monitor metrics alert create ` + --name "Node Memory High (80)" ` + --description "Node memory usage above 80%." ` + --resource-group $env:AZURE_RESOURCE_GROUP ` + --scopes $env:KUBERNETES_ID ` + --condition "avg node_memory_working_set_percentage > 80" ` + --window-size PT5M ` + --evaluation-frequency PT1M ` + --action $env:ACTION_GROUP ` + --severity 2; + +az monitor metrics alert create ` + --name "Node Memory Critical (95)" ` + --description "Node memory usage above 95%." ` + --resource-group $env:AZURE_RESOURCE_GROUP ` + --scopes $env:KUBERNETES_ID ` + --condition "avg node_memory_working_set_percentage > 95" ` + --window-size PT5M ` + --evaluation-frequency PT1M ` + --action $env:ACTION_GROUP ` + --severity 1; + +az monitor metrics alert create ` + --name "Node Disk High (80)" ` + --description "Node disk usage above 80%." ` + --resource-group $env:AZURE_RESOURCE_GROUP ` + --scopes $env:KUBERNETES_ID ` + --condition "avg node_disk_usage_percentage > 80" ` + --window-size PT5M ` + --evaluation-frequency PT1M ` + --action $env:ACTION_GROUP ` + --severity 2; + +az monitor scheduled-query create ` + --name "Container CPU High (Absolute NanoCores)" ` + --description "Container CPU high usage (requires limits set)." ` + --resource-group $env:AZURE_RESOURCE_GROUP ` + --scopes $env:LOG_ANALYTICS_WORKSPACE_ID ` + --condition "count 'InsightsMetrics | where TimeGenerated > ago(5m) | where Name == \'cpuUsageNanoCores\' | where Val > 800000000' > 0" ` + --window-size PT5M ` + --evaluation-frequency PT1M ` + --action $env:ACTION_GROUP ` + --location $env:AZURE_LOCATION ` + --severity 2; + +az monitor scheduled-query create ` + --name "Container Memory High (Absolute Bytes)" ` + --description "Container memory high usage (absolute threshold)." ` + --resource-group $env:AZURE_RESOURCE_GROUP ` + --scopes $env:LOG_ANALYTICS_WORKSPACE_ID ` + --condition "count 'InsightsMetrics | where TimeGenerated > ago(5m) | where Name == \'memoryWorkingSetBytes\' | where Val > 800000000' > 0" ` + --window-size PT5M ` + --evaluation-frequency PT1M ` + --action $env:ACTION_GROUP ` + --location $env:AZURE_LOCATION ` + --severity 2; + +az monitor scheduled-query create ` + --name "Node Not Ready" ` + --description "Node is not in Ready state." ` + --resource-group $env:AZURE_RESOURCE_GROUP ` + --scopes $env:LOG_ANALYTICS_WORKSPACE_ID ` + --condition "count 'KubePodInventory | where PodStatus != \'Running\' | summarize count() by Name' > 0" ` + --evaluation-frequency PT1M ` + --window-size PT5M ` + --action $env:ACTION_GROUP ` + --location $env:AZURE_LOCATION ` + --severity 1; + +az monitor scheduled-query create ` + --name "Pod CrashLoopBackOff" ` + --description "Pods in CrashLoopBackOff state." ` + --resource-group $env:AZURE_RESOURCE_GROUP ` + --scopes $env:LOG_ANALYTICS_WORKSPACE_ID ` + --condition "count 'KubePodInventory | where ContainerStatusReason == \'CrashLoopBackOff\'' > 0" ` + --evaluation-frequency PT5M ` + --window-size PT5M ` + --action $env:ACTION_GROUP ` + --location $env:AZURE_LOCATION ` + --severity 1; + +az monitor scheduled-query create ` + --name "Image Pull Failures" ` + --description "Image pull failures detected." ` + --resource-group $env:AZURE_RESOURCE_GROUP ` + --scopes $env:LOG_ANALYTICS_WORKSPACE_ID ` + --condition "count 'KubePodInventory | where ContainerStatusReason in (\'ImagePullBackOff\',\'ErrImagePull\')' > 0" ` + --evaluation-frequency PT5M ` + --window-size PT5M ` + --action $env:ACTION_GROUP ` + --location $env:AZURE_LOCATION ` + --severity 2; + +az monitor scheduled-query create ` + --name "OOM Killed Containers" ` + --description "Containers terminated due to OOM." ` + --resource-group $env:AZURE_RESOURCE_GROUP ` + --scopes $env:LOG_ANALYTICS_WORKSPACE_ID ` + --condition "count 'KubePodInventory | where ContainerStatusReason == \'OOMKilled\'' > 0" ` + --evaluation-frequency PT5M ` + --window-size PT5M ` + --action $env:ACTION_GROUP ` + --location $env:AZURE_LOCATION ` + --severity 1; + +az monitor scheduled-query create ` + --name "Pod Restart Rate High" ` + --description "High restart rate in last 5 minutes." ` + --resource-group $env:AZURE_RESOURCE_GROUP ` + --scopes $env:LOG_ANALYTICS_WORKSPACE_ID ` + --condition "count 'KubePodInventory | where TimeGenerated > ago(5m) | where ContainerRestartCount > 3' > 0" ` + --evaluation-frequency PT1M ` + --window-size PT5M ` + --action $env:ACTION_GROUP ` + --location $env:AZURE_LOCATION ` + --severity 2; + +az monitor scheduled-query create ` + --name "Pods Pending" ` + --description "Pods stuck in Pending state." ` + --resource-group $env:AZURE_RESOURCE_GROUP ` + --scopes $env:LOG_ANALYTICS_WORKSPACE_ID ` + --condition "count 'KubePodInventory | where PodStatus == \'Pending\' | summarize count()' > 0" ` + --evaluation-frequency PT1M ` + --window-size PT5M ` + --action $env:ACTION_GROUP ` + --location $env:AZURE_LOCATION ` + --severity 3; + +# Image Cleaner (Optional) +az aks update ` + -g $env:AZURE_RESOURCE_GROUP ` + -n $env:APP_NAME ` + --enable-image-cleaner ` + --image-cleaner-interval-hours 72; + +# Diagnostic Settings +$env:LOG_ANALYTICS_WORKSPACE_ID = az monitor log-analytics workspace list -g $env:AZURE_RESOURCE_GROUP_LOGS --query [0].[id] -o tsv; +$env:KUBERNETES_ID = az aks show -g $env:AZURE_RESOURCE_GROUP -n $env:APP_NAME --query id -o tsv; +$env:DIAGNOSTIC_SETTINGS_NAME = "diagnostics-" + $env:APP_NAME; + +az monitor diagnostic-settings create ` + --name $env:DIAGNOSTIC_SETTINGS_NAME ` + --workspace $env:LOG_ANALYTICS_WORKSPACE_ID ` + --resource $env:KUBERNETES_ID ` + --logs '@.diagnostic-settings/logs.json'; + +$env:LOAD_BALANCER_ID = az network lb show -g $env:AZURE_RESOURCE_GROUP_ASSETS -n kubernetes --query id -o tsv; +$env:DIAGNOSTIC_SETTINGS_LOAD_BALANCER_NAME = $env:DIAGNOSTIC_SETTINGS_NAME + "-load-balancer"; + +az monitor diagnostic-settings create ` + --name $env:DIAGNOSTIC_SETTINGS_LOAD_BALANCER_NAME ` + --workspace $env:LOG_ANALYTICS_WORKSPACE_ID ` + --resource $env:LOAD_BALANCER_ID ` + --logs '@.diagnostic-settings/load-balancer/logs.json' ` + --metrics '@.diagnostic-settings/load-balancer/metrics.json'; + +$env:ALB_LOAD_BALANCER_ID = az network alb list -g $env:AZURE_RESOURCE_GROUP_ASSETS --query [0].id -o tsv; +$env:DIAGNOSTIC_SETTINGS_ALB_LOAD_BALANCER_NAME = $env:DIAGNOSTIC_SETTINGS_NAME + "-alb"; + +az monitor diagnostic-settings create ` + --name $env:DIAGNOSTIC_SETTINGS_ALB_LOAD_BALANCER_NAME ` + --workspace $env:LOG_ANALYTICS_WORKSPACE_ID ` + --resource $env:ALB_LOAD_BALANCER_ID ` + --logs '@.diagnostic-settings/alb/logs.json'; + +# Microsoft Defender +$env:DEFENDER_CONFIG = @{logAnalyticsWorkspaceResourceId = $env:LOG_ANALYTICS_WORKSPACE_ID} | ConvertTo-Json -Compress +$env:DEFENDER_CONFIG_FILE_PATH = Join-Path $env:USERPROFILE "nano.azure.kuberentes.defender-config.json" + +Set-Content -Path $env:DEFENDER_CONFIG_FILE_PATH -Value $env:DEFENDER_CONFIG -Encoding utf8 + +az aks update ` + -g $env:AZURE_RESOURCE_GROUP ` + -n $env:APP_NAME ` + --enable-defender ` + --defender-config=$env:DEFENDER_CONFIG_FILE_PATH; + +# Policy (Optional) +az aks enable-addons ` + -n $env:APP_NAME ` + -g $env:AZURE_RESOURCE_GROUP ` + --addons azure-policy; diff --git a/Nano.Azure.Kubernetes/image-pull-secret.ps1 b/Nano.Azure.Kubernetes/image-pull-secret.ps1 new file mode 100644 index 0000000..075cd29 --- /dev/null +++ b/Nano.Azure.Kubernetes/image-pull-secret.ps1 @@ -0,0 +1,16 @@ +$env:AZURE_RESOURCE_GROUP_KUBERNETES = "Nano-Kubernetes"; +$env:CONTAINER_REGISTRY_HOST = "ghcr.io/{{organization-name}}"; +$env:CONTAINER_REGISTRY_USERNAME = ""; +$env:CONTAINER_REGISTRY_PASSWORD = ""; + +# Image Pull Secret +$env:KUBERNETES_NAME = az aks list -g $env:AZURE_RESOURCE_GROUP_KUBERNETES --query [0].name -o tsv; + +az aks command invoke ` + -g $env:AZURE_RESOURCE_GROUP_KUBERNETES ` + -n $env:KUBERNETES_NAME ` + --file $env:STORAGE_CLASS_PATH ` + -c "kubectl create secret docker-registry ghcr-pull-secret ` + --docker-server=$env:CONTAINER_REGISTRY_HOST ` + --docker-username=$env:CONTAINER_REGISTRY_USERNAME ` + --docker-password=$env:CONTAINER_REGISTRY_PASSWORD"; diff --git a/Nano.Azure.Kubernetes/namespace.ps1 b/Nano.Azure.Kubernetes/namespace.ps1 new file mode 100644 index 0000000..d2f1a8b --- /dev/null +++ b/Nano.Azure.Kubernetes/namespace.ps1 @@ -0,0 +1,10 @@ +$env:AZURE_RESOURCE_GROUP_KUBERNETES = "Nano-Kubernetes"; +$env:KUBERNETES_NAMESPACE = "apps"; + +# Namespace +$env:KUBERNETES_NAME = az aks list -g $env:AZURE_RESOURCE_GROUP_KUBERNETES --query [0].name -o tsv; + +az aks command invoke ` + -g $env:AZURE_RESOURCE_GROUP_KUBERNETES ` + -n $env:KUBERNETES_NAME ` + -c "kubectl create namespace $env:KUBERNETES_NAMESPACE --dry-run=client -o yaml | kubectl apply -f -"; diff --git a/Nano.Azure.Monitoring/README.md b/Nano.Azure.Monitoring/README.md new file mode 100644 index 0000000..35802cc --- /dev/null +++ b/Nano.Azure.Monitoring/README.md @@ -0,0 +1,62 @@ +# Nano.Azure.Monitoring + +> _Centralized monitoring - metrics and logging for Nano infrastructure and applications._ + +*** + +## Table of Contents +* **[Summary](#summary)** +* **[Registration](#registration)** + * **[Log Analytics Workspace](#log-analytics-workspace)** + * **[Monitor Workspace](#monitor-workspace)** + * **[Monitor Action Group](#monitor-action-group)** +* **[Dependencies](#dependencies)** + +## Summary +Monitoring deploys the foundational resources required to enable monitoring across other Azure resources. + +Azure Log Analytics is a service that collects, stores, and analyzes log and performance data from a wide range of sources. A Log Analytics workspace acts as the central +repository for this data, enabling powerful querying using Kusto Query Language (KQL), as well as integration with Azure Monitor and other Azure services. This provides a unified +platform for monitoring, troubleshooting, and gaining operational insights across your environment. + +An Azure Monitor workspace is a dedicated environment for storing and managing data collected by Azure Monitor. Each workspace maintains its own data store, configuration, and +access control, allowing you to isolate monitoring data per application, workload, or tenant while supporting scalable and flexible observability setups. + +Create a shared Log Analytics workspace to centralize logs and diagnostics across all Nano resources, alongside an Azure Monitor workspace aligned with modern monitoring practices +and used for Prometheus-based metrics. An Azure Monitor Action Group is also provisioned to route and dispatch alerts from Azure resources. + +> 📖 Learn more about **[Azure Log Analytics Workspace](https://learn.microsoft.com/en-us/azure/azure-monitor/logs/log-analytics-workspace-overview)**. +> 📖 Learn more about **[Azure Monitor Workspace](https://learn.microsoft.com/en-us/azure/azure-monitor/fundamentals/overview)**. + +## Registration +Start by registering the required Azure providers and creating the resource group, by executing the top part of the `deploy.ps1`. + +> ⚠️ Ensure all required variables are specified in the PowerShell script before execution. + +### Log Analytics Workspace +Execute the next part of the `deploy.ps1` to create the log-analytics workspace. + +### Monitor Workspace +Proceed to create the Azure Monitor workspace, which serves as the modern, recommended workspace for collecting and managing monitoring data. + +This will also create a managed resource group (`MA_nano-monitor-workspace...`) that contains the data collection rules and endpoints associated with the Azure Monitor workspace. +This resource group is system-managed, cannot be modified or moved, and should be left as is. + +### Monitor Action Group +Continue the script to create the Azure Monitor Action Group. + +This configuration ensures that all Nano alerts are routed through the `monitor-action-group`, with email notifications sent to members assigned to the specified roles + +| Role | Type | +| ------------------------ | ------ | +| Monitoring Reader | Email | +| Monitoring Contributor | Email | + +> 💡 This may be updated in the future to support more advanced or flexible configuration and notification scenarios. + +## Dependencies +Monitoring has the following dependencies that must be deployed or otherwise satisfied prior to setup. + +| Dependency | Description | +| ---------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------- | +| **[Nano.Azure.Account](https://github.com/Nano-Core/Nano.Azure/tree/master/Nano.Azure.Account/README.md#nanoazureaccount)** | The is the foundation or prerequites of the Nano Azure infrastructure. | diff --git a/Nano.Azure.Monitoring/deploy.ps1 b/Nano.Azure.Monitoring/deploy.ps1 new file mode 100644 index 0000000..9348d56 --- /dev/null +++ b/Nano.Azure.Monitoring/deploy.ps1 @@ -0,0 +1,49 @@ +$env:ENVIRONMENT = ""; +$env:AZURE_LOCATION = "North Europe"; +$env:AZURE_RESOURCE_GROUP = "Nano-Logs"; +$env:APP_NAME_LOG_ANALYTCS = "nano-log-analytics-workspace-" + $env:ENVIRONMENT.ToLower(); +$env:APP_NAME_MONITOR = "nano-monitor-workspace-" + $env:ENVIRONMENT.ToLower(); +$env:WORKSPACE_SKU = "PerGB2018"; +$env:ACTION_GROUP_NAME = "nano-monitor-action-group"; + +# Register Providers +az provider register --namespace microsoft.insights; +az provider register --namespace Microsoft.OperationalInsights; + +# Resurce Group +az group create ` + -n $env:AZURE_RESOURCE_GROUP ` + -l $env:AZURE_LOCATION; + +# Log Analytics Workspace +az monitor log-analytics workspace create ` + -n $env:APP_NAME_LOG_ANALYTCS ` + -g $env:AZURE_RESOURCE_GROUP ` + -l $env:AZURE_LOCATION ` + --sku $env:WORKSPACE_SKU ` + --identity-type SystemAssigned ` + --retention-time 30; + +# Monitor Workspace +az monitor account create ` + -g $env:AZURE_RESOURCE_GROUP ` + -l $env:AZURE_LOCATION ` + -n $env:APP_NAME_MONITOR; + +# Monitor Action Group +az monitor action-group create ` + -n $env:ACTION_GROUP_NAME ` + -g $env:AZURE_RESOURCE_GROUP ` + --short-name monitor; + +$env:MONITORING_READER_ROLE_ID = az role definition list --name "Monitoring Reader" --query [0].[name] -o tsv; +az monitor action-group update ` + -n $env:ACTION_GROUP_NAME ` + -g $env:AZURE_RESOURCE_GROUP ` + --add-action armrole "Monitoring Reader" $env:MONITORING_READER_ROLE_ID; + +$env:MONITORING_CONTRIBUTOR_ROLE_ID = az role definition list --name "Monitoring Contributor" --query [0].[name] -o tsv; +az monitor action-group update ` + -n $env:ACTION_GROUP_NAME ` + -g $env:AZURE_RESOURCE_GROUP ` + --add-action armrole "Monitoring Contributor" $env:MONITORING_CONTRIBUTOR_ROLE_ID; diff --git a/Nano.Azure.MySql/.diagnostic-settings/logs.json b/Nano.Azure.MySql/.diagnostic-settings/logs.json new file mode 100644 index 0000000..a5817c3 --- /dev/null +++ b/Nano.Azure.MySql/.diagnostic-settings/logs.json @@ -0,0 +1,10 @@ +[ + { + "category": "MySqlSlowLogs", + "enabled": true + }, + { + "category": "MySqlAuditLogs", + "enabled": true + } +] \ No newline at end of file diff --git a/Nano.Azure.MySql/.diagnostic-settings/metrics.json b/Nano.Azure.MySql/.diagnostic-settings/metrics.json new file mode 100644 index 0000000..551d2f1 --- /dev/null +++ b/Nano.Azure.MySql/.diagnostic-settings/metrics.json @@ -0,0 +1,7 @@ +[ + { + "category": "AllMetrics", + "enabled": true, + "timeGrain": "PT1M" + } +] \ No newline at end of file diff --git a/Nano.Azure.MySql/README.md b/Nano.Azure.MySql/README.md new file mode 100644 index 0000000..6f6fc7d --- /dev/null +++ b/Nano.Azure.MySql/README.md @@ -0,0 +1,173 @@ +# Nano.Azure.MySql + +> _Azure flexible MySQL database server for Nano applications._ + +*** + +## Table of Contents +* **[Summary](#summary)** +* **[Registration](#registration)** + * **[MySQL Flexble Server](#mysql-flexble-server)** + * **[Database Backup](#database-backup)** + * **[IOPS Auto Scaling](#iops-auto-scaling)** + * **[High Availability](#high-availability)** + * **[Maintenance](#maintenance)** + * **[Transaction Isolation Level](#transaction-isolation-level)** + * **[Alerts](#alerts)** + * **[Diagnostic Settings](#diagnostic-settings)** + * **[Network Rules](#network-rules)** + * **[Microsoft Defender](#microsoft-defender)** +* **[Dependencies](#dependencies)** + +## Summary +MySQL is an open-source relational database management system (RDBMS) known for its speed, reliability, and ease of use. Developed by Oracle, it is one of the most popular databases +for web applications and is a key component of the LAMP stack (Linux, Apache, MySQL, PHP/Perl/Python). MySQL supports SQL for managing and querying data and offers features like +replication, partitioning, and full-text search. It is highly scalable, making it suitable for both small-scale projects and large, high-traffic websites. MySQL’s strong community +support and wide range of tools make it a preferred choice for developers and businesses around the world. + +Create a MySQL Flexible Server to host databases used by Nano applications. + +> 📖 Learn more about **[Azure MySql](https://learn.microsoft.com/azure/mysql)**. + +## Registration +Start by registering the required Azure providers and creating the resource group, by executing the top part of the `deploy.ps1`. + +> ⚠️ Ensure all required variables are specified in the PowerShell script before execution. + +Add the following GitHub organization variables. + +| Secret | Type | Description | +| -------------------------------- | ------- |------------------------------------------------ | +| `AZURE_RESOURCE_GROUP_DATABASE` | vars | The Azure resource group of the SQL Database. | + +### MySQL Flexble Server +Execute the next part of the `deploy.ps1` to create a managed flexible MySQL database server on Azure. + +The default SKU is set to: `Standard_D2ads_v5`, but other SKUs can also be used depending on workload requirements and the availability in Azure regions. + +Azure virtual machine SKUs follow this general format: `[Family][Size][Features][Generation]`. + +The virtual machine familiy types is listed below. + +| Family | Meaning | Typical use | +| ------ | ------------------ | ------------------------------------------------------------------------------------------------------- | +| B | Burstable | Dev, low-cost, light workloads. ⚠️ Be aware that you cannot later upgrade to a different family type. | +| D | General Purpose | Most apps, balanced CPU/memory | +| E | Memory Optimized | Databases, in-memory workloads | +| F | Compute Optimized | High CPU workloads | + +And the SKU feature letters. + +| Letter | Meaning | +| ------ | -------------------------------------------------- | +| a | AMD CPU | +| i | Intel CPU | +| d | Local temporary SSD | +| s | Premium SSD support | +| e | Extra memory (memory optimized variants) | +| z | High memory / special variants | +| l | Low latency / local disk optimized (older series) | + +> ⚠️ Be aware that the chosen tier must match the chosen SKU. e.g. Family=B, Tier=Burstable. + +To see all available SKUs for a specific region use the command below. + +```powershell +az mysql flexible-server list-skus -l $env:AZURE_LOCATION -o table; +``` + +You can also check out the official list of available SKUs on the **[MySQL Service Tiers](https://learn.microsoft.com/en-us/azure/mysql/flexible-server/concepts-service-tiers-storage)** page. + +> ⚠️ Make sure to store the information returned during creation. The _admin password_ for instance cannot be retrieved later. + +Create the required secrets in GitHub for the MySQL server. They will be used later to securely connect your applications to the database. + +| Secret | Type | Description | +| --------------------------------------- | -------- | ------------------------------------------------------------------------------ | +| `{{environment}}_SQL_ADMIN_PASSWORD ` | Secret | The MySQL admin password, used when applying EF migrations during deployment. | + +The MySQL connection string has this format. + +``` +Server={server};Port={{port}};Database={database};Uid={username};Pwd={password};SslMode=Preferred; +``` + +### Database Backup +Database backups have also been configured to run every 24 hours, with a maximum retention period of 35 days. To enable geo-redundant backups, uncomment the `--geo-redundant-backup` +parameter in the create command. + +> ⚠️ MySQL database backups is not integrated with **[Nano.Azure.Backup](https://github.com/Nano-Core/Nano.Azure/tree/master/Nano.Azure.Backup/README.md#nanoazurebackup)**, +but has its own integrated configurable service. + +### IOPS Auto Scaling +Furthermoe, IOPS is set to auto-grow, allowing flexible a scaled database server. + +### High Availability +Optionally, you can enable high availability for the MySQL server. In Azure Database for MySQL Flexible Server, high availability provides automatic failover between zones to +improve resilience and reduce downtime in case of infrastructure or zone-level failures. To enable it, simply uncomment the `--high-availability`, `--zone`, and `--standby-zone` +parameters in the `deploy.ps1` script’s create command. + +### Maintenance +The Azure maintainance window is set to sunday at 04:00. + +### Transaction Isolation Level +The default transaction isolation level can optionally be changed to `READ-UNCOMMITTED` for improved performance. However, this should be used with caution, as it may lead to dirty +reads and less consistent query results in some applications. + +### Alerts +Last, a couple of default alerts have been configured for the MySQL server and are associated with the action group created in +**[Nano.Azure.Monitoring](https://github.com/Nano-Core/Nano.Azure/tree/master/Nano.Azure.Monitoring/README.md#nanoazuremonitoring)**. These alerts monitor High CPU Usage, +High Memory Usage, High Number Of Connections, High Storage IO, and High Storage Percent. + +### Diagnostics Settings +The diagnostic settings for MySQL includes both `AllMetrics` for metrics, and `MySqlSlowLogs` and `MySqlAuditLogs`, and the time-grain is set tot 1-minute aggregation interval. This +value can be adjusted if needed. You can retrieve the full list of supported metric and log categories for the MySQL resource using the following command. + +```powershell +az monitor diagnostic-settings categories list --resource $env:MYSQL_ID; +``` + +### Network Rules +The MySQL flexible server has no public access by default. + +A Private Endpoint is created for the Kubernetes virtual network, enabling applications running in Kubernetes to securely access the MySQL server over a private connection. + +To get the avaiable `group-id`, run the following command. + +```powershell +az network private-link-resource list --id $env:MYSQL_ID; +``` + +> ⚠️ The private endpoint must be deployed in the same Azure region as the virtual network (VNet) it is associated with. + +Optionally, IP address whitelisting can be configured to allow access to the MySQL server. By default, access is fully restricted, and no external connections are permitted. The +Nano system does not depend on IP whitelisting for connectivity, and using it is generally discouraged as it can negatively impact the overall cloud security score. If IP +whitelisting is required, it can be configured using the following command. + +```powershell +$env:NETWORK_RULE_NAME = ""; +$env:NETWORK_RULE_WHITE_LISTED_IP_ADDRESS_START = "0.0.0.0"; +$env:NETWORK_RULE_WHITE_LISTED_IP_ADDRESS_END = "255.255.255.255"; + +az mysql flexible-server firewall-rule create ` + -n $env:APP_NAME ` + -g $env:AZURE_RESOURCE_GROUP ` + --rule-name $env:NETWORK_RULE_NAME ` + --start-ip-address $env:NETWORK_RULE_WHITE_LISTED_IP_ADDRESS_START ` + --end-ip-address $env:NETWORK_RULE_WHITE_LISTED_IP_ADDRESS_END; +``` + +> ⚠️ Whitelisting IP addresses can reduce the overall security posture and negatively impact the security score. + +### Microsoft Defender +After successful registration, enable Defender directly on the MySQL resource in the Azure Portal. + +> ⚠️ This setting is not currently configurable via the Azure CLI. + +## Dependencies +MySQL has the following dependencies that must be deployed or otherwise satisfied prior to setup. + +| Dependency | Description | +| ------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------- | +| **[Nano.Azure.Monitoring](https://github.com/Nano-Core/Nano.Azure/tree/master/Nano.Azure.Monitoring/README.md#nanoazuremonitoring)** | Components for centralized monitoring and logging | +| **[Nano.Azure.Kubernetes](https://github.com/Nano-Core/Nano.Azure/tree/master/Nano.Azure.Kubernetes/README.md#nanoazurekubernetes)** | The Azure Kubernetes Service (AKS). | diff --git a/Nano.Azure.MySql/deploy.ps1 b/Nano.Azure.MySql/deploy.ps1 new file mode 100644 index 0000000..55f4f8e --- /dev/null +++ b/Nano.Azure.MySql/deploy.ps1 @@ -0,0 +1,166 @@ +$env:ENVIRONMENT = ""; +$env:AZURE_LOCATION = "North Europe"; +$env:AZURE_RESOURCE_GROUP = "Nano-Database"; +$env:AZURE_RESOURCE_GROUP_LOGS = "Nano-Logs"; +$env:AZURE_RESOURCE_GROUP_KUBERNETES_ASSETS = "Nano-Kubernetes-Assets"; +$env:MYSQL_VERSION = "8.4"; +$env:MYSQL_SKU = "Standard_D2ads_v5"; +$env:MYSQL_STORAGE_SIZE = "64"; +$env:MYSQL_TIER = "GeneralPurpose"; +$env:MYSQL_BACKUP_INTERVAL = 24 +$env:MYSQL_BACKUP_RETENTION = 35 +$env:MYSQL_ADMIN_USERNAME = "adminuser"; +$env:MYSQL_ADMIN_PASSWORD = ""; +$env:MYSQL_PORT = 3306; +$env:APP_NAME = "nano-mysql-" + $env:ENVIRONMENT.ToLower(); + +# Register Providers +az provider register -n Microsoft.DBforMySQL; + +# Resource Group +az group create ` + -n $env:AZURE_RESOURCE_GROUP ` + -l $env:AZURE_LOCATION; + +# Create MySql Server +az mysql flexible-server create ` + -n $env:APP_NAME ` + -g $env:AZURE_RESOURCE_GROUP ` + -l $env:AZURE_LOCATION ` + --sku-name $env:MYSQL_SKU ` + --tier $env:MYSQL_TIER ` + --version $env:MYSQL_VERSION ` + --storage-auto-grow Enabled ` + --storage-size $env:MYSQL_STORAGE_SIZE ` + --database-port $env:MYSQL_PORT ` + --auto-scale-iops Enabled ` + --backup-retention $env:MYSQL_BACKUP_RETENTION ` + --backup-interval $env:MYSQL_BACKUP_INTERVAL ` + --admin-user $env:MYSQL_ADMIN_USERNAME ` + --admin-password $env:MYSQL_ADMIN_PASSWORD ` + --public-access Disabled ` + --zone 1 ` + --high-availability ZoneRedundant ` + --standby-zone 2 ` + -y; + +# Maintenance +az mysql flexible-server update ` + -n $env:APP_NAME ` + -g $env:AZURE_RESOURCE_GROUP ` + --maintenance-window Sun:4:00; + +# Parameters (Optional) +az mysql flexible-server parameter set ` + -g $env:AZURE_RESOURCE_GROUP ` + -s $env:APP_NAME ` + -n transaction_isolation ` + -v "READ-UNCOMMITTED"; + +# Diagnostics Settings +$env:DIAGNOSTIC_SETTINGS_NAME = "diagnostics-" + $env:APP_NAME; +$env:WORKSPACE_ID = az monitor log-analytics workspace list -g $env:AZURE_RESOURCE_GROUP_LOGS --query [0].[id] -o tsv; +$env:MYSQL_ID = az mysql flexible-server show -g $env:AZURE_RESOURCE_GROUP -n $env:APP_NAME --query id -o tsv; + +az monitor diagnostic-settings create ` + --name $env:DIAGNOSTIC_SETTINGS_NAME ` + --resource $env:MYSQL_ID ` + --workspace $env:WORKSPACE_ID ` + --logs '@.diagnostic-settings/logs.json' ` + --metrics '@.diagnostic-settings/metrics.json'; + +# Alert Rules +$env:MYSQL_ID = az mysql flexible-server show -g $env:AZURE_RESOURCE_GROUP -n $env:APP_NAME --query id -o tsv; +$env:ACTION_GROUP = az monitor action-group list -g $env:AZURE_RESOURCE_GROUP_LOGS --query [0].[id] -o tsv; + +az monitor metrics alert create ` + --name "High CPU Usage" ` + --resource-group $env:AZURE_RESOURCE_GROUP ` + --scopes $env:MYSQL_ID ` + --condition "avg cpu_percent > 80" ` + --window-size PT5M ` + --evaluation-frequency PT1M ` + --action $env:ACTION_GROUP ` + --severity 2 ` + --description "Alert when CPU usage is above 80% for 5 minutes."; + +az monitor metrics alert create ` + --name "High Memory Usage" ` + --resource-group $env:AZURE_RESOURCE_GROUP ` + --scopes $env:MYSQL_ID ` + --condition "avg memory_percent > 80" ` + --window-size PT5M ` + --evaluation-frequency PT1M ` + --action $env:ACTION_GROUP ` + --severity 2 ` + --description "Alert when Memory usage is above 80% for 5 minutes."; + +az monitor metrics alert create ` + --name "High Number Of Connections" ` + --resource-group $env:AZURE_RESOURCE_GROUP ` + --scopes $env:MYSQL_ID ` + --condition "avg active_connections > 100" ` + --window-size PT5M ` + --evaluation-frequency PT1M ` + --action $env:ACTION_GROUP ` + --severity 2 ` + --description "Alert when the number of active connections exceeds 100 for 5 minutes."; + +az monitor metrics alert create ` + --name "High Storage IO" ` + --resource-group $env:AZURE_RESOURCE_GROUP ` + --scopes $env:MYSQL_ID ` + --condition "avg io_consumption_percent > 80" ` + --window-size PT5M ` + --evaluation-frequency PT1M ` + --action $env:ACTION_GROUP ` + --severity 2 ` + --description "Alert when Storage IO consumption is above 80% for 5 minutes."; + +az monitor metrics alert create ` + --name "High Storage Percent" ` + --resource-group $env:AZURE_RESOURCE_GROUP ` + --scopes $env:MYSQL_ID ` + --condition "avg storage_percent > 80" ` + --window-size PT5M ` + --evaluation-frequency PT1M ` + --action $env:ACTION_GROUP ` + --severity 2 ` + --description "Alert when Storage usage exceeds 80% for 5 minutes."; + +# Network Rules. +$env:PRIVATE_LINK = "privatelink.mysql.database.azure.com"; +$env:PRIVATE_ENDPOINT_NAME = $env:APP_NAME + "-private-endpoint"; +$env:MYSQL_ID = az mysql flexible-server show -g $env:AZURE_RESOURCE_GROUP -n $env:APP_NAME --query id -o tsv; +$env:VNET_ID = az network vnet list -g $env:AZURE_RESOURCE_GROUP_KUBERNETES_ASSETS --query [0].id -o tsv; +$env:VNET_NAME = az network vnet list -g $env:AZURE_RESOURCE_GROUP_KUBERNETES_ASSETS --query [0].name -o tsv; +$env:VNET_LOCATION = az network vnet list -g $env:AZURE_RESOURCE_GROUP_KUBERNETES_ASSETS --query [0].location -o tsv; +$env:SUBNET_ID = az network vnet subnet list -g $env:AZURE_RESOURCE_GROUP_KUBERNETES_ASSETS --vnet-name $env:VNET_NAME --query "[?name =='aks-subnet'].[id]" -o tsv; + +az network private-dns zone create ` + -g $env:AZURE_RESOURCE_GROUP ` + -n $env:PRIVATE_LINK; + +az network private-dns link vnet create ` + -g $env:AZURE_RESOURCE_GROUP ` + -n $env:PRIVATE_ENDPOINT_NAME-dns-link ` + -z $env:PRIVATE_LINK ` + -v $env:VNET_ID ` + -e false; + +az network private-endpoint create ` + -l $env:VNET_LOCATION ` + --name $env:PRIVATE_ENDPOINT_NAME ` + --connection-name $env:PRIVATE_ENDPOINT_NAME-connection ` + --nic-name $env:PRIVATE_ENDPOINT_NAME-nic ` + --resource-group $env:AZURE_RESOURCE_GROUP ` + --group-id mysqlServer ` + --subnet $env:SUBNET_ID ` + --private-connection-resource-id $env:MYSQL_ID; + +az network private-endpoint dns-zone-group create ` + -g $env:AZURE_RESOURCE_GROUP ` + -n $env:PRIVATE_ENDPOINT_NAME-dns-zone-group ` + --endpoint-name $env:PRIVATE_ENDPOINT_NAME ` + --private-dns-zone $env:PRIVATE_LINK ` + --zone-name mysql; diff --git a/Nano.Azure.PostgreSql/README.md b/Nano.Azure.PostgreSql/README.md new file mode 100644 index 0000000..fc014ef --- /dev/null +++ b/Nano.Azure.PostgreSql/README.md @@ -0,0 +1,35 @@ +# Nano.Azure.Postgres + +> _Azure PostgreSQL database server for Nano applications._ + +*** + +## Table of Contents +* **[Summary](#summary)** +* **[Registration](#registration)** +* **[Dependencies](#dependencies)** + +## Summary +PostgreSQL, commonly known as Postgres, is an open-source relational database management system (RDBMS) known for its robustness and advanced features. It supports SQL for querying +and managing data but also extends SQL with powerful features like support for JSON, XML, and custom data types, making it suitable for complex applications. Postgres is highly +extensible, allowing users to add custom functions, data types, and operators. It also offers strong ACID compliance, ensuring reliable transactions, and supports advanced indexing, +full-text search, and replication. Due to its flexibility, scalability, and active community support, PostgreSQL is widely used in various industries, from startups to large +enterprises. + +Create a PostgreSQL Flexible Server to host databases used by Nano applications. + +> 📖 Learn more about **[Azure PostgreSQL](https://learn.microsoft.com/en-us/azure/postgresql/overview)**. + +After successful registration, enable Defender directly on the PostgreSQL resource in the Azure Portal. This setting is not currently configurable via the Azure CLI. + +## Registration +Coming... + +## Dependencies +PostgreSQL has the following dependencies that must be deployed or otherwise satisfied prior to setup. + +| Dependency | Description | +| ------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------- | +| **[Nano.Azure.Monitoring](https://github.com/Nano-Core/Nano.Azure/tree/master/Nano.Azure.Monitoring/README.md#nanoazuremonitoring)** | Components for centralized monitoring and logging | +| **[Nano.Azure.Backup](https://github.com/Nano-Core/Nano.Azure/tree/master/Nano.Azure.Backup/README.md#nanoazurebackup)** | Backup and recovery services. | +| **[Nano.Azure.Kubernetes](https://github.com/Nano-Core/Nano.Azure/tree/master/Nano.Azure.Kubernetes/README.md#nanoazurekubernetes)** | The Azure Kubernetes Service (AKS). | diff --git a/Nano.Azure.SqlServer/README.md b/Nano.Azure.SqlServer/README.md new file mode 100644 index 0000000..052737e --- /dev/null +++ b/Nano.Azure.SqlServer/README.md @@ -0,0 +1,33 @@ +# Nano.Azure.SqlServer + +> _Azure SQL Server database server for Nano applications._ + +*** + +## Table of Contents +* **[Summary](#summary)** +* **[Registration](#registration)** +* **[Dependencies](#dependencies)** + +## Summary +SQL Server is a relational database management system (RDBMS) developed by Microsoft. It is designed to store, retrieve, and manage data for a wide range of applications, from +small-scale projects to large enterprise systems. SQL Server supports Structured Query Language (SQL) for querying and managing data and offers advanced features like data +replication, encryption, and full-text search. It integrates well with other Microsoft products and services, making it a popular choice for businesses using the Microsoft +ecosystem. SQL Server also includes tools for data analysis, reporting, and integration, making it a comprehensive platform for data management. + +Create a SQL Server to host databases used by Nano applications. + +> 📖 Learn more about **[Azure SQL Server](https://learn.microsoft.com/en-us/azure/azure-sql)**. + +After successful registration, enable Defender directly on the SQL Server resource in the Azure Portal. This setting is not currently configurable via the Azure CLI. + +## Registration +Coming... + +## Dependencies +PostgreSQL has the following dependencies that must be deployed or otherwise satisfied prior to setup. + +| Dependency | Description | +| ------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------- | +| **[Nano.Azure.Monitoring](https://github.com/Nano-Core/Nano.Azure/tree/master/Nano.Azure.Monitoring/README.md#nanoazuremonitoring)** | Components for centralized monitoring and logging | +| **[Nano.Azure.Kubernetes](https://github.com/Nano-Core/Nano.Azure/tree/master/Nano.Azure.Kubernetes/README.md#nanoazurekubernetes)** | The Azure Kubernetes Service (AKS). | diff --git a/Nano.Azure.Storage/.backup-policies/storage-backup-policy.json b/Nano.Azure.Storage/.backup-policies/storage-backup-policy.json new file mode 100644 index 0000000..19fed39 --- /dev/null +++ b/Nano.Azure.Storage/.backup-policies/storage-backup-policy.json @@ -0,0 +1,26 @@ +{ + "properties": { + "backupManagementType": "AzureStorage", + "workLoadType": "AzureFileShare", + "timeZone": "UTC", + "schedulePolicy": { + "schedulePolicyType": "SimpleSchedulePolicy", + "scheduleRunFrequency": "Daily", + "scheduleRunTimes": [ + "2024-08-25T04:00:00Z" + ] + }, + "retentionPolicy": { + "retentionPolicyType": "LongTermRetentionPolicy", + "dailySchedule": { + "retentionTimes": [ + "2024-08-25T04:00:00Z" + ], + "retentionDuration": { + "count": 30, + "durationType": "Days" + } + } + } + } +} \ No newline at end of file diff --git a/Nano.Azure.Storage/.diagnostic-settings/file/logs.json b/Nano.Azure.Storage/.diagnostic-settings/file/logs.json new file mode 100644 index 0000000..e593722 --- /dev/null +++ b/Nano.Azure.Storage/.diagnostic-settings/file/logs.json @@ -0,0 +1,14 @@ +[ + { + "category": "StorageRead", + "enabled": true + }, + { + "category": "StorageWrite", + "enabled": true + }, + { + "category": "StorageDelete", + "enabled": true + } +] \ No newline at end of file diff --git a/Nano.Azure.Storage/.diagnostic-settings/file/metrics.json b/Nano.Azure.Storage/.diagnostic-settings/file/metrics.json new file mode 100644 index 0000000..46586bf --- /dev/null +++ b/Nano.Azure.Storage/.diagnostic-settings/file/metrics.json @@ -0,0 +1,12 @@ +[ + { + "category": "Transaction", + "enabled": true, + "timeGrain": "PT1M" + }, + { + "category": "SLI", + "enabled": true, + "timeGrain": "PT1M" + } +] \ No newline at end of file diff --git a/Nano.Azure.Storage/.diagnostic-settings/metrics.json b/Nano.Azure.Storage/.diagnostic-settings/metrics.json new file mode 100644 index 0000000..d2beeaa --- /dev/null +++ b/Nano.Azure.Storage/.diagnostic-settings/metrics.json @@ -0,0 +1,12 @@ +[ + { + "category": "Capacity", + "enabled": true, + "timeGrain": "PT1M" + }, + { + "category": "Transaction", + "enabled": true, + "timeGrain": "PT1M" + } +] \ No newline at end of file diff --git a/Nano.Azure.Storage/.kubernetes/serviceaccount.yaml b/Nano.Azure.Storage/.kubernetes/serviceaccount.yaml new file mode 100644 index 0000000..a597949 --- /dev/null +++ b/Nano.Azure.Storage/.kubernetes/serviceaccount.yaml @@ -0,0 +1,7 @@ +apiVersion: v1 +kind: ServiceAccount +metadata: + name: %APP_NAME%-service-account + namespace: %KUBERNETES_NAMESPACE% + annotations: + azure.workload.identity/client-id: %CLIENT_ID% diff --git a/Nano.Azure.Storage/.kubernetes/storageclass.yaml b/Nano.Azure.Storage/.kubernetes/storageclass.yaml new file mode 100644 index 0000000..aff0a4c --- /dev/null +++ b/Nano.Azure.Storage/.kubernetes/storageclass.yaml @@ -0,0 +1,12 @@ +apiVersion: storage.k8s.io/v1 +kind: StorageClass +metadata: + name: azurefile-static +provisioner: file.csi.azure.com +reclaimPolicy: Retain +volumeBindingMode: Immediate +mountOptions: + - dir_mode=0777 + - file_mode=0777 + - uid=0 + - gid=0 \ No newline at end of file diff --git a/Nano.Azure.Storage/README.md b/Nano.Azure.Storage/README.md new file mode 100644 index 0000000..dc805f6 --- /dev/null +++ b/Nano.Azure.Storage/README.md @@ -0,0 +1,171 @@ +# Nano.Azure.Storage + +> _Azure storage account file shares for Nano applications._ + +*** + +## Table of Contents +* **[Summary](#summary)** +* **[Registration](#registration)** + * **[Storage Account](#storage-account)** + * **[Storage Security](#storage-security)** + * **[Backup Policy](#backup-policy)** + * **[Alerts](#alerts)** + * **[Diagnostic Settings](#diagnostic-settings)** + * **[Network Rules](#network-rules)** + * **[Managed Identity (Kubernetes)](#managed-identity-kubernetes)** + * **[Microsoft Defender](#microsoft-defender)** +* **[Dependencies](#dependencies)** + +## Summary +Azure Storage is a cloud service offering scalable and secure storage solutions for a variety of data types, including files, blobs, queues, and tables. It provides services such +as Azure Blob Storage for unstructured data, Azure File Storage for managed file shares, and Azure Queue Storage for message queuing. With features like high availability, data +redundancy, and advanced security, Azure Storage ensures that data is protected and accessible from anywhere. + +Create a storage account to host and provision Azure File Shares for Nano applications. + +> 📖 Learn more about **[Azure Storage](https://learn.microsoft.com/azure/storage)**. + +## Registration +Start by registering the required Azure providers and creating the resource group, by executing the top part of the `deploy.ps1`. + +> ⚠️ Ensure all required variables are specified in the PowerShell script before execution. + +Add the resource group name as GitHub organization variables. + +| Secret | Type | Description | +| ------------------------------- | ------- |-------------------------------------------------- | +| `AZURE_RESOURCE_GROUP_STORAGE` | vars | The Azure resource group of the storage account. | + +### Storage Account +Execute the next part of the `deploy.ps1` to create the storage account on Azure. + +Other available Storage SKUs. + +| SKU | Description | Data redundancy & availability | Typical use case | +| ---------------- | --------------------------------- |------------------------------------------------------------------- | ------------------------------------------------------ | +| Standard_LRS | Locally Redundant Storage | 3 copies of data within a single datacenter in a region. | Lowest cost, non-critical data, dev/test. | +| Standard_ZRS | Zone Redundant Storage | 3 copies across multiple availability zones in the same region. | Higher availability, protection against zone failure. | +| Standard_GRS | Geo-Redundant Storage | LRS in primary region + async replication to a secondary region. | Disaster recovery, read/write in primary only. | +| Standard_RAGRS | Read-Access Geo-Redundant Storage | GRS + read access to secondary region. | Disaster recovery with read-only secondary access. | + +To retrieve a clean and relevant list of available Storage SKUs for a specific region, use the following command. + +```powershell +$env:AZURE_LOCATION_ID = ($env:AZURE_LOCATION -replace '\s+', '').ToLower(); + +az storage sku list --query "[?kind=='StorageV2' && contains(locations, '$env:AZURE_LOCATION_ID')].[name, tier]" -o table +``` + +> ⚠️ Finding a compatible `--kind` and `--sku` combination for the selected region can sometimes be challenging due to SKU availability and regional support limitations. + +### Storage Security +The storage account is configured with public network access disabled (`--public-network-access Disabled`), ensuring that all access to storage services is restricted to private +network connectivity through approved virtual networks and private endpoints. This prevents exposure of the storage account to the public internet and strengthens the overall network +security posture. + +Shared key access is also disabled (`--allow-shared-key-access false`), preventing authentication through storage account access keys. As a result, access to the storage account must use +Microsoft Entra ID–based authentication and Azure role-based access control (RBAC), reducing the risk associated with long-lived shared secrets and supporting a more secure, identity-based +access model. + +Alternatively, shared key access can be enabled on the storage account and the storage account credentials stored as GitHub secrets. These credentials can later be used to mount Azure File +shares into Kubernetes applications. Run the command below to retrieve the access key for the newly created storage account. + +```powershell +az storage account keys list --account-name $env:APP_NAME --resource-group $env:AZURE_RESOURCE_GROUP --query "[].value" -o tsv; +``` + +| Secret | Type | Description | +| -------------------------------------------- | -------- | ---------------------------------------------------------------- | +| `{{environment}}_STORAGE_CREDENTIALS_ID` | Secrets | The name of the storage account. | +| `{{environment}}_STORAGE_CREDENTIALS_SECRET` | Secrets | The account key used to authenticate with the storage account. | + +> ⚠️ The GitHub secrets are only needed when shared access keys are enabled on the storage account. + +### Backup Policy +Continue the script by registering the backup policy for the storage account. + +The backup policy is scheduled daily at 04:00 UTC and uses **[Nano.Azure.Backup](https://github.com/Nano-Core/Nano.Azure/tree/master/Nano.Azure.Backup/README.md#nanoazurebackup)** to +store file share backups. Daily backups are retained for 30 days. These retention settings can be adjusted to meet specific requirements. The policy can be configured either through the +Azure portal or by using a JSON definition similar to the one provided here. + +The concrete file shares still needs to enable backup when created. Also, Azure File Shares protected by Azure Backup cannot be deleted while backup is enabled or recovery points exist. +When backup is configured, Azure may also apply a management lock (AzureBackupProtectionLock) on the storage account to prevent accidental deletion of protected data. This lock must be +removed before the storage account or file share can be deleted. + +To inspect or remove the lock from the storage account. + +```powershell +az lock list --resource $env:STORAGE_ACCOUNT_ID; + +az lock delete --name AzureBackupProtectionLock --resource $env:STORAGE_ACCOUNT_ID; +``` + +## Alerts +Next, create the default alert rules to ensure baseline monitoring and early detection of common issues across the storage account. + +The alerts have been configured for the storage account and are associated with the action group created in +**[Nano.Azure.Monitoring](https://github.com/Nano-Core/Nano.Azure/tree/master/Nano.Azure.Monitoring/README.md#nanoazuremonitoring)**. These alerts monitor low availability, +high latency, elevated egress traffic, and excessive transaction counts. + +### Diagnostic Settings +Proceed with configuring diagnostic settings for the storage account to enable logging and monitoring of its activity and performance. + +The diagnostic settings for backup metrics are configured to both `Capacity` and `Transaction` with a 1-minute aggregation interval, which can be adjusted if required. This ensures +high-resolution monitoring of backup performance data. Diagnostics settings are also configured for file shares for both metrics and logs. The full list of supported metric and log +categories for the storage account resource can be retrieved using the following command. + +```powershell +az monitor diagnostic-settings categories list --resource $env:STORAGE_ACCOUNT_ID; +``` + +## Network Rules +The storage account has public network access disabled by default. + +A Private Endpoint is created within the Kubernetes virtual network, enabling applications running in the cluster to securely access the Storage Account file shares over a private +connection. Nano defaults to creating a Private Endpoint for the file shares component of the storage account by using `--group-id file`. + +To list the available `--group-id` values for use with the `az network private-endpoint create` command, run the following command. + +```powershell +az network private-link-resource list --id $env:STORAGE_ACCOUNT_ID; +``` + +> ⚠️ The private endpoint must be deployed in the same Azure region as the virtual network (VNet) it is associated with. + +Optionally, IP address whitelisting can be configured to allow access to the storage file shares. By default, access is fully restricted, and no external connections are permitted. +The Nano system does not depend on IP whitelisting for connectivity, and using it is generally discouraged as it can negatively impact the overall cloud security score. If IP +whitelisting is required, it can be configured using the following command. + +```powershell +$env:NETWORK_RULE_WHITE_LISTED_IP_ADDRESS = ""; + +az storage account network-rule add ` + -n $env:APP_NAME ` + --ip-address $env:NETWORK_RULE_WHITE_LISTED_IP_ADDRESS; +``` + +> ⚠️ Whitelisting IP addresses can reduce the overall security posture and negatively impact the security score. + +### Managed Identity (Kubernetes) +When public network access is disabled on the storage account, Kubernetes requires a secure way to authenticate and access the Azure File Shares mounted into the application containers. +This configuration uses a managed identity together with workload identity federation, allowing pods in the cluster to access the storage account without relying on storage account keys +or connection strings. The script creates the managed identity, assigns the required Azure Files permissions, and links the Kubernetes service account to the identity through the +cluster’s OIDC issuer. This ensures secure, keyless access to the file shares while keeping authentication fully managed through Azure AD. + +In addition to the Azure identity setup, the required Kubernetes resources are also created, including a service account, and a storage class for Azure File Shares. These are applied to +the `apps` namespace, where Nano applications are also deployed. + +### Microsoft Defender +After successful registration, enable Microsoft Defender either directly on the storage resource or via the Defender for Cloud overview in the Azure Portal. + +> ⚠️ This setting is not currently configurable via the Azure CLI. + +## Dependencies +Storage has the following dependencies that must be deployed or otherwise satisfied prior to setup. + +| Dependency | Description | +| ------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------- | +| **[Nano.Azure.Monitoring](https://github.com/Nano-Core/Nano.Azure/tree/master/Nano.Azure.Monitoring/README.md#nanoazuremonitoring)** | Components for centralized monitoring and logging. | +| **[Nano.Azure.Backup](https://github.com/Nano-Core/Nano.Azure/tree/master/Nano.Azure.Backup/README.md#nanoazurebackup)** | Backup and recovery services. | +| **[Nano.Azure.Kubernetes](https://github.com/Nano-Core/Nano.Azure/tree/master/Nano.Azure.Kubernetes/README.md#nanoazurekubernetes)** | The Azure Kubernetes Service (AKS). | diff --git a/Nano.Azure.Storage/deploy.ps1 b/Nano.Azure.Storage/deploy.ps1 new file mode 100644 index 0000000..bc2a710 --- /dev/null +++ b/Nano.Azure.Storage/deploy.ps1 @@ -0,0 +1,197 @@ +$env:ENVIRONMENT = ""; +$env:AZURE_LOCATION = "North Europe"; +$env:AZURE_RESOURCE_GROUP = "Nano-Storage"; +$env:AZURE_RESOURCE_GROUP_LOGS = "Nano-Logs"; +$env:AZURE_RESOURCE_GROUP_BACKUP = "Nano-Backup"; +$env:AZURE_RESOURCE_GROUP_KUBERNETES = "Nano-Kubernetes"; +$env:AZURE_RESOURCE_GROUP_KUBERNETES_ASSETS = "Nano-Kubernetes-Assets"; +$env:AZURE_FILE_IDENTITY = "aks-azurefile"; +$env:KUBERNETES_NAMESPACE = "apps"; +$env:ACCESS_TIR = "Hot"; +$env:STORAGE_SKU = "Standard_ZRS"; +$env:APP_NAME = "nanostorage" + $env:ENVIRONMENT.ToLower(); + +# Register Providers +az provider register -n Microsoft.Storage; + +# Resource Group +az group create ` + -n $env:AZURE_RESOURCE_GROUP ` + -l $env:AZURE_LOCATION; + +# Storage Account +az storage account create ` + -n $env:APP_NAME ` + -g $env:AZURE_RESOURCE_GROUP ` + -l $env:AZURE_LOCATION ` + --sku $env:STORAGE_SKU ` + --kind StorageV2 ` + --access-tier $env:ACCESS_TIR ` + --default-action Deny ` + --https-only true ` + --enable-large-file-share ` + --public-network-access Disabled ` + --allow-blob-public-access false ` + --min-tls-version TLS1_2 ` + --require-infrastructure-encryption ` + --identity-type SystemAssigned ` + --allow-shared-key-access false; + +# Backup Policy +$env:STORAGE_ACCOUNT_BACKUP_POLICY_NAME = $env:APP_NAME + "-fileshare-backup-policy"; +$env:BACKUP_VAULT_NAME = az backup vault list -g $env:AZURE_RESOURCE_GROUP_BACKUP --query [0].name -o tsv; + +az backup policy create ` + -n $env:STORAGE_ACCOUNT_BACKUP_POLICY_NAME ` + -g $env:AZURE_RESOURCE_GROUP_BACKUP ` + --vault-name $env:BACKUP_VAULT_NAME ` + --backup-management-type AzureStorage ` + --workload-type AzureFileShare ` + --policy .backup-policies/storage-backup-policy.json; + +# Alerts +$env:STORAGE_ACCOUNT_ID = az storage account show -g $env:AZURE_RESOURCE_GROUP -n $env:APP_NAME --query id -o tsv; +$env:ACTION_GROUP = az monitor action-group list -g $env:AZURE_RESOURCE_GROUP_LOGS --query [0].[id] -o tsv; + +az monitor metrics alert create ` + --name "High Transaction Count" ` + --resource-group $env:AZURE_RESOURCE_GROUP ` + --scopes $env:STORAGE_ACCOUNT_ID ` + --condition "total transactions > 1000" ` + --window-size PT5M ` + --evaluation-frequency PT1M ` + --action $env:ACTION_GROUP ` + --severity 2 ` + --description "Alert when transaction count exceeds 1000 for 5 minutes."; + +az monitor metrics alert create ` + --name "Low Availability" ` + --resource-group $env:AZURE_RESOURCE_GROUP ` + --scopes $env:STORAGE_ACCOUNT_ID ` + --condition "avg availability < 99.9" ` + --window-size PT5M ` + --evaluation-frequency PT1M ` + --action $env:ACTION_GROUP ` + --severity 2 ` + --description "Alert when availability drops below 99.9% for 5 minutes."; + +az monitor metrics alert create ` + --name "High Latency" ` + --resource-group $env:AZURE_RESOURCE_GROUP ` + --scopes $env:STORAGE_ACCOUNT_ID ` + --condition "avg SuccessE2ELatency > 200" ` + --window-size PT5M ` + --evaluation-frequency PT1M ` + --action $env:ACTION_GROUP ` + --severity 2 ` + --description "Alert when End-to-End latency exceeds 200ms for 5 minutes."; + +az monitor metrics alert create ` + --name "High Egress" ` + --resource-group $env:AZURE_RESOURCE_GROUP ` + --scopes $env:STORAGE_ACCOUNT_ID ` + --condition "avg egress > 1000" ` + --window-size PT5M ` + --evaluation-frequency PT1M ` + --action $env:ACTION_GROUP ` + --severity 2 ` + --description "Alert when egress exceeds 1000MB for 5 minutes."; + +# Diagnostic Settings +$env:STORAGE_ACCOUNT_ID = az storage account list --query "[?name =='$env:APP_NAME'].[id]" -o tsv; +$env:DIAGNOSTIC_SETTINGS_NAME = "diagnostics-" + $env:APP_NAME; +$env:WORKSPACE_ID = az monitor log-analytics workspace list -g $env:AZURE_RESOURCE_GROUP_LOGS --query [0].[id] -o tsv; + +az monitor diagnostic-settings create ` + -n $env:DIAGNOSTIC_SETTINGS_NAME ` + --workspace $env:WORKSPACE_ID ` + --resource $env:STORAGE_ACCOUNT_ID ` + --metrics '@.diagnostic-settings/metrics.json'; + +az monitor diagnostic-settings create ` + -n $env:DIAGNOSTIC_SETTINGS_NAME ` + --workspace $env:WORKSPACE_ID ` + --resource $env:STORAGE_ACCOUNT_ID/fileServices/default ` + --logs '@.diagnostic-settings/file/logs.json' ` + --metrics '@.diagnostic-settings/file/metrics.json'; + +# Network Rules. +$env:PRIVATE_LINK = "privatelink.file.core.windows.net"; +$env:PRIVATE_ENDPOINT_NAME = $env:APP_NAME + "-private-endpoint"; +$env:STORAGE_ACCOUNT_ID = az storage account show -g $env:AZURE_RESOURCE_GROUP -n $env:APP_NAME --query id -o tsv; +$env:VNET_ID = az network vnet list -g $env:AZURE_RESOURCE_GROUP_KUBERNETES_ASSETS --query [0].id -o tsv; +$env:VNET_NAME = az network vnet list -g $env:AZURE_RESOURCE_GROUP_KUBERNETES_ASSETS --query [0].name -o tsv; +$env:VNET_LOCATION = az network vnet list -g $env:AZURE_RESOURCE_GROUP_KUBERNETES_ASSETS --query [0].location -o tsv; +$env:SUBNET_ID = az network vnet subnet list -g $env:AZURE_RESOURCE_GROUP_KUBERNETES_ASSETS --vnet-name $env:VNET_NAME --query "[?name =='aks-subnet'].[id]" -o tsv; + +az network private-dns zone create ` + -g $env:AZURE_RESOURCE_GROUP ` + -n $env:PRIVATE_LINK; + +az network private-dns link vnet create ` + -g $env:AZURE_RESOURCE_GROUP ` + -n $env:PRIVATE_ENDPOINT_NAME-dns-link ` + -z $env:PRIVATE_LINK ` + -v $env:VNET_ID ` + -e false; + +az network private-endpoint create ` + -l $env:VNET_LOCATION ` + --name $env:PRIVATE_ENDPOINT_NAME ` + --connection-name $env:PRIVATE_ENDPOINT_NAME-connection ` + --nic-name $env:PRIVATE_ENDPOINT_NAME-nic ` + --resource-group $env:AZURE_RESOURCE_GROUP ` + --group-id file ` + --subnet $env:SUBNET_ID ` + --private-connection-resource-id $env:STORAGE_ACCOUNT_ID; + +az network private-endpoint dns-zone-group create ` + -g $env:AZURE_RESOURCE_GROUP ` + -n $env:PRIVATE_ENDPOINT_NAME-dns-zone-group ` + --endpoint-name $env:PRIVATE_ENDPOINT_NAME ` + --private-dns-zone $env:PRIVATE_LINK ` + --zone-name file; + +# Managed Identity (Kubernetes) +$env:KUBERNETES_NAME = az aks list -g $env:AZURE_RESOURCE_GROUP_KUBERNETES --query [0].name -o tsv; +$env:KUBERNETES_ISSUER_URL = az aks list -g $env:AZURE_RESOURCE_GROUP_KUBERNETES --query [0].['oidcIssuerProfile.issuerUrl'] -o tsv; +$env:STORAGE_ACCOUNT_ID = az storage account show -g $env:AZURE_RESOURCE_GROUP -n $env:APP_NAME --query id -o tsv; + +az identity create ` + -g $env:AZURE_RESOURCE_GROUP ` + -n $env:AZURE_FILE_IDENTITY; + +$env:PRINCIPAL_ID = az identity show -g $env:AZURE_RESOURCE_GROUP -n $env:AZURE_FILE_IDENTITY --query principalId -o tsv; + +az role assignment create ` + --assignee-object-id $env:PRINCIPAL_ID ` + --assignee-principal-type ServicePrincipal ` + --role "Storage File Data SMB Share Contributor" ` + --scope $env:STORAGE_ACCOUNT_ID; + +$env:CLIENT_ID = az identity show -g $env:AZURE_RESOURCE_GROUP -n $env:AZURE_FILE_IDENTITY --query clientId -o tsv; +$env:SERVICE_ACCOUNT_PATH = Join-Path $env:USERPROFILE serviceaccount.yaml; +Get-Content .kubernetes/serviceaccount.yaml | foreach { [Environment]::ExpandEnvironmentVariables($_) } | Set-Content $env:SERVICE_ACCOUNT_PATH; + +az aks command invoke ` + -g $env:AZURE_RESOURCE_GROUP_KUBERNETES ` + -n $env:KUBERNETES_NAME ` + --file $env:SERVICE_ACCOUNT_PATH ` + -c "kubectl apply -f serviceaccount.yaml"; + +az identity federated-credential create ` + --name azurefile-federated ` + --resource-group $env:AZURE_RESOURCE_GROUP ` + --identity-name $env:AZURE_FILE_IDENTITY ` + --issuer $env:KUBERNETES_ISSUER_URL ` + --subject system:serviceaccount:$env:KUBERNETES_NAMESPACE:$env:APP_NAME-service-account ` + --audience api://AzureADTokenExchange; + +$env:STORAGE_CLASS_PATH = Join-Path $env:USERPROFILE storageclass.yaml; +Get-Content .kubernetes/storageclass.yaml | foreach { [Environment]::ExpandEnvironmentVariables($_) } | Set-Content $env:STORAGE_CLASS_PATH; + +az aks command invoke ` + -g $env:AZURE_RESOURCE_GROUP_KUBERNETES ` + -n $env:KUBERNETES_NAME ` + --file $env:STORAGE_CLASS_PATH ` + -c "kubectl apply -f storageclass.yaml"; diff --git a/Nano.Azure.sln.cmd b/Nano.Azure.sln.cmd new file mode 100644 index 0000000..0c8adc5 --- /dev/null +++ b/Nano.Azure.sln.cmd @@ -0,0 +1,2 @@ +@echo off +start "" devenv.exe "%~dp0" \ No newline at end of file diff --git a/README.md b/README.md index eba5718..667705e 100644 --- a/README.md +++ b/README.md @@ -1,2 +1,57 @@ # Nano.Azure -Azure services, infrastructure components, and deployment scripts. + +> _Cloud-native Azure services, infrastructure components, and deployment assets for Nano applications._ + +*** + +## Table of Contents +    📌 **[Summary](#-summary)** +    ⚙️ **[Required Tools](#-required-tools)** +    ⚖️ **[Licenses](#-licenses)** + +### Documentaion +    🔹 **[Nano.Azure](https://github.com/Nano-Core/Nano.Azure/tree/master/Nano.Azure.Account/README.md#nanoazureaccount)** +    🔹 **[Nano.Azure.Monitoring](https://github.com/Nano-Core/Nano.Azure/tree/master/Nano.Azure.Monitoring/README.md#nanoazuremonitoring)** +    🔹 **[Nano.Azure.Delivery](https://github.com/Nano-Core/Nano.Azure/tree/master/Nano.Azure.Delivery/README.md#nanoazuredelivery)** +    🔹 **[Nano.Azure.Dns](https://github.com/Nano-Core/Nano.Azure/tree/master/Nano.Azure.Dns/README.md#nanoazuredns)** +    🔹 **[Nano.Azure.Backup](https://github.com/Nano-Core/Nano.Azure/tree/master/Nano.Azure.Backup/README.md#nanoazurebackup)** +    🔹 **[Nano.Azure.Kubernetes](https://github.com/Nano-Core/Nano.Azure/tree/master/Nano.Azure.Kubernetes/README.md#nanoazurekubernetes)** +    🔹 **[Nano.Azure.Storage](https://github.com/Nano-Core/Nano.Azure/tree/master/Nano.Azure.Storage/README.md#nanoazurestorage)** +    🔹 **[Nano.Azure.MySql](https://github.com/Nano-Core/Nano.Azure/tree/master/Nano.Azure.MySql/README.md#nanoazuremysql)** +    🔹 **[Nano.Azure.PostgreSql](https://github.com/Nano-Core/Nano.Azure/tree/master/Nano.Azure.PostgreSql/README.md#nanoazurepostgresql)** +    🔹 **[Nano.Azure.SqlServer](https://github.com/Nano-Core/Nano.Azure/tree/master/Nano.Azure.SqlServer/README.md#nanoazuresqlserver)** + +## 📌 Summary +Nano.Azure provides a curated set of Azure services, infrastructure components, and deployment scripts designed to support +**[Nano Applications](https://github.com/Nano-Core/Nano.Library/blob/master/README.md#nanolibrary)**. The goal is to standardize how Nano applications are hosted and operated +in Azure by providing: + +- Pre-configured infrastructure patterns +- Managed service first architecture +- Reusable deployment components +- Opinionated but flexible cloud setup + +This repository focuses on leveraging _Azure managed services_ to reduce operational overhead, improve scalability, and simplify maintenance across environments. The key principles +in the Nano infrastructure are. + +- Managed-first architecture – prefer Azure-managed services over self-hosted infrastructure +- Consistency over configuration – standardized setups across all Nano applications +- Infrastructure as code – everything is reproducible and versioned +- Minimal operational overhead – reduce maintenance burden in production systems +- Composable architecture – services can be combined or used independently + +## ⚙️ Required Tools +Before continuing, make sure you have the following tools installed and configured. + +| Tool | Description | +| ------------------------------------------------------------------------- | ---------------------------------------------------------------- | +| **[Azure CLI](https://docs.microsoft.com/cli/azure/install-azure-cli)** | Azure CLI is a command-line tool for managing Azure resources. | +| **[Azure VPN Client](https://apps.microsoft.com/detail/9np355qt2sqb)** | Azure VPN client used to connect to private Azure resources. | + +> ⚠️ If Azure CLI is already installed, make sure it is updated to the latest version: `az upgrade`. + +## ⚖️ Licenses +Nano.Azure is free to use and released under the MIT License. + +The scripts in this repository use Azure CLI and interact with Microsoft Azure services. The Azure CLI and Azure services are governed by their respective Microsoft licensing terms and +service agreements.