-
Notifications
You must be signed in to change notification settings - Fork 1.9k
[OSDOCS-19523]: Setting up a management cluster for HCP on Azure #112346
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,201 @@ | ||
| //Module included in the following assemblies: | ||
| // hosted_control_planes/hcp-deploy/hcp-deploy-azure.adoc | ||
|
|
||
| :_mod-docs-content-type: PROCEDURE | ||
| [id="hcp-azure-mgmt-cluster_{context}"] | ||
| = Configuring an {azure-short} management cluster for {hcp} | ||
|
|
||
| [role="_abstract"] | ||
| To configure the management cluster for {hcp} on {azure-short}, you need to ensure that external DNS and the HyperShift Operator are set up on the cluster. | ||
|
|
||
| The configuration steps include configuring the DNS zone, delegating the DNS records for your cluster, and creating a dedicated service principal for external DNS. | ||
|
|
||
| .Prerequisites | ||
|
|
||
| * You installed the {mce-short} 2.5 or later on an {product-title} cluster. You can install {mce-short} as an Operator from the {product-title} software catalog. Or, if you already use {rh-rhacm-title}, the Operator is automatically installed. The HyperShift Operator is enabled by default in the operator package for {mce-short}. | ||
|
|
||
| * The {azure-short} command-line interface (CLI) is installed and configured. | ||
|
|
||
| * The {oc-first} is installed. | ||
|
|
||
| * You have an {product-title} management cluster on {azure-short}. | ||
|
|
||
| * If you are using external DNS, the `jq` command-line JSON processor is installed. | ||
|
|
||
| .Procedure | ||
|
|
||
| . Set the DNS configuration variables as shown in the following example: | ||
| + | ||
| [source,bash] | ||
| ---- | ||
| PARENT_DNS_RG="<my_parent_dns_resource_group>" | ||
| PARENT_DNS_ZONE="<my_parent.dns.zone.com>" | ||
| DNS_RECORD_NAME="<my_subdomain>" | ||
| RESOURCE_GROUP_NAME="<my_resource_group>" | ||
| DNS_ZONE_NAME="<my_subdomain.my_parent.dns.zone.com>" | ||
| LOCATION="<my_region>" | ||
| ---- | ||
|
|
||
| . Create the {azure-short} group by entering the following command: | ||
| + | ||
| [source,terminal] | ||
| ---- | ||
| $ az group create \ | ||
| --name $RESOURCE_GROUP_NAME \ | ||
| --location $LOCATION | ||
| ---- | ||
|
lahinson marked this conversation as resolved.
|
||
|
|
||
| . Create the {azure-short} DNS zone by entering the following command: | ||
| + | ||
| [source,terminal] | ||
| ---- | ||
| $ az network dns zone create \ | ||
| --resource-group $RESOURCE_GROUP_NAME \ | ||
| --name $DNS_ZONE_NAME | ||
| ---- | ||
|
|
||
| . If an existing name server record exists, delete it by entering the following command: | ||
| + | ||
| [source,terminal] | ||
| ---- | ||
| $ az network dns record-set ns delete \ | ||
| --resource-group $PARENT_DNS_RG \ | ||
| --zone-name $PARENT_DNS_ZONE \ | ||
| --name $DNS_RECORD_NAME -y | ||
| ---- | ||
|
|
||
| . Get the name servers from your DNS zone by entering the following command: | ||
| + | ||
| [source,bash] | ||
| ---- | ||
| name_servers=$(az network dns zone show \ | ||
| --resource-group $RESOURCE_GROUP_NAME \ | ||
| --name $DNS_ZONE_NAME \ | ||
| --query nameServers \ | ||
| --output tsv) | ||
| ---- | ||
|
|
||
| . Create an array of name servers as shown in the following example: | ||
| + | ||
| [source,bash] | ||
| ---- | ||
|
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @bryan-cox @Nirshal I thought the code on lines 76 - 79 looked a little unusual, so I searched the There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I don't know... I was following the upstream doc on that, it seemed a reasonable approach at the time. I think we need to wait for Bryan on that. |
||
| ns_array=() | ||
| while IFS= read -r ns; do | ||
| ns_array+=("$ns") | ||
| done <<< "$name_servers" | ||
| ---- | ||
|
|
||
| . Add name server records to the parent zone as shown in the following example: | ||
| + | ||
| [source,bash] | ||
| ---- | ||
|
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Similar to my earlier comment, the code on lines 86 - 92 looks unconventional for the official docs. Suggestions for alternatives are appreciated. |
||
| for ns in "${ns_array[@]}"; do | ||
| az network dns record-set ns add-record \ | ||
| --resource-group $PARENT_DNS_RG \ | ||
| --zone-name $PARENT_DNS_ZONE \ | ||
| --record-set-name $DNS_RECORD_NAME \ | ||
| --nsdname "$ns" | ||
| done | ||
| ---- | ||
|
|
||
| . Set the external DNS configuration variables as shown in the following example: | ||
| + | ||
| [source,bash] | ||
| ---- | ||
| EXTERNAL_DNS_NEW_SP_NAME="<external_dns_service_principal>" | ||
| SERVICE_PRINCIPAL_FILEPATH="<path_to_azure_mgmt_json_file>" | ||
| RESOURCE_GROUP_NAME="<my_resource_group>" | ||
| ---- | ||
|
lahinson marked this conversation as resolved.
|
||
|
|
||
| . Create the service principal for external DNS by entering the following command: | ||
| + | ||
| [source,bash] | ||
| ---- | ||
| DNS_SP=$(az ad sp create-for-rbac --name ${EXTERNAL_DNS_NEW_SP_NAME}) | ||
| EXTERNAL_DNS_SP_APP_ID=$(echo "$DNS_SP" | jq -r '.appId') | ||
| EXTERNAL_DNS_SP_PASSWORD=$(echo "$DNS_SP" | jq -r '.password') | ||
| ---- | ||
|
|
||
| . Get the DNS zone ID by entering the following command: | ||
| + | ||
| [source,bash] | ||
| ---- | ||
|
lahinson marked this conversation as resolved.
|
||
| DNS_ID=$(az network dns zone show \ | ||
| --name ${DNS_ZONE_NAME} \ | ||
| --resource-group ${RESOURCE_GROUP_NAME} \ | ||
| --query "id" \ | ||
| --output tsv) | ||
| ---- | ||
|
|
||
| . Assign the `Reader` role to the service principal by entering the following command: | ||
| + | ||
| [source,terminal] | ||
| ---- | ||
| $ az role assignment create \ | ||
| --role "Reader" \ | ||
| --assignee "${EXTERNAL_DNS_SP_APP_ID}" \ | ||
| --scope "${DNS_ID}" | ||
| ---- | ||
|
|
||
| . Assign the `Contributor` role to the service principal by entering the following command: | ||
| + | ||
| [source,terminal] | ||
| ---- | ||
| $ az role assignment create \ | ||
| --role "Contributor" \ | ||
| --assignee "${EXTERNAL_DNS_SP_APP_ID}" \ | ||
| --scope "${DNS_ID}" | ||
| ---- | ||
|
|
||
| . Create the content for the {azure-short} credentials file as shown in the following example: | ||
| + | ||
| [source,json] | ||
| ---- | ||
|
lahinson marked this conversation as resolved.
|
||
| { | ||
|
lahinson marked this conversation as resolved.
|
||
| "tenantId": "$(az account show --query tenantId -o tsv)", | ||
| "subscriptionId": "$(az account show --query id -o tsv)", | ||
| "resourceGroup": "$RESOURCE_GROUP_NAME", | ||
| "aadClientId": "$EXTERNAL_DNS_SP_APP_ID", | ||
| "aadClientSecret": "$EXTERNAL_DNS_SP_PASSWORD" | ||
| } | ||
| ---- | ||
|
|
||
| . Create the credentials file by entering the following command: | ||
| + | ||
| [source,terminal] | ||
| ---- | ||
| $ oc apply -f <service_principal_filepath>.json | ||
| ---- | ||
|
lahinson marked this conversation as resolved.
|
||
|
|
||
| . If an existing Kubernetes secret for the {azure-short} credentials exists, delete it by entering the following command: | ||
| + | ||
| [source,terminal] | ||
| ---- | ||
| $ oc delete secret/azure-config-file --namespace "default" || true | ||
| ---- | ||
|
|
||
| . Create the Kubernetes secret for the {azure-short} credentials by entering the following command: | ||
| + | ||
| [source,terminal] | ||
| ---- | ||
| $ oc create secret generic azure-config-file \ | ||
| --namespace "default" \ | ||
| --from-file ${SERVICE_PRINCIPAL_FILEPATH} | ||
| ---- | ||
|
|
||
| .Verification | ||
|
|
||
| * Verify that both the HyperShift Operator and the external DNS are running by entering the following command: | ||
| + | ||
| [source,terminal] | ||
| ---- | ||
| $ oc get pods -n hypershift | ||
| ---- | ||
| + | ||
| .Example output | ||
|
lahinson marked this conversation as resolved.
|
||
| [source,terminal] | ||
| ---- | ||
| NAME READY STATUS RESTARTS AGE | ||
| external-dns-xxxxx-xxxxx 1/1 Running 0 1m | ||
| operator-xxxxx-xxxxx 1/1 Running 0 1m | ||
| ---- | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The procedure creates the K8s secret but there doesn't seem to be a step that configures the HyperShift Operator to actually use external DNS. When I set up external DNS via MCE for ACM-30209 testing, I had to apply a ConfigMap to pass the install flags to the operator: apiVersion: v1
kind: ConfigMap
metadata:
name: hypershift-operator-install-flags
namespace: local-cluster
data:
installFlagsToAdd: "--external-dns-provider=azure --external-dns-credentials <secret> --external-dns-domain-filter <dns-zone>"
installFlagsToRemove: ""Without this, I suspect the HO wouldn't pick up the external DNS configuration and the
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I'll wait for Bryan's confirmation, but I have no problem adding those details if they are needed. |
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The user should be made aware that if
--external-dns-domainmatches{cluster-name}.{base-domain}, the CPO creates a private DNS zone that shadows the cluster domain, causing*.appsDNS resolution to fail (console and ingress become unavailable). See hypershift#8480 for details.@bryan-cox do you think it's worth adding a caveat here about DNS zone naming to help users avoid this condition? Or is this better covered in the private clusters doc?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I plan to add the private clusters procedures in a separate PR, so that caveat will be covered there. If you'd like me to to mention it here, too, let me know.