Skip to content

README.md

Latest commit

 

History

History
230 lines (157 loc) · 8.75 KB

README.md

File metadata and controls

230 lines (157 loc) · 8.75 KB

Openshift on prem deployment playbooks!

Below is the procedure for running a disconnected install.

Sample inventory file is under inventories/inventory_os

Prerequisites:

  1. Collect the images in the vars/image_list.yml and import to local docker registry
  2. Setup local yum repo with the following repos (under "repos" directory):

rhel-7-server-rpms

rhel-7-server-extras-rpms

rhel-7-server-ansible-2.6-rpms

rhel-7-server-ose-3.11-rpms 3. Deploy/Setup the RHEL 7.6 Machines that you will be using for your cluster and set IP address

Procedure:

  1. Login to the deployer machine where you will run the playbooks from

  2. Copy the playbooks/templates/setup_internal_repos.yml file to /etc/yum.repos.d/ocp.repo and update the {{ yumrepo_url}} with the IP of the internal repo

  3. Install ansible yum install ansible -y

  4. Insure ssh connection to all instances without login by sharing ssh keys (ssh-copy-id)

  5. Update inventory file with desired parameters

  6. Run pre-install.yml

    cd ocpprepost/playbooks

    ansible-playbook -i inventory pre-install.yml

  7. Install openshfit-ansible on deployer:

    sudo yum install openshift-ansible -y

  8. Run the installation playbooks:

    cd /usr/share/ansible/openshift-ansible

    ansible-playbook -i inventory playbooks/prerequisites.yml

    ansible-playbook -i inventory playbooks/deploy_cluster.yml

  9. Run post-jobs.yml

    cd ocpprepost/playbooks

    ansible-playbook -i inventory post-jobs.yml

############################################################

Playbooks Details:

pre-install.yml:

  • Sets up yum repo configuration to point to internal repo
  • Sets ntp
  • Sets dns
  • Updates hosts file
  • Installs docker (including storage on separate disk)
  • Sets internal docker registry as insecure in daemon.json
  • Sets up etcd storage on separate disk (optional)
  • Install support packages
  • Sets hostname
  • Sets keepalived VIP in front of LBs (optional)

post-jobs.yml:

  • Adds cluster data store to external grafana server

Additional parameters in inventory file for this playbook are as follows:

Under all:vars

  • dns = uncomment and add a list of dns servers (optional)
  • searchdomain = default search domain to add to resolv.conf
  • ntp = uncomment and add a list of ntp servers (optional)
  • openshift_docker_insecure_registries = set docker registry IP
  • yumrepo_url = set the IP and path if needed for the internal yum repo (ie. 10.0.0.1 or 10.0.0.1/repos)
  • routervialb = configures haproxy on lb node(s) to handle traffic to router as well on port 80 and 443. If using this parameter, master API port MUST be 8443. (Optional)

############################ (Optional) In the case of using 2 internal load balancers, you can use the following parameters to deploy keepalived VIP in front of them. If not then comment them out

  • keepalived_vip = virtual floating ip to be used
  • keepalived_interface = interface on the loadblancers to be used (ie. eth0)
  • keepalived_vrrpid = random unique integer ############################

Following are for adding the cluster to external Grafana Server as part of the post-jobs.yml. (Optional)

  • grafanaURL= full grafana URL (ie. http://10.142.15.244:3000)
  • grafanaPass = grafana admin password
  • grafanaClusterName = desired data source name as it will appear in grafana
  • grafmas = FQDN of one of the master nodes
  • prometheusURL = URL of prometheus route as given in oc get routes command under openshift-monitoring project. Must be routable to an infra node running the router.

Under OSEv3

  • oreg_url: set the IP for the internal docker registry

Under Each host entry:

  • hostname = sets the hostname

Under node:vars

  • pv_device = Sets the extra disk to be used for docker storage. (ie. sdb)

Under etcd:vars

  • pv_etcd = value under etcd:vars. Sets the extra disk to be used for etcd storage (ie. sdc) (optional)

###########################################

Online Option

In order to use this deployment option with the RHEL online registries do the following:

  1. Comment out the yumrepo_url param
  2. Add the following parameters to the all section of the inventory file and populate the values
  • rheluser
  • rhelpass
  • rhelpool
  1. Add the following with in the OSEv3 section
  • oreg_auth_password
  • oreg_auth_user
  1. Run procedure as above

##############################################################################################

Project ScaleUp via Ansible Tower Workflow

In order to scaleup the cluster for dedicated project nodes, the following procedure should be followed:

Note: In this section, all references to Virtualization Automation is Based on the following project:

https://github.com/rockocoop/vmware-ansible.git

Prerequisite Configurations (should be done when setting up the environment):

  1. Add group "new_nodes" into your ansible tower inventory file as an associated group of OSEv3
  2. Create a project in git and configure in ansible tower for source inventory files

Scaling Up

  1. Create an inventory.ini file in the source inventory git with a unique name for the cluster
  • This file will be used for all subsequent Scale-Ups
  • Use the Source from Project option, select the project you created earlier and specify the file to use
  • Insure that the following settings ONLY are set when creating the source:
    Overwrite
    Overwrite Variables
    Update On Launch
  1. Populate as follows for first scale up

  • all:vars section

    [all:vars]
    projectName=flintstones ##name of project to be deployed. This will be the node label

  • new_nodes (example)

    [new_nodes]
    ocpnode7.ocp1.test.com ansible_ssh_host=10.35.76.240 netmask=255.255.255.128 gateway=10.35.76.254 hostname=ocpnode7.ocp1.test.com vlan="VM Network" disks=[30] openshift_node_group_name='node-config-compute' openshift_node_problem_detector_install=true
    ocpnode8.ocp1.test.com ansible_ssh_host=10.35.76.241 netmask=255.255.255.128 gateway=10.35.76.254 hostname=ocpnode8.ocp1.test.com vlan="VM Network" disks=[30] openshift_node_group_name='node-config-compute' openshift_node_problem_detector_install=true

  • new_nodes:vars

    [new_nodes:vars]
    vmCPUs=4
    vmMemory=16384
    vmDisk=40
    pv_device=sdb

  1. Run the OCP New Project Deploy WorfkFlow with your Inventory File

    NOTE:
    I have noticed cases where I needed to SYNCH the Project Containing the Source File Before Running

  • Deploy Virtual Machines: Same playbook as for deploying all hosts in virtual envrionment, but utilizes LIMIT option for 'new_nodes'
  • Distribute SSH Keys: Distributes SSH Keys to ALL hosts (same as in full new cluster deploy)
  • OCP Pre-Install: Runs Pre-install.yml on whole environment (same as in full new cluster deploy)
  • OCP Scale-Up: Runs openshift-ansible playbook for scale up playbooks/openshift-node/scaleup.yml
  • Post Scale-Up: Runs the scaleuppost.yml in this project for adding node labels for the specifid projectName
  • Delete SSH Keys: Removes the shared ssh key from the cluster
  1. Once the Deployment is complete, for the next scale up make the following updates to your inventory.ini file:
  • Create a nodes section [nodes] if it doesnt exist and move the created nodes to that section
  • Copy the [new_nodes:vars] section and rename [nodes:vars] if the section doesnt exist yet
  • Create the new nodes in the new_nodes section
  • Update the projectName parameter
  • Run the Workflow

Below an example:

[all:vars]
projectName=jetsons

[new_nodes]
ocpnode7.ocp1.test.com ansible_ssh_host=10.35.76.240 netmask=255.255.255.128 gateway=10.35.76.254 hostname=ocpnode7.ocp1.test.com vlan="VM Network" disks=[30] openshift_node_group_name='node-config-compute' openshift_node_problem_detector_install=true
ocpnode8.ocp1.test.com ansible_ssh_host=10.35.76.241 netmask=255.255.255.128 gateway=10.35.76.254 hostname=ocpnode8.ocp1.test.com vlan="VM Network" disks=[30] openshift_node_group_name='node-config-compute' openshift_node_problem_detector_install=true

[new_nodes:vars]
vmCPUs=4
vmMemory=16384
vmDisk=40
pv_device=sdb
[nodes]
ocpnode5.ocp1.test.com ansible_ssh_host=10.35.76.238 netmask=255.255.255.128 gateway=10.35.76.254 hostname=ocpnode5.ocp1.test.com vlan="VM Network" disks=[30] openshift_node_group_name='node-config-compute' openshift_node_problem_detector_install=true
ocpnode6.ocp1.test.com ansible_ssh_host=10.35.76.239 netmask=255.255.255.128 gateway=10.35.76.254 hostname=ocpnode6.ocp1.test.com vlan="VM Network" disks=[30] openshift_node_group_name='node-config-compute' openshift_node_problem_detector_install=true

[nodes:vars]
vmCPUs=4
vmMemory=16384
vmDisk=40
pv_device=sdb

#################################################################

mmpost.yaml

Applies templates to catalog, as well as blocks users from creating projects