Skip to content

[WIP] JS2 support #56

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
almahmoud wants to merge 14 commits into from
Closed

[WIP] JS2 support #56

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
34337ec
Longer wait on rke
almahmoud Apr 22, 2022
310abd0
Try latest k8s
almahmoud Jun 21, 2022
7ab3579
false intree provider
almahmoud Jun 21, 2022
a4a83d7
hostname for js2
almahmoud Jun 21, 2022
59de0f4
Update links for openstack manifests
almahmoud Jun 21, 2022
c36ccb1
Update daemon patch
almahmoud Jun 21, 2022
37c68ae
Revert "Try latest k8s"
almahmoud Jun 21, 2022
a3e6f4e
Revert "Revert "Try latest k8s""
almahmoud Jun 21, 2022
60233b8
Bump nginx
almahmoud Jun 21, 2022
392cf8c
Remove cvmfs
almahmoud Jun 21, 2022
dbc947a
Revert "Remove cvmfs"
almahmoud Jun 21, 2022
7d4c1d8
Modified gist for opentack-manager daemonset
almahmoud Jun 21, 2022
67e6e1e
Revert "Revert "Revert "Try latest k8s"""
almahmoud Jun 21, 2022
d3e67cb
change volume az
almahmoud Jun 21, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion roles/rke/defaults/main.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
kube_cloud_provider: "{{ lookup('env', 'KUBE_CLOUD_PROVIDER') }}"
kube_cloud_conf: "{{ lookup('env', 'KUBE_CLOUD_CONF') | b64decode }}"
# Whether to use the in-tree cloud provider or external cloud-provider
kube_in_tree_provider: true
kube_in_tree_provider: false
Copy link
Copy Markdown
Contributor

@nuwang nuwang Jun 22, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Won't this break the other clouds? Did you check the csi drivers as well?

Copy link
Copy Markdown
Collaborator Author

@almahmoud almahmoud Jun 22, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CSI drivers work for openstack JS2. Haven't looked at other clouds, but I also imagine they're a lot less needed given they have k8s services, so at least personally I think I'm only using this stack for JS


# cluster settings
cluster_hostname: "{{ lookup('env', 'CLUSTER_HOSTNAME') | default(inventory_hostname, true) }}"
Expand Down
2 changes: 1 addition & 1 deletion roles/rke/tasks/common.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -101,5 +101,5 @@
wait_for:
path: /etc/rancher/node/password
state: present
timeout: 180
timeout: 1800
Copy link
Copy Markdown
Contributor

@nuwang nuwang Jun 22, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Half an hour for the password seems a fairly long time. Does it take that long?

Copy link
Copy Markdown
Collaborator Author

@almahmoud almahmoud Jun 22, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It did on JS1 on TACC, this is an old change


6 changes: 3 additions & 3 deletions roles/rke/tasks/registration.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,9 +26,9 @@
- name: Install OpenStack Cloud Controller Manager
shell: kubectl apply -f {{ item }}
with_items:
- https://raw.githubusercontent.com/kubernetes/cloud-provider-openstack/master/cluster/addons/rbac/cloud-controller-manager-roles.yaml
- https://raw.githubusercontent.com/kubernetes/cloud-provider-openstack/master/cluster/addons/rbac/cloud-controller-manager-role-bindings.yaml
- https://raw.githubusercontent.com/kubernetes/cloud-provider-openstack/master/manifests/controller-manager/openstack-cloud-controller-manager-ds.yaml
- https://raw.githubusercontent.com/kubernetes/cloud-provider-openstack/master/manifests/controller-manager/cloud-controller-manager-roles.yaml
- https://raw.githubusercontent.com/kubernetes/cloud-provider-openstack/master/manifests/controller-manager/cloud-controller-manager-role-bindings.yaml
- https://gist.githubusercontent.com/almahmoud/b02942eb2b18ea14123e163e069a79e2/raw/b59139b17d697c9075ce3870ad7e88c567851d8a/openstack-ds.yaml
Copy link
Copy Markdown
Contributor

@nuwang nuwang Jun 22, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is the custom gist needed? Can we check this in somewhere, maybe into the cloudman-boot repo itself?

Copy link
Copy Markdown
Collaborator Author

@almahmoud almahmoud Jun 22, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The control-plane vs master change for the daemonset node selector was the main issue. The custom gist was to make it work with 1.19 as before. With 1.24, the newer one from their repo works, but the patch under needs to change, and helm versions (notably for nginx) need to be upgraded. This was the easiest way to get it working in the meantime

when: not kube_in_tree_provider and kube_cloud_provider == "openstack"

- name: Patch node-selector on OpenStack because of https://github.com/rancher/k3s/issues/1869
Expand Down
2 changes: 1 addition & 1 deletion roles/rke/templates/ebs_storage_class.j2
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ provisioner: kubernetes.io/cinder
provisioner: cinder.csi.openstack.org
{% endif %}
parameters:
availability: melbourne-qh2
availability: nova
Copy link
Copy Markdown
Contributor

@nuwang nuwang Jun 22, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we parameterize this?

{% else %}
provisioner: rancher.io/local-path
{% endif %}
Expand Down
2 changes: 1 addition & 1 deletion roles/rke/templates/rke2_config.j2
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
node-name: "{{ ansible_hostname if 'jetstream' in ansible_fqdn else ansible_fqdn }}"
node-name: "{{ ansible_hostname if 'jetstream' in ansible_fqdn or 'js2local' in ansible_fqdn else ansible_fqdn }}"
token: {{ rke_registration_token }}

{% if 'controllers' in group_names %}
Expand Down