Skip to content

Additional ceph disks for clusters #893

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
dahoat-sprecher wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Additional ceph disks for clusters #893

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 27 additions & 1 deletion library/cluster_vm.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -201,6 +201,13 @@
- Optional parameter relevant only if I(command) is C(clone)
type: bool
default: false
additional_disks:
description:
- List of qcow2 file paths for additional disks
- Device names (vdb, vdc, ...) are assigned automatically
- Optional parameter relevant only if I(command) is C(create)
type: list
default: []
disk_bus:
description:
- Disk bus type to use for the VM's disk (virtio, scsi, ide, etc.)
Expand Down Expand Up @@ -311,6 +318,16 @@
cpu: 4
memory: 2048

# Create a VM with additional disks
- name: Create guest0 with an extra data disk
cluster_vm:
name: guest0
command: create
system_image: my_disk.qcow2
xml: "{{ lookup('template', 'my_vm_config.xml.j2') }}"
additional_disks:
- additional_data.qcow2

# Remove a VM
- name: Remove guest0
cluster_vm:
Expand Down Expand Up @@ -548,7 +565,7 @@ def check_parameters(parameters, commands_list):
command=dict(type="str", required=True, choices=commands_list),
name=dict(type="str", required=False, aliases=["guest"]),
xml=dict(type="str", required=False),
data_disk=dict(type="str", required=False),
additional_disks=dict(type="list", required=False, default=[]),
force=dict(type="bool", required=False, default=False),
enable=dict(type="bool", required=False, default=True),
system_image=dict(type="str", required=False),
Expand Down Expand Up @@ -668,6 +685,7 @@ def check_parameters(parameters, commands_list):
"clear_pacemaker_utilization", False
)
disk_bus = args.get("disk_bus", "virtio")
additional_disks = args.get("additional_disks", [])

vm_name_command_list = commands_list.copy()
vm_name_command_list.remove("list_vms")
Expand All @@ -689,6 +707,13 @@ def check_parameters(parameters, commands_list):
module.fail_json(
msg="`system_image` doesn't exist or is not a file`"
)
for i, filepath in enumerate(additional_disks):
if not os.path.isfile(filepath):
module.fail_json(
msg="additional_disks[{}] '{}' doesn't exist or is not a file".format(
i, filepath
)
)
vm_options = {
"name": vm_name,
"base_xml": vm_config,
Expand All @@ -710,6 +735,7 @@ def check_parameters(parameters, commands_list):
"pacemaker_params": pacemaker_params,
"pacemaker_utilization": pacemaker_utilization,
"disk_bus": disk_bus,
"additional_disks": additional_disks,
}
vm_manager.create(vm_options)
elif command == "clone":
Expand Down
2 changes: 1 addition & 1 deletion roles/deploy_vm_manager/files/vm_manager
Open in desktop
Comment thread
dahoat-sprecher marked this conversation as resolved.
Submodule vm_manager updated 43 files
+4 −1 .cqfd/docker/Dockerfile
+4 −1 .cqfdrc
+4 −1 .flake8
+63 −0 .github/workflows/ci.yml
+3 −0 .gitignore
+93 −0 CLAUDE.md
+50 −0 README.md
+31 −0 docs/api.rst
+23 −0 docs/cli.rst
+28 −0 docs/conf.py
+16 −0 docs/index.rst
+52 −0 docs/overview.rst
+4 −0 pyproject.toml
+45 −0 tests/conftest.py
+113 −0 tests/test_libvirt_manager.py
+886 −0 tests/test_vm_manager_cluster.py
+76 −0 tests/test_vm_manager_cmd.py
+126 −0 tests/test_vm_manager_cmd_cluster.py
+190 −0 tests/test_vm_manager_libvirt.py
+6 −0 vm_manager/__init__.py
+10 −0 vm_manager/exceptions.py
+28 −3 vm_manager/helpers/libvirt.py
+9 −1 vm_manager/helpers/libvirt_cmd.py
+15 −6 vm_manager/helpers/pacemaker.py
+1 −1 vm_manager/helpers/rbd_manager.py
+2 −0 vm_manager/helpers/tests/pacemaker/add_vm.py
+2 −0 vm_manager/helpers/tests/pacemaker/remove_vm.py
+2 −0 vm_manager/helpers/tests/pacemaker/start_vm.py
+2 −0 vm_manager/helpers/tests/pacemaker/stop_vm.py
+2 −0 vm_manager/helpers/tests/rbd_manager/clone_rbd.py
+2 −0 vm_manager/helpers/tests/rbd_manager/create_rbd_group.py
+2 −0 vm_manager/helpers/tests/rbd_manager/create_rbd_namespace.py
+2 −0 vm_manager/helpers/tests/rbd_manager/metadata_rbd.py
+2 −0 vm_manager/helpers/tests/rbd_manager/purge_rbd.py
+2 −0 vm_manager/helpers/tests/rbd_manager/rollback_rbd.py
+2 −0 vm_manager/helpers/tests/rbd_manager/write_rbd.py
+3 −1 vm_manager/testdata/vm.xml
+3 −1 vm_manager/testdata/wrong_vm_config.xml
+1 −0 vm_manager/vm_manager_api.py
+428 −118 vm_manager/vm_manager_cluster.py
+119 −12 vm_manager/vm_manager_cmd.py
+41 −18 vm_manager/vm_manager_libvirt.py
+56 −0 vm_manager/xml_utils.py
64 changes: 32 additions & 32 deletions roles/deploy_vms_cluster/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -66,38 +66,38 @@ This templated XML file offers default behaviors and configurations to launch VM
It can be a good starting point if you plan to deploy your VM on SEAPATH, however, for any production setup, it is recommended to create your own XML file.
Below is a list of the VMs member variables that can be used with this XML file. None of these variables are required.

| Member variable | Derived variable | Type | Default | Comments |
|-----------------|------------------|-----------------|-----------|-----------------------------------------------------------------------------------------------------------------------------|
| uuid | | Integer | random | Libvirt UUID of the VM |
| description | | String | Test VM | Libvirt description of the VM |
| memory | | Integer | 2048 | RAM of the VM in MiB |
| additional_disk | | List of strings | | Additional disks to give to the VM. The main disk is given by the vm_disk variable |
| vm_features | | List of strings | | List of vm features to enable. Possible values are "rt", "isolated", "secure-boot", "dpdk", "memballoon", "graphic-console" |
| | rt | | | Enable real time tweaks (priority, cgroup, scheduler, etc ...). Depends on `cpuset` |
| | isolated | | | Pin vCPU to hypervisor CPUs. Depends on `cpuset` |
| | secure-boot | | | Enable secure boot |
| | dpdk | | | Connect the VM to a DPDK OVS bridge port. Depends on `dpdk` |
| | memballoon | | | Enable memory ballooning for the VM. |
| | graphic-console | | | Add a graphic console (VNC) with video and tablet input to the VM |
| graphics_listen | | String | 127.0.0.1 | Address on which the VNC server listens. Depends on `graphic-console` in `vm_features` |
| cpuset | | List of int | | List of hypervisor CPU cores to use in the case of an isolated/RT VM |
| emulatorpin | | Integer | | Hypervisor CPU on which to pin the QEMU thread running the VM. If not set, the thread is not pinned. |
| nb_cpu | | Integer | 1 | Number of vCPU for the VM. Fallback to `cpuset` size if defined. |
| sriov | | List of strings | | List of SRIOV pools to use. |
| pci_passthrough | | List of dict | | List of dictionaries defining devices to passthrough to the VM. Each entry must contain: |
| | domain | Integer | | PCI domain of the device |
| | bus | Integer | | PCI bus of the device |
| | slot | Integer | | PCI slot of the device |
| | function | Integer | | PCI function of the device |
| bridges | | List of dicts | | List of Linux bridges to use. Each entry must define: |
| | name | String | | Name of the bridge to connect to |
| | mac_address | String | | Mac address of the virtual NIC of the VM on this bridge |
| ovs | | List of dicts | | List of OVS ports to use. Each element must contain: |
| | ovs_port | String | | OVS port to use for this interface |
| | mad_address | String | | Mac address of this interface |
| dpdk | | List of dicts | | List of Open vSwitch ports on which to enable dpdk. Depends on `dpdk` in `vm_features`. Each element must contain: |
| | ovs_port | | | OVS port on which to enable DPDK |
| | cpu_nb | | | Hypervisor CPU to use for this port (100% of the cpu time will be used) |
| Member variable | Derived variable | Type | Default | Comments |
|-----------------|------------------|-----------------|-----------|------------------------------------------------------------------------------------------------------------------------------------------|
| uuid | | Integer | random | Libvirt UUID of the VM |
| description | | String | Test VM | Libvirt description of the VM |
| memory | | Integer | 2048 | RAM of the VM in MiB |
| additional_disk | | List of strings | | Additional disks to give to the VM. Each entry is a path to a qcow2 disk image. Device names (vdb, vdc, ...) are assigned automatically. |
| vm_features | | List of strings | | List of vm features to enable. Possible values are "rt", "isolated", "secure-boot", "dpdk", "memballoon", "graphic-console" |
| | rt | | | Enable real time tweaks (priority, cgroup, scheduler, etc ...). Depends on `cpuset` |
| | isolated | | | Pin vCPU to hypervisor CPUs. Depends on `cpuset` |
| | secure-boot | | | Enable secure boot |
| | dpdk | | | Connect the VM to a DPDK OVS bridge port. Depends on `dpdk` |
| | memballoon | | | Enable memory ballooning for the VM. |
| | graphic-console | | | Add a graphic console (VNC) with video and tablet input to the VM |
| graphics_listen | | String | 127.0.0.1 | Address on which the VNC server listens. Depends on `graphic-console` in `vm_features` |
| cpuset | | List of int | | List of hypervisor CPU cores to use in the case of an isolated/RT VM |
| emulatorpin | | Integer | | Hypervisor CPU on which to pin the QEMU thread running the VM. If not set, the thread is not pinned. |
| nb_cpu | | Integer | 1 | Number of vCPU for the VM. Fallback to `cpuset` size if defined. |
| sriov | | List of strings | | List of SRIOV pools to use. |
| pci_passthrough | | List of dict | | List of dictionaries defining devices to passthrough to the VM. Each entry must contain: |
| | domain | Integer | | PCI domain of the device |
| | bus | Integer | | PCI bus of the device |
| | slot | Integer | | PCI slot of the device |
| | function | Integer | | PCI function of the device |
| bridges | | List of dicts | | List of Linux bridges to use. Each entry must define: |
| | name | String | | Name of the bridge to connect to |
| | mac_address | String | | Mac address of the virtual NIC of the VM on this bridge |
| ovs | | List of dicts | | List of OVS ports to use. Each element must contain: |
| | ovs_port | String | | OVS port to use for this interface |
| | mad_address | String | | Mac address of this interface |
| dpdk | | List of dicts | | List of Open vSwitch ports on which to enable dpdk. Depends on `dpdk` in `vm_features`. Each element must contain: |
| | ovs_port | | | OVS port on which to enable DPDK |
| | cpu_nb | | | Hypervisor CPU to use for this port (100% of the cpu time will be used) |

## Example Playbook

Expand Down
23 changes: 22 additions & 1 deletion roles/deploy_vms_cluster/tasks/main.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,18 @@
vars:
ansible_remote_tmp: "{{ deploy_vms_cluster_qcow2tmpuploadfolder | default(omit) }}"
when: deploy_vms_cluster_disk_copy | bool
- name: "Copy additional disk on target for {{ item }}"
ansible.builtin.copy:
src: "{{ add_disk }}"
dest: "{{ deploy_vms_cluster_qcow2tmpuploadfolder }}/additional_{{ add_disk_idx }}.qcow2"
mode: "0644"
vars:
ansible_remote_tmp: "{{ deploy_vms_cluster_qcow2tmpuploadfolder | default(omit) }}"
loop: "{{ hostvars[item].additional_disk | default([]) }}"
loop_control:
loop_var: add_disk
index_var: add_disk_idx
when: deploy_vms_cluster_disk_copy | bool
- name: "Create {{ item }}"
cluster_vm:
name: "{{ item }}"
Expand All @@ -86,17 +98,26 @@
preferred_host: "{{ hostvars[item].preferred_host | default(omit) }}"
crm_config_cmd: "{{ hostvars[item].crm_config_cmd | default(omit) }}"
disk_bus: "{{ hostvars[item].disk_bus | default(omit) }}"
additional_disks: "{{ range(hostvars[item].additional_disk | default([]) | length) | map('string') | map('regex_replace', '^(.+)$', deploy_vms_cluster_qcow2tmpuploadfolder ~ '/additional_\\1.qcow2') | list }}"
xml: >-
{{ lookup('file', hostvars[item].xml_path)
if hostvars[item].xml_path is defined
else lookup('template',hostvars[item].vm_template,template_vars=dict(vm=hostvars[item]))
if hostvars[item].vm_template is defined
else lookup('file',deploy_vms_cluster_vms_disks_directory + '/' + item + '.xml')
| replace('\n', '') }}
- name: Remove temporary file
- name: Remove temporary system disk file
ansible.builtin.file:
path: "{{ vm_file_dest }}"
state: absent
- name: "Remove temporary additional disk files for {{ item }}"
Comment thread
dahoat-sprecher marked this conversation as resolved.
ansible.builtin.file:
path: "{{ deploy_vms_cluster_qcow2tmpuploadfolder }}/additional_{{ add_disk_idx }}.qcow2"
state: absent
loop: "{{ hostvars[item].additional_disk | default([]) }}"
loop_control:
loop_var: add_disk
index_var: add_disk_idx
- name: Wait for VM connections
ansible.builtin.wait_for_connection:
delegate_to: "{{ item }}"
Expand Down
Loading