setup_os

Root OS Install

We will use Xubuntu 15.10 as our basic image. You could use any OS you want, but there will be many copies so you want the image to be relatively small.

Note about EFI vs BIOS

In theory, EFI should provide more features, better boot times, and a generally more modern environment. I have had some bad luck with EFI, especially on xarc0. This guide will use the legacy BIOS mode (set in the motherboard settings).

Boot Loaders

See this guide: https://www.gnu.org/software/grub/manual/html_node/Multi_002dboot-manual-config.html

Since we are going to have many OSs on the same machine, each wanting to do it's own thing for the boot loader, grub configuration gets complicated. For simple dual-boot this isn't a big deal, but we need something more flexible.

The solution is to use grub's recursive multiboot features to write a custom grub.cfg in the rootvg/boot LV that lets you choose between the various OS installs. Each OS will have it's own grub.cfg in /boot as if it were the only OS on the system. In this way, all OS boot menu tools continue to work since they don't even know about xarcvg/boot/grub.cfg.

Step-by-Step

1. Boot the live xubuntu install and select "try xubuntu"
2. setup the following LVM partitions using the CLI (optional GUI tool: "sudo apt-get install system-config-lvm")
3. Volume Group: rootvg (I used the whole first disk)
4. LV: root (10G)
5. LV: boot (256M)
6. LV: swap (8G)
7. Volume Group: vmvg (I used the whole second disk)
8. No LVs for now
9. Install Xubuntu with custom disk layout (select "Something Else" when asked about partitioning)
10. mount / to rootvg/root
11. make sure rootvg/swap is marked as swap
12. Reboot and make sure the basic OS is working

Installing more OSs

Now that we have a stable base OS, we can make experimental clones to play with. First, make sure your root OS is set up how you want since we are going to be cloning it. It's much harder to make changes after cloning since you have to repeat everything. I installed vim, git, openssh-server, eclipse...

Backup and Clone the OS

All experimental OSs will be a clone of a snapshot of your root OS

1. Create a snapshot of your root partition (I made mine 5G and called it root_bak)
2. You will likely need to add the dm-snapshot module to your kernel.
3. Add "dm-snapshot" to /etc/modules
4. modprobe dm-snapshot
5. Create a new logical volume of the same size os root (I'll call this os0)
6. Copy the snapshot of root to os0 "sudo dd if=/dev/rootvg/root_bak of=/dev/rootvg/os0 bs=4M"
7. Make the clone is aware of its new location:
8. sudo mount /dev/rootvg/os0 /mnt
9. change /mnt/etc/fstab to point to /dev/rootvg/os0 instead of root
10. sudo chmod +x /mnt/boot/grub/grub.cfg
11. change all occurances of rootvg-root to rootvg-os0 in /mnt/boot/grub/grub.cfg

Set up Grub Multiboot

We will install a layer of indirection to grub so it can boot any OS

1. Mount rootvg/boot ("sudo mount /dev/rootvg/boot /mnt")
2. create the file "/mnt/grub/grub.cfg" with the contents listed below.
3. Now re-install grub so that it looks for this new grub.cfg: "sudo grub-install --boot-directory=/mnt/ /dev/sda"
4. If this fails, try mounting a livecd/usb and running the same command
5. Reboot and pick os0, check the output of "mount" to make sure /dev/rootvg/os0 is mounted as /

Centos

Centos is a bit touchier than Ubuntu when copying. This is because it relies more on UUIDs and xfs. In addition to the above you must also:

1. the root LV cannot be mounted when creating the snapshot (xfs doesn't like being coppied live)
2. You might have to do this from a liveCD or a different OS if you already have a few multiboots setup
3. After copying, change the uuid of the new filesystem: xfs_admin -U generate /dev/rootvg/os0
4. It will print out the new UUID, record this
5. Update boot/''grub2''/grub.cfg (note the new path)
6. Change rootvg-root to rootvg-os0 as before
7. Change rootvg/root to rootvg/os0
8. Replace the old UUID with the new one recorded during step 2
9. Change /etc/default/grub
10. GRUB_CMDLINE_LINUX change "rootvg/root" to "rootvg/os0"
11. Run "grub2-mkconfig --output=/boot/grub2/grub.cfg" after messing with /etc/default/grub to regenerate the grub.cfg

You can now repeat this as many times as you like!

grub.cfg

menuentry "root" {
  insmod lvm
  set root=lvm/rootvg-root
  configfile /boot/grub/grub.cfg
}

menuentry "os0" {
  insmod lvm
  set root=lvm/rootvg-os0
  configfile /boot/grub/grub.cfg
}

Notes and Gotchas

• make sure the partition table is MBR (a.k.a MSDOS) or else the installer will fail with a cryptic message about failing to install grub
• You might want to add "GRUB_DISABLE_OS_PROBER=true" to /etc/default/grub to prevent grub trying to dual boot in the os-level grub config (it's redundant and slow).