Preparing the Provider Template
Requirements for templates and step-by-step preparation process.
Virtual machine templates for all providers (VMware vSphere, oVirt, OpenStack, Basis Dynamix, vCloud Director, Yandex Cloud) are prepared in one way. The process does not depend on the type of provider: you create a VM, prepare it with the stc utility, convert it to a template and load it into the provider instance.
When creating a VM from a template, the Shturval platform independently installs Kubernetes of the selected version and the necessary configurations.
How this relates to deploying the entire cluster (Cluster API, steps before registering nodes), see Creating nodes when deploying a client cluster.
What the template contains
The template stores:
- Operating system and its version - from list of supported OS;
- Disk size limitation - created VMs cannot be smaller than the disk capacity of the original VM.
The template does not store information about the resources (CPU, RAM) of the source VM.
Requirements for the VM before creating the template
On the virtual machine from which the template is generated:
| Requirement | Description |
|---|---|
| Time Synchronization | NTP on all nodes. Time discrepancies can cause crashes |
| Mirror Resolve | The host with the mirror (registry) must resolve using the current FQDN |
| SELinux | Disabled or permissive |
| Repositories | Inaccessible repositories disabled or removed (distribution native repositories) |
| Packages | iptables and gnupg set |
| Network and DHCP | On Debian/Ubuntu (where /etc/network/interfaces is used), configure the eth0 interface to obtain an IP address via DHCP. Do not set a static IP in the image; when creating the base VM, connect it to a network with working DHCP. If DHCP is configured for a previous interface name (not eth0), nodes will not receive IP addresses |
| VM Resources | At least 4 vCPU and 8 GB RAM. If stc fails to prepare the template with a node check failure due to insufficient resources, add VM resources and rerun the command |
| User namespaces for rootless Podman | Enabled kernel.unprivileged_userns_clone permission to create unprivileged user namespaces. Execute sysctl kernel.unprivileged_userns_clone. If the value is 0, set 1 (enable the option) to /etc/sysctl.conf and execute sysctl -p. If the command returns the error cannot stat /proc/sys/kernel/unprivileged_userns_clone: No such file or directory, the parameter may not be configured |
When using a mirror, specify the registry name in the same way as when deploying the platform.
| Provider | Description |
|---|---|
| VK Cloud | When creating images, set the access type to public public, private images - private will not be available for selecting a VM template in the provider instance. How to change the access type of the created image, see page. |
| oVirt, zVirt, HostVM, RED Virtualization, ROSA Virtualization | The VM template must have a disk with the Bootable flag enabled. Check and enable the flag in the disk properties so that the VM can use this disk to run the operating system. |
Disk allocation
Recommended: use LVM and store data in the root partition. As the VM disk grows, the root partition will expand. Minimum disk size is 60 GB.
Note: To correctly expand the disk, follow the requirements for the selected partitioning scheme. When LVM is used, the PV that is part of a volume group must be the active and last partition in the partition table.
The top-level use of partitions other than the root one in the minimal configuration is as follows:
| Section | Directories | Purpose | Volume |
|---|---|---|---|
| /var/ | /var/csi/rawfile/ | Local storage, custom PVs | 5 GB + PV |
| /var/log | Logs | 20 GB | |
| /var/lib/kubelet/ | Kubelet (emptydir) | 10 GB | |
| /var/lib/containerd/ | Container Images | 10 GB | |
| /home | — | Home directories | 500 MB |
| /boot | — | Boot section | 1 GB |
| / | — | Root partition | 20 GB |
If necessary, allocate /var/log - allocate 20 GB.
If LVM is not used
The root partition / must be the active and last partition in the partition table. If other partitions are located behind the root partition, disk expansion will not work.
Step-by-step process for creating a template
Step 1: Create a VM in the virtualization platform
Create a virtual machine in oVirt, VMware vSphere, OpenStack, or another environment. Connect the VM to a network with working DHCP — cluster nodes obtain IP addresses via DHCP. Install supported OS, configure the disk according to recommendations above, fulfill VM requirements.
Step 2: Preparing the template with the stc utility
Note: stc runs on a working machine (not on the cluster host) running Linux OS. When using a mirror, specify the registry name in the same way as when deploying the platform.
- Download stc https://public.shturval.tech/stc-2.14.0 and make it executable:
chmod +x ./stc-2.14.0
- Start preparing the template on the VM (specify additionally the registry parameter if you are using a mirror instead of r.shturval.tech):
./stc-2.14.0 template --license="YOUR-LICENSE"
Note: If during template preparation it is reported thatA node reboot is required to enable cgroup v2, wait until it finishes. Then run the stc template command again with the same parameters. The template will be ready only after the command has been successfully executed again without requiring a command restart.
Stc template command parameters:
| Parameter | Required | Description | Default |
|---|---|---|---|
--license | Yes | License key for downloading packages | - |
--registry | No | Registry address (by default r.shturval.tech). Must match registry when installing the platform | - |
--platform-version | No | Shturval platform version. Version compliance - in documentation | - |
--keep-cloud-init | No | Leave cloud-init (required for Yandex Cloud) | false |
--fix-vmware-network | No | Fixing network on VMs for VMware vSphere, VMware vCloud Director providers. Without a fix on vmxnet3, the checksum of tunnel traffic does not work correctly, which leads to problems with the Kubernetes overlay network. On non-VMware provider VMs, the patch is skipped | true |
--dhcp-no-domain-name | No | Disable the ability of DHCP to accept the domain-name parameter (configurable in netplan or dhclient). Can be used when a VM is on a corporate network with DHCP that distributes an organization’s domain, and the cluster nodes must resolve without a suffix. | false |
Examples of stc commands
Open loop:
./stc-2.14.0 template --license="YOUR-LICENSE"
Closed loop (mirror):
./stc-2.14.0 template --license="YOUR-LICENSE" --registry="YOUR-REGISTRY-ADDRESS"
Yandex Cloud (with cloud-init):
./stc-2.14.0 template --license="YOUR-LICENSE" --keep-cloud-init
oVirt (with dhcp-no-domain-name):
./stc-2.14.0 template --license="YOUR-LICENSE" --dhcp-no-domain-name
- Shut down the VM in the virtualization platform. After entering the “off” state, convert the VM into a template using the provider’s tools.
Step 3: Uploading the template to the Shturval platform
Add the created template to the provider instance:
- VMware vSphere
- oVirt (zVirt, HostVM, ED Virtualization, ROSA Virtualization)
- OpenStack
- Basis Dynamix
- VMware vCloud Director
- Yandex Cloud
You can add multiple templates to one provider instance, including templates with the same Kubernetes version. Each template specifies one version of Kubernetes. The mapping of template name and Kubernetes version is stored on the instance InfrastructureProvider, in spec.templates. The Kubernetes version in the template determines the available platform versions when creating client clusters. Before selecting the Kubernetes version, please read correspondence between Shturval and Kubernetes versions.
If the instance has several templates for one Kubernetes version, the Basic data step of creating a client cluster shows the required VM template field. The selected template is applied to all nodes.
During an upgrade, if the instance has several VM templates with the same Kubernetes version, the first one in alphabetical order is selected and applied to all node groups. To make sure the intended template is used on upgrade, add one template of the next version. If there are several templates, the first one in alphabetical order is selected.
On the provider instance page in the Virtual machine templates block, click + to add a template. In the side window, select name (as in virtualization) and Kubernetes version. The corresponding Shturval platform version will be filled in automatically and will be available for viewing only.
InfraMachineTemplate
VM templates are the basis for InfraMachineTemplates, from which nodes in groups are created. Based on one VM template, you can configure several InfraMachineTemplates with different parameters (CPU, RAM, etc.).
For a general description of the steps involved in the emergence of nodes from a template when creating a client cluster, see also help on nodes during deployment.