Installation process

Installation of a management cluster via an installation wizard or console, closed loop parameters, first steps and diagnostics.

In a closed loop, installation is performed from the local administrator’s workstation with the shtil utility. Cluster nodes and the shtil machine must have access to a pre-prepared mirror of images.

Before installation, check the Closed Loop Requirements, the Installation Preparations (Mirror, shtil Machine, Hosts, SSH), and the applicable scenarios in the Internetworking Diagrams. The list of repository services can be found on the repository services page.

The shtil executable of the current version is supplied by the vendor’s manager together with the platform bundle and artifacts for the corporate mirror. After unpacking the bundle, copy the utility to shtil machine, make the file executable (chmod +x ./shtil or the current binary name). For the procedure for working with the bundle and mirror, see the page preparing for installation in a closed loop. The shtil version must match the version of your bundle.

Available installation scripts:

  • with GUI — install-gui, then the installation wizard in the browser;
  • with flags — install-cli and the provider subcommand;
  • with YAML - install-cli … --yaml-path / -f.

Installation from GUI

GUI installation consists of two stages: first shtil creates a local installation cluster, then the administrator configures the management cluster in the browser-based installation wizard.

The install-gui command is also available through the create-cluster and create synonyms.

Step 1: Start a local installation cluster

For a closed loop, be sure to indicate the license and mirror address:

./shtil install-gui \
  --bootstrap-license="YOUR-LICENSE" \
  --bootstrap-registry="MIRROR ADDRESS"

If the mirror has a self-signed certificate, use one of the options.

Skip TLS mirror check:

./shtil install-gui \
  --bootstrap-license="YOUR-LICENSE" \
  --bootstrap-registry="MIRROR-ADDRESS" \
  --bootstrap-insecure

Pass the directory with trusted mirror certificates:

./shtil install-gui \
  --bootstrap-license="YOUR-LICENSE" \
  --bootstrap-registry="MIRROR-ADDRESS" \
  --bootstrap-registry-certs-dir="/path/to/certs"

Note: Important for Docker: if Docker is used on the shtil machine, the --bootstrap-insecure flag does not disable TLS checking at the primary stage docker pull: this check is performed by the Docker daemon before the bootstrap cluster is launched. Before running shtil configure mirror trust: add CA to /etc/docker/certs.d/<registry-host:port>/ca.crt or configure insecure-registries to /etc/docker/daemon.json, then restart Docker.

To immediately set the platform administrator password, add --password. The password must be at least 8 characters and contain a lowercase letter, an uppercase letter and a number.

./shtil install-gui \
  --bootstrap-license="YOUR-LICENSE" \
  --bootstrap-registry="MIRROR-ADDRESS" \
  --password="PASSWORD-ADMIN"

Basic launch parameters:

ParameterDescription
--bootstrap-licensePlatform license. Required for local installation script
--bootstrap-registryImage mirror address. Mandatory in a closed loop
--bootstrap-insecureDisables TLS checking when accessing a mirror. If you use Docker additionally configure trust in the mirror
--bootstrap-registry-certs-dirDirectory with trusted mirror certificates; does not replace the /etc/docker/certs.d/ or insecure-registries setting for Docker
--bootstrap-kubeconfig-pathLocal installation cluster kubeconfig path
--passwordPlatform administrator password
--name, -nThe name of the local installation cluster. The default is shturval-installer
--waitWaiting for the Control Plane of the local cluster node to be ready
--retryNumber of retries for installation errors. Values ​​that are too low are automatically increased to 10
--debugSaves the container upon unsuccessful installation for diagnostic purposes

After a successful launch, the command displays a message about the start of the installer, the administrator account and the URL of the GUI.

Browser for the GUI - see browser requirements.

If shtil install-gui is launched on a remote Linux machine, open the installation wizard in a browser using the IP address or hostname of this machine, without first forwarding ports, for example, http://10.11.12.13:30080/.

Going by hostname is only possible if the DNS is configured to resolve the domain name to an IP address.

Life cycle of a local installer: the “Cancel installation” button in the installation wizard resets the completed cluster creation form and returns to the initialization screen; refreshing a page without canceling field data usually doesn’t reset. After successful installation of a permanent cluster, the temporary (bootstrap) cluster is not deleted automatically - if necessary, clean it according to the instructions Uninstalling a temporary cluster. If there is an error without --debug, the local installer container may be deleted; for diagnostics use ./shtil install-gui ... --debug.

Step 2: Open the installation wizard and fill out the parameters

  1. Open the GUI URL from the output of shtil.
  2. Log in as an administrator.
  3. Click Create Management Cluster.
  4. Complete the installation wizard and click Create cluster at the data verification step.
  5. Wait for the deployment to complete. Installation usually takes 20-60 minutes.

Do not close the browser tab until the installation is complete.

Screenshot of the local installer start screen

Local installer start screen

Step 1. Selecting a provider

Screenshot and step fields

Management Cluster Installation Wizard - Provider Selection

FieldDescription
Cluster nameThe unique name of the management cluster. Used in service objects and FQDN. Indicate the Latin alphabet, numbers and hyphens.
Select providerSelect the infrastructure provider prepared before installation: Shturval v2, VMware vSphere or oVirt and similar.

Note: the provider with which the platform is deployed will be installed in the platform as a critical service - it cannot be disabled. The remaining infrastructure providers in the platform will be disabled.

Before filling out the form, check the requirements of the selected provider: Shturval v2, VMware vSphere, oVirt and similar providers.

Step 2. Provider configuration

Shturval v2

The Shturval v2 provider uses pre-prepared physical or virtual hosts. A VM template is not needed for this provider: the installation wizard connects to hosts via SSH and installs components on the selected hosts.

Screenshot and step fields

Configuration of the Shturval v2 provider in the install-gui installation wizard

FieldDescription
Instance nameThe name of the provider configuration by which the installation wizard will refer to provisioned hosts.
IP PoolPool of free addresses for the API server and VIP Ingress. If the pool is not selected or there are no free addresses in it, the addresses are specified manually in the basic data step.
SSH usernameUser to connect to hosts. The user must have sudo rights without a password.
Private SSH KeyPrivate key for connecting to hosts. The public part of the key must be added to all target hosts.
HostsIP addresses or FQDN of hosts, roles (controlplane, workers, infra and others) and, if necessary, labels for selecting nodes in the installation wizard.

By default, the provider uses common credentials for all hosts. If necessary, you can specify private credentials for some or all hosts.

Credentials are saved in the Secret of the management cluster in the namespace shturval-capsm. The Secret name is formed from the name of the provider instance with the suffix capsm-v2-provider-secret. An example of Secret is given on the page Provider Shturval v2 (shturval-capsm).

VMware vSphere

For the VMware vSphere provider, use vCenter and the settings prepared before installation. Management cluster nodes will be created from VM template; if the template was not created in advance, prepare it according to the instructions and return to installation.

Screenshot and step fields

VMware vSphere provider configuration in the install-gui installation wizard

FieldDescription
Instance nameThe name of the vSphere configuration in the installation wizard.
IP PoolPool for API server and VIP Ingress. Without a pool, addresses are entered manually.
vCenter AddressIP address or FQDN of the vCenter server.
UsernamevCenter account in username@domain format.
PasswordvCenter account password.
DatacenterVirtual data center, available after checking the connection.
Resource PoolResource pool in which management cluster VMs will be created.
NetworkNetwork for VM. DHCP must be enabled on the network.
DatastoreStorage for VM disks.
VM storage pathFolder or path to Datastore for VM files.

oVirt, zVirt, HostVM, ED Virtualization, ROSA Virtualization

For oVirt and similar providers, use the Engine, network, and virtualization settings prepared before installation. Management cluster nodes will be created from VM template; if the template was not created in advance, prepare it according to the instructions and return to installation.

Screenshot and step fields

oVirt provider configuration in the install-gui installation wizard

FieldDescription
Instance nameThe name of the provider configuration in the installation wizard.
URL EngineFull API Engine address, for example https://ovirt.example.com/ovirt-engine/api.
UsernameAn Engine account with a domain such as admin@internal.
PasswordEngine account password.
Certificate VerificationEngine TLS certificate verification. Disable only if trust in an enterprise or self-signed CA is not configured.
DatacenterVirtual data center from the list after successful connection verification.
ClustersVirtualization cluster where VMs will be created.
VNIC ProfileNetwork profile for VM interfaces. DHCP must be enabled on the network.
Allow CSIAllows you to enable CSI integration for Persistent Volumes in the basic data step.
IP PoolPool for API server and VIP Ingress. Without a pool, addresses are entered manually.

Step 3. Basic data

Shturval v2

Screenshots and step fields

Shturval v2 - basic data, basic mode

Shturval v2 - basic data, advanced settings

Shturval v2 - external API balancer

Shturval v2 - external Ingress balancer

FieldDescription
Kubernetes version / platform versionCompatible version of Kubernetes and platform for the management cluster.
cluster API server addressIP or FQDN for KubeAPI. You can select a free address from the pool or enter it manually.
VIP address for IngressIPv4 address for system Ingress if an external Ingress balancer is not used.
DNS record IngressWildcard FQDN for Ingress when using DNS instead of VIP.
Use external balancer for IngressLeaves the DNS record field for Ingress. The external load balancer must be provisioned before the deployment starts.
Use an external balancer for the API serverFor the API server, the FQDN of the external balancer is specified. The balancer must be ready before installation.
Set advanced information security settingsIncludes Secret Encryption for etcd, TLS connection to Kubelet and pre-configured audit policy.
PodSubnetCIDR for Pod network. Should not intersect with the network of nodes and services.
Service SubnetCIDR for Service ClusterIP. Should not interfere with the network of nodes and pods.
Cluster ProfileA set of policies and settings that apply to a cluster.

VIPs are needed if external balancers for KubeAPI and Ingress are not used. Both addresses must be free. The API address must be on the Control Plane subnet, and VIP Ingress must be accessible from the administrator workstation.

VMware vSphere

Screenshots and step fields

VMware vSphere - basic data, basic mode

VMware vSphere -basic data, advanced settings

FieldDescription
Provider TemplateA VM template for the required version of Kubernetes, prepared before installation according to instructions. If there is only one template, it is selected automatically.
cluster API server addressVIP or FQDN for KubeAPI. Select an address from the pool or enter it manually.
VIP address for IngressIPv4 address for system Ingress in basic mode.
VIP address for Ingress or DNS record for IngressIn advanced mode, you can specify an IPv4 address or Wildcard FQDN.
Platform VersionCompatible platform version for the selected template.
Enable CSI IntegrationEnables vSphere CSI for Persistent Volumes if allowed by the template and vCenter permissions.
Use external balancerTransfers Ingress traffic to an external balancer; for Ingress, a DNS record is specified.
Set advanced information security settingsIncludes Secret Encryption for etcd, TLS connection to Kubelet and pre-configured audit policy.
PodSubnetCIDR for Pod network.
Service SubnetCIDR for Service ClusterIP.
Cluster ProfileA set of policies and settings that apply to a cluster.

VIPs are needed if external balancers for KubeAPI and Ingress are not used. Both addresses must be free. The API address must be on the Control Plane subnet, and VIP Ingress must be accessible from the administrator workstation.

oVirt, zVirt, HostVM, ED Virtualization, ROSA Virtualization

Screenshots and step fields

oVirt - basic data, basic mode

oVirt - basic data, advanced settings

FieldDescription
Provider TemplateA VM template for the required version of Kubernetes, prepared before installation according to instructions.
Platform VersionCompatible platform version for the selected template.
cluster API server addressVIP or FQDN for KubeAPI. Select an address from the pool or enter it manually.
VIP address for IngressIPv4 address for system Ingress in basic mode.
VIP address for Ingress or DNS record for IngressIn advanced mode, you can specify an IPv4 address or Wildcard FQDN.
Enable CSI IntegrationEnables CSI integration if it is enabled on the provider instance.
Usecall external balancerTransfers Ingress traffic to an external balancer; for Ingress, a DNS record is specified.
Set advanced information security settingsIncludes Secret Encryption for etcd, TLS connection to Kubelet and pre-configured audit policy.
PodSubnetCIDR for Pod network.
Service SubnetCIDR for Service ClusterIP.
Cluster ProfileA set of policies and settings that apply to a cluster.

VIPs are needed if external balancers for KubeAPI and Ingress are not used. Both addresses must be free. The API address must be on the Control Plane subnet, and VIP Ingress must be accessible from the administrator workstation.

Step 4. Configuring Control Plane nodes

Shturval v2

Screenshots and step fields

Shturval v2 - Control Plane

Shturval v2 - Control Plane, advanced settings

FieldDescription
Number of Control Plane nodesOdd value, no more than 5.
Potentially available hostsHosts with role controlplane or suitable labels.
Host SelectorExpanded selection of hosts by labels and expressions.

VMware vSphere

Screenshot and step fields

VMware vSphere - Control Plane

FieldDescription
Number of Control Plane nodesAn odd value, typically 1 for test or 3 for failover cluster.
CPU/memory/diskControl Plane VM resources. The values must not be below the GUI minimum limits.

oVirt, zVirt, HostVM, ED Virtualization, ROSA Virtualization

Screenshot and step fields

oVirt - Control Plane

FieldDescription
Number of Control Plane nodesOdd value, no more than 5.
Cores/Sockets/ThreadsComputing resources of the Control Plane VM.
MemoryThe amount of RAM of the VM.
DiskVM disk capacity; for local storage, not less than the minimum GUI value.

Step 5. Configuration of Worker nodes

Shturval v2

Screenshots and step fields

Shturval v2 - Worker groups

Shturval v2 - Worker parameters

Shturval v2 - withelector Worker

FieldDescription
Worker node groupA group of hosts, such as default or infra.
ZoomManual number of nodes or automatic range.
Host RoleRole for selecting hosts in basic mode, such as workers or infra.
Host SelectorExpanded selection of Worker hosts by label.

VMware vSphere

Screenshots and step fields

VMware vSphere - Worker groups

VMware vSphere - Worker parameters

FieldDescription
Worker node groupVM group, for example default or infra.
ZoomManual number of nodes or automatic range.
CPU/memory/diskResources of VM Worker nodes.
RolesRoles of nodes separated by commas; they will be added to the node labels.

oVirt, zVirt, HostVM, ED Virtualization, ROSA Virtualization

Screenshots and step fields

oVirt - Worker groups

oVirt - Worker parameters

FieldDescription
Worker node groupVM group, for example default or infra.
ZoomManual number of nodes or automatic range.
Cores/Sockets/ThreadsComputing resources of VM Worker nodes.
MemoryThe amount of RAM of Worker nodes.
DiskDisk capacity of Worker nodes.
RolesRoles of nodes separated by commas; they will be added to the node labels.

Step 6. Install the certificate

Shturval v2

Screenshot and step fields

Step “Installing a certificate”

OptionDescription
Self-SubscribedThe certificate is issued by the platform. Suitable for test and indoor benches.
IntermediateAn intermediate CA is used; download the certificate chain and private key.
ACMEACME server is used; specify the root CA and server URL.
CorporateUses a ready-made Ingress certificate and private key issued by your CA.
Let’s EncryptUses Let’s Encrypt; enter your account email.

VMware vSphere

Screenshot and step fields

Step “Installing a certificate”

OptionDescription
Self-SubscribedThe certificate is issued by the platform. Suitable for test and indoor benches.
IntermediateAn intermediate CA is used; download the certificate chain and private key.
ACMEACME server is used; specify the root CA and server URL.
CorporateUses a ready-made Ingress certificate and private key issued by your CA.
Let’s EncryptUses Let’s Encrypt; enter your account email.

oVirt, zVirt, HostVM, ED Virtualization, ROSA Virtualization

Screenshot and step fields

Step “Installing a certificate”

OptionDescription
Self-SubscribedThe certificate is issued by the platform. Suitable for test and indoor benches.
IntermediateAn intermediate CA is used; download the certificate chain and private key.
ACMEACME server is used; specify the root CA and server URL.
CorporateUses a ready-made Ingress certificate and private key issued by your CA.
Let’s EncryptUses Let’s Encrypt; enter your account email.

Step 7. Selecting services

Shturval v2

Screenshots and step fields

Selecting services by group

Extended selection of services

FieldDescription
Critical ServicesRequired platform components. Always enabled and cannot be disabled.
MonitoringMetrics, dashboards and metrics export.
LoggingCollection and processing of logs.
Backup and RestoreBackup and snapshot services.
Application DeliveryComponents of continuous delivery and deployment strategies.
Load BalancerLoad balancing and traffic management components.
Information Security ToolsConfiguration analysis and image scanning.
Show advanced settingsSwitches the selection from groups of services to individual services and charts.

VMware vSphere

Screenshots and step fields

Selecting services by group

Extended selection of services

FieldDescription
Critical ServicesRequired componentsyou are the platform. Always enabled and cannot be disabled.
MonitoringMetrics, dashboards and metrics export.
LoggingCollection and processing of logs.
Backup and RestoreBackup and snapshot services.
Application DeliveryComponents of continuous delivery and deployment strategies.
Load BalancerLoad balancing and traffic management components.
Information Security ToolsConfiguration analysis and image scanning.
Show advanced settingsSwitches the selection from groups of services to individual services and charts.

oVirt, zVirt, HostVM, ED Virtualization, ROSA Virtualization

Screenshots and step fields

Selecting services by group

Extended selection of services

FieldDescription
Critical ServicesRequired platform components. Always enabled and cannot be disabled.
MonitoringMetrics, dashboards and metrics export.
LoggingCollection and processing of logs.
Backup and RestoreBackup and snapshot services.
Application DeliveryComponents of continuous delivery and deployment strategies.
Load BalancerLoad balancing and traffic management components.
Information Security ToolsConfiguration analysis and image scanning.
Show advanced settingsSwitches the selection from groups of services to individual services and charts.

Step 8. Data verification

Shturval v2

Step description

Check the summary of parameters: cluster name, provider, API and Ingress addresses, network CIDRs, Control Plane configuration, Worker groups, certificate and selected services. To correct a parameter, return to the desired step through the left panel or the Back button. After verification, click Create Cluster.

VMware vSphere

Step description

Check the summary of parameters: cluster name, provider, API and Ingress addresses, network CIDRs, Control Plane configuration, Worker groups, certificate and selected services. To correct a parameter, return to the desired step through the left panel or the Back button. After verification, click Create Cluster.

oVirt, zVirt, HostVM, ED Virtualization, ROSA Virtualization

Step description

Check the summary of parameters: cluster name, provider, API and Ingress addresses, network CIDRs, Control Plane configuration, Worker groups, certificate and selected services. To correct a parameter, return to the desired step through the left panel or the Back button. After verification, click Create Cluster.

Management cluster installation status

For a detailed description of status stages, color indication, node and service tables, see Cluster Deployment Status.

After the deployment starts, the management cluster installation status and stages are displayed on the screen.

Upon completion of the installation, you will automatically move to the “Cluster ready” stage. At this stage the following are available:

  • static kubeconfig of the persistent management cluster - download and save it;
  • link to switch to the permanent management cluster. To log in, use the platform administrator login and password received when you started the installation.

The management cluster installation is considered complete when the minimum requirements are met:

  • infrastructure is ready;
  • at least 1 Control Plane node is ready;
  • services shturval-backend, shturval-auth, shturval-frontend, shturval-ingress-controller were successfully installed.

When kubeconfig and the link to the persistent cluster interface are already available, certain steps may still be in progress: for example, not all services may be installed or not all NCIs may be applied. You can wait for all stages to complete in the temporary cluster interface or check the status in the permanent cluster: NCI - on the Node Configuration page, if necessary, edit nodes through creating and working with NCI; services - on the Installed services page.

Installation of Shturval repository services is carried out in two stages: first, critical services are installed, then the services selected in the “Selecting services” step. The installation of Shturval repository services stage is considered completed only after installation of all services: critical and user-selected (for the composition and updating of registry artifacts, see repository services). For the management cluster, the status also displays the “Replication” stage with a list of replication conditions.

Installation with flags

Installation with flags is performed with the install-cli command without the installation wizard. The command is also available through the install synonym.

Providers

In a closed loop, use on-premise providers:

ProvaiderSubcommand
Shturval v2shturvalv2
oVirt, zVirt, HostVM, ED Virtualization, ROSA Virtualizationovirt
VMware vSpherevsphere

Scenarios and environment settings are also described in the documentation of Shturval v2, oVirt and VMware vSphere.

General parameters

For a closed loop, enter --bootstrap-registry. If the mirror has a self-signed certificate, add --bootstrap-insecure or pass certificates through --bootstrap-registry-certs-dir. If the shtil machine is running Docker, also configure mirror trust in advance: --bootstrap-insecure does not affect the primary docker pull. Network Context - Internetworking Patterns and Closed Loop Requirements.

GroupOptionsPurpose
Basic--api-endpoint, -a, --use-external-kubeapi-lb, --provider-name, --cluster-nameManagement API Address, KubeAPI External Balancer, Provider Instance Name, and Cluster Name
Network--pod-subnet, --service-subnetPod and Service subnets by CIDR; for errors, see address space (subnet) conflict
Ingress--ingress-vip, --ingress-default-fqdn, --use-external-ingress-lbExternal access options for Ingress
Security and certificates--secure, PEM, ACME and TLS optionsCluster creation request policies and Ingress certificates
System Services--enabled-system-services, --minimalList of included components and minimum configuration; see installing services
Bootstrap--existing-bootstrap-kubeconfig, --bootstrap-kubeconfig-path, --bootstrap-license, --bootstrap-registry, --bootstrap-insecure, --bootstrap-registry-certs-dirUsing an existing bootstrap cluster or creating a local one, license and mirror settings
Other--yaml-path, -f, --platform-version, --overwrite (instead of overwrite you can write the command -y), --cluster-kubeconfig-pathSettings file, platform version, overwriting conflicting resources and final kubeconfig path

Note: --existing-bootstrap-kubeconfig and --bootstrap-kubeconfig-path cannot be used together. --bootstrap-license is not required if the kubeconfig of a valid bootstrap cluster is passed.

Shturval v2

./shtil install-cli shturvalv2 \
  --bootstrap-registry="MIRROR-ADDRESS" \
  --bootstrap-license="YOUR-LICENSE" \
  --ssh-user="USERNAME" \
  --path-ssh-private-key="/path/to/private-key" \
  --controlplane-nodes="IP-CP1,IP-CP2,IP-CP3" \
  --worker-nodes="IP-W1,IP-W2,IP-W3" \
  --api-endpoint="IP-API" \
  --ingress-vip="IP-INGRESS" \
  --cluster-name="CLUSTER-NAME"

Required parameters for Shturval v2:

ParameterDescription
--api-endpoint, -aAPI server address
--path-ssh-private-keyPath to the private SSH key file
--ssh-userSSH username
--controlplane-nodes,-cIP or FQDN of Control Plane hosts separated by commas
--worker-nodes, -wIP or FQDN of Worker hosts separated by commas. Not required with --minimal

An example of a minimal configuration on one Control Plane node:

./shtil install-cli shturvalv2 \
  --bootstrap-registry="mirror.corp.local" \
  --bootstrap-license="YOUR-LICENSE" \
  --ssh-user="shturval" \
  --path-ssh-private-key="/home/admin/.ssh/id_ed25519" \
  --controlplane-nodes="10.11.12.1" \
  --api-endpoint="10.11.12.100" \
  --ingress-vip="10.11.12.101" \
  --cluster-name="mgmt-cluster" \
  --minimal=true

An example of a rerun with overwriting conflicting resources on an existing bootstrap cluster:

bin/shtil install-cli shturvalv2 \
  --overwrite\
  --ssh-user="shturval" \
  --path-ssh-private-key="/home/username/sh/kubeconfigs/certs/id_ed25519" \
  --api-endpoint="10.XX.XXX.246" \
  --ingress-vip="10.XX.XXX.252" \
  --controlplane-nodes="10.XX.XXX.145" \
  --worker-nodes="10.XX.XXX.69" \
  --bootstrap-kubeconfig-path="/home/username/sh/kubeconfigs/01bt" \
  -v=10\
  --cluster-name="mgmt617c1" \
  --provider-name="sht617p1"

oVirt / zVirt / HostVM / ED Virtualization / ROSA

./shtil install-cli ovirt \
  --bootstrap-registry="MIRROR-ADDRESS" \
  --bootstrap-license="YOUR-LICENSE" \
  --url-provider="https://OVIRT-FQDN/ovirt-engine/api" \
  --user-provider="USER" \
  --password-provider="PASSWORD" \
  --datacenter-provider="DATA CENTER" \
  --datacenter-cluster-provider="CLUSTER" \
  --vnic-profile-provider="VNIC-PROFILE" \
  --template="TEMPLATE-VM" \
  --provider-csi-storage="STORAGE-DOMAIN" \
  --api-endpoint="IP-API" \
  --ingress-vip="IP-INGRESS" \
  --cluster-name="CLUSTER-NAME"

Basic provider parameters:

ParameterDescription
--url-provideroVirt API URL with /ovirt-engine/api suffix
--user-provider, --password-provideroVirt credentials
--datacenter-provider, --datacenter-cluster-providerData center and cluster
--vnic-profile-providerVNIC profile
--template, -tVM template name
--provider-csi-storageStorage Domain for CSI
--sub-type-providerCompatible virtualization type: zvirt, hostvm, redvirt, rosavirt
--control-plane, --workersNumber of nodes
--cores-*, --memory-*, --disk-*VM Resources

VMware vSphere

./shtil install-cli vsphere \
  --bootstrap-registry="MIRROR-ADDRESS" \
  --bootstrap-license="YOUR-LICENSE" \
  --address-vcenter="VCENTER-FQDN" \
  --user-vcenter="USER" \
  --password-vcenter="PASSWORD" \
  --provider-datacenter="DATA CENTER" \
  --provider-datastore="DATASTORE" \
  --provider-resource-pool="RESOURCE-POOL" \
  --provider-network="NETWORK" \
  --provider-folder="FOLDER" \
  --template="TEMPLATE-VM" \
  --api-endpoint="IP-API" \
  --ingress-vip="IP-INGRESS" \
  --cluster-name="CLUSTER-NAME"

Basic provider parameters:

ParameterDescription
--address-vcenter, --user-vcenter, --password-vcenterConnecting to vCenter
--provider-datacenter, --provider-datastoreData center and storage
--provider-resource-pool, --provider-network, --provider-folderResource pool, network and VM folder
--template, -tVM template name
--control-plane, --workersNumber of nodes
--cpu-*, --memory-*, --disk-*VM Resources
--enable-csi, --provider-clone-mode, --provider-storage-policyAdditional options

Installation with YAML

Installation with YAML is performed with the install-cli command without the installation wizard. The command is also available through the install synonym.

Parameters are passed through the YAML manifest using --yaml-path or -f.

When using YAML, the parameters from the file are first applied, then only those command line flags that are explicitly specified at startup are applied on top of them.

Providers

In a closed loop, use on-premise providers:

ProviderSubcommand
Shturval v2shturvalv2
oVirt, zVirt, HostVM, ED Virtualization, ROSA Virtualizationovirt
VMware vSpherevsphere

Scenarios and environment settings are also described in the documentation of Shturval v2, oVirt and VMware vSphere.

General parameters

For a closed loop, specify --bootstrap-registry. If the mirror has a self-signed certificate, add --bootstrap-insecure or pass the certificates through --bootstrap-registry-certs-dir. If the shtil machine is running Docker, also configure mirror trust in advance: --bootstrap-insecure does not affect the primary docker pull. Network Context - Internetworking Patterns and Closed Loop Requirements.

GroupOptionsPurpose
Basic--api-endpoint, -a, --use-external-kubeapi-lb, --provider-name, --cluster-nameManagement API Address, KubeAPI External Balancer, Provider Instance Name, and Cluster Name
Network--pod-subnet, --service-subnetPod and Service subnets by CIDR; for errors, see address space (subnet) conflict
Ingress--ingress-vip, --ingress-default-fqdn, --use-external-ingress-lbExternal access options for Ingress
Security and certificates--secure, PEM, ACME and TLS optionsCluster creation request policies and Ingress certificates
System Services--enabled-system-services, --minimalList of included components and minimum configuration; see installing services
Bootstrap--existing-bootstrap-kubeconfig, --bootstrap-kubeconfig-path, --bootstrap-license, --bootstrap-registry, --bootstrap-insecure, --bootstrap-registry-certs-dirUsing an existing bootstrap cluster or creating a local one, license and mirror settings
Other--yaml-path, -f, --platform-version, --overwrite (instead of overwrite you can write the command -y), --cluster-kubeconfig-pathSettings file, platform version, overwriting conflicting resources and final kubeconfig path

Note: --existing-bootstrap-kubeconfig and --bootstrap-kubeconfig-path cannot be used together. --bootstrap-license is not required if the kubeconfig of a valid bootstrap cluster is passed.

YAML constructors

Interactive forms to prepare manifests (two files: provider and cluster):

Installation using YAML manifest

  1. Get a template for the desired provider:

    Shturval v2:

    ./shtil install-cli shturvalv2 example > install.yaml
    

    oVirt (and compatible virtualizations):

    ./shtil install-cli ovirt example > install.yaml
    

    VMware vSphere:

    ./shtil install-cli vsphere example > install.yaml
    
  2. Fill out the template: provider, cluster, nodes, VIP, subnets and, if necessary, certificates. Check network values and VIPs against Closed Loop Requirements and Interconnection Schemes. Eliminate CIDR intersections, see the section on address space conflict. Frequent provider fields are also described in the documentation Shturval v2, oVirt, VMware vSphere.

  3. Run the installation:

    ./shtil install-cli PROVIDER-TYPE \
      --yaml-path="PATH/TO/FILE.yaml" \
      --bootstrap-registry="MIRROR-ADDRESS" \
      --bootstrap-license="YOUR-LICENSE"
    

    For a self-signed mirror certificate, add --bootstrap-insecure or --bootstrap-registry-certs-dir. When using Docker on the shtil machine, additionally configure trust in the mirror before running the command.

Configuration in YAML - two documents (separated by ---): provider configuration and cluster configuration.

ProviderProvider ConfigurationCluster configuration
Shturval v2CapsmProviderConfigCapsmClusterConfig
oVirtCapovProviderConfigCapovClusterConfig
VMware vSphereCapvsProviderConfigCapvsClusterConfig

Example for Shturval v2: in the first document - type: CapsmProviderConfig, credentials.SSH (privateKey, userName), hosts (address, labelsSelector.role). In the second - type: CapsmClusterConfig, in cluster: apiEndpoint, clusterName, ingressvip, podSubnet, serviceSubnet, minimal, controlPlaneNodesCount; if necessary - provider.controlPlaneNodeConfig, provider.workerNodeConfigs.

The first document is the provider (CapsmProviderConfig):

---
type: CapsmProviderConfig
name: myprovidername
credentials:
  SSH:
    privateKey: /home/username/.ssh/id_ed25519
    userName: shturval
hosts:
  - address: 10.11.11.11
    labelsSelector: { role: controlplane}
    credentials: null
  - address: 10.11.12.12
    labelsSelector: { role: workers }
    credentials: null

The second document is a cluster (CapsmClusterConfig). In the working install.yaml place it after the provider with a --- delimiter between the documents:

---
type: CapsmClusterConfig
cluster:
  apiEndpoint: 10.11.12.13
  clusterName: myclustername
  controlPlaneNodesCount: 1
  ingressvip: 10.11.12.14
  podSubnet: 172.16.0.0/16
  serviceSubnet: 10.96.0.0/12
  minimal: true
provider:
  controlPlaneNodeConfig:
    controlPlaneSelector: { matchLabels: { role: controlplane } }
    nodeConfig: { groupName: control-plane, roles: [{ name: node-role.kubernetes.io/control-plane }] }
  workerNodeConfigs:
    - nodeConfig: { groupName: default, roles: [{ name: node-role.kubernetes.io/workers }] }
      workersCount: 1
      workerSelector: { matchLabels: { role: workers } }

Example for oVirt with multiple groups of Worker nodes: in provider.workerNodeConfigs, set a separate element for each group, for example default and infra.

---
type: CapovProviderConfig
name: sht-capov-mgmt-prov
credentials:
  oVirt:
    url: https://ovirt.example.com/ovirt-engine/api
    username: ENTER-USER
    password: ENTER-PASSWORD
connection:
  tlsInsecure: true
providerConfig:
  datacenter:DCname
  datacenterCluster: RHVLab
  vnicProfile: CHLab_LAN_3260
  template: template-name-in-ovirt-infrastructure
  csiStorage: CHLab_Pure_VM_Store
ingressCerts:
  ##ca:
    ##crtPath:
    ##keyPath:
  ##tls:
    ##crtPath:
    ##keyPath:

---
type: CapovClusterConfig
cluster:
  apiEndpoint: 10.XX.XXX.236
  clusterName: sht-capov-mgmt7
  controlPlaneNodesCount: 1
  ingressvip: 10.XX.XXX.237
  minimal: true
provider:
  name: sht-capov-mgmt-prov
  controlPlaneNodeConfig:
    nodeConfig:
      groupName: control-plane
    roles:
      - name: node-role.kubernetes.io/control-plane
    capovNodeConfig:
      CPU:
        cores: 8
        sockets: 1
        threads: 1
      memory:
        sizeMB: 9000
        guaranteedMB: 9000
      osDiskSizeGB: 60
  workerNodeConfigs:
    - nodeConfig:
        groupName: default
        roles:
          - name: node-role.kubernetes.io/workers
      workersCount: 1
      capovNodeConfig:
        CPU:
          cores: 8
          sockets: 1
          threads: 1
        memory:
          sizeMB: 12000
          guaranteedMB: 12000
        osDiskSizeGB: 60
    - nodeConfig:
        groupName:infra
        roles:
          - name: node-role.kubernetes.io/workers
      workersCount: 1
      capovNodeConfig:
        CPU:
          cores: 8
          sockets: 1
          threads: 1
        memory:
          sizeMB: 9000
          guaranteedMB: 9000
        osDiskSizeGB: 60

For oVirt and VMware vSphere, use the current command template install-cli ovirt example and install-cli vsphere example; see pages deployment via oVirt and deployment via VMware vSphere.

Minimal example of launching Shturval v2 using YAML:

./shtil install-cli shturvalv2 \
  --yaml-path="install.yaml" \
  --bootstrap-registry="mirror.corp.local" \
  --bootstrap-license="YOUR-LICENSE"

First steps after installation

ActionDescription
Check accessOpen the GUI URL. Login: admin, password - from user.conf in the installer data directory or the value specified in --password. Ensure that the management cluster status is Ready
Save your credentialsSave your administrator password and kubeconfig. In the GUI, download kubeconfig from the user menu → Download Kubeconfig
Set up base entitiesConfigure user delivery, IP pool, provider and VM template. For additional components and installation scripts, see the service installation page; platform registry and artifactsrms - repository services; permanent configuration of nodes - via node configuration (NCI)

The default installer data directory is ~/.shturval/installer. If the user’s home directory is not accessible, /tmp/shturval/installer is used. The administrator credentials file is user.conf inside the data directory.

Before removing the local installation cluster or management cluster, save user.conf if you need it for further access to the management GUI.

Diagnostics

Useful commands

./shtil --help
./shtil install-gui --help
./shtil install-cli --help
./shtil install-cli shturvalv2 --help
./shtil version

For more verbose output, use the global flag --verbosity or -v.

Errors connecting tothe mirror

SymptomPossible reasonSolution
Registry unreachable errorNo TCP access to mirror host or portCheck DNS, routing, firewall and --bootstrap-registry value
TLS errors when accessing a mirrorUntrusted or self-signed certificateUse --bootstrap-insecure or pass trusted certificates via --bootstrap-registry-certs-dir
TLS Docker error on primary docker pullThe Docker daemon does not trust the mirror certificate; shtil flags are not yet applied at this stageSet up Docker on the shtil machine: put the CA in /etc/docker/certs.d/<registry-host>/ca.crt or add a mirror in insecure-registries, then restart Docker and run again. See configuring Docker mirror trust
401 UnauthorizedLicense or mirror settings errorCheck the license and mirror settings

Checking mirror availability:

curl -v https://mirror.corp.local

Workstation errors

SymptomSolution
There is no container runtime in PATHCheck for the presence of docker from the bundle. It is needed for install-gui and local installation cluster operations
Need to diagnose an unsuccessful GUI installationRerun install-gui with --debug to save the container

SSH errors with Shturval v2

SymptomSolution
Permission deniedCheck SSH key, user and access to each host
sudo: no tty presentAdd the user to sudoers, for example shturval ALL=(ALL) NOPASSWD: ALL
SELinux blockingSet SELinux to permissive mode or disable it according to your organization’s requirements

Errors when creating a persistent cluster

SymptomSolution
VIP is already in useUse free non-pinging addresses
Subnets do not match / CIDR conflictCheck that the API address is in the Control Plane subnet and that podSubnet and serviceSubnet do not overlap with the cluster network; see address space conflict
Error loading imagesCheck node access to the mirror (repository services)

Additional diagnostics

  • Time discrepancy - check NTP or Chrony on all nodes and the mirror host.
  • Not enough space - check that node resources meet requirements.
  • CLI check - the version command displays the shturval-installer version; the output mode depends on the --verbosity level.