Client cluster manager (shturval-cluster-manager)

About the service #

1. Review #

cluster-manager is a component of the Shturval platform, responsible for installing, updating and synchronizing system services in the management cluster. It does not require manual launch of helm upgrade - the state of services is described by ShturvalServiceConfig (SSC) objects, and customization by the administrator is done through ShturvalServicePatch (SSP).

During the initial deployment of a client cluster, cluster-manager also participates in coordinating the installation of platform services and is associated with the application of node configurations through the Node Config (NCI) mechanism - see Node Configuration Module and Node Configuration. An overview of infrastructure providers and links to documentation for each provider - Providers.

Each service comes as one or more Helm charts. Charts with CRD (*-crds) are installed separately and marked as Hidden - they are managed automatically.

2. Managing services through the platform #

2.1. How cluster manager works #

• Cluster-manager reads ShturvalServiceConfig objects and ensures that the state of the charts in the cluster matches them.
• SSC objects are created and updated by the platform during updates; do not edit SSC manually - changes may be overwritten.
• For customization, use ShturvalServicePatch (SSP), which is overlaid on top of SSC.

2.2. Enabling and disabling services #

Service mode control - via the mode field in SSC or SSP:

Infrastructure providers are marked as critical Critical (disabling the service in the platform is not available) only for those environments where the management cluster is deployed with the corresponding provider: OpenStack (shturval-capos), Shturval v2 (shturval-capsm), vSphere (shturval-capvs) or oVirt (shturval-capov). Providers Basis Dynamix (shturval-capbd), Yandex Cloud (shturval-capy) and vCloud Director (shturval-capvcd) are not critical services.

2.3. Dependencies between services #

Cluster-manager respects the installation order. Main dependencies:

3. Diagnostics #

3.1. Service status #

3.2. Typical problems #

3.3. Diagnostics CAPSM #

A detailed guide to diagnosing the manual provider is in shturval-capsm §4.

4. Service life cycle #

4.1. Service installation #

1. Cluster-manager reads the SSC for the service.
2. Checks dependencies (all dependent charts must be installed).
3. Applies accumulated SSP (merges spec.customvalues).
4. Sets or updates the Helm chart with the resulting values.
5. Waits for the pods to move to Running.

4.2. Platform update #

When the platform is updated, Shturval cluster-manager updates SSC with new versions of charts. SSPs are not affected, but the resulting values may change (if a new version of the chart has added or renamed fields).

4.3. Critical Services #

Services with the Critical flag (for example, all CAPI providers) are restarted by cluster-manager automatically when they fail. Manually removing the pod to restart is acceptable.

5. Configuration via SSP #

General instructions for creating and applying patches - How to manage CustomValues of services. Below are examples in the context of cluster-manager and CAPSM.

5.1. What is SSP #

ShturvalServicePatch is a Kubernetes object with which the administrator overrides the parameters of the system service on top of the platform SSC.

Basic SSP fields:

5.2. Extending cluster-manager rights #

Cluster-manager, when rolling out a cluster, applies manifests from ClusterProfile in the management cluster. If resources are specified in the profile, for example, users API groups auth.shturval.tech or pods, standard rights may not be sufficient. In this case, checking Secret with manifests will fail, and the cluster profile will receive the status False (in the GUI Not Ready).

Creating profiles and diagnosing rights errors are described on the page How to create and apply cluster profiles (ClusterProfile).

Cluster-manager default rights are specified in ClusterRole - shturval-cluster-manager-role.

To add rights, create a separate ClusterRole and aggregate its rights into the role shturval-cluster-manager-role in one of the following ways:

• add the label rbac.shturval.tech/shturval-cluster-manager-aggregated-role: "true";
• add a custom label to ClusterRole and specify this label in spec.customvalues.roleAggregateLabels for shturval-cluster-manager.

Example ShturvalServicePatch:

apiVersion: ops.shturval.tech/v1beta2
kind: ShturvalServicePatch
metadata:
  name: cluster-manager-custom
spec:
  shturvalServiceConfigName: shturval-cluster-manager
  customvalues:
    roleAggregateLabels:
      custom/auth-user-role: "true"

The detailed procedure for adding rights for cluster profiles is described in the section If there are not enough rights to platform manifests.

5.3. Example: Change CAPSM Controller Resources #

apiVersion: ops.shturval.tech/v1beta1
kind: ShturvalServicePatch
metadata:
  name: capsm-resources-patch
  labels:
    ops.shturval.tech/ssc: shturval-capsm
spec:
  shturvalServiceConfigName: shturval-capsm
  customvalues:
    controllerManager:
      manager:
        resources:
          limits:
            cpu: "500m"
            memory: "512Mi"

5.4. Example: Enable ServiceMonitor for CAPSM #

apiVersion: ops.shturval.tech/v1beta1
kind: ShturvalServicePatch
metadata:
  name: capsm-servicemonitor
spec:
  shturvalServiceConfigName: shturval-capsm
  customvalues:
    serviceMonitor:
      enabled: true

5.5. What not to do through SSP #

• Do not change the mode field of CAPI providers through SSP without approval from the platform.
• Do not set nodeSelector or affinity, which conflicts with the management cluster’s placement policy.
• Do not change namespace through SSP - it is set in SSC and is part of the service identification.

6. Reference: SSC and SSP structure #

6.1. ShturvalServiceConfig (for reference) #

The SSC is created and updated by the platform. Key fields:

6.2. ShturvalServicePatch #

The spec.customvalues field in SSP overlays the spec.customvalues in SSC - does not replace it completely. Use only those keys that need to be overridden.

6.3. Correspondence of charts and parameters #

Parameters spec.customvalues (inSSC and SSP) correspond to values.yaml of a specific chart:

• CAPSM: the structure of values.yaml is described in shturval-capsm §6.
• Other services: the values structure is described in the documentation of the corresponding chart.