Node configuration management module (shturval-node-config)

About the service #

Configuring NCI in the GUI, selectors, priorities and examples of sections can be found on the Node Configuration (NCI) page.

The service status can be viewed in the GUI: Services and repositories → Installed services → Node configuration management module (shturval-node-config) → Manage.

Contents #

1. Review
2. Architecture
3. Installation/deployment
4. Diagnostics
5. Lifecycle
6. Operator and Daemon Parameters
7. CRD/API reference

1. Review #

shturval-node-config ensures that the Kubernetes cluster node configuration is delivered and applied. The operator manages configuration objects and secure reboot, and the std daemon applies the configuration to the node, signals the need for a reboot, and completes it when authorized by the operator.

Place in the chain of command #

The administrator creates and modifies NodeConfigItem objects, the operator maps them to the NodeConfig of specific nodes, and the std daemon on each node receives the resulting configuration, applies it, and reflects the state back to NodeConfig.status. If necessary, the operator performs a cordon/drain and only then allows the node to reboot.

End-to-end sequence (NCI → NodeConfig → daemon, drain, reboot, CSR) in the diagram below; Detailed steps are in §5.

2. Architecture #

In the working environment, the controller is represented by two SSCs:

• shturval-node-config-crds — CRD chart of the group node.shturval.tech; (see the screenshot below)
• shturval-node-config — the main chart of the module. (see the screenshot below)

General architecture diagram, see the illustration below.

shturval-node-config service #

The operator manages the configuration of cluster nodes through the CRDs NodeConfig and NodeConfigItem, orchestrates drains and reboots, and automatically approves the CSR for kubelet-serving.

Key Features #

Components #

Demon std #

std (shturvald) runs on every Linux node in the cluster. It applies the merged node configuration, monitors the local init config, synchronizes the state with the NodeConfig object, and reboots only after permission from the operator.

Key Features #

Main tasks #

Objects in the cluster #

External dependencies #

Demon shurvald (std) #

The daemon is deployed directly on a Linux node and runs as a system service. It uses local files in /etc/std/, kubeconfig kubelet and sync queue to apply the configuration and exchange state with the operator.

Demon Circuits #

Logical architecture of the std daemon on the node, see the illustration below.

Processing flow: configuration, application, interaction with NodeConfig:

Components on a node #

Operator-daemon communication #

3. Installation / deployment #

shturval-node-config is a critical platform service. The activation and life cycle of the controller in the cluster is provided by cluster-manager (platform service); ShturvalServiceConfig (SSC) objects for associated charts are created and maintained as part of the platform configuration, rather than as an arbitrary manual installation of a Helm chart by an administrator.

The std agent (shturvald) is installed by the platform on nodes automatically as part of the node preparation process (bootstrap). No separate manual installation of the agent is required.

CRDs, controllers and resource ownership:

4. Diagnostics #

shturval-node-config service #

Quick checks #

kubectl get nc
kubectl get nci

The operator publishes the following printable columns for NodeConfig:

The following columns are available for NodeConfigItem:

Conditions of the NodeConfig resource #

Note: NodeDrained only appears with rebootscheduled=true, and RebootScheduled only appears with rebootrequired=true.

Demon std #

Quick checks #

systemctl status shturvald
std node status
std node info

If the daemon has not yet joined the cluster, an additional guideline is the presence of the /etc/std/nci.json and /etc/kubernetes/kubelet.conf files.

Demon states #

Annotations affecting diagnostics #

Typical operating situations #

If, after reconciling the local init NCI and the expected cluster configuration, you need to re-run the configuration application, remove the node.shturval.tech/initchecksum annotation written to NodeConfig:

kubectl get nodeconfig <node-name> -o jsonpath='{.status}'
kubectl annotate nodeconfig <node-name> node.shturval.tech/initchecksum-

5. Life cycle #

shturval-node-config service #

Complete scenario on one diagram: see also §1 - place in the chain.

Node Circuit → NodeConfig #

Matching Node and NodeConfig, see the illustration below.

Outline NodeConfigItem → NodeConfig #

Matching NodeConfigItem and selected NodeConfig, see the illustration below.

Application and reload circuit #

Application and reload reconcile flow for NodeConfig, see the illustration below.

Agreement NodeConfig: drain:

Negotiation NodeConfig: reboot queue:

Agreement NodeConfig: conditions:

Drain Policy #

Finalizers #

Demon std #

Startup order #

Configuration application cycle #

Reboot and reapply #

6. Operator and daemon parameters #

shturval-node-config service #

Process flags #

Environment Variables #

Demon std #

Global CLI flags #

Basic commands #

Operating restrictions #

7. CRD/API reference #

Loop API group: node.shturval.tech, version v1beta2.

NodeConfig #

NodeConfig is created automatically for each Node and is not intended to be created manually.

Resource characteristics #

spec #

status #

Conditions #

Example #

apiVersion: node.shturval.tech/v1beta2
kind: NodeConfig
metadata:
  name: node-1
  labels:
    kubernetes.io/os: linux
    node.shturval.tech/applygroup: workers
spec:
  applygroup: workers
  donotdrain: false
  nodeconfigitems:
    - packages-basic-install
    - kubelet-basic

NodeConfigItem #

NodeConfigItem is created by the administrator and specifies a reusable configuration template for a group of nodes selected by tags.

Resource characteristics #

spec #

status #

Examples #

Minimal example of installing packages:

apiVersion: node.shturval.tech/v1beta2
kind: NodeConfigItem
metadata:
  name: packages-basic-install
spec:
  nodeconfigselector:
    node-role.kubernetes.io/worker: ""
  packages:
    - name: chrony
      state: present
    - name: curl
      state: present

Example of kubelet setup:

apiVersion: node.shturval.tech/v1beta2
kind: NodeConfigItem
metadata:
  name: kubelet-basic
spec:
  nodeconfigselector:
    node-role.kubernetes.io/worker: ""
  kubelet:
    cpuManagerPolicy: none
    memoryManagerPolicy: None
    maxPods: 110
    hairpinMode: promiscuous-bridge

OIDC and AuthenticationConfiguration #

kube-apiserver can admit users with a JWT token from an external OIDC provider.

Previously, OIDC on kube-apiserver was set with startup flags: --oidc-issuer-url, --oidc-client-id, and the other --oidc-* flags. That allowed only one provider.

Since Kubernetes 1.34 the same settings go into an AuthenticationConfiguration file (apiserver.config.k8s.io/v1). The file has a jwt list: each item is a separate provider with its issuer address (issuer.url) and, if needed, its own discovery address (issuer.discoveryURL).

Do not set --oidc-* flags together with the path to this file (--authentication-config): kube-apiserver will not start.

Fields for a single provider in the GUI are on the Node Configuration (NCI) page. Several providers are set in the manifest only.

In NCI:

• spec.kubeapi.oidc — one provider, same as the GUI. The field is still supported: on merge and apply it becomes one JWT entry.
• spec.kubeapi.authenticationConfiguration.jwt — a list of providers.

Through the platform API, oidc can list several providers: each item is a separate issuer. An item can have several client IDs (clientid), plus exist and discoveryURL. In the cluster NCI object those items are stored in authenticationConfiguration.jwt.

After the switch to the file, the daemon does the following on the node:

1. Writes the configuration to /etc/kubernetes/auth-config/authentication-config.yaml.
2. In the kube-apiserver manifest, points to that file with --authentication-config= and attaches the /etc/kubernetes/auth-config/ directory so the API server can read it.
3. Removes the old --oidc-* flags.

If later the file or arguments on the node no longer match the NCI, the daemon brings them back in line.

If several NCI objects apply to a node, their JWT lists are combined.

• The same issuer.url in two NCI objects — the description from the NCI with the higher priority is kept.
• A new issuer.url — the provider is added to the list.
• exist: false on the entry with that URL — the provider is not in the resulting configuration. If exist is omitted or true, the provider stays.

One NCI cannot list the same issuer.url twice. The CA certificate must be PEM. Otherwise the object is not saved.

An example with a single oidc field is on the Node Configuration (NCI) page. Several providers:

apiVersion: node.shturval.tech/v1beta2
kind: NodeConfigItem
metadata:
  name: auth-config-nci
spec:
  nodeconfigselector:
    node-role.kubernetes.io/control-plane: ""
  kubeapi:
    authenticationConfiguration:
      jwt:
        - issuer:
            url: https://auth.example.com
            discoveryURL: https://discovery.example.com/.well-known/openid-configuration
            audiences:
              - backend
          claimMappings:
            username:
              claim: sub
              prefix: "shturval:"
            groups:
              claim: groups
              prefix: "shturval:"
        - issuer:
            url: https://sso.example.com
            audiences:
              - backend
          claimMappings:
            username:
              claim: sub
        - exist: false
          issuer:
            url: https://old-idp.example.com
          claimMappings:
            username:
              claim: sub
  priority: 100

Data the daemon writes to NodeConfig #

Annotations #

Application states #