Local alerting

In the “Shturval” platform, centralized alerting is implemented by default for the management cluster with placement of metrics and alert rules in the management cluster. When using centralized alerting, setting up alert rules from the cluster interface is available. If necessary, you can configure local alerting. In that case, the GUI for configuring notification rules, routes, and silences is unavailable.

Configuring local alerting is possible using a custom resource VMRule and applying changes to the specification (ssc) monitoring module control component (VM Agent) (shturval-metrics-collector).

  1. Prepare a ShturvalServicePatch to apply to the Monitoring Module Control Component (shturval-metrics-collector) specification (ssc) where include the local vmsingle metrics storage database and requiredcomponents as shown in the example.
Example ShturvalServicePatch
apiVersion: ops.shturval.tech/v1beta2
kind: ShturvalServicePatch
metadata:
  name: metrics-collector-vmsingle
spec:
  shturvalServiceConfigName: shturval-metrics-collector
  customvalues:
    defaultRules:
      create: true
    vmalert:
      enabled: true
    vmsingle:
      enabled: true
  1. Load the prepared ShturvalServicePatch into the management cluster using import manifests.
Screenshot

collectormgmt_alertpatch

  1. Upload the custom VMRule resource to the management cluster using manifest import.
VMRule example
apiVersion: operator.victoriametrics.com/v1beta1
kind: VMRule
metadata:
  labels:
    cluster: clustername
    monitoringid: "0"
    role: user
  name: <your parameter value>
  namespace: victoria-metrics
spec:
  groups:
    - name: <your parameter value>
      rules:
        - alert: <your parameter value>
          annotations:
            message: <your parameter value>
          expr: <your parameter value>
          for: <your parameter value>
          labels:
            <your value>
ParameterDescriptionData typeExample
metadata.nameRule namestringshturval-backup
groups.nameRule group namestringshturval-backup
rules.alertAlert namestringVeleroBackupPartialFailures
rules.annotations.messageText of the message in the notification to the userstringVelero backup {{ $labels.schedule }} has {{ $value | humanizePercentage }} partialy failed backups
rules.exprRule that will trigger an alertstringvelero_backup_partial_failure_total{schedule!=""} / velero_backup_attempt_total{schedule!=""} > 0.25
rules.forTimeout period before sending an alertstring15m
rules.labelsLabels rulesstringseverity: warning
  1. Set up the alerting configuration, for example, route and recipient.

You can check local alerting is working.