Monitoring/logging module component (VLogs) (shturval-metrics-collector)

About the service

ParameterDescription
GUI nameMonitoring module. Monitoring module control component
System name (identifier)shturval-metrics-collector
Cluster AssignmentCollection of system metrics, sending to VictoriaMetrics; routing events and sending alerts (VM Alert and VM Alert Manager components); deployment of a log storage instance VictoriaLogs (VM Agent component).
CriticalNo
Public repositoryVictoria Metrics Agent (vmagent)
Dependenciesshturval-local-csi, shturval-cert-manager, shturval-metrics-collector-crds, shturval-monitoring-crds
Namespacevictoria-metrics

Purpose

shturval-metrics-collector - the component provides local collection of metrics and sending them to VictoriaMetrics of the management cluster. The chart also includes VM Alert (alert generation) and VM Alert Manager (notification routing and delivery). The component controls the deployment of VictoriaLogs instances for storing logs (more details - below).

More details: Monitoring architecture.

Features

Installation and dependencies

Already installed service: Services and repositories → Installed services → Monitoring module. Monitoring module control component → Manage.

If the service is missing: Services and repositories → Available charts → shturval tab → chart shturval-metrics-collector → Install. Select the chart version and namespace victoria-metrics. Specify the parameters in the Service Specification (customvalues) block or via ShturvalServicePatch after installation. A common scenario is How to deploy a new service.

Note: Important For correct operation of monitoring in the client cluster:

  • shturval-metrics-collector-crds (Monitoring module. CRD of the CR management component of the monitoring module) must be running in the client cluster;
  • in the management cluster - shturval-metrics-collector and shturval-metrics-collector-crds, shturval-monitoring.

VM Agent configuration

Basic VM Agent setup:

  • defines collection tasks (scrape jobs) with a frequency of scrapeInterval and a timeout of scrapeTimeout;
  • defines CPU/RAM limits and requests for VM Agent, VictoriaMetrics Operator and auxiliary services (node-exporter, kube-state-metrics).
Example: collection frequency and timeout
vmagent:
  enabled: true
  spec:
    scrapeInterval: <your parameter value>
vmScrape:
  kind: VMNodeScrape
  spec:
    scrapeTimeout: <your parameter value>
ParameterDescriptionData typeDefault
scrapeIntervalFrequency of metrics collectionstring20s
scrapeTimeoutTimeout after which metrics collection will endstring5s
Customvalues example: Compute resources for VM Agent
vmagent:
  enabled: true
  resources:
    limits:
      cpu: <your parameter value>
      memory: <your parameter value>
    requests:
      cpu: <your parameter value>
      memory: <your parameter value>
ParameterDescriptionData typeExample
limits.cpuCPU limit for VM Agentstring1
limits.memoryMemory limit for VM Agentstring1Gi
requests.cpuCPU request for VM Agentstring50m
requests.memoryMemory request for VM Agentstring50Mi

Selectors and Labels

Basic monitoring is configured using labels. If necessary, assign a label to the service so that it is included in the list of objects for monitoring, or configure itlecturers on namespaces and goals.

namespace and target selectors
vmagent:
  spec:
    selectAllByDefault: false
    nodeScrapeNamespaceSelector: # Selector for namespaces from which metrics will be collected
      matchLabels:
        shturval.tech/system-namespace: ''
    nodeScrapeSelector: {} # Selector for selecting cluster nodes from which metrics will be collected
    podScrapeNamespaceSelector: # Selector for selecting pods from which metrics will be collected
      matchLabels:
        shturval.tech/system-namespace: ''podScrapeSelector: {} # Selector for selecting pods from which metrics will be collected
    probeNamespaceSelector: 
      matchLabels:
        shturval.tech/system-namespace: ''
    probeSelector: {} 
    serviceScrapeNamespaceSelector: # Selector for services from which metrics will be collected
      matchLabels:
        shturval.tech/system-namespace: ''
    serviceScrapeSelector: {} # Selector for services from which metrics will be collected
    staticScrapeNamespaceSelector: # Selector for static objects from which metrics will be collected
      matchLabels:
        shturval.tech/system-namespace: ''
    staticScrapeSelector: {} ## Selector for static objects from which metrics will be collected

VictoriaLogs

Centralized storage of logs is performed in VictoriaLogs. Architecturally, the data is located in the management cluster: for each cluster (client and management) a separate instance of VictoriaLogs is created - VLSingle in the namespace with the name of the corresponding cluster. Instance deployment is controlled by shturval-metrics-collector. Log collection is performed by shturval-log-collector (Vector).

More details: Architecture of logging services, How to view logs in the platform, How to set up log rotation, Troubleshooting log collection problems

Additional

The shturval-metrics-collector chart is used in the management cluster and in each client cluster. Below are the instructions in which it appears.

Configuration via ShturvalServicePatch

Alert

Application Monitoring

Visualization

← Local log collection module (shturval-log-collector)
Monitoring module. Component for centralized storage of metrics (shturval-monitoring) →