Architecture of logging services
Data flow and components diagram.
In the Shturval platform, logs are collected by Vector and stored in VictoriaLogs. This page contains components and data flows.
General scheme
Illustration: General scheme
Components
Local log collection module (Vector)
| Component | System name | Purpose |
|---|---|---|
| Local log collection module | shturval-log-collector | Collecting logs from stdout pods, Kubernetes audit, journald |
The Vector configuration consists of sources, transforms, sinks. More details: service card shturval-log-collector.
Scheme of interaction of collection components
Log storage component (VictoriaLogs)
| Component | Description |
|---|---|
| VictoriaLogs | Log storage. For each cluster - a separate instance in the cluster namespace |
Logs are stored in the management cluster, and an instance of VictoriaLogs is created in the namespace of each cluster in the management cluster.
Deployment management
| Component | Purpose |
|---|---|
| shturval-metrics-collector | Manages the deployment of VictoriaLogs instances |
“Logs” tab in the GUI
Display conditions and working with filters are in the section How to view logs in the platform.
Pre-configured ClusterVectorPipeline
Pipeline table, example spec and recommendations - Collecting and processing logs (ClusterVectorPipeline).
Log rotation
The storage period and disk limits are set in the custom resource VLSingle (VictoriaLogs) in the cluster namespace. Parameters, UI path and example - How to configure log rotation.
Types of logs collected
- Authentication (shturval-auth);
- Backend (shturval-backend);
- Audit;
- System components;
- User load applications.