Exporting security logs to external systems

Integration with external security monitoring systems.

Introduction

To integrate with SIEM, Local log collection module (Vector, shturval-log-collector) must be enabled in the cluster. Log redirection is configured via ShturvalServicePatch, which applies to the service specification shturval-log-collector; The general procedure for working with custom values is described in the section How to manage CustomValues of services.

Vector uses:

sources — data sources;
transforms — data transformation and filtering;
sinks — recipients to which logs are sent.

What logs can be sent

In customvalues.shturval specify the types of logs that need to be redirected. Supported:

auditd — system audit logs;
journald — logs containerd, kubelet, shturvald;
k8s_audit — Kubernetes API audit logs;
k8s_events — Kubernetes events;
k8s_logs — container logs.

The shturval block must list all log types that require redirection. For each log type, set recipients to sinks. If you need to change records before sending, set transforms.

General setup procedure

Prepare ShturvalServicePatch for shturval-log-collector.
In customvalues.shturval specify the log types, transforms if necessary and sinks with connection parameters.
Upload the manifest to the cluster via import manifests.
Wait for the changes to apply.

Sending to SIEM via socket

In this example, k8s_audit, journald, and k8s_logs are sent to the SIEM. To change the list of sent logs, remove the unnecessary block with the log type from the shturval block.

For releases 2.11-2.14, the inputs values in the supplied platform-logs sink did not change: k8s_audit and journald use critical_level, and k8s_logs uses post_transforms_system_namespaces.

apiVersion: ops.shturval.tech/v1beta2
kind: ShturvalServicePatch
metadata:
  name: <resource name>
spec:
  shturvalServiceConfigName: shturval-log-collector
  customvalues:
    shturval:
      k8s_audit:
        enable: true
        sinks:
          siem:
            type: socket
            inputs:
              - critical_level
            address: "<your parameter value>"
            mode: <your parameter value>
            encoding:
              codec: json
      journald:
        sinks:
          siem:
            type: socket
            inputs:
              - critical_level
            address: "<your parameter value>"
            mode: <your parameter value>
            encoding:
              codec: json
      k8s_logs:
        transforms:
          filter_backend:
            condition: |
              .kubernetes.pod_namespace == "shturval-backend" &&
              (
                (.kubernetes.container_name == "opa" && starts_with(string!(.kubernetes.pod_name), "shturval-backend-")) ||
                (.kubernetes.container_name == "authn" && starts_with(string!(.kubernetes.pod_name), "authn-"))
              )              
            inputs:
              - post_transforms_system_namespaces
            type: filter
        sinks:
          siem:
            type: socket
            inputs:
              - filter_backend
            address: "<your parameter value>"
            mode: <your parameter value>
            encoding:
              codec: json

Parameter	Description	Data type	Example
`metadata.name`	Name `ShturvalServicePatch`	`string`	`shturval-log-collector-siem`
`address`	SIEM socket: IP address and port	`string`	`10.11.12.13:514`
`mode`	Connection protocol	`string`	`udp`

Vector supports confirmation of data delivery. In case of incorrect delivery, a re-send is performed, which reduces the impact of the UDP protocol features.

Send to Loki

Prepare ShturvalServicePatch for application to the ssc Local Log Collection Module (shturval-log-collector) specification. In customvalues add the shturval block with the required parameters.

apiVersion: ops.shturval.tech/v1beta2
kind: ShturvalServicePatch
metadata:
  name: shturval-log-collector-logs-to-loki
spec:
  shturvalServiceConfigName: shturval-log-collector
  customvalues:
    shturval:
      platform_logs_enable: true
      k8s_logs:
        enable: true
        sinks:
          loki_external:
            type: loki
            inputs:
              - post_transforms_system_namespaces
            endpoint: <your parameter value>
            path: /loki/api/v1/push
            compression: snappy
            tenant_id: "<your parameter value>"
            encoding:
              codec: json
            tls:
              verify_certificate: false
            labels:
              __tenant_id__: "<your parameter value>"
              resourceRealm: "<your parameter value>"
              log_type: "kubernetes"
      k8s_audit:
        enable: true
        sinks:
          loki_external:
            type: loki
            inputs:
              - critical_level
            endpoint: <your parameter value>
            path: /loki/api/v1/push
            compression: snappy
            tenant_id: "<your parameter value>"
            encoding:
              codec: json
            tls:
              verify_certificate: false
            labels:
              __tenant_id__: "<your parameter value>"
              resourceRealm: "<your parameter value>"
              log_type: "k8s_audit"
      k8s_events:
        enable: true
        sinks:
          loki_external:
            type: loki
            inputs:
              - critical_level
            endpoint: <your parameter value>
            path: /loki/api/v1/push
            compression: snappy
            tenant_id: "<your parameter value>"
            encoding:
              codec: json
            tls:
              verify_certificate: false
            labels:
              __tenant_id__: "<your parameter value>"
              resourceRealm: "<your parameter value>"
              log_type: "k8s_events"
      journald:
        enable: true
        sinks:
          loki_external:
            type: loki
            inputs:
              - critical_level
            endpoint: <your parameter value>
            path: /loki/api/v1/push
            compression: snappy
            tenant_id: "<your parameter value>"
            encoding:
              codec: json
            tls:
              verify_certificate: false
            labels:
              __tenant_id__: "<your parameter value>"
              resourceRealm: "<your parameter value>"
              log_type: "journald"
      auditd:
        enable: true
        sinks:
          loki_external:
            type: loki
            inputs:
              - auditd
            endpoint: <your parameter value>
            path: /loki/api/v1/push
            compression: snappy
            tenant_id: "<your parameter value>"
            encoding:
              codec: json
            tls:
              verify_certificate: false
            labels:
              __tenant_id__: "<your parameter value>"
              resourceRealm: "<your parameter value>"
              log_type: "auditd"

Parameter	Description	Data type	Example
`metadata.name`	Name `ShturvalServicePatch`	`string`	`shturval-log-collector-logs-to-loki`
`platform_logs_enable`	Enables sending platform logs	`boolean`	`true`
`<log-type>.enable`	Enables sendingof the selected log type: `auditd`, `journald`, `k8s_audit`, `k8s_events`, `k8s_logs`	`boolean`	`true`
`loki_external.type`	Data recipient type	`string`	`loki`
`loki_external.inputs`	List of log sources or transformations to send to Loki	`array`	`post_transforms_system_namespaces`
`loki_external.endpoint`	Loki URL	`string`	`https://loki.example.com`
`loki_external.path`	API path for sending logs to Loki	`string`	`/loki/api/v1/push`
`loki_external.compression`	Data compression algorithm before sending	`string`	`snappy`
`loki_external.tenant_id`	Tenant ID in Loki	`string`	`tenant-example`
`loki_external.encoding.codec`	Log encoding format	`string`	`json`
`loki_external.tls.verify_certificate`	Verifying TLS certificate upon connection	`boolean`	`false`
`loki_external.labels.__tenant_id__`	Tenant label for log entries	`string`	`tenant-example`
`loki_external.labels.resourceRealm`	Label of the environment or cluster from which logs are sent	`string`	`cluster-example`
`loki_external.labels.log_type`	Log Type Label	`string`	`kubernetes`

In the example above, the following logs will be sent to Loki: auditd, journald, k8s_audit, k8s_events and k8s_logs. To change the list of sent logs, remove the unnecessary block with the log type from shturval.

Load the prepared ShturvalServicePatch into the cluster via import manifests. After downloading, it will take some time for the changes to take effect.

Posting to Elasticsearch or OpenSearch

General template

apiVersion: ops.shturval.tech/v1beta2
kind: ShturvalServicePatch
metadata:
  name: <resource name>
spec:
  shturvalServiceConfigName: shturval-log-collector
  customvalues:
    shturval:
      <your value>: # Type of logs to redirect
        enable: true
        transforms: {}
        sinks:
          elastic_logs:
            type: <your parameter value>
            auth:
              user: <your parameter value>
              password: <your parameter value>
              strategy: <your parameter value>
            endpoints:
              - <your parameter value>
            tls:
              verify_certificate: true
            api_version: <your parameter value>
            buffer:
              max_size: <your parameter value>
              type: <your parameter value>

Parameter	Description	Data type	Example
`metadata.name`	Name `ShturvalServicePatch`	`string`	`shturval-log-collector-logs`
`elastic_logs.type`	Data recipient type	`string`	`elasticsearch`
`elastic_logs.auth.user`	Account name to connect to Elasticsearch	`string`	`username`
`elastic_logs.auth.password`	Account password for connecting to Elasticsearch	`string`	`1234qwe`
`elastic_logs.auth.strategy`	Elasticsearch Authentication Strategy	`string`	`basic`
`elastic_logs.endpoints`	List of URLs to connect to	`array`	`https://elastic.bank.local:9200/`
`elastic_logs.api_version`	Elasticsearch API Version	`string`	`v8`
`elastic_logs.buffer.max_size`	Buffer size for storing logs before sending	`int`	`268435488`
`elastic_logs.buffer.type`	Buffer type for storing logs before sending	`string`	`disk`

audit

apiVersion: ops.shturval.tech/v1beta2
kind: ShturvalServicePatch
metadata:
  name: shturval-log-collector-auditd
spec:
  shturvalServiceConfigName: shturval-log-collector
  customvalues:
    shturval:
      auditd:
        sinks:
          elastic_logs:
            type: elasticsearch
            inputs:
              - auditd
            bulk:
              index: "{{ .Values.CLUSTER_NAME }}_auditd-%Y.%m.%d"
            auth:
              user: my_username
              password: "my_password"
              strategy: basic
            endpoints:
              - "https://external-elastic.address:9200"
            tls:
              verify_certificate: false
            api_version: "v8"
            buffer:
              max_size: 268435488
              type: disk

k8s_audit

apiVersion: ops.shturval.tech/v1beta2
kind: ShturvalServicePatch
metadata:
  name: shturval-log-collector-kaudit
spec:
  shturvalServiceConfigName: shturval-log-collector
  customvalues:
    shturval:
      k8s_audit:
        sinks:
          elastic_logs:
            type: elasticsearch
            inputs:
              - critical_level
            bulk:
              index: "{{ .Values.CLUSTER_NAME }}_kaudit-%Y.%m.%d"
            auth:
              user: my_username
              password: "my_password"
              strategy: basic
            endpoints:
              - "https://external-elastic.address:9200"
            tls:
              verify_certificate: false
            api_version: "v8"
            buffer:
              max_size: 268435488
              type: disk

k8s_events

apiVersion: ops.shturval.tech/v1beta2
kind: ShturvalServicePatch
metadata:
  name: shturval-log-collector-kevents
spec:
  shturvalServiceConfigName: shturval-log-collector
  customvalues:
    shturval:
      k8s_events:
        sinks:
          elastic_logs:
            type: elasticsearch
            inputs:
              - critical_level
            bulk:
              index: "{{ .Values.CLUSTER_NAME }}_events-%Y.%m.%d"
            auth:
              user: my_username
              password: "my_password"
              strategy: basic
            endpoints:
              - "https://external-elastic.address:9200"
            tls:
              verify_certificate: false
            api_version: "v8"
            buffer:
              max_size: 268435488
              type: disk

k8s_logs

apiVersion: ops.shturval.tech/v1beta2
kind: ShturvalServicePatch
metadata:
  name: shturval-log-collector-klogs
spec:
  shturvalServiceConfigName: shturval-log-collector
  customvalues:
    shturval:
      k8s_logs:
        sinks:
          elastic_logs:
            type: elasticsearch
            inputs:
              - post_transforms_system_namespaces
            bulk:
              index: "{{ .Values.CLUSTER_NAME }}_logs-%Y.%m.%d"
            auth:
              user: my_username
              password: "my_password"
              strategy: basic
            endpoints:
              - "https://external-elastic.address:9200"
            tls:
              verify_certificate: false
            api_version: "v8"
            buffer:
              max_size: 268435488
              type: disk

journald

apiVersion: ops.shturval.tech/v1beta2
kind: ShturvalServicePatch
metadata:
  name: shturval-log-collector-journald
spec:
  shturvalServiceConfigName: shturval-log-collector
  customvalues:
    shturval:
      journald:
        sinks:
          elastic_logs:
            type: elasticsearch
            inputs:
              - critical_level
            bulk:
              index: "{{ .Values.CLUSTER_NAME }}_journald-%Y.%m.%d"
            auth:
              user: my_username
              password: "my_password"
              strategy: basic
            endpoints:
              - "https://external-elastic.address:9200"
            tls:
              verify_certificate: false
            api_version: "v8"
            buffer:
              max_size: 268435488
              type: disk

Sending to syslog

To redirect logs to syslog, prepare ShturvalServicePatch for shturval-log-collector and add the shturval block with the required parameters to customvalues.

apiVersion: ops.shturval.tech/v1beta2
kind: ShturvalServicePatch
metadata:
  name: shturval-log-collector-syslog
spec:
  shturvalServiceConfigName: shturval-log-collector
  customvalues:
    shturval:
      k8s_audit:
        sinks:
          syslog:
            type: socket
            inputs:
              - critical_level
            address: <your parameter value>
            mode: <your parameter value>

Parameter	Description	Data type	Example
`k8s_audit.sinks.syslog.inputs`	Transform name in the `k8s-audit` pipeline; in the supplied values, this is `critical_level`	`array`	`critical_level`
`k8s_audit.sinks.syslog.address`	Syslog server address	`string`	`11.12.13.14:5000`
`k8s_audit.sinks.syslog.mode`	Transfer protocol type	`string`	`tcp`

The example uses socket. Other ways to configure redirection to the syslog server are described in Vector documentation.

Loading manifest

Load the prepared ShturvalServicePatch into the cluster via import manifests.

Screenshot: import ShturvalServicePatch for SIEM

Screenshot: importing ShturvalServicePatch for log redirection

Screenshot: import ShturvalServicePatch for syslog

Once ShturvalServicePatch is loaded, it will take some time for the changes to take effect.

Application check

To check that ShturvalServicePatch is applied:

In the cluster, open the Services and Repositories section.
Go to the Installed services page.
Find Local log collection module (shturval-log-collector) and open service management.
Check the Service tab: after applying the patch, the status should update to Patched.
Check the Applied ShturvalServicePatch tab: the list should show the loaded ShturvalServicePatch.

Screenshots: applied ShturvalServicePatch

Exporting security logs to external systems

Introduction

What logs can be sent

General setup procedure

Sending to SIEM via socket

Send to Loki

Posting to Elasticsearch or OpenSearch

General template

audit

k8s_audit

k8s_events

k8s_logs

journald

Sending to syslog

Loading manifest

Application check

See also

Text feedback

Exporting security logs to external systems

Introduction #

What logs can be sent #

General setup procedure #

Sending to SIEM via socket #

Send to Loki #

Posting to Elasticsearch or OpenSearch #

General template #

audit #

k8s_audit #

k8s_events #

k8s_logs #

journald #

Sending to syslog #

Loading manifest #

Application check #

See also #

Introduction

What logs can be sent

General setup procedure

Sending to SIEM via socket

Send to Loki

Posting to Elasticsearch or OpenSearch

General template

audit

k8s_audit

k8s_events

k8s_logs

journald

Sending to syslog

Loading manifest

Application check

See also