Seccomp

Seccomp profiles for Kubernetes pods and containers, hosting profiles on nodes via NCI, and securely implementing syscall restrictions.

General information #

Seccomp (Secure computing mode) is a Linux kernel security mechanism that limits system calls from user space to the kernel. In Kubernetes, Seccomp profiles are applied to pods and containers to reduce the attack surface: a process inside a container receives only those kernel capabilities that are allowed by the selected profile.

A Seccomp profile is a JSON file in a format compatible with the OCI runtime specification. It specifies the default action, lists of system calls, and, if necessary, conditions on system call arguments.

Basic profile actions:

• SCMP_ACT_ALLOW — allow the call (white list).
• SCMP_ACT_ERRNO - return an error to the process for a blocked call.
• SCMP_ACT_LOG — allow the call, but record it in the log (audit).
• SCMP_ACT_KILL_PROCESS — kill the process.
• SCMP_ACT_KILL_THREAD or SCMP_ACT_KILL — terminate the thread.
• SCMP_ACT_TRAP — send the SIGSYS signal to the process.
• SCMP_ACT_TRACE — notify the tracing process.

For universal profiles, SCMP_ACT_LOG, SCMP_ACT_ERRNO and SCMP_ACT_ALLOW are usually used.

The defaultAction parameter determines what to do with system calls that are not included in any rules. For example:

• SCMP_ACT_LOG is suitable for auditing: calls are executed but recorded in the log.
• SCMP_ACT_ERRNO is suitable for a hard profile: unauthorized calls are blocked.

Blocking and audit logs are looked at on the node where the Pod is running, in /var/log/messages or /var/log/syslog by the keyword SECCOMP or by process name.

How Kubernetes applies Seccomp #

The profile is specified in securityContext.seccompProfile. It can be set:

• for the entire Pod: spec.securityContext.seccompProfile;
• for a separate container: spec.containers[*].securityContext.seccompProfile;
• for the init container: spec.initContainers[*].securityContext.seccompProfile;
• for an ephemeral container: spec.ephemeralContainers[*].securityContext.seccompProfile.

The container level profile takes precedence over the Pod level profile. If a container does not set its seccompProfile, it inherits the Pod profile.

Available values for seccompProfile.type:

You cannot apply the Seccomp profile to a Pod or container with privileged: true: privileged containers always run as Unconfined.

In new manifests, use securityContext.seccompProfile fields. The Seccomp annotations used in older versions of Kubernetes are no longer needed for the current configuration.

Basic option: RuntimeDefault #

For most loads, it’s worth checking the RuntimeDefault profile first. It includes the restrictions set by the container runtime and does not require a separate JSON file to be placed on the nodes.

apiVersion: v1
kind: Pod
metadata:
  name: runtime-default-pod
spec:
  securityContext:
    seccompProfile:
      type: RuntimeDefault
  containers:
  - name: app
    image: hashicorp/http-echo:1.0
    args:
    - "-text=hello"
    securityContext:
      allowPrivilegeEscalation: false

If seccompDefault is enabled in the kubelet configuration or the kubelet is launched with the --seccomp-default flag, a Pod without an explicit profile will receive RuntimeDefault. If this behavior is not enabled, a Pod without seccompProfile runs as Unconfined.

To implement RuntimeDefault by default, enable the setting step by step: first on some nodes, then check the launch and main application scenarios. Some workloads may require less stringent restrictions. For such cases, use one of the options: explicitly specify Unconfined, prepare a separate Localhost profile, or run the workload on nodes without seccompDefault.

You can check the actually applied profile using container runtime, for example through crictl inspect on the node.

Creating a local profile via NCI #

For a custom profile, create a JSON file and place it on Linux nodes. In the Shturval platform, this is done using the NCI (NodeConfigItem) resource. More information about creating and using NCI can be found on the Node Configuration page.

You can create an NCI by importing a manifest or in the GUI: Administration → Node Configuration. When creating an object, select kubernetes.io/os: linux in the node selector, select Files and directories in the custom sections, create a directory and place the JSON profile file in it.

Below is an example NCI that creates a directory /var/lib/kubelet/seccomp/ and a file /var/lib/kubelet/seccomp/myprofile.json.

apiVersion: node.shturval.tech/v1beta2
kind: NodeConfigItem
metadata:
  name: seccomp-profile
spec:
  files:
  - path: /var/lib/kubelet/seccomp/
    type: directory
    owner: root
    group: root
    mode: "0755"
  - path: /var/lib/kubelet/seccomp/myprofile.json
    type: file
    owner: root
    group: root
    mode: "0644"
    content: |-
      {
        "defaultAction": "SCMP_ACT_LOG",
        "architectures": ["SCMP_ARCH_X86_64", "SCMP_ARCH_X86", "SCMP_ARCH_X32"],
        "syscalls": [
          {
            "names": ["arch_prctl", "sched_yield", "futex", "write", "mmap", "exit_group", "madvise", "rt_sigprocmask", "getpid", "gettid", "tgkill", "rt_sigaction", "read", "getpgrp"],
            "action": "SCMP_ACT_ALLOW"
          },
          {
            "names": ["add_key", "keyctl", "ptrace"],
            "action": "SCMP_ACT_ERRNO"
          }
        ]
      }            
  nodeconfigselector:
    kubernetes.io/os: linux
  priority: 100

After applying NCI, check that the profile appears on the required nodes:

kubectl get nci seccomp-profile -o yaml
kubectl get nodes --selector kubernetes.io/os=linux

If necessary, check the file directly on the node using OS administration tools. Kubernetes does not store the contents of the Localhost profile in the Pod: when creating a container, runtime looks for the file on the node where the Pod was assigned.

Running workloads with the Seccomp profile #

To apply a profile,specify type: Localhost and the file name in localhostProfile. The value of localhostProfile is set relative to /var/lib/kubelet/seccomp.

If the file is located as /var/lib/kubelet/seccomp/myprofile.json, specify:

securityContext:
  seccompProfile:
    type: Localhost
    localhostProfile: myprofile.json

If the file is located in a subdirectory of /var/lib/kubelet/seccomp/profiles/myprofile.json, specify localhostProfile: profiles/myprofile.json.

Example Pod with a local profile at the container level:

apiVersion: v1
kind: Pod
metadata:
  name: nettool
  namespace: namespacename
  labels:
    run: nettool
spec:
  containers:
  - name: nettool
    image: r.shturval.tech/network-multitool:alpine-extra
    securityContext:
      seccompProfile:
        type: Localhost
        localhostProfile: myprofile.json

Example Pod, where the general profile is set at the Pod level and the individual container overrides it:

apiVersion: v1
kind: Pod
metadata:
  name: seccomp-priority
spec:
  securityContext:
    seccompProfile:
      type: RuntimeDefault
  containers:
  - name: app
    image: hashicorp/http-echo:1.0
    args:
    - "-text=hello"
  - name: custom
    image: hashicorp/http-echo:1.0
    args:
    - "-text=custom"
    securityContext:
      seccompProfile:
        type: Localhost
        localhostProfile: myprofile.json

In the GUI, when creating or editing a Pod or Deployment, the profile is configured in the Security context block at the Pod or container level. For a local profile, choose to use a custom profile and provide a file name.

Creating a profile based on audit results #

For application containers, it’s safer to move from observing to constraining:

1. Create an audit profile with defaultAction: SCMP_ACT_LOG.
2. Run the workload with this profile.
3. Execute basic user and service application scripts.
4. Collect system calls from node logs.
5. Create a profile with defaultAction: SCMP_ACT_ERRNO and allow only necessary calls through SCMP_ACT_ALLOW.
6. Check startup, readiness/liveness tests, network calls, file handling, graceful shutdown and application version update.

Minimum audit profile:

{
  "defaultAction": "SCMP_ACT_LOG"
}

An example of a strict profile that by default blocks calls and allows only those listed:

{
  "defaultAction": "SCMP_ACT_ERRNO",
  "defaultErrnoRet": 38,
  "architectures": [
    "SCMP_ARCH_X86_64",
    "SCMP_ARCH_X86",
    "SCMP_ARCH_X32"
  ],
  "syscalls": [
    {
      "names": [
        "accept4",
        "bind",
        "brk",
        "close",
        "epoll_ctl",
        "epoll_wait",
        "exit",
        "exit_group",
        "futex",
        "getpid",
        "gettid",
        "mmap",
        "munmap",
        "openat",
        "read",
        "rt_sigaction",
        "rt_sigprocmask",
        "socket",
        "write",
        "writev"
      ],
      "action": "SCMP_ACT_ALLOW"
    }
  ]
}

This approach requires testing on a specific application. Audit logs do not guarantee that a single test run will capture all the system calls that an application might make in rare scenarios.

Diagnostics #

kubectl describe pod <pod-name> -n <namespace>
kubectl get pod <pod-name> -n <namespace> -o yaml
kubectl get events -n <namespace> --field-selector involvedObject.name=<pod-name>

For local profiles, additionally check which node the Pod is assigned to:

kubectl get pod <pod-name> -n <namespace> -o wide

Recommendations #

1. Start with RuntimeDefault. This is a basic security hardening without the accompanying JSON profile.
2. Use Localhost for critical and well-studied workloads. A custom profile is useful when you need to explicitly limit the set of system calls for a particular application.
3. Go from auditing to blocking. First collect system calls with SCMP_ACT_LOG, then move on to SCMP_ACT_ERRNO and SCMP_ACT_ALLOW.
4. Set the profile at the Pod level if it is common to all containers. Override the profile at the container level only for exceptions.
5. Don’t use Unconfined without reason. This mode disables Seccomp limits for the load.
6. Don’t rely on Seccomp for privileged containers. A container with privileged: true always works like Unconfined.
7. Test the profile on the full application scenario. A profile that is too strict can block legitimate calls after the container has successfully started.
8. Monitor the same profiles on nodes. For Localhost, the file must be available on every node that the Pod can access.
9. Be aware of runtime differences. The RuntimeDefault profile and support for individual Seccomp actions may differ between container runtimes and their versions.