Skip to content

KOF Retention and Replication#

KOF stores metrics in VictoriaMetrics, logs in VictoriaLogs, and traces in VictoriaTraces.

Recommendations#

Configure the storage class, space, retention, and replication to balance cost, durability, and compliance, for example:

  • Management cluster: short-term retention (1–30 days)
  • Regional clusters: long-term retention (30–365 days)
  • Increase replicationFactor where higher availability is required; this field enables you to determine how many copies are stored, usually on different nodes.

Review the next guides to make informed decision:

Examples of values#

Metrics#

victoriametrics:
  vmcluster:
    spec:
      retentionPeriod: "30d"
      replicationFactor: 2
      vmstorage:
        resources:
          limits:
            cpu: "8"
            memory: 8000Mi
        storage:
          volumeClaimTemplate:
            spec:
              storageClassName: <EXAMPLE_STORAGE_CLASS>
              resources:
                requests:
                  storage: "100Gi"

Details: VMClusterSpec

Logs#

victoria-logs-cluster:
  vlstorage:
    extraArgs:
      retentionPeriod: "30d"
    persistentVolume:
      storageClassName: <EXAMPLE_STORAGE_CLASS>
      size: "100Gi"

Details: Values of victoria-logs-cluster

Audit Logs#

KOF collects Kubernetes audit logs by default and stores them in a dedicated VictoriaLogs cluster (vlcluster_audit), separate from the regular logs cluster, to allow a longer retention period (default: 1y).

To change the retention period, storage class, or volume size, adjust the following in kof-storage values:

victoriametrics:
  vlcluster_audit:
    enabled: true
    spec:
      vlstorage:
        retentionPeriod: "1y"
        storage:
          volumeClaimTemplate:
            spec:
              storageClassName: <EXAMPLE_STORAGE_CLASS>
              resources:
                requests:
                  storage: "100Gi"

Details: VLClusterSpec

To disable audit log storage, set vlcluster_audit.enabled: false in kof-storage values.

To disable audit log collection, update the kof-collectors values to remove the filelog/k8s_audit receiver from the daemon collector pipeline:

opentelemetry-kube-stack:
  collectors:
    daemon:
      config:
        service:
          pipelines:
            logs:
              receivers:
                - otlp
                - receiver_creator
                - filelog/syslog
                - journald

Traces#

victoria-traces-cluster:
  vtstorage:
    extraArgs:
      retentionPeriod: "30d"
    persistentVolume:
      storageClassName: <EXAMPLE_STORAGE_CLASS>
      size: "100Gi"

Details: Values of victoria-traces-cluster

Places to customize#

There are multiple places where the storage can be customized using these values.

All regional clusters at once#

Add these values to the kof-values.yaml file under the kof-regional.values.storage key as described in the step 5 of the Management Cluster section.

Specific regional cluster#

Add these values to the regional ClusterDeployment as the spec.config.clusterAnnotations."k0rdent.mirantis.com/kof-storage-values" described in the step 10 of the Regional Cluster section.

Regionless setup#

Add these values to the placeholders in the Regionless section.

Management to Management case#

Add these values to the placeholders in the From Management to Management section.