Architecture#
High-level#
From a high-level perspective, KOF consists of three layers:
- the Collection layer, where the statistics and events are gathered,
- the Regional layer, which includes storage to keep track of those statistics and events,
- and the Management layer, where you interact through the UI.
flowchart TD
A[Management UI, promxy]
A --> C[Storage Region 1]
A --> D[Storage Region 2]
C --> E[Collect Child 1]
C --> F[Collect Child 2]
D ==> G[...]
Mid-level#
Getting a little bit more detailed, it's important to undrestand that data flows upwards, from observed objects to centralized Grafana on the Management layer:
Management Cluster
kof-operators chart
grafana-operator
opentelemetry-operator
prometheus-operator-crds
kof-mothership chart
victoria-metrics-operator
cluster-api-visualizer
sveltos-dashboard
dex
k0rdent service templates
kof-dashboards
kof-operator
promxy
kof-collectors chart
opencost
opentelemetry-kube-stack
Either kof-istio
Certificates
ClusterProfiles
Or kof-regional and kof-child
MultiClusterServices
Cloud 1..N
Region 1..M
Regional Cluster
kof-operators chart
grafana-operator
opentelemetry-operator
prometheus-operator-crds
kof-storage chart
victoria-metrics-operator
victoria-logs-cluster
external-dns
jaeger-operator
dex
kof-dashboards
kof-collectors chart
opencost
opentelemetry-kube-stack
cert-manager
ingress-nginx
istio/gateway
kof-istio chart
cert-manager-istio-csr
istio/base
istiod
Child Cluster 1
cert-manager
Optional kof-istio
kof-operators chart
Disabled grafana-operator
opentelemetry-operator
prometheus-operator-crds
kof-collectors chart
opencost
opentelemetry-kube-stack
observed objects
Helm Charts#
KOF is deployed as a series of Helm charts at various levels.
kof-operators#
- Grafana dashboards platform, managed by grafana-operator
- OpenTelemetry collectors below, managed by opentelemetry-operator
- prometheus-operator-crds required to create OpenTelemetry collectors, also required to monitor
kof-mothership
itself
kof-mothership#
- Local VictoriaMetrics storage for alerting rules only, managed by victoria-metrics-operator
- cluster-api-visualizer for insight into multicluster configuration
- Sveltos dashboard, automatic secret distribution
- Dex SSO chart
- k0rdent service templates used by
kof-regional
andkof-child
charts - kof-dashboards for Grafana
- kof-operator (don't confuse it with the
kof-operators
chart) for auto-configuration - Promxy for aggregating Prometheus metrics from regional clusters
kof-regional#
- MultiClusterService
which configures and installs
kof-storage
and other charts to regional clusters
kof-child#
- MultiClusterService
which configures and installs
kof-collectors
and other charts to child clusters
kof-istio#
- Optional Istio support for secure connectivity between clusters without external DNS
kof-storage#
- Regional VictoriaMetrics storage with main data, managed by victoria-metrics-operator
- vmauth entrypoint proxy for VictoriaMetrics components
- vmcluster for high-available fault-tolerant version of VictoriaMetrics database
- victoria-logs-cluster for high-performance, cost-effective, scalable logs storage
- external-dns to communicate with other clusters
- Jaeger tracing platform, managed by jaeger-operator
- Dex SSO chart
- kof-dashboards for Grafana
kof-collectors#
- opentelemetry-kube-stack for hardware, OS, and Kubernetes metrics
- OpenCost "shines a light into the black box of Kubernetes spend"