Files

timotheereausanofi b91c35c410 gitops: observability stack (Loki/Promtail/Grafana), Grafana Ingress, Argo multi-source

- Add gitops/observability umbrella chart with vendored Helm deps
- Grafana Ingress: Traefik, letsencrypt-prod, grafana.k8s.selair.it + root_url
- Argo Application: spec.sources (onelab + onelab-obs)
- OneLab: configuration secret override, compliance/LDAP values, logs.path /logs
- Docs: OBSERVABILITY, BOOTSTRAP, README, instance-overrides example

Made-with: Cursor

2026-03-20 11:10:06 +01:00

2.3 KiB

Raw Blame History

Observability (Loki / Promtail / Grafana)

The umbrella chart under gitops/observability/ deploys:

Loki — log storage (SingleBinary, filesystem PVC, 7-day retention by default).
Promtail — DaemonSet: Kubernetes pod logs (/var/log/pods) plus OneLab file logs from the same host path the app chart uses (/opt/onelab/logs by default).
Grafana — explore logs; datasource points at this release’s Loki gateway.

It is synced by the same Argo CD Application as the OneLab chart (gitops/argocd/application.yaml): second sources entry, Helm release name onelab-obs (so services are like onelab-obs-loki-gateway).

First-time setup

Change the Grafana admin password in gitops/observability/values.yaml (grafana.adminPassword) or switch to admin.existingSecret per the upstream Grafana chart.
Align host paths — if you change persistence.hostPath.logs for OneLab, update promtail.extraVolumes / extraVolumeMounts in the same values.yaml so Promtail still reads the shared log directory.
Multi-node — with hostPath logs, each node only sees its own files; Promtail runs on every node, so you still get coverage when pods move.

Access Grafana

An Ingress is enabled by default (Traefik + cert-manager), matching the OneLab web UI pattern in gitops/values/k3s-example.yaml:

Host: grafana.k8s.selair.it (edit in gitops/observability/values.yaml alongside grafana.ini.server domain / root_url).
TLS Secret: grafana-tls-k8s-selair (cert-manager with letsencrypt-prod).

Point DNS at your ingress, sync the app, then open https://<grafana-host>/ (user admin until you change values).

For debugging without DNS:

kubectl -n onelab port-forward svc/onelab-obs-grafana 3000:80

Upgrading chart dependencies

From gitops/observability/:

helm dependency update

Commit updated Chart.lock and charts/*.tgz if you want Argo to render without calling remote Helm repos at sync time.

OneLab `logs.path`

The OneLab chart now sets onelab.logs.path: "/logs" in the generated configuration so application file logs match the /logs volume mount (see Enterprise guide §7.2).

2.3 KiB Raw Blame History Unescape Escape