diff --git a/gitops/README.md b/gitops/README.md index 6eaacf0..e657534 100644 --- a/gitops/README.md +++ b/gitops/README.md @@ -8,7 +8,8 @@ This directory holds the **Helm chart** that replaces `docker stack deploy` from |------|---------| | `charts/onelab` | Helm chart (StatefulSets, Deployments, Services, ConfigMaps, Secrets) | | `values/*.yaml` | Environment-specific overrides (non-secret defaults; use sealed/external secrets for prod) | -| `argocd/application.yaml` | Example `Application` — set `repoURL` / `targetRevision` to your remote | +| `argocd/application.yaml` | `Application` (multi-source): OneLab chart + [`observability/`](observability/) (Loki/Promtail/Grafana) | +| `observability/` | Umbrella Helm chart for log aggregation (same Argo app, release `onelab-obs`) | ## Prerequisites @@ -36,8 +37,14 @@ helm upgrade --install onelab . -n onelab --create-namespace \ 2. Edit `argocd/application.yaml`: `repoURL`, `targetRevision`, and values file as needed. 3. `kubectl apply -f gitops/argocd/application.yaml` (from a machine with a working kubeconfig). +The Application uses **`spec.sources`** (Argo CD 2.6+): source 1 is the OneLab chart (`releaseName: onelab`), source 2 is [`observability/`](observability/) (`releaseName: onelab-obs`). Both deploy to namespace **`onelab`**. + Sync waves order Postgres → Redis/Rabbit/config → application pods. +### Logs / Grafana + +See [docs/OBSERVABILITY.md](docs/OBSERVABILITY.md). Change `grafana.adminPassword` in `observability/values.yaml` before relying on it in production. + ## kubectl / credentials If `kubectl` reports *You must be logged in*, refresh your kubeconfig (e.g. copy `/etc/rancher/k3s/k3s.yaml` from the server or re-run your auth plugin) before applying manifests. @@ -50,6 +57,15 @@ See [docs/BOOTSTRAP.md](docs/BOOTSTRAP.md) for Argo CD access to `git.luneski.fr Helm 3.19 may return empty content for `.Files.Get` on Windows; this chart uses `fromYaml (.Files.AsConfig)` as a workaround so packaged files still render correctly. +## Application configuration (`configurations.yml`) + +Do **not** need to edit `app/configurations.yml` in Git for Kubernetes. The chart builds `configurations.yml` from `charts/onelab/files/configurations.gotmpl` and stores it in Secret **`onelab-configurations`** (mounted by app pods and `ldap-worker`). + +1. **Values (recommended)** — set `onelab.compliance.enabled`, `onelab.ldap.enabled`, and related fields. See `values/instance-overrides.example.yaml`. Point Helm/Argo at an extra values file for your site (Argo: add another path under `spec.source.helm.valueFiles`, relative to the chart directory). +2. **Bring your own Secret** — set `configuration.existingSecretName` to a Secret you manage (SealedSecrets, External Secrets, `kubectl create secret ... --from-file=configurations.yml=...`). The chart will **not** create `onelab-configurations` in that case; the Secret must contain key **`configurations.yml`**. + +A **ConfigMap** alone is fine if you mount it yourself, but this chart expects a **Secret** for the config file (same as Swarm-style sensitivity). LDAP TLS file paths in values are container paths; mount PEMs with extra volumes on `ldap-worker` if you use them. + ## Ingress (web UI) Enable `ingress.enabled` and set `ingress.host` (and optional TLS). Traffic is sent to Service **`revproxy`** (internal nginx). On k3s, `ingress.className: traefik` matches the default controller. diff --git a/gitops/argocd/application.yaml b/gitops/argocd/application.yaml index 50b5ae7..198861f 100644 --- a/gitops/argocd/application.yaml +++ b/gitops/argocd/application.yaml @@ -1,4 +1,5 @@ -# Syncs chart from Git; ensure Argo CD can clone repoURL (add credentials in Argo if private). +# Syncs OneLab app + observability (Loki/Promtail/Grafana) into namespace onelab. +# Requires Argo CD 2.6+ (spec.sources). Ensure repoURL matches your remote. apiVersion: argoproj.io/v1alpha1 kind: Application metadata: @@ -8,13 +9,21 @@ metadata: - resources-finalizer.argocd.argoproj.io spec: project: default - source: - repoURL: https://git.luneski.fr/luneski/onelab-k8s.git - targetRevision: main - path: gitops/charts/onelab - helm: - valueFiles: - - ../../values/k3s-example.yaml + sources: + - repoURL: https://git.luneski.fr/luneski/onelab-k8s.git + targetRevision: main + path: gitops/charts/onelab + helm: + releaseName: onelab + valueFiles: + - ../../values/k3s-example.yaml + - repoURL: https://git.luneski.fr/luneski/onelab-k8s.git + targetRevision: main + path: gitops/observability + helm: + releaseName: onelab-obs + valueFiles: + - values.yaml destination: server: https://kubernetes.default.svc namespace: onelab diff --git a/gitops/charts/onelab/files/configurations.gotmpl b/gitops/charts/onelab/files/configurations.gotmpl index 3235418..bc7e798 100644 --- a/gitops/charts/onelab/files/configurations.gotmpl +++ b/gitops/charts/onelab/files/configurations.gotmpl @@ -2,6 +2,7 @@ onelab: domain: {{ .Values.onelab.domain | quote }} logs: + path: "/logs" level: info assets: purge: 1d @@ -41,6 +42,15 @@ onelab: remember_me: true lab: creation_policy: many +{{- if .Values.onelab.compliance.enabled }} + compliance: + require_electronic_signature: {{ .Values.onelab.compliance.requireElectronicSignature }} + execution_operator_restriction_policy: {{ .Values.onelab.compliance.executionOperatorRestrictionPolicy | quote }} + execution_admin_expert_restriction_policy: {{ .Values.onelab.compliance.executionAdminExpertRestrictionPolicy | quote }} + prevent_csv_import: {{ .Values.onelab.compliance.preventCsvImport }} + prevent_manual_metadata_edit: {{ .Values.onelab.compliance.preventManualMetadataEdit }} + device_restart: {{ .Values.onelab.compliance.deviceRestart }} +{{- end }} signup: false {{- if .Values.onelab.intercom.appid }} intercom: @@ -56,7 +66,39 @@ onelab: maxtries: 3 timeout: 60 ldap: - enabled: {{ .Values.features.ldapWorker }} + enabled: {{ if or .Values.onelab.ldap.enabled .Values.features.ldapWorker }}true{{ else }}false{{ end }} +{{- if or .Values.onelab.ldap.enabled .Values.features.ldapWorker }} + {{- if .Values.onelab.ldap.timeout }} + timeout: {{ .Values.onelab.ldap.timeout | int }} + {{- end }} + {{- if .Values.onelab.ldap.encryption }} + encryption: {{ .Values.onelab.ldap.encryption | quote }} + {{- end }} + {{- if .Values.onelab.ldap.policy }} + policy: {{ .Values.onelab.ldap.policy | quote }} + {{- end }} + {{- if kindIs "bool" .Values.onelab.ldap.verifyCertificates }} + verify_certificates: {{ .Values.onelab.ldap.verifyCertificates }} + {{- end }} + {{- if or .Values.onelab.ldap.tlsCaPath .Values.onelab.ldap.tlsCertPath .Values.onelab.ldap.tlsKeyPath .Values.onelab.ldap.tlsCiphers .Values.onelab.ldap.tlsSslVersion }} + tls: + {{- if .Values.onelab.ldap.tlsCaPath }} + ca: {{ .Values.onelab.ldap.tlsCaPath | quote }} + {{- end }} + {{- if .Values.onelab.ldap.tlsCertPath }} + cert: {{ .Values.onelab.ldap.tlsCertPath | quote }} + {{- end }} + {{- if .Values.onelab.ldap.tlsKeyPath }} + key: {{ .Values.onelab.ldap.tlsKeyPath | quote }} + {{- end }} + {{- if .Values.onelab.ldap.tlsCiphers }} + ciphers: {{ .Values.onelab.ldap.tlsCiphers | quote }} + {{- end }} + {{- if .Values.onelab.ldap.tlsSslVersion }} + ssl_version: {{ .Values.onelab.ldap.tlsSslVersion | quote }} + {{- end }} + {{- end }} +{{- end }} services: db: host: db diff --git a/gitops/charts/onelab/templates/_helpers.tpl b/gitops/charts/onelab/templates/_helpers.tpl index 19334ec..a538910 100644 --- a/gitops/charts/onelab/templates/_helpers.tpl +++ b/gitops/charts/onelab/templates/_helpers.tpl @@ -31,3 +31,7 @@ app.kubernetes.io/version: {{ .Chart.AppVersion | quote }} {{- $entry := dict "username" $user "password" $pass "auth" $auth -}} {{- dict "auths" (dict $server $entry) | toJson -}} {{- end }} + +{{- define "onelab.configurationSecretName" -}} +{{- .Values.configuration.existingSecretName | default "onelab-configurations" }} +{{- end }} diff --git a/gitops/charts/onelab/templates/deployment-optional-workers.yaml b/gitops/charts/onelab/templates/deployment-optional-workers.yaml index 13e2bf2..1a7fadd 100644 --- a/gitops/charts/onelab/templates/deployment-optional-workers.yaml +++ b/gitops/charts/onelab/templates/deployment-optional-workers.yaml @@ -1,5 +1,5 @@ {{- $root := . }} -{{- if .Values.features.ldapWorker }} +{{- if or .Values.onelab.ldap.enabled .Values.features.ldapWorker }} --- apiVersion: apps/v1 kind: Deployment @@ -43,7 +43,7 @@ spec: volumes: - name: configurations secret: - secretName: onelab-configurations + secretName: {{ include "onelab.configurationSecretName" $root }} {{- if eq $root.Values.persistence.mode "hostPath" }} - name: logs hostPath: @@ -98,7 +98,7 @@ spec: volumes: - name: configurations secret: - secretName: onelab-configurations + secretName: {{ include "onelab.configurationSecretName" $root }} {{- if eq $root.Values.persistence.mode "hostPath" }} - name: logs hostPath: diff --git a/gitops/charts/onelab/templates/secret-configurations.yaml b/gitops/charts/onelab/templates/secret-configurations.yaml index 6523394..631c903 100644 --- a/gitops/charts/onelab/templates/secret-configurations.yaml +++ b/gitops/charts/onelab/templates/secret-configurations.yaml @@ -1,3 +1,4 @@ +{{- if not .Values.configuration.existingSecretName }} {{- $cfg := fromYaml (.Files.AsConfig) }} apiVersion: v1 kind: Secret @@ -11,3 +12,4 @@ type: Opaque stringData: configurations.yml: | {{- tpl (index $cfg "configurations.gotmpl") . | nindent 4 }} +{{- end }} diff --git a/gitops/charts/onelab/templates/workloads.yaml b/gitops/charts/onelab/templates/workloads.yaml index 3e20a01..a1e684d 100644 --- a/gitops/charts/onelab/templates/workloads.yaml +++ b/gitops/charts/onelab/templates/workloads.yaml @@ -76,7 +76,7 @@ spec: {{- if .config }} - name: configurations secret: - secretName: onelab-configurations + secretName: {{ include "onelab.configurationSecretName" $root }} {{- end }} {{- if eq $root.Values.persistence.mode "hostPath" }} {{- if has "logs" .mounts }} diff --git a/gitops/charts/onelab/values.yaml b/gitops/charts/onelab/values.yaml index 34f5e35..30aec72 100644 --- a/gitops/charts/onelab/values.yaml +++ b/gitops/charts/onelab/values.yaml @@ -3,6 +3,11 @@ nameOverride: "" fullnameOverride: "" +# If non-empty, workloads mount this Secret instead of chart-generated onelab-configurations. +# Secret must contain key `configurations.yml`. Chart will NOT create onelab-configurations. +configuration: + existingSecretName: "" + images: registry: hub.andrewalliance.com/releases tag: "1.27.0" @@ -71,11 +76,32 @@ onelab: authTokenKey: "TokenAuthPlaceholder" monitoringToken: "TokenMonitoringPlaceholder" rabbitToken: "TokenRabbitPlaceholder" + # Mirrors app/configurations.yml params.compliance (enable without editing app/). + compliance: + enabled: false + requireElectronicSignature: true + executionOperatorRestrictionPolicy: "reviewed" + executionAdminExpertRestrictionPolicy: "reviewed" + preventCsvImport: true + preventManualMetadataEdit: true + deviceRestart: true + # Set enabled: true to turn on LDAP in configurations.yml and deploy ldap-worker (or use features.ldapWorker). + ldap: + enabled: false + timeout: "" + encryption: "" + policy: "" + tlsCaPath: "" + tlsCertPath: "" + tlsKeyPath: "" + tlsCiphers: "" + tlsSslVersion: "" intercom: appid: "zxvgsagz" secret: "QUw2jEV8utIpe9DeYjOqBjhBY9VxjXddKUCISUNu" features: + # Deprecated for LDAP: prefer onelab.ldap.enabled (either enables ldap-worker + ldap.enabled in config). ldapWorker: false mailerWorker: false diff --git a/gitops/docs/BOOTSTRAP.md b/gitops/docs/BOOTSTRAP.md index 19746eb..f8b722a 100644 --- a/gitops/docs/BOOTSTRAP.md +++ b/gitops/docs/BOOTSTRAP.md @@ -47,3 +47,9 @@ kubectl apply -f gitops/argocd/application.yaml ## 3. RabbitMQ TLS Secret `onelab-rabbit-tls` must exist before RabbitMQ starts (created once from `app/rabbit/ssl/` or your own PEMs). + +## 4. Argo CD version + observability stack + +`gitops/argocd/application.yaml` uses **`spec.sources`** (two Helm charts in one Application). Use **Argo CD 2.6 or newer**. + +The second source installs Loki/Promtail/Grafana from `gitops/observability/` (`releaseName: onelab-obs`). Set a strong **`grafana.adminPassword`** in `gitops/observability/values.yaml` before production. Details: [OBSERVABILITY.md](OBSERVABILITY.md). diff --git a/gitops/docs/OBSERVABILITY.md b/gitops/docs/OBSERVABILITY.md new file mode 100644 index 0000000..eb797be --- /dev/null +++ b/gitops/docs/OBSERVABILITY.md @@ -0,0 +1,44 @@ +# Observability (Loki / Promtail / Grafana) + +The umbrella chart under [`gitops/observability/`](../observability/) deploys: + +- **Loki** — log storage (SingleBinary, filesystem PVC, 7-day retention by default). +- **Promtail** — DaemonSet: Kubernetes pod logs (`/var/log/pods`) plus **OneLab file logs** from the same host path the app chart uses (`/opt/onelab/logs` by default). +- **Grafana** — explore logs; datasource points at this release’s Loki gateway. + +It is synced by the **same** Argo CD Application as the OneLab chart ([`gitops/argocd/application.yaml`](../argocd/application.yaml)): second `sources` entry, Helm release name **`onelab-obs`** (so services are like `onelab-obs-loki-gateway`). + +## First-time setup + +1. **Change the Grafana admin password** in [`gitops/observability/values.yaml`](../observability/values.yaml) (`grafana.adminPassword`) or switch to `admin.existingSecret` per the upstream Grafana chart. +2. **Align host paths** — if you change `persistence.hostPath.logs` for OneLab, update `promtail.extraVolumes` / `extraVolumeMounts` in the same `values.yaml` so Promtail still reads the shared log directory. +3. **Multi-node** — with `hostPath` logs, each node only sees its own files; Promtail runs on every node, so you still get coverage when pods move. + +## Access Grafana + +An **Ingress** is enabled by default (Traefik + cert-manager), matching the OneLab web UI pattern in `gitops/values/k3s-example.yaml`: + +- Host: **`grafana.k8s.selair.it`** (edit in `gitops/observability/values.yaml` alongside `grafana.ini.server` `domain` / `root_url`). +- TLS Secret: **`grafana-tls-k8s-selair`** (cert-manager with `letsencrypt-prod`). + +Point DNS at your ingress, sync the app, then open `https:///` (user `admin` until you change values). + +For debugging without DNS: + +```bash +kubectl -n onelab port-forward svc/onelab-obs-grafana 3000:80 +``` + +## Upgrading chart dependencies + +From `gitops/observability/`: + +```bash +helm dependency update +``` + +Commit updated `Chart.lock` and `charts/*.tgz` if you want Argo to render without calling remote Helm repos at sync time. + +## OneLab `logs.path` + +The OneLab chart now sets `onelab.logs.path: "/logs"` in the generated configuration so application file logs match the `/logs` volume mount (see Enterprise guide §7.2). diff --git a/gitops/observability/Chart.lock b/gitops/observability/Chart.lock new file mode 100644 index 0000000..287aaae --- /dev/null +++ b/gitops/observability/Chart.lock @@ -0,0 +1,12 @@ +dependencies: +- name: loki + repository: https://grafana.github.io/helm-charts + version: 6.55.0 +- name: promtail + repository: https://grafana.github.io/helm-charts + version: 6.17.1 +- name: grafana + repository: https://grafana.github.io/helm-charts + version: 10.5.15 +digest: sha256:5b34192a8db9d940587777fbc62a13503c21217da814308654ce73fca2ed5d56 +generated: "2026-03-20T11:06:47.9376325+01:00" diff --git a/gitops/observability/Chart.yaml b/gitops/observability/Chart.yaml new file mode 100644 index 0000000..599dee9 --- /dev/null +++ b/gitops/observability/Chart.yaml @@ -0,0 +1,16 @@ +apiVersion: v2 +name: onelab-observability +description: Loki + Promtail + Grafana for OneLab (same Argo Application as app chart via multi-source). +type: application +version: 0.1.0 +appVersion: "1.0" +dependencies: + - name: loki + version: 6.55.0 + repository: https://grafana.github.io/helm-charts + - name: promtail + version: 6.17.1 + repository: https://grafana.github.io/helm-charts + - name: grafana + version: 10.5.15 + repository: https://grafana.github.io/helm-charts diff --git a/gitops/observability/README.md b/gitops/observability/README.md new file mode 100644 index 0000000..2904905 --- /dev/null +++ b/gitops/observability/README.md @@ -0,0 +1,7 @@ +# OneLab observability (Helm umbrella) + +Loki + Promtail + Grafana dependencies are pinned in `Chart.lock`; packaged charts live in `charts/*.tgz`. + +Deployed by Argo CD as the second `sources` entry in `gitops/argocd/application.yaml` with **`releaseName: onelab-obs`**. + +See [../docs/OBSERVABILITY.md](../docs/OBSERVABILITY.md) for operations and security notes. diff --git a/gitops/observability/charts/grafana-10.5.15.tgz b/gitops/observability/charts/grafana-10.5.15.tgz new file mode 100644 index 0000000..057cb67 Binary files /dev/null and b/gitops/observability/charts/grafana-10.5.15.tgz differ diff --git a/gitops/observability/charts/loki-6.55.0.tgz b/gitops/observability/charts/loki-6.55.0.tgz new file mode 100644 index 0000000..d8b8ab2 Binary files /dev/null and b/gitops/observability/charts/loki-6.55.0.tgz differ diff --git a/gitops/observability/charts/promtail-6.17.1.tgz b/gitops/observability/charts/promtail-6.17.1.tgz new file mode 100644 index 0000000..5af8374 Binary files /dev/null and b/gitops/observability/charts/promtail-6.17.1.tgz differ diff --git a/gitops/observability/values.yaml b/gitops/observability/values.yaml new file mode 100644 index 0000000..178b2d4 --- /dev/null +++ b/gitops/observability/values.yaml @@ -0,0 +1,128 @@ +# Umbrella chart: Loki (SingleBinary + filesystem) + Promtail + Grafana. +# Keep hostPath below in sync with persistence.hostPath.logs in gitops/values/k3s-example.yaml. + +loki: + deploymentMode: SingleBinary + loki: + auth_enabled: false + commonConfig: + replication_factor: 1 + storage: + type: filesystem + schemaConfig: + configs: + - from: "2024-04-01" + store: tsdb + object_store: filesystem + schema: v13 + index: + prefix: loki_index_ + period: 24h + limits_config: + retention_period: 168h + ingestion_rate_mb: 16 + ingestion_burst_size_mb: 32 + singleBinary: + replicas: 1 + persistence: + enabled: true + size: 10Gi + backend: + replicas: 0 + read: + replicas: 0 + write: + replicas: 0 + ingester: + replicas: 0 + querier: + replicas: 0 + queryFrontend: + replicas: 0 + queryScheduler: + replicas: 0 + distributor: + replicas: 0 + compactor: + replicas: 0 + indexGateway: + replicas: 0 + bloomCompactor: + replicas: 0 + bloomGateway: + replicas: 0 + ruler: + replicas: 0 + minio: + enabled: false + lokiCanary: + enabled: false + test: + enabled: false + chunksCache: + enabled: false + resultsCache: + enabled: false + +promtail: + config: + clients: + - url: http://{{ .Release.Name }}-loki-gateway.{{ .Release.Namespace }}.svc.cluster.local/loki/api/v1/push + snippets: + extraScrapeConfigs: | + - job_name: onelab-host-log-files + static_configs: + - targets: + - localhost + labels: + job: onelab-files + __path__: /onelab-host-logs/**/* + extraVolumes: + - name: onelab-host-logs + hostPath: + path: /opt/onelab/logs + type: DirectoryOrCreate + extraVolumeMounts: + - name: onelab-host-logs + mountPath: /onelab-host-logs + readOnly: true + +# Grafana Ingress: align host/TLS with gitops/values/k3s-example.yaml ingress (Traefik + cert-manager). +grafana: + adminUser: admin + adminPassword: changeme + persistence: + enabled: true + size: 2Gi + service: + type: ClusterIP + # Required when served behind Ingress (redirects, OAuth callbacks). + grafana.ini: + server: + domain: grafana.k8s.selair.it + root_url: https://grafana.k8s.selair.it/ + ingress: + enabled: true + ingressClassName: traefik + annotations: + cert-manager.io/cluster-issuer: letsencrypt-prod + hosts: + - grafana.k8s.selair.it + path: / + pathType: Prefix + tls: + - secretName: grafana-tls-k8s-selair + hosts: + - grafana.k8s.selair.it + datasources: + datasources.yaml: + apiVersion: 1 + datasources: + - name: Loki + type: loki + uid: loki + url: http://{{ .Release.Name }}-loki-gateway.{{ .Release.Namespace }}.svc.cluster.local + access: proxy + isDefault: true + jsonData: + maxLines: 1000 diff --git a/gitops/values/instance-overrides.example.yaml b/gitops/values/instance-overrides.example.yaml new file mode 100644 index 0000000..c07207a --- /dev/null +++ b/gitops/values/instance-overrides.example.yaml @@ -0,0 +1,34 @@ +# Copy to a private file (e.g. gitops/values/private-k3s.yaml, gitignored) or merge into your env values. +# Reference from Helm: -f ../../values/k3s-example.yaml -f ../../values/private-k3s.yaml +# Argo CD: add a second entry under helm.valueFiles (paths relative to chart path). + +onelab: + compliance: + enabled: true + # Optional tweaks (defaults match chart values.yaml): + # requireElectronicSignature: true + # executionOperatorRestrictionPolicy: "reviewed" + # executionAdminExpertRestrictionPolicy: "reviewed" + # preventCsvImport: true + # preventManualMetadataEdit: true + # deviceRestart: true + + ldap: + enabled: true + # timeout: 30 + # encryption: "start_tls" + # policy: "your-policy" + # verifyCertificates: true + # Paths inside the ldap-worker container (mount certs via extraVolumes if needed): + # tlsCaPath: "/ldap/ca.crt" + # tlsCertPath: "/ldap/client.crt" + # tlsKeyPath: "/ldap/client.key" + # tlsCiphers: "" + # tlsSslVersion: "" + +# Alternative: supply the full YAML yourself (no Helm templating of compliance/LDAP blocks). +# 1. kubectl create secret generic onelab-configurations-custom -n onelab \ +# --from-file=configurations.yml=./my-configurations.yml +# 2. Set in values: +# configuration: +# existingSecretName: onelab-configurations-custom