obmp-docker/docs/PORTABILITY-FINDINGS.md

Portability findings — what breaks on a greenfield deploy

Every artifact hit while standing this stack up on a fresh host (WSL2 NWE-LT02, Ubuntu 24.04, native docker-ce 29.x — 2026-07), ordered roughly as encountered. Each one is something a clean deploy has to survive; most are now handled automatically by deploy.sh/setup.sh. Kept as a checklist for the next new host and as the rationale for the deploy tooling.

1. Bind-mounted volumes need pre-existing source dirs

docker-compose.yml binds ${OBMP_DATA_ROOT}/postgres/{data,ts} with type:none,o:bind. Docker will not create these; if absent, Postgres fails with no such file or directory. setup.sh creates them — the error only appears when setup.sh wasn't run on the target. deploy.sh double-checks the dirs exist after setup.sh.

2. Stale HOST_IP survives a copied .env

A .env copied from another host carries that host's HOST_IP; setup.sh validation only checks non-empty/not-"changeme", so the stale IP passes and gets rendered into gobgpd.conf and the Grafana root URL. This is the core reason deploy.sh exists — it reconciles HOST_IP (and warns when the .env value isn't a local address) before setup.sh renders anything.

3. A populated .env is not a deployed host

The same copied .env had late-stage artifacts (Authelia secrets) while early-stage host state (Postgres dirs) was missing — proof that setup.sh completed on a different host. Run setup.sh per target, always.

4. WSL: HOST_IP is the NAT address; routers can't reach it

hostname -I in WSL returns the VM's NAT address — it changes on wsl --shutdown and physical routers cannot reach it. Routers must target the Windows LAN IP, forwarded into WSL via netsh portproxy (scripts/wsl-portproxy.ps1). deploy.sh separates "router-facing IP:port" from HOST_IP and auto-detects the Windows LAN IP via powershell.exe interop.

5. Cold-start contention masquerades as component bugs

Bringing the whole stack up at once crash-looped Kafka on a preflight "writable" check that was actually resource contention, not permissions. deploy.sh stages the bring-up: infra (zookeeper/kafka) -> psql -> core -> feeders.

6. Size for BMP burst, not steady state

Generic docs say the collector is light — true, and misleading: the store saturates under burst (full table x monitored sessions, Postgres write amplification). See DEPLOYMENT-TYPES.md for the full model.

7. Ports 5000 and 3000 are crowded

The collector's 5000 and Grafana's 3000 are commonly already occupied. deploy.sh runs a port-collision preflight before touching anything. The router-facing BMP port default moved 5000 -> 1790 (the IANA BMP port); the collector still listens on 5000 inside the container.

8. Docker Desktop breaks host bind mounts on WSL (the big one)

Symptom: Postgres mount fails no such file or directory on a path ls shows exists. Cause: Docker Desktop runs the daemon in its own WSL distro (docker-desktop VM), so host bind mounts resolve against that filesystem, not the distro you ran compose from. No mkdir fixes it. Fix: native docker-in-WSL — disable Desktop's WSL integration, install docker-ce, systemctl enable --now docker, and confirm docker info | grep 'Operating System' does not say Docker Desktop. deploy.sh detects Docker Desktop and warns/refuses.

8b. Desktop residue splits the mount namespace

After installing native docker, the CLI couldn't reach /run/docker.sock even though dockerd held it — a stale mount-namespace split left by Desktop. wsl --shutdown and reopening the distro cleared it.

9. Grafana customizations must live in the repo, not grafana.db

Dashboards hand-built in the UI exist only in that host's grafana.db and vanish on a greenfield deploy. All dashboards are now committed as JSON under obmp-grafana/dashboards/ and setup.sh syncs both provisioning YAML and dashboard JSONs to the data root. If you edit a dashboard in the UI, export and commit the JSON.

10. Blanket chmod -R 777 on the data root breaks Postgres re-deploys

setup.sh's lab-permissive chmod over an existing data tree made psql_server.key world-accessible; Postgres refuses to boot (FATAL: private key file ... has group or world access) and the whole core cascades down. Fresh installs never hit it (the key is generated after the chmod) — it is strictly a re-deploy bug. setup.sh now skips postgres/ (the image entrypoint owns those perms) and re-tightens the key/cert to 0600 if a previous run already clobbered them.

11. docker-ce 29 rejects Docker API < 1.40 — old pinned clients break

telegraf 1.28's [[inputs.docker]] hardcodes Docker API version 1.24, which docker-ce >= 29 refuses (client version 1.24 is too old). Every per-container stack metric silently never reached InfluxDB; the stack dashboards sat empty while disk/postgresql_* inputs worked fine. DOCKER_API_VERSION env is ignored (version pinned in code) — the fix is telegraf >= 1.30, which negotiates the API version (telegraf/Dockerfile now pins 1.33). Watch for the same failure in any other tooling that talks to the daemon with an old vendored client.

12. Router-side automation targeted the old lab's values

cml/proxmox_bmp_config.py sent routers at HOST_IP port 5000, and cml/xrd-node-definition.yaml baked in 10.40.40.202:5000 — both correct on the old native-Linux dev lab, wrong on a WSL host (finding 4: routers must target the router-facing address). Resolved 2026-07: the router-facing port is 1790 everywhere; compose publishes ${ROUTER_FACING_PORT:-1790}:5000, setup.sh renders the same port into gobgp's BMP export (gobgp is host-networked and must hit the published port), and the cml tooling reads ROUTER_FACING_IP/ROUTER_FACING_PORT from .env. Remaining hands-on step: re-apply the config to routers still pointed at 5000. See router-bmp-config.md.