obmp-portability-test/docs/design.md

Design: Two-VM Portability Test

Goal

Prove that obmp-docker deploys and ingests on a fresh host with only the inputs a new operator would have (repo URL, a hypervisor, an SSH key). The existing deploy.sh claims portability; this test is what turns the claim into evidence.

Topology

   +------------------------+           BMP TCP :ROUTER_FACING_PORT
   |     store VM (VM-A)    |  <----------------------------------+
   |  ubuntu 24.04 cloud    |                                     |
   |  docker + deploy.sh    |                                     |
   |  full-stack profile    |                                     |
   |                        |                                     |
   |  ROUTER_FACING_IP =    |                                     |
   |    <VM-A ip>           |                                     |
   |  ROUTER_FACING_PORT =  |                          +----------+---------+
   |    1790 (IANA BMP)     |                          |  bmpgen VM (VM-B) |
   |                        |                          |  ubuntu 24.04     |
   +------------------------+                          |  docker           |
                                                       |  gobgp container  |
                                                       |  (BMP client)     |
                                                       +-------------------+

           |                                                     |
           +---- shared L2 (Proxmox bridge) or routed subnet ----+

Two VMs, both stock Ubuntu 24.04 cloud images, both provisioned entirely by cloud-init. No manual step between terraform apply and the assertions running.

Why two VMs, not one

The deploy.sh --scope full-stack --profile test option already spins up GoBGP inside the same compose network as the collector. That proves the compose is internally consistent, but it doesn't exercise the actual router-integration surface (ROUTER_FACING_IP/ROUTER_FACING_PORT, compose's ${ROUTER_FACING_PORT:-1790}:5000 publish, cross-host TCP, management-plane reachability). A second VM is the smallest topology that makes those things real.

If the two-VM test passes, "deploy.sh works AND routers can reach it" holds. If it fails, we know exactly which of the two.

What VM-A runs

Cloud-init on VM-A does:

1. Install docker.io, docker-compose-plugin, git.
2. git clone the obmp-docker repo at a pinned branch/commit.
3. Write a minimal .env from .env.example with:
   • HOST_IP = VM-A's own primary IP
   • ROUTER_FACING_IP = same
   • ROUTER_FACING_PORT = 1790 (test uses IANA port, not the lab transitional 5000)
   • OBMP_AUTH_MODE=local (no reverse proxy in this topology)
4. Run ./deploy.sh --scope full-stack --auth local --yes.
5. Write a completion marker to /var/lib/cloud/portability-test.done with exit status so the test harness can poll it.

What VM-B runs

Cloud-init on VM-B does:

1. Install docker.io.
2. docker run a GoBGP container with a config file that:
   • Sets local AS to a private test ASN (65099).
   • Establishes a BMP session to <VM-A ip>:1790.
   • Injects a small synthetic route set so ip_rib gets non-zero rows.
3. Write a completion marker.

GoBGP is minimal on purpose — a real router would test more, but each extra dial-in path (BGP-LS, gNMI, NETCONF) needs a real IOS-XR environment or an emulator like XRd. The minimum route-monitoring path is enough to catch the portability failure modes.

Assertion set (run by scripts/run-test.sh)

Once cloud-init on both VMs is done (poll the completion markers, or 15 min timeout), SSH into VM-A and run:

#	Assertion	Query / check
1	All obmp containers healthy	docker ps --format '{{.Names}} {{.Status}}' grep unhealthy = 0
2	Collector accepted a BMP session from VM-B	docker logs obmp-collector contains "peer up" for VM-B's IP
3	routers table has 1 row	SELECT count(*) FROM routers = 1
4	bgp_peers has ≥ 1 UP session	SELECT count(*) FROM bgp_peers WHERE state='UP' ≥ 1
5	ip_rib has ≥ 1 row (GoBGP injected some routes)	SELECT count(*) FROM ip_rib ≥ 1
6	Grafana healthcheck returns 200	curl -sf http://<VM-A>:3000/api/health

Pass = all six green. Failure emits the first failing assertion + a tarball of docker logs, .env, deploy.sh output for diagnosis.

Why bpg/proxmox

Two candidate providers exist for Proxmox: Telmate/proxmox (original, slower moving) and bpg/proxmox (fork, active, modern cloud-init handling). We pick bpg because:

• Cloud-init user_data_file_id is first-class — we can hand-craft the cloud-init YAML and reference it by snippet ID rather than shoving it through the provider's per-field API.
• proxmox_virtual_environment_download_file handles the cloud-image download idempotently.
• The type system + docs are noticeably better on 0.60+.

Why cloud-init and not Ansible

Adding Ansible would double the surface. Cloud-init is enough for "install a few packages, drop a repo, run a script" — anything more complex is a smell that we're testing the wrong thing.

Why a separate repo

Test infra should not gate product commits. If the two-VM test suite grows (multi-vendor router VMs, prod-like tuning tests, longer-running soak variants), it earns its own release cadence. Keeping the code separate keeps that door open.