obmp-portability-test/docs/design.md

# Design: Two-VM Portability Test

## Goal

Prove that `obmp-docker` deploys and ingests on a fresh host with only the
inputs a new operator would have (repo URL, a hypervisor, an SSH key). The
existing `deploy.sh` claims portability; this test is what turns the claim
into evidence.

## Topology

```
   +------------------------+           BMP TCP :ROUTER_FACING_PORT
   |     store VM (VM-A)    |  <----------------------------------+
   |  ubuntu 24.04 cloud    |                                     |
   |  docker + deploy.sh    |                                     |
   |  full-stack profile    |                                     |
   |                        |                                     |
   |  ROUTER_FACING_IP =    |                                     |
   |    <VM-A ip>           |                                     |
   |  ROUTER_FACING_PORT =  |                          +----------+---------+
   |    1790 (IANA BMP)     |                          |  bmpgen VM (VM-B) |
   |                        |                          |  ubuntu 24.04     |
   +------------------------+                          |  docker           |
                                                       |  gobgp container  |
                                                       |  (BMP client)     |
                                                       +-------------------+

           |                                                     |
           +---- shared L2 (Proxmox bridge) or routed subnet ----+
```

Two VMs, both stock Ubuntu 24.04 cloud images, both provisioned entirely
by cloud-init. No manual step between `terraform apply` and the
assertions running.

## Why two VMs, not one

The `deploy.sh --scope full-stack --profile test` option already spins up
GoBGP *inside* the same compose network as the collector. That proves the
compose is internally consistent, but it doesn't exercise the actual
router-integration surface (`ROUTER_FACING_IP`/`ROUTER_FACING_PORT`,
compose's `${ROUTER_FACING_PORT:-1790}:5000` publish, cross-host TCP,
management-plane reachability). A second VM is the smallest topology that
makes those things real.

If the two-VM test passes, "deploy.sh works AND routers can reach it"
holds. If it fails, we know exactly which of the two.

## What VM-A runs

Cloud-init on VM-A does:

1. Install `docker.io`, `docker-compose-plugin`, `git`.
2. `git clone` the obmp-docker repo at a pinned branch/commit.
3. Write a minimal `.env` from `.env.example` with:
   - `HOST_IP` = VM-A's own primary IP
   - `ROUTER_FACING_IP` = same
   - `ROUTER_FACING_PORT` = 1790 (test uses IANA port, not the lab
     transitional 5000)
   - `OBMP_AUTH_MODE=local` (no reverse proxy in this topology)
4. Run `./deploy.sh --scope full-stack --auth local --yes`.
5. Write a completion marker to `/var/lib/cloud/portability-test.done`
   with exit status so the test harness can poll it.

## What VM-B runs

Cloud-init on VM-B does:

1. Install `docker.io`.
2. `docker run` a GoBGP container with a config file that:
   - Sets local AS to a private test ASN (65099).
   - Establishes a BMP session to `<VM-A ip>:1790`.
   - Injects a small synthetic route set so `ip_rib` gets non-zero rows.
3. Write a completion marker.

GoBGP is minimal on purpose — a real router would test more, but each
extra dial-in path (BGP-LS, gNMI, NETCONF) needs a real IOS-XR
environment or an emulator like XRd. The minimum route-monitoring path
is enough to catch the portability failure modes.

## Assertion set (run by `scripts/run-test.sh`)

Once cloud-init on both VMs is done (poll the completion markers, or 15
min timeout), SSH into VM-A and run:

| # | Assertion                                                        | Query / check                                                   |
|---|------------------------------------------------------------------|-----------------------------------------------------------------|
| 1 | All obmp containers healthy                                      | `docker ps --format '{{.Names}} {{.Status}}'` grep unhealthy = 0 |
| 2 | Collector accepted a BMP session from VM-B                       | `docker logs obmp-collector` contains "peer up" for VM-B's IP    |
| 3 | `routers` table has 1 row                                        | `SELECT count(*) FROM routers` = 1                              |
| 4 | `bgp_peers` has ≥ 1 UP session                                   | `SELECT count(*) FROM bgp_peers WHERE state='UP'` ≥ 1           |
| 5 | `ip_rib` has ≥ 1 row (GoBGP injected some routes)                | `SELECT count(*) FROM ip_rib` ≥ 1                               |
| 6 | Grafana healthcheck returns 200                                  | `curl -sf http://<VM-A>:3000/api/health`                        |

Pass = all six green. Failure emits the first failing assertion + a
tarball of `docker logs`, `.env`, `deploy.sh` output for diagnosis.

## Why bpg/proxmox

Two candidate providers exist for Proxmox: `Telmate/proxmox` (original,
slower moving) and `bpg/proxmox` (fork, active, modern cloud-init
handling). We pick bpg because:

- Cloud-init `user_data_file_id` is first-class — we can hand-craft the
  cloud-init YAML and reference it by snippet ID rather than shoving it
  through the provider's per-field API.
- `proxmox_virtual_environment_download_file` handles the cloud-image
  download idempotently.
- The type system + docs are noticeably better on 0.60+.

## Why cloud-init and not Ansible

Adding Ansible would double the surface. Cloud-init is enough for
"install a few packages, drop a repo, run a script" — anything more
complex is a smell that we're testing the wrong thing.

## Why a separate repo

Test infra should not gate product commits. If the two-VM test suite
grows (multi-vendor router VMs, prod-like tuning tests, longer-running
soak variants), it earns its own release cadence. Keeping the code
separate keeps that door open.