sam
d30ec2074a
Terraform (bpg/proxmox) provisions two Ubuntu 24.04 cloud-init VMs on a Proxmox host, one running obmp-docker's `deploy.sh --scope full-stack` and the other running GoBGP as a BMP-source stand-in. `scripts/run-test.sh` runs the six assertions from docs/design.md (containers healthy, BMP peer up, ip_rib/bgp_peers non-zero, Grafana healthcheck). Layout: terraform/ provider + VM + cloud-init snippet resources terraform/cloud-init/ user-data templates (store + bmpgen) scripts/ run-test.sh, teardown.sh docs/design.md topology, assertion set, and rationale Cred handling: terraform.tfvars is gitignored; endpoint + api_token can also come from PROXMOX_VE_ENDPOINT / PROXMOX_VE_API_TOKEN env vars. Not validated locally: `terraform init && terraform validate` -- the terraform binary is not on the authoring host. YAML + shell + HCL brace balance verified. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
124 lines
5.8 KiB
Markdown
124 lines
5.8 KiB
Markdown
# Design: Two-VM Portability Test
|
|
|
|
## Goal
|
|
|
|
Prove that `obmp-docker` deploys and ingests on a fresh host with only the
|
|
inputs a new operator would have (repo URL, a hypervisor, an SSH key). The
|
|
existing `deploy.sh` claims portability; this test is what turns the claim
|
|
into evidence.
|
|
|
|
## Topology
|
|
|
|
```
|
|
+------------------------+ BMP TCP :ROUTER_FACING_PORT
|
|
| store VM (VM-A) | <----------------------------------+
|
|
| ubuntu 24.04 cloud | |
|
|
| docker + deploy.sh | |
|
|
| full-stack profile | |
|
|
| | |
|
|
| ROUTER_FACING_IP = | |
|
|
| <VM-A ip> | |
|
|
| ROUTER_FACING_PORT = | +----------+---------+
|
|
| 1790 (IANA BMP) | | bmpgen VM (VM-B) |
|
|
| | | ubuntu 24.04 |
|
|
+------------------------+ | docker |
|
|
| gobgp container |
|
|
| (BMP client) |
|
|
+-------------------+
|
|
|
|
| |
|
|
+---- shared L2 (Proxmox bridge) or routed subnet ----+
|
|
```
|
|
|
|
Two VMs, both stock Ubuntu 24.04 cloud images, both provisioned entirely
|
|
by cloud-init. No manual step between `terraform apply` and the
|
|
assertions running.
|
|
|
|
## Why two VMs, not one
|
|
|
|
The `deploy.sh --scope full-stack --profile test` option already spins up
|
|
GoBGP *inside* the same compose network as the collector. That proves the
|
|
compose is internally consistent, but it doesn't exercise the actual
|
|
router-integration surface (`ROUTER_FACING_IP`/`ROUTER_FACING_PORT`,
|
|
compose's `${ROUTER_FACING_PORT:-1790}:5000` publish, cross-host TCP,
|
|
management-plane reachability). A second VM is the smallest topology that
|
|
makes those things real.
|
|
|
|
If the two-VM test passes, "deploy.sh works AND routers can reach it"
|
|
holds. If it fails, we know exactly which of the two.
|
|
|
|
## What VM-A runs
|
|
|
|
Cloud-init on VM-A does:
|
|
|
|
1. Install `docker.io`, `docker-compose-plugin`, `git`.
|
|
2. `git clone` the obmp-docker repo at a pinned branch/commit.
|
|
3. Write a minimal `.env` from `.env.example` with:
|
|
- `HOST_IP` = VM-A's own primary IP
|
|
- `ROUTER_FACING_IP` = same
|
|
- `ROUTER_FACING_PORT` = 1790 (test uses IANA port, not the lab
|
|
transitional 5000)
|
|
- `OBMP_AUTH_MODE=local` (no reverse proxy in this topology)
|
|
4. Run `./deploy.sh --scope full-stack --auth local --yes`.
|
|
5. Write a completion marker to `/var/lib/cloud/portability-test.done`
|
|
with exit status so the test harness can poll it.
|
|
|
|
## What VM-B runs
|
|
|
|
Cloud-init on VM-B does:
|
|
|
|
1. Install `docker.io`.
|
|
2. `docker run` a GoBGP container with a config file that:
|
|
- Sets local AS to a private test ASN (65099).
|
|
- Establishes a BMP session to `<VM-A ip>:1790`.
|
|
- Injects a small synthetic route set so `ip_rib` gets non-zero rows.
|
|
3. Write a completion marker.
|
|
|
|
GoBGP is minimal on purpose — a real router would test more, but each
|
|
extra dial-in path (BGP-LS, gNMI, NETCONF) needs a real IOS-XR
|
|
environment or an emulator like XRd. The minimum route-monitoring path
|
|
is enough to catch the portability failure modes.
|
|
|
|
## Assertion set (run by `scripts/run-test.sh`)
|
|
|
|
Once cloud-init on both VMs is done (poll the completion markers, or 15
|
|
min timeout), SSH into VM-A and run:
|
|
|
|
| # | Assertion | Query / check |
|
|
|---|------------------------------------------------------------------|-----------------------------------------------------------------|
|
|
| 1 | All obmp containers healthy | `docker ps --format '{{.Names}} {{.Status}}'` grep unhealthy = 0 |
|
|
| 2 | Collector accepted a BMP session from VM-B | `docker logs obmp-collector` contains "peer up" for VM-B's IP |
|
|
| 3 | `routers` table has 1 row | `SELECT count(*) FROM routers` = 1 |
|
|
| 4 | `bgp_peers` has ≥ 1 UP session | `SELECT count(*) FROM bgp_peers WHERE state='UP'` ≥ 1 |
|
|
| 5 | `ip_rib` has ≥ 1 row (GoBGP injected some routes) | `SELECT count(*) FROM ip_rib` ≥ 1 |
|
|
| 6 | Grafana healthcheck returns 200 | `curl -sf http://<VM-A>:3000/api/health` |
|
|
|
|
Pass = all six green. Failure emits the first failing assertion + a
|
|
tarball of `docker logs`, `.env`, `deploy.sh` output for diagnosis.
|
|
|
|
## Why bpg/proxmox
|
|
|
|
Two candidate providers exist for Proxmox: `Telmate/proxmox` (original,
|
|
slower moving) and `bpg/proxmox` (fork, active, modern cloud-init
|
|
handling). We pick bpg because:
|
|
|
|
- Cloud-init `user_data_file_id` is first-class — we can hand-craft the
|
|
cloud-init YAML and reference it by snippet ID rather than shoving it
|
|
through the provider's per-field API.
|
|
- `proxmox_virtual_environment_download_file` handles the cloud-image
|
|
download idempotently.
|
|
- The type system + docs are noticeably better on 0.60+.
|
|
|
|
## Why cloud-init and not Ansible
|
|
|
|
Adding Ansible would double the surface. Cloud-init is enough for
|
|
"install a few packages, drop a repo, run a script" — anything more
|
|
complex is a smell that we're testing the wrong thing.
|
|
|
|
## Why a separate repo
|
|
|
|
Test infra should not gate product commits. If the two-VM test suite
|
|
grows (multi-vendor router VMs, prod-like tuning tests, longer-running
|
|
soak variants), it earns its own release cadence. Keeping the code
|
|
separate keeps that door open.
|