router extension (dormant) + v2 redesign doc after OOM'd first run
Adds the real-router BMP extension code and a redesign doc capturing
what changed in the test's intent.
scripts/router_bmp.py push/dry-run/remove `bmp server 2`
on all 18 lab routers via paramiko
SSH shell. Inventory + cred pattern
lifted from obmp-rib-poller/poller.py.
Activation scope: bmp-activate server 2
on the existing BMP-MONITORED
neighbor-group -- mirrors production.
terraform/router_bmp.tf terraform_data resource with two-step
safety (dry-run first, apply on
confirm), triggers_replace on action
change, destroy provisioner calls
`remove` (idempotent, on_failure=
continue so a stuck router doesn't
block VM teardown).
terraform/terraform.tfvars enable_router_bmp / confirm_router_push
both default false (dormant).
docs/redesign-real-routers.md captures the pivot from v1 "prove
deploy.sh works" to v2 "canary
collector receiving real router BMP".
Documents that v1 store VM OOM'd at
4 GB under the internal --profile
test feed alone, proposes 32 GB /
8 vCPU for v2 and dropping --profile
test so the collector receives ONLY
real router BMP.
State on prox940 as of this commit:
- v1 VMs (9001 store, 9002 bmpgen) destroyed via PVE API
(terraform destroy hung on the OOM'd guest agent)
- Terraform state cleaned
- Cached Ubuntu image kept for a fast next apply
- Lab routers untouched (extension code dormant)
Not yet done, awaiting user input:
- v2 sizing landed in terraform.tfvars
- Compose scope decision (add --no-feeders to deploy.sh vs
workaround in cloud-init)
- Re-apply
- Router extension dry-run + confirm
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
parent
d0b82c4973
commit
6fc92541e9
175
docs/redesign-real-routers.md
Normal file
175
docs/redesign-real-routers.md
Normal file
@ -0,0 +1,175 @@
|
||||
# Redesign: Real-Router BMP Ingest Test (v2)
|
||||
|
||||
## Where we landed after the first run
|
||||
|
||||
The original two-VM design (`docs/design.md`) proved `deploy.sh` runs to
|
||||
completion on a fresh Ubuntu host and that a synthetic GoBGP peer can
|
||||
open a BMP session to the collector across a real network boundary.
|
||||
That work found and fixed three real obmp-docker bugs
|
||||
(`bb71c9f`, `745f066`, `6df67a7` on `deploy-refinement`) plus one design
|
||||
issue on the bmpgen VM (`d0b82c4` here). Portability of the deploy is
|
||||
now demonstrated.
|
||||
|
||||
Then the intent shifted: **use the test collector to receive BMP from
|
||||
the real lab routers**, running alongside the production collector via
|
||||
IOS-XR's second-server slot (`bmp server 2`). The router-side extension
|
||||
was written (`scripts/router_bmp.py`, `terraform/router_bmp.tf`) but is
|
||||
dormant (`enable_router_bmp=false`). The 4 GB store VM OOM'd under the
|
||||
existing internal ingest (251 K routes from Bromirski via `--profile
|
||||
test`) before we ever turned the extension on.
|
||||
|
||||
That's the pivot: v1 was *"prove deploy works"*. v2 is *"receive real
|
||||
router BMP as a canary collector"*. Different workload, different
|
||||
sizing, different scope.
|
||||
|
||||
## Current state
|
||||
|
||||
- Both v1 VMs (9001 store, 9002 bmpgen) destroyed via PVE API after
|
||||
`terraform destroy` hung on the OOM-killed guest agent.
|
||||
- Terraform state cleaned; only the cached Ubuntu image resource
|
||||
remains (kept for a fast next apply).
|
||||
- Router extension code committed but disabled by default.
|
||||
- Nothing on prox940 or the lab routers has been touched.
|
||||
|
||||
## Proposed v2 topology
|
||||
|
||||
**Single VM**, not two. Bmpgen has no role once real routers are the
|
||||
BMP source — it was a stand-in for exactly that.
|
||||
|
||||
```
|
||||
+---------------------------------------------------+
|
||||
| store VM (v2) |
|
||||
| ubuntu 24.04 cloud + deploy.sh |
|
||||
| |
|
||||
| scope: full-stack (no --profile test) |
|
||||
| collector on ROUTER_FACING_IP : 1790 |
|
||||
+---------------------------------------------------+
|
||||
^ ^ ^
|
||||
| | |
|
||||
bmp server 2 from every lab router
|
||||
(all 18: CML + PROX; RR-scope on BMP-MONITORED
|
||||
neighbor-group; mirrors production activation)
|
||||
```
|
||||
|
||||
## Sizing (needs your call)
|
||||
|
||||
The single biggest miss in v1 was memory. Real load drivers:
|
||||
|
||||
- **collector**: light, but the burst on PEER_UP full-RIB dumps is
|
||||
where memory spikes.
|
||||
- **kafka**: 4 GB heap plus page cache; 8 GB minimum for the ingest
|
||||
spool under a full-table burst.
|
||||
- **psql**: `shared_buffers = 6 GB` in the shipped image. Under real
|
||||
ingest, `work_mem` + autovacuum churn adds several GB more.
|
||||
- **psql-app**: 3 GB JVM heap per current defaults.
|
||||
- **grafana + monitors + rib-poller**: ~1 GB combined.
|
||||
|
||||
Production (`fleet-feed-stress-test` in your memory notes) is on 62 GB
|
||||
after the RAM bump. For the *duplicate* stream, we're not asking for
|
||||
production sizing — the test collector doesn't need to sustain the
|
||||
same peak indefinitely, just enough to receive, ingest, and let you
|
||||
run diff queries. My best guess:
|
||||
|
||||
- **Minimum viable**: 16 GB / 4 vCPU / 64 GB disk. Handles all 18
|
||||
RR-scope BMP sessions if none are pushing full internet tables at
|
||||
the same instant. Likely to swap under a coordinated PEER UP storm.
|
||||
- **Comfortable**: 32 GB / 8 vCPU / 128 GB disk. Matches the
|
||||
production sizing for the store tier per `docs/DEPLOYMENT-TYPES.md`
|
||||
("full-stack: prod-realistic 16 vCPU / 48-64 GB / NVMe >=250 GB").
|
||||
- **Overkill for a canary**: 64 GB. Only if you plan to leave it up
|
||||
long-term ingesting real routes for extended comparison studies.
|
||||
|
||||
Recommendation: **32 GB / 8 vCPU / 128 GB**. Matches your "canary that
|
||||
sees what prod sees" intent without dominating prox940 (which has 754
|
||||
GB and 96 cores, so 32 GB is 4%).
|
||||
|
||||
## Compose scope
|
||||
|
||||
**Skip `--profile test`.** The internal gobgp there was what pulled the
|
||||
251 K routes from Bromirski that filled up v1's 4 GB. For a real-router
|
||||
test, we want the collector receiving *only* real router BMP — no
|
||||
synthetic feed muddying the diffs.
|
||||
|
||||
Current deploy.sh only offers scope `full-stack` (implies `--profile
|
||||
test`), `remote`, or `central-store`. Fresh portability find:
|
||||
**`full-stack` conflates "the full stack" with "the lab feeders",
|
||||
which is fine for a demo but not for a canary collector.**
|
||||
|
||||
Two options:
|
||||
|
||||
1. Extend `deploy.sh --scope full-stack --no-feeders` as a new option
|
||||
(adds a real portability enhancement to obmp-docker; landing on
|
||||
`deploy-refinement`).
|
||||
2. Sidestep in cloud-init: run `deploy.sh --scope full-stack --yes`,
|
||||
then explicitly `docker compose --profile test down` after to remove
|
||||
the synthetic feeders.
|
||||
|
||||
Option 1 is cleaner and lands a real feature. Option 2 is a workaround
|
||||
we could remove once option 1 exists.
|
||||
|
||||
## Router extension (unchanged in intent)
|
||||
|
||||
`scripts/router_bmp.py` + `terraform/router_bmp.tf` are already written
|
||||
and correct for v2. Two-step safety still holds: first apply with
|
||||
`enable_router_bmp=true` runs a dry-run (prints per-router pending
|
||||
diffs, does NOT commit); set `confirm_router_push=true` and re-apply to
|
||||
actually push. Destroy provisioner removes it cleanly, `on_failure =
|
||||
continue` so a single unreachable router doesn't block VM teardown.
|
||||
|
||||
Scope: `bmp-activate server 2` on `BMP-MONITORED` neighbor-group per
|
||||
router — matches production activation pattern (enables diff-based
|
||||
validation between the two collectors).
|
||||
|
||||
Router creds: read via `ROUTER_USER`/`ROUTER_PASS` env for CML;
|
||||
hardcoded `admin/cisco` for PROX (matches the inventory pattern in
|
||||
`obmp-docker/obmp-rib-poller/poller.py`).
|
||||
|
||||
## Assertion set (needs updating)
|
||||
|
||||
`scripts/run-test.sh`'s six assertions were written for the v1
|
||||
synthetic-feed topology. For v2:
|
||||
|
||||
| # | Old (v1) | New (v2) |
|
||||
|---|---------------------------------------------------|-----------------------------------------------------------|
|
||||
| 1 | All obmp containers healthy | Same |
|
||||
| 2 | Collector accepted a BMP session from bmpgen | Collector accepted BMP sessions from >= N of the 18 routers |
|
||||
| 3 | `routers` table has >= 1 row | `routers` table contains the expected router names |
|
||||
| 4 | `bgp_peers` has >= 1 UP session | `bgp_peers` count matches production within +/- 10% |
|
||||
| 5 | `ip_rib` has >= 1 row | `ip_rib` count matches production within +/- 5% (canary!) |
|
||||
| 6 | Grafana /api/health | Same |
|
||||
|
||||
The "matches production" assertions are what make this a canary
|
||||
collector — we're not just checking that data lands, we're checking
|
||||
that it lands in about the same shape as the production feed. Needs
|
||||
the production DB to be queryable from the same host that runs the
|
||||
assertions.
|
||||
|
||||
## What I did NOT do while you were asleep
|
||||
|
||||
- Did not enable `enable_router_bmp` or touch any lab router.
|
||||
- Did not create new VMs.
|
||||
- Did not change compose scope / deploy.sh flags in obmp-docker.
|
||||
- Did not push v2 sizing changes to `terraform.tfvars`.
|
||||
|
||||
## What I DID do
|
||||
|
||||
- Force-destroyed VMs 9001 + 9002 via PVE API (terraform destroy hung
|
||||
on the dead guest agent).
|
||||
- Cleaned terraform state to match reality.
|
||||
- Committed the dormant router extension code + the earlier bug-fix
|
||||
commit set from the v1 run.
|
||||
- Wrote this doc.
|
||||
|
||||
## Decisions you'll want to make when you're back
|
||||
|
||||
1. **Sizing**: 16 GB / 32 GB / other for the v2 store VM?
|
||||
2. **Compose scope**: add `--no-feeders` to deploy.sh (option 1
|
||||
above), or workaround in cloud-init (option 2)?
|
||||
3. **Ready to enable router extension?** Once v2 store VM is up and
|
||||
healthy, first apply is a safe dry-run.
|
||||
4. **Assertion updates**: leave the v1 assertions in place for now, or
|
||||
evolve to the "matches production" set right away? (The latter
|
||||
requires network reach + creds for the production Postgres.)
|
||||
|
||||
Any of these can wait until we've stood up the v2 VM and confirmed
|
||||
it's ingesting fine.
|
||||
242
scripts/router_bmp.py
Executable file
242
scripts/router_bmp.py
Executable file
@ -0,0 +1,242 @@
|
||||
#!/usr/bin/env python3
|
||||
"""router_bmp.py -- push/remove `bmp server 2` on lab routers.
|
||||
|
||||
Adds a SECOND BMP feed to every lab router pointing at the
|
||||
obmp-portability-test collector, WITHOUT touching `bmp server 1` (the
|
||||
production feed). Activation scope mirrors production: `bmp-activate
|
||||
server 2` on the `BMP-MONITORED` neighbor-group that already exists on
|
||||
every router.
|
||||
|
||||
Usage (called by terraform via local-exec; also runnable standalone):
|
||||
router_bmp.py dry-run --target-ip 10.40.40.241 --target-port 1790
|
||||
router_bmp.py apply --target-ip 10.40.40.241 --target-port 1790
|
||||
router_bmp.py remove --target-ip 10.40.40.241 --target-port 1790
|
||||
|
||||
Inventory + credential pattern lifted verbatim from obmp-docker's
|
||||
obmp-rib-poller/poller.py (keep in sync manually; drift silently misses
|
||||
routers). CML routers use ROUTER_USER/ROUTER_PASS env (defaults
|
||||
webui/cisco); PROX cores + clients use admin/cisco -- this asymmetry is
|
||||
inherent to the PROX build and captured in the inventory below.
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import os
|
||||
import sys
|
||||
import time
|
||||
from typing import List, Tuple
|
||||
|
||||
import paramiko
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Inventory. Mirror of obmp-rib-poller/poller.py:50 with per-router ASN added
|
||||
# so the activation line can name the right `router bgp <asn>` context.
|
||||
# ---------------------------------------------------------------------------
|
||||
ROUTER_USER = os.environ.get("ROUTER_USER", "webui")
|
||||
ROUTER_PASS = os.environ.get("ROUTER_PASS", "cisco")
|
||||
|
||||
# (name, mgmt_ip, ssh_user, ssh_pass, local_asn)
|
||||
ROUTERS: List[Tuple[str, str, str, str, int]] = [
|
||||
("CML-CORE-01", "10.100.0.100", ROUTER_USER, ROUTER_PASS, 65020),
|
||||
("CML-CORE-02", "10.100.0.200", ROUTER_USER, ROUTER_PASS, 65020),
|
||||
("CML-R9K-01", "10.100.0.1", ROUTER_USER, ROUTER_PASS, 65020),
|
||||
("CML-R9K-02", "10.100.0.2", ROUTER_USER, ROUTER_PASS, 65020),
|
||||
("CML-R9K-03", "10.100.0.3", ROUTER_USER, ROUTER_PASS, 65020),
|
||||
("CML-R9K-04", "10.100.0.4", ROUTER_USER, ROUTER_PASS, 65020),
|
||||
("CML-R9K-05", "10.100.0.5", ROUTER_USER, ROUTER_PASS, 65020),
|
||||
("CML-R9K-06", "10.100.0.6", ROUTER_USER, ROUTER_PASS, 65020),
|
||||
("CML-R9K-07", "10.100.0.7", ROUTER_USER, ROUTER_PASS, 65020),
|
||||
("PROX-CORE-01", "10.100.1.100", "admin", "cisco", 65021),
|
||||
("PROX-CORE-02", "10.100.1.200", "admin", "cisco", 65021),
|
||||
("PROX-R9K-01", "10.100.1.1", "admin", "cisco", 65021),
|
||||
("PROX-R9K-02", "10.100.1.2", "admin", "cisco", 65021),
|
||||
("PROX-R9K-03", "10.100.1.3", "admin", "cisco", 65021),
|
||||
("PROX-R9K-04", "10.100.1.4", "admin", "cisco", 65021),
|
||||
("PROX-R9K-05", "10.100.1.5", "admin", "cisco", 65021),
|
||||
("PROX-R9K-06", "10.100.1.6", "admin", "cisco", 65021),
|
||||
("PROX-R9K-07", "10.100.1.7", "admin", "cisco", 65021),
|
||||
]
|
||||
|
||||
SERVER_NUM = 2 # bmp server 2 -- keeps server 1 (production) untouched.
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Config lines we push. Each list is written line-by-line at (config)#,
|
||||
# which puts them straight into IOS-XR's flat-formal command layer without
|
||||
# needing to enter the bmp submode.
|
||||
# ---------------------------------------------------------------------------
|
||||
def bmp_add_lines(collector_ip: str, collector_port: int) -> List[str]:
|
||||
return [
|
||||
f"bmp server {SERVER_NUM} host {collector_ip} port {collector_port}",
|
||||
f"bmp server {SERVER_NUM} description OpenBMP-Test-Collector",
|
||||
f"bmp server {SERVER_NUM} update-source MgmtEth0/RP0/CPU0/0",
|
||||
f"bmp server {SERVER_NUM} initial-delay 60",
|
||||
f"bmp server {SERVER_NUM} stats-reporting-period 300",
|
||||
f"bmp server {SERVER_NUM} initial-refresh delay 60 spread 30",
|
||||
]
|
||||
|
||||
|
||||
def bmp_activate_line(asn: int) -> str:
|
||||
return f"router bgp {asn} neighbor-group BMP-MONITORED bmp-activate server {SERVER_NUM}"
|
||||
|
||||
|
||||
def bmp_remove_lines(asn: int) -> List[str]:
|
||||
# Order matters: deactivate on the group first, then remove the server.
|
||||
return [
|
||||
f"router bgp {asn} neighbor-group BMP-MONITORED no bmp-activate server {SERVER_NUM}",
|
||||
f"no bmp server {SERVER_NUM}",
|
||||
]
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Paramiko shell helpers (interactive IOS-XR CLI, same pattern as
|
||||
# cml/proxmox_bmp_config.py -- NETCONF YANG doesn't cover bmp server).
|
||||
# ---------------------------------------------------------------------------
|
||||
def _drain(shell, settle=1.0, limit=15.0) -> str:
|
||||
buf = b""
|
||||
start = time.time()
|
||||
last = start
|
||||
while time.time() - start < limit:
|
||||
if shell.recv_ready():
|
||||
buf += shell.recv(65535)
|
||||
last = time.time()
|
||||
elif time.time() - last > settle:
|
||||
break
|
||||
else:
|
||||
time.sleep(0.05)
|
||||
return buf.decode(errors="replace")
|
||||
|
||||
|
||||
def _run(shell, cmd: str, settle: float = 0.5, limit: float = 6.0) -> str:
|
||||
shell.send(cmd + "\n")
|
||||
return _drain(shell, settle, limit)
|
||||
|
||||
|
||||
def _connect(host: str, user: str, pwd: str):
|
||||
ssh = paramiko.SSHClient()
|
||||
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
|
||||
ssh.connect(host, port=22, username=user, password=pwd, timeout=10,
|
||||
allow_agent=False, look_for_keys=False)
|
||||
shell = ssh.invoke_shell()
|
||||
_drain(shell, 1.0, 6)
|
||||
return ssh, shell
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Per-router work.
|
||||
# ---------------------------------------------------------------------------
|
||||
def process_router(name: str, ip: str, user: str, pwd: str, asn: int,
|
||||
target_ip: str, target_port: int, action: str
|
||||
) -> Tuple[bool, str]:
|
||||
"""Return (ok, message). action = 'apply' | 'dry-run' | 'remove'."""
|
||||
print(f"\n== {name} ({ip}) [{action}] ==", flush=True)
|
||||
try:
|
||||
ssh, shell = _connect(ip, user, pwd)
|
||||
except Exception as e:
|
||||
return False, f"SSH connect failed: {e}"
|
||||
|
||||
try:
|
||||
_run(shell, "terminal length 0")
|
||||
_run(shell, "configure terminal")
|
||||
|
||||
if action == "remove":
|
||||
lines = bmp_remove_lines(asn)
|
||||
else:
|
||||
lines = bmp_add_lines(target_ip, target_port) + [bmp_activate_line(asn)]
|
||||
|
||||
for line in lines:
|
||||
_run(shell, line, 0.3, 4)
|
||||
|
||||
# Show the pending diff before commit. If nothing changes, this
|
||||
# is a no-op and we abort out of config mode cleanly.
|
||||
diff = _run(shell, "show configuration", 1.0, 10)
|
||||
has_change = False
|
||||
for want in [f"bmp server {SERVER_NUM}", f"bmp-activate server {SERVER_NUM}"]:
|
||||
if want in diff:
|
||||
has_change = True
|
||||
break
|
||||
if not has_change:
|
||||
print(" no-op (already in target state)")
|
||||
_run(shell, "abort")
|
||||
ssh.close()
|
||||
return True, "no-op"
|
||||
|
||||
print(" ---- pending diff ----")
|
||||
for l in diff.splitlines():
|
||||
s = l.rstrip()
|
||||
if s and not s.startswith("!") and "show configuration" not in s.lower():
|
||||
print(" ", s)
|
||||
print(" ----------------------")
|
||||
|
||||
if action == "dry-run":
|
||||
_run(shell, "abort")
|
||||
ssh.close()
|
||||
return True, "dry-run (not committed)"
|
||||
|
||||
commit_out = _run(shell, "commit", 1.5, 10)
|
||||
_run(shell, "end")
|
||||
|
||||
# Verify. `show run formal bmp` gives us the server-side state;
|
||||
# activation is verified separately on the router-bgp block.
|
||||
verify = _run(shell, "show run formal bmp", 1.5, 10)
|
||||
if action == "apply":
|
||||
ok = f"host {target_ip} port {target_port}" in verify
|
||||
msg = "committed + verified" if ok else "committed but verify NOT FOUND"
|
||||
else: # remove
|
||||
ok = f"bmp server {SERVER_NUM}" not in verify
|
||||
msg = "removed + verified" if ok else "commit ran but server still present"
|
||||
|
||||
ssh.close()
|
||||
return ok, msg
|
||||
except Exception as e:
|
||||
try:
|
||||
ssh.close()
|
||||
except Exception:
|
||||
pass
|
||||
return False, f"error: {e}"
|
||||
|
||||
|
||||
def main():
|
||||
p = argparse.ArgumentParser(description=__doc__,
|
||||
formatter_class=argparse.RawDescriptionHelpFormatter)
|
||||
p.add_argument("action", choices=["apply", "dry-run", "remove"])
|
||||
p.add_argument("--target-ip", required=True,
|
||||
help="collector IP that routers should target")
|
||||
p.add_argument("--target-port", type=int, required=True,
|
||||
help="collector port (typically 1790)")
|
||||
p.add_argument("--routers",
|
||||
help="comma-separated router names to include (default: all 18)")
|
||||
p.add_argument("--fail-fast", action="store_true",
|
||||
help="stop on first router failure (default: process all, report at end)")
|
||||
args = p.parse_args()
|
||||
|
||||
inv = ROUTERS
|
||||
if args.routers:
|
||||
wanted = {n.strip() for n in args.routers.split(",")}
|
||||
inv = [r for r in inv if r[0] in wanted]
|
||||
if not inv:
|
||||
print(f"error: no routers matched {args.routers}", file=sys.stderr)
|
||||
print(f" known: {', '.join(r[0] for r in ROUTERS)}", file=sys.stderr)
|
||||
sys.exit(2)
|
||||
|
||||
print(f"routers: {len(inv)} action: {args.action} "
|
||||
f"target: {args.target_ip}:{args.target_port}", flush=True)
|
||||
|
||||
results = []
|
||||
for name, ip, user, pwd, asn in inv:
|
||||
ok, msg = process_router(name, ip, user, pwd, asn,
|
||||
args.target_ip, args.target_port, args.action)
|
||||
results.append((name, ok, msg))
|
||||
if args.fail_fast and not ok:
|
||||
break
|
||||
|
||||
print("\n== summary ==", flush=True)
|
||||
for name, ok, msg in results:
|
||||
status = "OK " if ok else "FAIL"
|
||||
print(f" {status} {name:15s} {msg}")
|
||||
|
||||
fails = sum(1 for _, ok, _ in results if not ok)
|
||||
sys.exit(1 if fails else 0)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
91
terraform/router_bmp.tf
Normal file
91
terraform/router_bmp.tf
Normal file
@ -0,0 +1,91 @@
|
||||
# Real-router extension: push `bmp server 2` to every lab router pointing
|
||||
# at the test collector, so real-router BMP output flows into the test
|
||||
# stack alongside the internal gobgp synthetic feed. Nothing touches
|
||||
# `bmp server 1` (production).
|
||||
#
|
||||
# Two-step safety: first apply is a DRY RUN (prints per-router pending
|
||||
# diffs, does not commit). Set `confirm_router_push=true` and re-apply
|
||||
# to actually commit the config on all routers.
|
||||
#
|
||||
# Lifecycle:
|
||||
# enable_router_bmp=false -> resource absent, no router-side changes
|
||||
# enable=true, confirm=false -> dry-run provisioner (safe; abort at end)
|
||||
# enable=true, confirm=true -> apply provisioner (commits + verifies)
|
||||
# flip either back -> destroy provisioner runs `remove` (idempotent)
|
||||
#
|
||||
# The destroy provisioner uses `on_failure = continue` so a partial router
|
||||
# outage never blocks the VM teardown in scripts/teardown.sh.
|
||||
|
||||
variable "enable_router_bmp" {
|
||||
description = "Enable the real-router BMP extension (push bmp server 2 to lab routers)."
|
||||
type = bool
|
||||
default = false
|
||||
}
|
||||
|
||||
variable "confirm_router_push" {
|
||||
description = "Set to true on second apply to actually commit `bmp server 2` on routers. First apply (false) is a dry-run only."
|
||||
type = bool
|
||||
default = false
|
||||
}
|
||||
|
||||
locals {
|
||||
router_bmp_action = var.confirm_router_push ? "apply" : "dry-run"
|
||||
router_bmp_script = abspath("${path.module}/../scripts/router_bmp.py")
|
||||
|
||||
# The collector's address on the routers' management path. With DHCP,
|
||||
# this only becomes known after the store VM's guest-agent reports.
|
||||
router_bmp_target = try(
|
||||
proxmox_virtual_environment_vm.store.ipv4_addresses[1][0],
|
||||
""
|
||||
)
|
||||
}
|
||||
|
||||
resource "terraform_data" "router_bmp" {
|
||||
count = var.enable_router_bmp ? 1 : 0
|
||||
|
||||
# Any change to these values REPLACES the resource -- destroy provisioner
|
||||
# runs first (removes any prior state on routers), then create provisioner
|
||||
# runs (applies the new state). That gives us clean transitions:
|
||||
# dry-run -> apply, or apply -> dry-run.
|
||||
triggers_replace = [
|
||||
local.router_bmp_action,
|
||||
local.router_bmp_target,
|
||||
var.router_facing_port,
|
||||
]
|
||||
|
||||
# `input` is available to provisioners as `self.input`.
|
||||
input = {
|
||||
action = local.router_bmp_action
|
||||
script = local.router_bmp_script
|
||||
target_ip = local.router_bmp_target
|
||||
target_port = tostring(var.router_facing_port)
|
||||
}
|
||||
|
||||
# CREATE (or replace): push (or dry-run) `bmp server 2` to all routers.
|
||||
provisioner "local-exec" {
|
||||
command = "python3 ${self.input.script} ${self.input.action} --target-ip ${self.input.target_ip} --target-port ${self.input.target_port}"
|
||||
}
|
||||
|
||||
# DESTROY: remove `bmp server 2` from all routers. Idempotent: routers
|
||||
# without the config are no-ops. `on_failure = continue` so a single
|
||||
# unreachable router does not block the VM teardown.
|
||||
provisioner "local-exec" {
|
||||
when = destroy
|
||||
on_failure = continue
|
||||
command = "python3 ${self.input.script} remove --target-ip ${self.input.target_ip} --target-port ${self.input.target_port}"
|
||||
}
|
||||
}
|
||||
|
||||
output "router_bmp_status" {
|
||||
description = "Real-router BMP extension state."
|
||||
value = var.enable_router_bmp ? {
|
||||
enabled = true
|
||||
action = local.router_bmp_action
|
||||
confirm_needed = !var.confirm_router_push
|
||||
target = "${local.router_bmp_target}:${var.router_facing_port}"
|
||||
hint = var.confirm_router_push ? "committed on last apply; destroy will remove" : "set confirm_router_push=true and re-apply to commit"
|
||||
} : {
|
||||
enabled = false
|
||||
hint = "set enable_router_bmp=true (and later confirm_router_push=true) to add bmp server 2 to lab routers"
|
||||
}
|
||||
}
|
||||
Loading…
x
Reference in New Issue
Block a user