68 lines
3.0 KiB
Markdown
68 lines
3.0 KiB
Markdown
|
# Deployment types
|
||
|
|
|
||
|
|
`deploy.sh` deploys one of three types (`--scope`). This is the reference for
|
||
|
|
what each includes, when to use it, and what to size it for. Old scope names
|
||
|
|
(`core`/`feeders`/`full`/`standalone`) are accepted as aliases.
|
||
|
|
|
||
|
|
## 1. full-stack
|
||
|
|
|
||
|
|
Everything on one host: collector + Kafka + Postgres/TimescaleDB + Grafana +
|
||
|
|
whois + lab feeders (`--profile test`: ExaBGP, GoBGP, traffic-gen, InfluxDB,
|
||
|
|
telegraf, rib-poller, churn/kafka-lag monitors). BMP terminates here and data
|
||
|
|
is stored here.
|
||
|
|
|
||
|
|
- **Use when:** one node monitors the whole fabric (lab, or small prod).
|
||
|
|
- **Excludes:** nothing except Authelia (separate toggle, `--auth authelia` +
|
||
|
|
`--profile auth`).
|
||
|
|
- **Resources:** prod-realistic 16 vCPU / 48-64 GB / NVMe >=250 GB;
|
||
|
|
lab-only 4 vCPU / 16 GB / SSD.
|
||
|
|
|
||
|
|
## 2. remote
|
||
|
|
|
||
|
|
Collector + local Kafka only. Forwards parsed/raw data to a **central store's
|
||
|
|
Kafka** (`--central-kafka HOST:PORT`, written to `KAFKA_FQDN`). No local
|
||
|
|
Postgres/psql-app/Grafana.
|
||
|
|
|
||
|
|
- **Use when:** scaling ingest out — run these near the routers so each node's
|
||
|
|
Kafka spool absorbs only its own routers' RIB dumps instead of every table
|
||
|
|
hitting one host at once.
|
||
|
|
- **Resources:** light — 2-4 vCPU / 4-8 GB / 20-40 GB fast disk (the Kafka
|
||
|
|
spool must buffer a full connect/flap burst from its local routers).
|
||
|
|
|
||
|
|
## 3. central-store
|
||
|
|
|
||
|
|
The store half: Kafka + Postgres + Grafana + feeders, **no local collector**.
|
||
|
|
Ingest arrives from remote collectors over Kafka. Run one of these behind N
|
||
|
|
`remote` nodes.
|
||
|
|
|
||
|
|
- **Use when:** you've split ingest with remote collectors.
|
||
|
|
- **Resources:** same as the full-stack store tier — 16 vCPU / 48-64 GB /
|
||
|
|
NVMe >=250 GB (it carries all remotes' data).
|
||
|
|
|
||
|
|
## Sizing: dimension for BMP burst, not steady state
|
||
|
|
|
||
|
|
BMP is event-driven and bursty. Steady state is trivial; the sizing events are
|
||
|
|
(1) the initial RIB dump on PEER_UP, (2) all routers reconnecting at once
|
||
|
|
(collector restart / RR failover), (3) reconvergence churn.
|
||
|
|
|
||
|
|
The multiplier that actually bites is **full table x monitored sessions**. A
|
||
|
|
full v4+v6 table is ~1.18M paths; in an RR topology the same table is
|
||
|
|
reflected to every client, so BMP ingests it once **per monitored session**.
|
||
|
|
Monitoring the RR-clients group means absorbing full-table x (RRs x clients)
|
||
|
|
simultaneously — that is what saturates a single host. The wall is Postgres
|
||
|
|
write latency + write amplification backpressuring psql-app -> Kafka -> the
|
||
|
|
BMP TCP sessions; the collector process itself is light and is *not* the
|
||
|
|
sizing driver.
|
||
|
|
|
||
|
|
Mitigations: NVMe with real IOPS headroom (>=5000); BMP-monitor pre-policy on
|
||
|
|
the RRs only (not every client session); stagger connects
|
||
|
|
(`initial-refresh delay/spread` on the routers); raise `PSQL_MEM_LIMIT` /
|
||
|
|
Kafka memory; or split ingest with `remote` collectors.
|
||
|
|
|
||
|
|
Storage scales with prefixes: ~1 GB per peer with a full internet table
|
||
|
|
(+~50 MB/day of timeseries); internal peers far less.
|
||
|
|
|
||
|
|
> All figures are engineering estimates to calibrate against your own
|
||
|
|
> watermarking, not measured specs. Replace them with real numbers once a
|
||
|
|
> prod-realistic node has been load-tested (see docs/production-sizing.md).
|