Add platform roadmap: multi-lab CML integration and production deployment

Four-track roadmap covering configuration centralization (inventory.yaml),
CML API automation (virl2_client), production ISP deployment (multi-vendor
IOS-XR + Junos), and packaging for distribution.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

.gitignore

@@ -2,4 +2,6 @@
 *.log
 .env
 .claude/
+__pycache__/
+*.pyc
 

DB_SCHEMA.md

@@ -0,0 +1,387 @@
+# OpenBMP Database Schema Reference
+
+PostgreSQL database `openbmp` with TimescaleDB extension for time-series data.
+
+## Entity Relationship Diagram
+
+```
+collectors
+  └── routers (collector_hash_id)
+        └── bgp_peers (router_hash_id)
+              ├── ip_rib (peer_hash_id) ──► base_attrs (base_attr_hash_id)
+              ├── ip_rib_log (peer_hash_id)
+              ├── l3vpn_rib (peer_hash_id) ──► base_attrs
+              ├── ls_nodes (peer_hash_id)
+              ├── ls_links (peer_hash_id) ──► ls_nodes (local/remote_node_hash_id)
+              ├── ls_prefixes (peer_hash_id) ──► ls_nodes (local_node_hash_id)
+              ├── peer_event_log (peer_hash_id)
+              ├── stat_reports (peer_hash_id)
+              └── stats_* tables (peer_hash_id)
+
+ip_rib.prefix ◄──► global_ip_rib.prefix (aggregated view)
+                     ├── rpki_origin_as ◄── rpki_validator
+                     └── irr_origin_as ◄── info_route
+
+base_attrs.origin_as ──► info_asn.asn (ASN enrichment)
+routers.geo_ip_start ──► geo_ip.ip (geolocation)
+```
+
+---
+
+## BMP Core Tables
+
+### routers
+BMP-monitored routers (one row per monitored device).
+
+| Column | Type | Description |
+|--------|------|-------------|
+| hash_id | uuid | Primary key |
+| name | varchar(200) | Router hostname |
+| ip_address | inet | Router management IP |
+| router_as | bigint | Router ASN |
+| bgp_id | inet | BGP router-id |
+| collector_hash_id | uuid | FK to collectors |
+| state | opstate | up / down |
+| timestamp | timestamp | Last update time |
+| description | varchar(255) | Router description |
+| init_data | text | BMP init message data |
+| term_reason_code | int | BMP termination reason |
+
+### collectors
+BMP collector instances.
+
+| Column | Type | Description |
+|--------|------|-------------|
+| hash_id | uuid | Primary key |
+| admin_id | varchar(64) | Admin identifier |
+| name | varchar(200) | Collector name |
+| ip_address | varchar(40) | Collector IP |
+| state | opstate | up / down |
+| router_count | smallint | Number of monitored routers |
+
+### bgp_peers
+BGP sessions per router (one row per peer per router).
+
+| Column | Type | Description |
+|--------|------|-------------|
+| hash_id | uuid | Primary key (composite with router_hash_id) |
+| router_hash_id | uuid | FK to routers |
+| peer_addr | inet | Peer IP address |
+| peer_as | bigint | Peer ASN |
+| peer_bgp_id | inet | Peer BGP router-id |
+| name | varchar(200) | Peer name |
+| state | opstate | up / down |
+| isl3vpnpeer | boolean | L3VPN peer flag |
+| isipv4 | boolean | IPv4 peer |
+| isprepolicy | boolean | Pre-policy RIB |
+| islocrib | boolean | Local RIB |
+| local_ip | inet | Local IP |
+| local_asn | bigint | Local ASN |
+| local_hold_time | smallint | Local hold time |
+| remote_hold_time | smallint | Remote hold time |
+| sent_capabilities | varchar(4096) | BGP capabilities sent |
+| recv_capabilities | varchar(4096) | BGP capabilities received |
+| table_name | varchar(255) | VRF/table name |
+
+### peer_event_log (TimescaleDB)
+Historical BGP session state changes.
+
+| Column | Type | Description |
+|--------|------|-------------|
+| id | bigint | Event sequence |
+| peer_hash_id | uuid | FK to bgp_peers |
+| state | opstate | up / down |
+| timestamp | timestamp | Event time (partition key) |
+| bmp_reason | smallint | BMP reason code |
+| bgp_err_code | smallint | BGP error code |
+| bgp_err_subcode | smallint | BGP error subcode |
+| error_text | varchar(255) | Error description |
+
+---
+
+## BGP Path Attributes
+
+### base_attrs
+BGP path attributes shared across routes.
+
+| Column | Type | Description |
+|--------|------|-------------|
+| hash_id | uuid | Primary key |
+| peer_hash_id | uuid | FK to bgp_peers |
+| origin | varchar(16) | IGP / EGP / Incomplete |
+| as_path | bigint[] | AS path array |
+| as_path_count | smallint | AS path length |
+| origin_as | bigint | Origin ASN |
+| next_hop | inet | BGP next-hop |
+| med | bigint | Multi-Exit Discriminator |
+| local_pref | bigint | Local preference |
+| community_list | varchar(15)[] | Standard communities |
+| ext_community_list | varchar(50)[] | Extended communities (RT, etc.) |
+| large_community_list | varchar(40)[] | Large communities (RFC 8092) |
+| cluster_list | varchar(40)[] | Route reflector cluster list |
+| isatomicagg | boolean | Atomic aggregate flag |
+| originator_id | inet | RR originator ID |
+| aggregator | varchar(64) | Aggregator |
+
+**Indexes**: GIN on as_path, community_list, ext_community_list, large_community_list
+
+---
+
+## IP RIB Tables
+
+### ip_rib
+Current IPv4/IPv6 unicast routing table.
+
+| Column | Type | Description |
+|--------|------|-------------|
+| hash_id | uuid | Route hash |
+| peer_hash_id | uuid | FK to bgp_peers (composite PK) |
+| base_attr_hash_id | uuid | FK to base_attrs |
+| prefix | inet | IP prefix |
+| prefix_len | smallint | Prefix length |
+| origin_as | bigint | Origin ASN |
+| isipv4 | boolean | IPv4 flag |
+| iswithdrawn | boolean | Withdrawn flag |
+| labels | varchar(255) | MPLS labels |
+| path_id | bigint | Add-Path ID |
+| isprepolicy | boolean | Pre-policy flag |
+| isadjribin | boolean | Adj-RIB-In flag |
+| timestamp | timestamp | Last update |
+| first_added_timestamp | timestamp | First seen |
+
+### ip_rib_log (TimescaleDB)
+Historical RIB changes — every advertisement and withdrawal.
+
+| Column | Type | Description |
+|--------|------|-------------|
+| id | bigint | Change event ID |
+| peer_hash_id | uuid | FK to bgp_peers |
+| base_attr_hash_id | uuid | FK to base_attrs |
+| prefix | inet | IP prefix |
+| prefix_len | smallint | Prefix length |
+| origin_as | bigint | Origin ASN |
+| iswithdrawn | boolean | Withdrawal flag |
+| timestamp | timestamp | Event time (partition key) |
+
+### global_ip_rib
+Aggregated prefix summary across all peers.
+
+| Column | Type | Description |
+|--------|------|-------------|
+| prefix | inet | IP prefix (composite PK) |
+| prefix_len | smallint | Prefix length |
+| recv_origin_as | bigint | Received origin AS |
+| rpki_origin_as | bigint | RPKI-validated origin AS |
+| irr_origin_as | bigint | IRR-registered origin AS |
+| irr_source | varchar(32) | IRR source (RADB, RIPE, etc.) |
+| num_peers | int | Total advertising peers |
+| iswithdrawn | boolean | Withdrawn flag |
+
+---
+
+## L3VPN Tables
+
+### l3vpn_rib
+L3VPN (RFC 4364) routes with Route Distinguisher.
+
+| Column | Type | Description |
+|--------|------|-------------|
+| hash_id | uuid | Route hash |
+| peer_hash_id | uuid | FK to bgp_peers |
+| base_attr_hash_id | uuid | FK to base_attrs |
+| rd | varchar(128) | Route Distinguisher |
+| prefix | inet | VPN prefix |
+| prefix_len | smallint | Prefix length |
+| origin_as | bigint | Origin ASN |
+| labels | varchar(255) | MPLS VPN labels |
+| ext_community_list | varchar(50)[] | Route Targets |
+| path_id | bigint | Add-Path ID |
+| iswithdrawn | boolean | Withdrawn flag |
+
+### l3vpn_rib_log (TimescaleDB)
+Historical L3VPN route changes.
+
+---
+
+## Link-State Tables (BGP-LS / RFC 7752)
+
+### ls_nodes
+IS-IS / OSPF node information from BGP-LS.
+
+| Column | Type | Description |
+|--------|------|-------------|
+| hash_id | uuid | Node hash |
+| peer_hash_id | uuid | FK to bgp_peers (composite PK) |
+| base_attr_hash_id | uuid | FK to base_attrs |
+| asn | bigint | Node ASN |
+| bgp_ls_id | bigint | BGP-LS Identifier |
+| igp_router_id | varchar(46) | IGP Router ID |
+| router_id | varchar(46) | BGP Router ID |
+| protocol | ls_proto | IS-IS_L1, IS-IS_L2, OSPFv2, OSPFv3 |
+| isis_area_id | varchar(46) | IS-IS area |
+| ospf_area_id | varchar(16) | OSPF area |
+| name | varchar(255) | Node hostname |
+| flags | varchar(20) | Node flags |
+| mt_ids | varchar(128) | Multi-Topology IDs |
+| **sr_capabilities** | **varchar(255)** | **SR Global Block (SRGB) ranges** |
+| iswithdrawn | boolean | Withdrawn flag |
+
+### ls_links
+IS-IS / OSPF links with full TE and SR attributes.
+
+| Column | Type | Description |
+|--------|------|-------------|
+| hash_id | uuid | Link hash |
+| peer_hash_id | uuid | FK to bgp_peers (composite PK) |
+| local_node_hash_id | uuid | FK to ls_nodes (local end) |
+| remote_node_hash_id | uuid | FK to ls_nodes (remote end) |
+| local_router_id | varchar(46) | Local BGP Router ID |
+| remote_router_id | varchar(46) | Remote BGP Router ID |
+| local_igp_router_id | varchar(46) | Local IGP Router ID |
+| remote_igp_router_id | varchar(46) | Remote IGP Router ID |
+| interface_addr | inet | Local interface IP |
+| neighbor_addr | inet | Remote interface IP |
+| igp_metric | bigint | IGP metric |
+| protocol | ls_proto | IGP protocol |
+| mt_id | int | Multi-Topology ID |
+| local_link_id | bigint | Local link identifier |
+| remote_link_id | bigint | Remote link identifier |
+| name | varchar(255) | Link name |
+| **admin_group** | **bigint** | **TE admin group / link color bitmap** |
+| **max_link_bw** | **bigint** | **Maximum link bandwidth (bytes/sec)** |
+| **max_resv_bw** | **bigint** | **Maximum reservable bandwidth** |
+| **unreserved_bw** | **varchar(128)** | **Unreserved BW per priority (8 values)** |
+| **te_def_metric** | **bigint** | **TE default metric (for CSPF)** |
+| **protection_type** | **varchar(60)** | **Link protection (FRR type)** |
+| **mpls_proto_mask** | **ls_mpls_proto_mask** | **MPLS protocol support flags** |
+| **srlg** | **varchar(128)** | **Shared Risk Link Group** |
+| **peer_node_sid** | **varchar(128)** | **SR Peer Node SID (EPE, RFC 9086)** |
+| **sr_adjacency_sids** | **varchar(255)** | **SR Adjacency SIDs** |
+| iswithdrawn | boolean | Withdrawn flag |
+
+**Bold** = TE/SR fields available via BGP-LS but not used by default dashboards.
+
+### ls_prefixes
+IS-IS / OSPF prefix information.
+
+| Column | Type | Description |
+|--------|------|-------------|
+| hash_id | uuid | Prefix hash |
+| peer_hash_id | uuid | FK to bgp_peers (composite PK) |
+| local_node_hash_id | uuid | FK to ls_nodes |
+| prefix | inet | Advertised prefix |
+| prefix_len | smallint | Prefix length |
+| protocol | ls_proto | IGP protocol |
+| metric | bigint | Prefix metric |
+| mt_id | int | Multi-Topology ID |
+| ospf_route_type | ospf_route_type | Intra/Inter/Ext-1/Ext-2/NSSA |
+| igp_flags | varchar(20) | IGP flags |
+| route_tag | bigint | Route tag |
+| **sr_prefix_sids** | **varchar(255)** | **SR Prefix SIDs (node SIDs)** |
+| iswithdrawn | boolean | Withdrawn flag |
+
+### ls_nodes_log, ls_links_log, ls_prefixes_log (TimescaleDB)
+Historical link-state changes. Same columns as parent tables plus `id` (bigint) and timestamp as partition key.
+
+---
+
+## Statistics Tables (TimescaleDB)
+
+| Table | Purpose | Key Columns |
+|-------|---------|-------------|
+| **stat_reports** | BMP stat messages per peer | prefixes_rejected, known_dup_prefixes, num_routes_adj_rib_in, num_routes_local_rib |
+| **stats_chg_byprefix** | Per-prefix update/withdrawal counts | interval_time, prefix, updates, withdraws |
+| **stats_chg_byasn** | Per-ASN update/withdrawal counts | interval_time, origin_as, updates, withdraws |
+| **stats_chg_bypeer** | Per-peer update/withdrawal counts | interval_time, updates, withdraws |
+| **stats_peer_rib** | Per-peer RIB size over time | interval_time, v4_prefixes, v6_prefixes |
+| **stats_peer_update_counts** | Update rate statistics | interval_time, advertise_avg/min/max, withdraw_avg/min/max |
+| **stats_ip_origins** | Per-ASN IP prefix counts | interval_time, asn, v4_prefixes, v6_prefixes, v4_with_rpki, v4_with_irr |
+| **stats_l3vpn_chg_byprefix** | L3VPN per-prefix stats | interval_time, rd, prefix, updates, withdraws |
+| **stats_l3vpn_chg_bypeer** | L3VPN per-peer stats | interval_time, updates, withdraws |
+| **stats_l3vpn_chg_byrd** | L3VPN per-RD stats | interval_time, rd, updates, withdraws |
+
+---
+
+## Reference & Enrichment Tables
+
+| Table | Purpose | Key Columns |
+|-------|---------|-------------|
+| **rpki_validator** | RPKI ROAs | prefix, prefix_len, prefix_len_max, origin_as |
+| **info_asn** | ASN WHOIS/IRR data | asn, as_name, org_name, country, source |
+| **info_route** | Route IRR data | prefix, origin_as, descr, source |
+| **geo_ip** | IP geolocation (DB-IP) | ip, country, city, latitude, longitude, isp_name |
+| **pdb_exchange_peers** | PeeringDB IXP peering | ix_name, peer_name, peer_asn, speed, peer_ipv4/ipv6 |
+
+---
+
+## Views
+
+| View | Joins | Purpose |
+|------|-------|---------|
+| **v_peers** | bgp_peers + routers + info_asn | Complete peer info with router name and ASN details |
+| **v_ip_routes** | ip_rib + bgp_peers + base_attrs + routers | Full route detail with path attributes |
+| **v_ip_routes_geo** | v_ip_routes + geo_ip | Routes with geolocation |
+| **v_ip_routes_history** | ip_rib_log + base_attrs + bgp_peers + routers | Historical route changes with attributes |
+| **v_l3vpn_routes** | l3vpn_rib + bgp_peers + base_attrs + routers | L3VPN routes with path attributes |
+| **v_l3vpn_routes_history** | l3vpn_rib_log + base_attrs + bgp_peers + routers | Historical L3VPN changes |
+| **v_ls_nodes** | ls_nodes + base_attrs + bgp_peers + routers | Link-state nodes with peer/router info |
+| **v_ls_links** | ls_links + ls_nodes(x2) + routers | Links with local/remote node names + all TE/SR fields |
+| **v_ls_prefixes** | ls_prefixes + ls_nodes + routers | LS prefixes with originating node info |
+
+---
+
+## Custom Enum Types
+
+| Type | Values |
+|------|--------|
+| **opstate** | up, down |
+| **ls_proto** | IS-IS_L1, IS-IS_L2, OSPFv2, OSPFv3, Direct, Static |
+| **ospf_route_type** | Intra, Inter, Ext-1, Ext-2, NSSA-1, NSSA-2 |
+| **ls_mpls_proto_mask** | MPLS protocol bitmask |
+| **user_role** | admin, oper |
+
+---
+
+## Key Query Patterns
+
+### Get all active routes with full attributes
+```sql
+SELECT r.prefix, r.prefix_len, ba.origin_as, ba.as_path,
+       ba.med, ba.local_pref, ba.community_list, ba.next_hop
+FROM ip_rib r
+JOIN base_attrs ba ON ba.hash_id = r.base_attr_hash_id
+WHERE r.iswithdrawn = false AND r.isipv4 = true
+```
+
+### Get link-state topology with TE attributes
+```sql
+SELECT local_router_name, remote_router_name,
+       igp_metric, te_def_metric, max_link_bw, admin_group, srlg,
+       sr_adjacency_sids
+FROM v_ls_links
+WHERE peer_hash_id = '<peer_hash>' AND iswithdrawn = false
+```
+
+### Time-series RIB changes
+```sql
+SELECT date_trunc('minute', timestamp) as time,
+       SUM(CASE WHEN iswithdrawn = false THEN 1 ELSE 0 END) as ads,
+       SUM(CASE WHEN iswithdrawn = true THEN 1 ELSE 0 END) as withdrawals
+FROM ip_rib_log
+WHERE timestamp > NOW() - INTERVAL '24 hours'
+GROUP BY 1 ORDER BY 1
+```
+
+### RPKI validation status
+```sql
+SELECT CASE
+         WHEN rv.origin_as IS NOT NULL AND rv.origin_as = r.origin_as THEN 'Valid'
+         WHEN rv.origin_as IS NOT NULL THEN 'Invalid'
+         ELSE 'NotFound'
+       END as status,
+       COUNT(*)
+FROM ip_rib r
+LEFT JOIN rpki_validator rv ON rv.prefix = r.prefix AND rv.prefix_len = r.prefix_len
+WHERE r.iswithdrawn = false
+GROUP BY 1
+```

DOCS.md

@@ -16,6 +16,8 @@
 12. [Troubleshooting](#12-troubleshooting)
 13. [Data Retention](#13-data-retention)
 14. [Environment Variables Reference](#14-environment-variables-reference)
+15. [gNMI Streaming Telemetry (Phase 4)](#15-gnmi-streaming-telemetry-phase-4)
+16. [Traffic Generator (Phase 4)](#16-traffic-generator-phase-4)
 
 
 ---
@@ -28,7 +30,7 @@ This is a **BGP Monitoring Platform (BMP) lab stack** deployed via Docker Compos
 
 - Receives BMP (BGP Monitoring Protocol, RFC 7854) telemetry from routers on TCP port 5000
 - Streams BMP data through Kafka into a TimescaleDB/PostgreSQL database
-- Provides **23 Grafana dashboards** (17 operational + 6 learning-focused) for real-time and historical BGP analysis
+- Provides **30 Grafana dashboards** (17 operational + 6 learning + 4 advanced analytics + 3 streaming telemetry) for real-time and historical BGP analysis
 - Includes an **ExaBGP route injector** that peers with the two CORE routers and injects synthetic BGP routes, enabling testing of BGP policy, route propagation, and Grafana dashboards without needing internet connectivity
 - Provides a **Vue 3 web UI** at `:5001` for point-and-click scenario management, live route tables, and peer monitoring
 
@@ -64,7 +66,7 @@ IOS-XR Routers (9x, AS 65020)
     PostgreSQL 14 + TimescaleDB
          |
          +---------> obmp-grafana (grafana/grafana:9.1.7)  :3000
-         |              23 dashboards, PostgreSQL datasource
+         |              30 dashboards, PostgreSQL + InfluxDB datasources
          +---------> obmp-whois (openbmp/whois:2.2.0)      :4300
                        WHOIS query server backed by the DB
 
@@ -73,6 +75,24 @@ ExaBGP (obmp-exabgp, built locally)
   Peers eBGP to CORE-01 and CORE-02 (AS 65100 -> AS 65020)
   HTTP API on :5050 — inject/withdraw routes on demand
   Routes propagate via iBGP mesh to all 9 routers -> BMP -> DB -> Grafana
+
+gNMI Streaming Telemetry (Phase 4):
+  IOS-XR Routers (gRPC :57400)
+         |
+         v
+  obmp-telegraf (telegraf:1.28 + gnmi plugin)
+         |
+         v
+  obmp-influxdb (influxdb:2.7)  :8086
+         |
+         v
+  obmp-grafana (InfluxDB datasource -> Telemetry dashboards)
+
+Traffic Generator (Phase 4):
+  obmp-traffic-gen (python:3.11 + Scapy + Flask)  :5051
+    Dual-mode: sender (generate traffic) / responder (echo/log)
+    RFC 2544 testing, custom packet flows
+  obmp-traffic-gen-ui (Vue 3 + NGINX)  :5002
 ```
 
 ### Container Summary
@@ -87,7 +107,11 @@ ExaBGP (obmp-exabgp, built locally)
 | obmp-grafana | grafana/grafana:9.1.7 | 3000 | Visualization |
 | obmp-whois | openbmp/whois:2.2.0 | 4300 | WHOIS query server |
 | obmp-exabgp | local build | 5050 (host net) | BGP route injector |
-| obmp-exabgp-ui | local build | 5001 (host net) | Vue 3 web control panel |
+| obmp-exabgp-ui | local build | 5001 (host net) | Route injector web UI |
+| obmp-influxdb | influxdb:2.7 | 8086 | Time-series DB for telemetry |
+| obmp-telegraf | local build | - (host net) | gNMI telemetry collector |
+| obmp-traffic-gen | local build | 5051 (host net) | Scapy traffic generator |
+| obmp-traffic-gen-ui | local build | 5002 (host net) | Traffic generator web UI |
 
 ---
 
@@ -312,6 +336,9 @@ python3 inject.py scenarios
 | `convergence_test` | 10 | Prefixes for timing BGP convergence — announce then check ip_rib_log timestamps |
 | `route_leak` | 10 | Real prefixes re-announced with short AS paths — simulates a route leak (community 65100:999) |
 | `hijack_simulation` | 10 | Prefixes claimed directly by AS 65100 — simulates a prefix hijack (community 65100:hijack) |
+| `te_community_steering` | 15 | Routes tagged with TE communities for color-based steering (65020:100=red, 65020:200=blue, 65020:300=green) |
+| `origin_shift` | 5 | Prefixes with changed origin AS — simulates origin migration for anomaly detection |
+| `path_diversity` | 10 | Same prefixes with different AS paths/MEDs — demonstrates best-path selection |
 
 ### 7.4 Load a scenario
 
@@ -495,6 +522,23 @@ Six learning-focused dashboards in a separate folder, designed to teach BGP conc
 
 > **RPKI note:** The `rpki_validator` table is populated by a cron job in `psql-app` every 2 hours. Dashboard `obmp-learn-04` will show zero counts until the cron runs — check `ENABLE_RPKI=1` in `docker-compose.yml`.
 
+### Advanced Analytics Dashboards (folder: `OBMP-Learning`)
+
+Four advanced dashboards that go beyond basic BMP monitoring, unlocking TE/SR data and providing heuristic analysis.
+
+| Dashboard | UID | What it provides |
+|-----------|-----|-----------------|
+| Database Schema Map | `obmp-learn-07` | Interactive schema reference — live table row counts, entity relationships, column details for all 33 tables and 11 views |
+| TE & Segment Routing Analytics | `obmp-learn-08` | Exposes TE/SR fields from BGP-LS: link bandwidth, admin groups, SRLG, SR SIDs, adjacency SIDs, protection types |
+| Topology Change & Anomaly Detection | `obmp-learn-09` | Heuristic analysis: link state changes over time, origin AS hijack detection, convergence timeline, route consistency |
+| Link Utilization & TE Thought Experiment | `obmp-learn-10` | BGP-LS capacity data (bandwidth, TE metrics) + integration guide for streaming telemetry (gNMI/MDT) |
+
+> **TE/SR data note:** Some TE fields (admin_group, max_link_bw, srlg, sr_adjacency_sids) may be NULL if routers don't advertise those TLVs. Enable `mpls traffic-eng` under IS-IS and `segment-routing mpls` for full data.
+
+### Database Schema Reference
+
+A standalone database schema reference is also available at `DB_SCHEMA.md` in the repo root. It documents all 33 tables, 11 views, TE/SR columns, enum types, and common query patterns.
+
 ---
 
 ## 10. Sanity Checks
@@ -810,3 +854,215 @@ Adjust in `docker-compose.yml` under the `psql-app` service environment block.
 | Variable | Default | Description |
 |----------|---------|-------------|
 | `EXABGP_API` | `http://localhost:5050` | ExaBGP API base URL |
+
+---
+
+## 15. gNMI Streaming Telemetry (Phase 4)
+
+### Overview
+
+gNMI (gRPC Network Management Interface) adds **data-plane visibility** alongside BMP's control-plane monitoring. Telegraf collects real-time interface counters from all 9 IOS-XR routers via gNMI subscriptions and stores them in InfluxDB. Grafana queries InfluxDB for telemetry dashboards.
+
+### Architecture
+
+```
+IOS-XR Routers (9x, gRPC port 57400)
+         |
+    gNMI subscriptions (10s sample)
+         |
+         v
+  obmp-telegraf (telegraf:1.28 + gnmi input plugin)
+    host networking → reaches routers on 10.100.0.x
+         |
+         v
+  obmp-influxdb (influxdb:2.7, port 8086)
+    bucket: "telemetry", org: "openbmp"
+         |
+         v
+  obmp-grafana (InfluxDB datasource, Flux queries)
+    3 dashboards in OBMP-Telemetry folder
+```
+
+### Enabling gRPC on Routers
+
+The routers need gRPC enabled before Telegraf can collect telemetry. A NETCONF script is provided:
+
+```bash
+# From the host (requires ncclient: pip install ncclient)
+cd /home/user/obmp-docker/gnmi
+python3 gnmi_grpc_config.py
+```
+
+This connects to all 9 routers via NETCONF (port 830, credentials webui/cisco) and pushes:
+```
+grpc
+ port 57400
+ no-tls
+```
+
+**Verify on router:**
+```
+show grpc status
+```
+Expected: gRPC listening on port 57400.
+
+### Telemetry Data Collected
+
+Telegraf subscribes to two IOS-XR YANG paths at 10-second intervals:
+
+| Subscription | YANG Path | Data |
+|-------------|-----------|------|
+| interface_counters | `Cisco-IOS-XR-infra-statsd-oper:infra-statistics/interfaces/interface/latest/generic-counters` | bytes/packets in/out, errors, drops, CRC |
+| interface_rates | `Cisco-IOS-XR-infra-statsd-oper:infra-statistics/interfaces/interface/latest/data-rate` | bits/sec in/out, packet rate |
+
+### InfluxDB Access
+
+- **URL:** `http://localhost:8086`
+- **Org:** `openbmp`
+- **Bucket:** `telemetry`
+- **Token:** `openbmp-telemetry-token`
+- **Retention:** 30 days
+
+### Grafana Telemetry Dashboards
+
+Three dashboards in the **OBMP-Telemetry** folder:
+
+| Dashboard | UID | Description |
+|-----------|-----|-------------|
+| Interface Utilization | obmp-telem-01 | Input/output bytes rate, packets rate, top interfaces by throughput |
+| Interface Errors | obmp-telem-02 | CRC errors, input/output errors, drops, overruns |
+| Combined BMP + Telemetry | obmp-telem-03 | Mixed datasource — BGP peer status (PostgreSQL) alongside interface counters (InfluxDB) |
+
+All dashboards have `$router` and `$interface` template variables for filtering.
+
+### Troubleshooting gNMI
+
+```bash
+# Check Telegraf logs for gNMI connection status
+docker logs obmp-telegraf --tail 50
+
+# Verify InfluxDB has data
+curl -s -H "Authorization: Token openbmp-telemetry-token" \
+  "http://localhost:8086/api/v2/query?org=openbmp" \
+  --data-urlencode 'q=from(bucket:"telemetry") |> range(start: -5m) |> limit(n:5)'
+
+# Check InfluxDB health
+curl http://localhost:8086/health
+```
+
+---
+
+## 16. Traffic Generator (Phase 4)
+
+### Overview
+
+A portable, containerized traffic generator with a web UI for RFC 2544 testing and custom packet flows. Built with Scapy + Flask (backend) and Vue 3 + NGINX (frontend). The container supports **dual-mode operation**: sender (generate traffic) or responder (receive/echo packets).
+
+### Accessing the UI
+
+- **Web UI:** `http://localhost:5002`
+- **API:** `http://localhost:5051`
+
+### Dual-Mode Operation
+
+Set via `TRAFFIC_GEN_MODE` environment variable in `docker-compose.yml`:
+
+| Mode | Description |
+|------|-------------|
+| `sender` (default) | Generates traffic, runs RFC 2544 tests, sends custom flows |
+| `responder` | Listens for incoming test packets, echoes/timestamps them, reports receive stats |
+
+**Typical deployment:** One instance as `sender` on the host, optionally a second instance as `responder` on another endpoint. Without a responder, the sender uses ICMP echo for latency measurement (routers respond natively).
+
+### Creating Flows
+
+Use the **Flow Builder** panel (left sidebar) in the UI:
+
+| Field | Default | Description |
+|-------|---------|-------------|
+| Name | - | Human-readable flow name |
+| Destination IP | `10.100.0.100` | Target router IP |
+| Source IP | `10.40.40.202` | Host IP |
+| Protocol | UDP | UDP, TCP, or ICMP |
+| Source Port | 50000 | (UDP/TCP only) |
+| Destination Port | 5001 | (UDP/TCP only) |
+| Frame Size | 512 | Packet size in bytes |
+| Rate (pps) | 1000 | Packets per second |
+| Duration | 30 | Seconds (0 = infinite) |
+| DSCP | 0 | Differentiated Services Code Point |
+
+After creating a flow, use the **Flows** tab to Start/Stop/Delete flows.
+
+### RFC 2544 Testing
+
+Use the **Tests** tab to configure and run RFC 2544 tests:
+
+| Test Type | Description |
+|-----------|-------------|
+| **Throughput** | Binary search for maximum zero-loss forwarding rate |
+| **Latency** | Measure round-trip time at determined throughput rate |
+| **Frame Loss** | Loss percentage vs. offered load curve |
+| **Back-to-Back** | Maximum burst length at line rate with zero loss |
+
+**Parameters:**
+- **Base Flow:** Select a previously created flow as the test template
+- **Frame Sizes:** Standard sizes: 64, 128, 256, 512, 1024, 1280, 1518 bytes
+- **Trial Duration:** Per-frame-size test duration (5–300 sec)
+- **Max Rate (pps):** Upper bound for binary search
+- **Acceptable Loss %:** Threshold for pass/fail
+
+### Quick Presets
+
+Six built-in presets are available in the **Tests** tab:
+
+| Preset | Description |
+|--------|-------------|
+| quick_icmp | ICMP ping to CORE-01 at 10 pps |
+| udp_flood_small | 64-byte UDP at 5000 pps |
+| udp_flood_large | 1518-byte UDP at 1000 pps |
+| rfc2544_throughput | Full throughput test with standard frame sizes |
+| rfc2544_latency | Latency measurement with standard frame sizes |
+| tcp_session | TCP flow at 500 pps |
+
+### API Reference
+
+| Method | Path | Description |
+|--------|------|-------------|
+| GET | `/healthz` | Health check + engine status |
+| GET | `/interfaces` | Available network interfaces |
+| GET | `/mode` | Current mode (sender/responder) |
+| GET/POST | `/flows` | List / create flows |
+| GET/PUT/DELETE | `/flows/<id>` | Get / update / delete flow |
+| POST | `/flows/<id>/start` | Start sending |
+| POST | `/flows/<id>/stop` | Stop sending |
+| GET | `/flows/<id>/stats` | Real-time stats for a flow |
+| GET/POST | `/tests` | List / create RFC 2544 tests |
+| GET | `/tests/<id>` | Test details + results |
+| POST | `/tests/<id>/start` | Start test execution |
+| POST | `/tests/<id>/stop` | Abort test |
+| GET | `/tests/<id>/results` | Exportable results |
+| GET | `/presets` | Available test presets |
+| POST | `/presets/<name>` | Create flow + test from preset |
+| GET | `/stats/history` | Stats ring buffer (300 samples) |
+| GET | `/responder/stats` | Responder-mode receive stats |
+| POST | `/responder/reset` | Reset responder counters |
+
+### Integration with gNMI Telemetry
+
+The key value of combining the traffic generator with gNMI: **send traffic while watching real-time interface counters**.
+
+1. Create a UDP flow targeting a router (e.g., R9K-01 at 10.100.0.1)
+2. Open the Grafana **Interface Utilization** dashboard, select that router
+3. Start the flow — gNMI counters show traffic appearing on the interface
+4. Run an RFC 2544 throughput test — Grafana shows the stepped traffic pattern from binary search iterations
+5. Compare Scapy-reported stats with gNMI-reported counters for cross-validation
+
+The **Combined BMP + Telemetry** dashboard shows both control-plane (BMP BGP updates) and data-plane (gNMI interface counters) side by side, enabling correlation of BGP changes with traffic impact.
+
+### Environment Variables
+
+| Variable | Default | Description |
+|----------|---------|-------------|
+| `TRAFFIC_GEN_API_PORT` | `5051` | Flask API listen port |
+| `TRAFFIC_GEN_MODE` | `sender` | Operating mode: `sender` or `responder` |
+| `INFLUXDB_TOKEN` | `openbmp-telemetry-token` | InfluxDB auth token (Telegraf) |

cml/build-cml-image.sh

@@ -0,0 +1,53 @@
+#!/bin/bash
+# Build the ExaBGP Docker image and export it for CML 2.9 import.
+#
+# Usage:
+#   ./cml/build-cml-image.sh
+#
+# Output:
+#   /tmp/obmp-exabgp.tar  — upload this to CML via:
+#     Tools > Node and Image Definitions > Image Definitions > Manage Image Uploads
+#
+# After upload, also import the node + image definition YAMLs:
+#   Tools > Node and Image Definitions > Import > cml/exabgp-node-definition.yaml
+#   Tools > Node and Image Definitions > Import > cml/exabgp-image-definition.yaml
+
+set -e
+cd "$(dirname "$0")/.."
+
+echo "=== Building ExaBGP Docker image ==="
+docker build -t obmp-exabgp:latest ./exabgp/
+
+echo ""
+echo "=== Exporting image to /tmp/obmp-exabgp.tar ==="
+docker save -o /tmp/obmp-exabgp.tar obmp-exabgp:latest
+
+echo ""
+echo "=== Image details ==="
+SIZE=$(du -h /tmp/obmp-exabgp.tar | cut -f1)
+echo "  File: /tmp/obmp-exabgp.tar ($SIZE)"
+
+SHA=$(sha256sum /tmp/obmp-exabgp.tar | awk '{print $1}')
+echo "  SHA256: $SHA"
+
+IMAGE_ID=$(docker image inspect obmp-exabgp:latest --format='{{.Id}}')
+echo "  Image ID: $IMAGE_ID"
+
+echo ""
+echo "=== Next steps ==="
+echo "1. Update cml/exabgp-image-definition.yaml with:"
+echo "   sha256: $SHA"
+echo ""
+echo "2. Upload to CML:"
+echo "   a. Tools > Node and Image Definitions > Import"
+echo "      Upload: cml/exabgp-node-definition.yaml"
+echo "   b. Tools > Node and Image Definitions > Import"
+echo "      Upload: cml/exabgp-image-definition.yaml"
+echo "   c. Tools > Node and Image Definitions > Image Definitions > Manage Image Uploads"
+echo "      Upload: /tmp/obmp-exabgp.tar"
+echo ""
+echo "3. In your CML lab topology:"
+echo "   a. Drag 'ExaBGP Route Injector' from the node palette"
+echo "   b. Draw links to CORE-01 and CORE-02"
+echo "   c. Edit the boot.sh in the node config to set correct IPs"
+echo "   d. Start the node"

cml/build-xrd-image.sh

@@ -0,0 +1,62 @@
+#!/bin/bash
+# Export the XRd control-plane Docker image for CML 2.9 import.
+#
+# Usage:
+#   ./cml/build-xrd-image.sh
+#
+# The XRd image already exists locally (ios-xr/xrd-control-plane:25.1.1).
+# This script just exports it to a .tar file for CML upload.
+
+set -e
+
+IMAGE="ios-xr/xrd-control-plane:25.1.1"
+OUTPUT="/tmp/xrd-control-plane.tar"
+
+echo "=== Verifying XRd image exists ==="
+if ! docker image inspect "$IMAGE" >/dev/null 2>&1; then
+    echo "ERROR: Image $IMAGE not found locally."
+    echo "Check with: docker images | grep xrd"
+    exit 1
+fi
+
+echo "  Image: $IMAGE"
+SIZE=$(docker image inspect "$IMAGE" --format='{{.Size}}' | numfmt --to=iec 2>/dev/null || echo "unknown")
+echo "  Size: $SIZE"
+
+echo ""
+echo "=== Exporting image to $OUTPUT ==="
+echo "  (This may take a minute for ~1.3GB image...)"
+docker save -o "$OUTPUT" "$IMAGE"
+
+echo ""
+echo "=== Export complete ==="
+TAR_SIZE=$(du -h "$OUTPUT" | cut -f1)
+echo "  File: $OUTPUT ($TAR_SIZE)"
+
+SHA=$(sha256sum "$OUTPUT" | awk '{print $1}')
+echo "  SHA256: $SHA"
+
+echo ""
+echo "=== Next steps ==="
+echo "1. Update cml/xrd-image-definition.yaml with:"
+echo "   sha256: $SHA"
+echo ""
+echo "2. Upload to CML:"
+echo "   a. Tools > Node and Image Definitions > Import"
+echo "      Upload: cml/xrd-node-definition.yaml"
+echo "   b. Tools > Node and Image Definitions > Import"
+echo "      Upload: cml/xrd-image-definition.yaml"
+echo "   c. Tools > Node and Image Definitions > Image Definitions > Manage Image Uploads"
+echo "      Upload: $OUTPUT"
+echo "      (For large files, consider SCP to CML server instead)"
+echo ""
+echo "3. In your CML lab topology:"
+echo "   a. Drag 'XRd Control-Plane (IOS-XR)' from the node palette"
+echo "   b. Draw links to CORE-01 (→Gi0/0/0/0) and CORE-02 (→Gi0/0/0/1)"
+echo "   c. Edit xrd-startup.cfg if needed (IPs, BMP target, etc.)"
+echo "   d. Start the node (allow ~3-5 min for XRd boot)"
+echo ""
+echo "4. After boot, verify via XRd console:"
+echo "   show isis adjacency"
+echo "   show bgp summary"
+echo "   show bmp server 1"

cml/exabgp-image-definition.yaml

@@ -0,0 +1,10 @@
+id: obmp-exabgp.latest
+node_definition_id: obmp-exabgp
+description: |-
+  OpenBMP ExaBGP Route Injector
+  Python 3.11 + ExaBGP + Flask API for BGP route injection testing.
+label: ExaBGP Route Injector
+disk_image: obmp-exabgp.tar
+read_only: false
+schema_version: 0.0.1
+# sha256: <UPDATE after running: sha256sum /tmp/obmp-exabgp.tar>

cml/exabgp-node-definition.yaml

@@ -0,0 +1,112 @@
+id: obmp-exabgp
+boot:
+  timeout: 60
+  completed:
+    - "ExaBGP Route Injector"
+  uses_regex: false
+sim:
+  linux_native:
+    libvirt_domain_driver: docker
+    driver: ubuntu
+    ram: 512
+    cpus: 1
+    cpu_limit: 100
+    video:
+      memory: 1
+general:
+  nature: server
+  description: OpenBMP ExaBGP Route Injector (Docker container)
+  read_only: false
+configuration:
+  generator:
+    driver: null
+  provisioning:
+    files:
+      - editable: false
+        name: config.json
+        content: |-
+          {
+            "docker": {
+              "image": "obmp-exabgp:latest",
+              "mounts": [
+                "type=bind,source=cfg/boot.sh,target=/cml-boot.sh"
+              ],
+              "misc_args": [],
+              "env": [
+                "EXABGP_LOCAL_AS=65100",
+                "EXABGP_PEER_AS=65020",
+                "EXABGP_API_PORT=5050"
+              ]
+            },
+            "shell": "/bin/bash",
+            "day0cmd": [ "/bin/bash", "/cml-boot.sh" ],
+            "busybox": false
+          }
+      - editable: true
+        name: boot.sh
+        content: |-
+          #!/bin/bash
+          # CML boot script for ExaBGP container
+          # Configures data-plane interfaces before starting ExaBGP
+          #
+          # Interface mapping (assigned by CML topology links):
+          #   eth0 = first connected interface (data-plane link 1)
+          #   eth1 = second connected interface (data-plane link 2)
+          #   ...additional interfaces as connected in topology
+          #
+          # Edit the IPs below to match your topology addressing.
+          # These are examples using 10.120.x.x/30 point-to-point links.
+
+          # --- Data-plane interface configuration ---
+          # Link to CORE-01: ExaBGP=10.120.1.2/30, CORE-01=10.120.1.1/30
+          ip address add 10.120.1.2/30 dev eth0
+          ip link set dev eth0 up
+
+          # Link to CORE-02: ExaBGP=10.120.2.2/30, CORE-02=10.120.2.1/30
+          ip address add 10.120.2.2/30 dev eth1
+          ip link set dev eth1 up
+
+          # --- Set environment for ExaBGP peering ---
+          export EXABGP_LOCAL_IP=10.120.1.2
+          export EXABGP_PEER_1=10.120.1.1
+          export EXABGP_PEER_2=10.120.2.1
+
+          # --- Start ExaBGP ---
+          exec /bin/bash /exabgp/startup.sh
+    media_type: raw
+    volume_name: cfg
+device:
+  interfaces:
+    serial_ports: 1
+    physical:
+      - eth0
+      - eth1
+      - eth2
+      - eth3
+    has_loopback_zero: false
+    default_count: 2
+ui:
+  label_prefix: exabgp-
+  icon: server
+  label: ExaBGP Route Injector
+  visible: true
+  group: Others
+  description: |-
+    OpenBMP ExaBGP Route Injector
+    BGP route injection for OpenBMP testing.
+    AS 65100 (eBGP) peering with IOS-XR routers (AS 65020).
+    Flask API on port 5050 for route management.
+inherited:
+  image:
+    ram: true
+    cpus: false
+    data_volume: false
+    boot_disk_size: false
+    cpu_limit: false
+  node:
+    ram: true
+    cpus: false
+    data_volume: false
+    boot_disk_size: false
+    cpu_limit: false
+schema_version: 0.0.1

cml/xrd-image-definition.yaml

@@ -0,0 +1,10 @@
+id: xrd-control-plane.25.1.1
+node_definition_id: xrd-control-plane-rr
+description: |-
+  Cisco XRd Control-Plane 25.1.1
+  IOS-XR containerized routing daemon for BGP/IS-IS/BMP workloads.
+label: XRd Control-Plane 25.1.1
+disk_image: xrd-control-plane.tar
+read_only: false
+schema_version: 0.0.1
+# sha256: <UPDATE after running: sha256sum /tmp/xrd-control-plane.tar>

cml/xrd-node-definition.yaml

@@ -0,0 +1,179 @@
+id: xrd-control-plane-rr
+boot:
+  timeout: 300
+  completed:
+    - "IOS XR RUN"
+  uses_regex: false
+sim:
+  linux_native:
+    libvirt_domain_driver: docker
+    driver: ubuntu
+    ram: 2048
+    cpus: 2
+    cpu_limit: 100
+    video:
+      memory: 1
+general:
+  nature: router
+  description: Cisco XRd Control-Plane - IOS-XR containerized routing daemon
+  read_only: false
+configuration:
+  generator:
+    driver: null
+  provisioning:
+    files:
+      - editable: false
+        name: config.json
+        content: |-
+          {
+            "docker": {
+              "image": "ios-xr/xrd-control-plane:25.1.1",
+              "mounts": [
+                "type=bind,source=cfg/boot.sh,target=/cml-boot.sh",
+                "type=bind,source=cfg/xrd-startup.cfg,target=/etc/xrd/startup.cfg"
+              ],
+              "misc_args": [
+                "--privileged"
+              ],
+              "env": [
+                "XR_STARTUP_CFG=/etc/xrd/startup.cfg",
+                "XR_MGMT_INTERFACES=linux:eth0,chksum",
+                "XR_INTERFACES=linux:eth1,xr_name=Gi0/0/0/0;linux:eth2,xr_name=Gi0/0/0/1;linux:eth3,xr_name=Gi0/0/0/2;linux:eth4,xr_name=Gi0/0/0/3"
+              ]
+            },
+            "shell": "/bin/bash",
+            "day0cmd": [ "/bin/bash", "/cml-boot.sh" ],
+            "busybox": false
+          }
+      - editable: true
+        name: boot.sh
+        content: |-
+          #!/bin/bash
+          # CML boot wrapper for XRd control-plane.
+          # XRd handles its own init — this script configures
+          # data-plane interfaces before XRd starts.
+          #
+          # Interface mapping (set via XR_INTERFACES env var):
+          #   eth0  = MgmtEth0/RP0/CPU0/0 (CML mgmt)
+          #   eth1  = Gi0/0/0/0 (data-plane link 1, e.g. to CORE-01)
+          #   eth2  = Gi0/0/0/1 (data-plane link 2, e.g. to CORE-02)
+          #   eth3+ = Gi0/0/0/2+ (additional links)
+          #
+          # Linux-level IP config is handled by XRd via startup.cfg.
+          # Just ensure interfaces are up.
+          for iface in eth0 eth1 eth2 eth3 eth4; do
+            [ -d /sys/class/net/$iface ] && ip link set dev $iface up
+          done
+
+          # XRd entrypoint
+          exec /usr/sbin/xrd
+      - editable: true
+        name: xrd-startup.cfg
+        content: |-
+          !! XRd Control-Plane - Third Route Reflector (RR3)
+          !! Peers with CORE-01 and CORE-02 as RR mesh (non-client iBGP)
+          !! Sends BMP to OpenBMP collector at 10.40.40.202:5000
+          !!
+          hostname XRd-RR3
+          !
+          interface Loopback0
+           ipv4 address 10.10.255.30 255.255.255.255
+          !
+          interface Gi0/0/0/0
+           description to-CORE-01
+           ipv4 address 10.120.3.2 255.255.255.252
+           no shutdown
+          !
+          interface Gi0/0/0/1
+           description to-CORE-02
+           ipv4 address 10.120.4.2 255.255.255.252
+           no shutdown
+          !
+          router isis 1
+           is-type level-2-only
+           net 49.0001.0100.1000.0030.00
+           address-family ipv4 unicast
+            metric-style wide
+           !
+           interface Loopback0
+            passive
+            address-family ipv4 unicast
+            !
+           !
+           interface Gi0/0/0/0
+            point-to-point
+            address-family ipv4 unicast
+            !
+           !
+           interface Gi0/0/0/1
+            point-to-point
+            address-family ipv4 unicast
+            !
+           !
+          !
+          router bgp 65020
+           bgp router-id 10.10.255.30
+           address-family ipv4 unicast
+           !
+           neighbor 10.10.255.0
+            remote-as 65020
+            update-source Loopback0
+            address-family ipv4 unicast
+            !
+           !
+           neighbor 10.10.255.20
+            remote-as 65020
+            update-source Loopback0
+            address-family ipv4 unicast
+            !
+           !
+          !
+          bmp server 1
+           host 10.40.40.202 port 5000
+           description OpenBMP
+           update-source Gi0/0/0/0
+           flapping-delay 60
+           initial-delay 5
+           stats-reporting-period 300
+           initial-refresh delay 30 spread 2
+          !
+          ssh server v2
+          end
+    media_type: raw
+    volume_name: cfg
+device:
+  interfaces:
+    serial_ports: 1
+    physical:
+      - eth0
+      - eth1
+      - eth2
+      - eth3
+      - eth4
+    has_loopback_zero: false
+    default_count: 3
+ui:
+  label_prefix: xrd-
+  icon: router
+  label: XRd Control-Plane (IOS-XR)
+  visible: true
+  group: Cisco
+  description: |-
+    Cisco XRd Control-Plane (IOS-XR 25.1.1)
+    Containerized IOS-XR routing daemon for control-plane workloads.
+    Full BGP, IS-IS, BMP, NETCONF support.
+    Configured as third Route Reflector (RR3) with BMP to OpenBMP.
+inherited:
+  image:
+    ram: true
+    cpus: true
+    data_volume: false
+    boot_disk_size: false
+    cpu_limit: false
+  node:
+    ram: true
+    cpus: true
+    data_volume: false
+    boot_disk_size: false
+    cpu_limit: false
+schema_version: 0.0.1

docker-compose.yml

@@ -92,7 +92,13 @@ services:
       - ${OBMP_DATA_ROOT}/grafana/provisioning:/etc/grafana/provisioning/
     environment:
       - GF_SECURITY_ADMIN_PASSWORD=openbmp
-      - GF_AUTH_ANONYMOUS_ENABLED=true
+      - GF_AUTH_ANONYMOUS_ENABLED=false
+      - GF_SERVER_ROOT_URL=https://bmp.apodacalab.com/grafana/
+      - GF_SERVER_SERVE_FROM_SUB_PATH=true
+      - GF_AUTH_PROXY_ENABLED=true
+      - GF_AUTH_PROXY_HEADER_NAME=Remote-User
+      - GF_AUTH_PROXY_HEADER_PROPERTY=username
+      - GF_AUTH_PROXY_AUTO_SIGN_UP=true
       - GF_USERS_HOME_PAGE=d/obmp-home/obmp-home
       - GF_INSTALL_PLUGINS=agenty-flowcharting-panel,grafana-piechart-panel,grafana-worldmap-panel,grafana-simple-json-datasource,vonage-status-panel
 
@@ -231,6 +237,83 @@ services:
     network_mode: host
     # Serves on port 5001 (host network, defined in nginx.conf)
 
+  # --- Phase 4: gNMI Streaming Telemetry ---
+
+  influxdb:
+    restart: unless-stopped
+    container_name: obmp-influxdb
+    image: influxdb:2.7
+    ports:
+      - "8086:8086"
+    volumes:
+      - ${OBMP_DATA_ROOT}/influxdb:/var/lib/influxdb2
+    environment:
+      - DOCKER_INFLUXDB_INIT_MODE=setup
+      - DOCKER_INFLUXDB_INIT_USERNAME=openbmp
+      - DOCKER_INFLUXDB_INIT_PASSWORD=openbmp123
+      - DOCKER_INFLUXDB_INIT_ORG=openbmp
+      - DOCKER_INFLUXDB_INIT_BUCKET=telemetry
+      - DOCKER_INFLUXDB_INIT_ADMIN_TOKEN=openbmp-telemetry-token
+      - DOCKER_INFLUXDB_INIT_RETENTION=30d
+
+  telegraf:
+    restart: unless-stopped
+    container_name: obmp-telegraf
+    build:
+      context: ./telegraf
+      dockerfile: Dockerfile
+    network_mode: host
+    depends_on:
+      - influxdb
+    environment:
+      - INFLUXDB_TOKEN=openbmp-telemetry-token
+
+  # --- Phase 4: Traffic Generator ---
+
+  traffic-gen:
+    restart: unless-stopped
+    container_name: obmp-traffic-gen
+    build:
+      context: ./traffic-gen
+      dockerfile: Dockerfile
+    network_mode: host
+    cap_add:
+      - NET_RAW
+      - NET_ADMIN
+    environment:
+      - TRAFFIC_GEN_PORT=5051
+      - TRAFFIC_GEN_MODE=sender
+      - RESPONDER_URL=http://172.30.0.10:5053
+
+  traffic-gen-ui:
+    restart: unless-stopped
+    container_name: obmp-traffic-gen-ui
+    build:
+      context: ./traffic-gen-ui
+      dockerfile: Dockerfile
+    network_mode: host
+    # Serves on port 5002 (host network, defined in nginx.conf)
+
+  traffic-gen-responder:
+    restart: unless-stopped
+    container_name: obmp-traffic-gen-responder
+    build:
+      context: ./traffic-gen
+      dockerfile: Dockerfile
+    cap_add:
+      - NET_RAW
+      - NET_ADMIN
+    environment:
+      - TRAFFIC_GEN_PORT=5053
+      - TRAFFIC_GEN_MODE=responder
+      - TRAFFIC_GEN_RESPONDER_MODE=echo
+      - TRAFFIC_GEN_INTERFACE=eth0
+    networks:
+      traffic-test-net:
+        ipv4_address: 172.30.0.10
+    ports:
+      - "5053:5053"
+
   whois:
     restart: unless-stopped
     container_name: obmp-whois
@@ -249,3 +332,30 @@ services:
       - POSTGRES_DB=openbmp
       - POSTGRES_HOST=obmp-psql
       - POSTGRES_PORT=5432
+
+  authelia:
+    restart: unless-stopped
+    container_name: obmp-authelia
+    image: authelia/authelia:4.38
+    ports:
+      - "9091:9091"
+    volumes:
+      - ${OBMP_DATA_ROOT}/authelia:/config
+    environment:
+      - TZ=UTC
+
+  portal:
+    restart: unless-stopped
+    container_name: obmp-portal
+    image: nginx:alpine
+    ports:
+      - "8080:80"
+    volumes:
+      - ./portal:/usr/share/nginx/html:ro
+
+networks:
+  traffic-test-net:
+    driver: bridge
+    ipam:
+      config:
+        - subnet: 172.30.0.0/24

docs/ROADMAP.md

@@ -0,0 +1,269 @@
+# OpenBMP Platform Roadmap
+
+## Context
+
+This BMP monitoring platform is being developed against CML virtual labs (IOS-XR) and will be deployed into an ISP production network running IOS-XR and Juniper routers/route reflectors. The two tracks share a common foundation: configuration must be environment-agnostic so the same stack runs identically against virtual or production routers.
+
+Currently, router IPs, AS numbers, and credentials are hardcoded across 8+ files, tightly coupling the stack to a single CML lab. This roadmap addresses both the multi-lab development workflow and production deployment.
+
+---
+
+## Track A: Configuration Centralization (Foundation for Both Tracks)
+
+### A1. Create `inventory.yaml` — unified topology inventory
+
+**File**: `inventory.yaml` (new)
+
+Single source of truth for all environments. Structure:
+
+```yaml
+platform:
+  host_ip: 10.40.40.202
+  bmp_port: 5000
+  exabgp_port: 5050
+
+environments:
+  cml-lab1:
+    type: cml            # cml | production
+    description: "CML RR cluster - 9 IOS-XR virtual routers"
+    cml_server: "https://10.40.40.174"
+    cml_user: webui
+    bgp_as: 65020
+    netconf: { user: webui, password: cisco, port: 830 }
+    exabgp:
+      local_as: 65100
+      peers:
+        - { ip: 10.100.0.100, name: CORE-01, peer_as: 65020 }
+        - { ip: 10.100.0.200, name: CORE-02, peer_as: 65020 }
+    routers:
+      CORE-01:  { mgmt: 10.100.0.100, loopback: 10.10.255.0, role: rr, vendor: iosxr, gnmi: true }
+      CORE-02:  { mgmt: 10.100.0.200, loopback: 10.10.255.20, role: rr, vendor: iosxr, gnmi: true }
+      R9K-01:   { mgmt: 10.100.0.1, loopback: 10.10.255.1, role: client, vendor: iosxr }
+      # ...
+
+  cml-lab2:
+    type: cml
+    description: "Second CML Lab (TBD topology)"
+    cml_server: "https://<lab2-ip>"
+    routers: {}
+
+  production:
+    type: production
+    description: "ISP production network"
+    bgp_as: <prod-as>
+    netconf: { user: <prod-user>, port: 830 }
+    routers:
+      # IOS-XR and Juniper RRs + routers
+      PROD-RR1: { mgmt: x.x.x.x, role: rr, vendor: iosxr, gnmi: true }
+      PROD-RR2: { mgmt: x.x.x.x, role: rr, vendor: junos }
+      # ...
+```
+
+Key design decisions:
+- `vendor: iosxr | junos` — drives NETCONF dialect, gNMI paths, and config templates
+- `type: cml | production` — CML environments have `cml_server` for API automation; production does not
+- Credentials in `inventory.yaml` (gitignored) or pulled from env vars
+
+### A2. Create `config_loader.py` — Python inventory helper
+
+**File**: `config_loader.py` (new)
+
+Functions: `get_env(name)`, `get_all_routers()`, `get_routers_by_vendor(vendor)`, `get_exabgp_peers()`, `get_gnmi_targets()`, `get_routers_for_env(env_name)`
+
+### A3. Refactor hardcoded Python scripts
+
+Replace `ROUTERS` dicts/lists with `config_loader` calls:
+- `exabgp/route_diversity_config.py` (line 47)
+- `exabgp/bgpls_config.py` (line 35)
+- `gnmi/gnmi_grpc_config.py` (line 25)
+
+### A4. Expand `.env` and parameterize `docker-compose.yml`
+
+Add to `.env`:
+```env
+OBMP_DATA_ROOT=/var/openbmp
+DOCKER_HOST_IP=10.40.40.202
+EXABGP_LOCAL_IP=10.40.40.202
+EXABGP_LOCAL_AS=65100
+EXABGP_PEER_AS=65020
+EXABGP_PEER_1=10.100.0.100
+EXABGP_PEER_2=10.100.0.200
+```
+
+Replace hardcoded IPs in `docker-compose.yml` (Kafka listener, ExaBGP env vars).
+
+### A5. Telegraf config parameterization
+
+Replace hardcoded gNMI addresses in `telegraf/telegraf.conf` with env var substitution. Pass `GNMI_TARGETS` from docker-compose.yml.
+
+### A6. Fix InfluxDB datasource URL
+
+`obmp-grafana/provisioning/datasources/influxdb-ds.yml`: replace `http://10.40.40.202:8086` with `http://obmp-influxdb:8086`.
+
+---
+
+## Track B: Multi-Lab CML Development
+
+### B1. Dynamic ExaBGP multi-peer support
+
+**File**: `exabgp/startup.sh`
+
+Accept `EXABGP_PEERS` env var (comma-separated `ip:as:description`), generate N neighbor blocks. Keep `PEER_1`/`PEER_2` fallback.
+
+### B2. CML API client module
+
+**File**: `cml/cml_client.py` (new)
+
+Python module using `virl2_client` SDK:
+- Connect to CML server (creds from `inventory.yaml`)
+- Upload node/image definitions
+- Import/export topology YAML
+- Start/stop/destroy labs
+- Get node status
+
+### B3. Topology template system
+
+**File**: `cml/templates/xrd_rr.j2` (new)
+
+Jinja2 templates for XRd startup config. Parameterize: hostname, loopback, link IPs, IS-IS NET, BGP AS, neighbor IPs, BMP target.
+
+### B4. CLI deployment tool
+
+**File**: `cml/deploy.py` (new)
+
+```bash
+python3 cml/deploy.py --env cml-lab1 status
+python3 cml/deploy.py --env cml-lab1 upload-images
+python3 cml/deploy.py --env cml-lab2 create
+python3 cml/deploy.py --env cml-lab2 start
+python3 cml/deploy.py --env cml-lab2 destroy
+```
+
+### B5. Update build scripts with API push
+
+`cml/build-cml-image.sh` and `cml/build-xrd-image.sh` get `--push <env-name>` flag.
+
+---
+
+## Track C: Production ISP Deployment
+
+### C1. Multi-vendor NETCONF support
+
+Current scripts assume IOS-XR NETCONF only. For Juniper RRs:
+- `config_loader.py` provides `vendor` field per router
+- NETCONF scripts branch on vendor for dialect differences (`device_params='iosxr'` vs `device_params='junos'`)
+- Route diversity, BGP-LS config scripts get Junos templates alongside IOS-XR
+
+### C2. Multi-vendor gNMI paths
+
+Telegraf gNMI subscriptions currently use OpenConfig paths which work for both IOS-XR and Junos, but:
+- Verify Juniper gNMI support on target hardware
+- Add vendor-specific path overrides in `inventory.yaml` if needed
+- Telegraf can subscribe to multiple targets with different configs via `[[inputs.gnmi]]` blocks
+
+### C3. BMP considerations for production
+
+- BMP collector (port 5000) accepts connections from any router — no changes needed
+- Production routers need BMP config pushed (manual or via NETCONF automation)
+- Consider: separate BMP server IDs per environment for dashboard filtering
+- Juniper BMP config differs from IOS-XR — add Junos BMP config templates
+
+### C4. Dashboard multi-environment awareness
+
+- Add a Grafana template variable for environment filtering (by router name prefix or a tag)
+- Consider a "Network Overview" dashboard that shows all environments side-by-side
+- Existing dashboards work as-is — router dropdowns will show all BMP-reporting routers
+
+### C5. Security hardening for production
+
+- Move credentials out of `inventory.yaml` into environment variables or a secrets manager
+- Authelia config: stronger passwords, TOTP enforcement, session timeouts
+- PostgreSQL: restrict access, enable SSL
+- Kafka: consider authentication if exposed beyond localhost
+- BMP port: firewall to only accept connections from known router management IPs
+
+### C6. Scalability considerations
+
+- Monitor PostgreSQL disk usage and query performance with production-scale RIBs
+- TimescaleDB compression policies for historical data (ip_rib_log, ls_*_log)
+- Kafka topic partitioning if message throughput is high
+- Consider read replicas or materialized views for heavy Grafana queries
+
+---
+
+## Track D: Packaging & Distribution
+
+### D1. Configuration templates
+
+- `inventory.yaml.example` — documented example with placeholder values
+- `.env.example` — all environment variables with descriptions
+
+### D2. Bootstrap script
+
+`setup.sh` that:
+- Creates required directories (`$OBMP_DATA_ROOT/authelia`, etc.)
+- Copies example configs if originals don't exist
+- Validates inventory.yaml syntax
+- Generates Telegraf config from inventory
+
+### D3. Published Docker images
+
+Push custom images to a registry (Docker Hub or GHCR):
+- `obmp-exabgp`
+- `obmp-exabgp-ui`
+- `obmp-traffic-gen`
+- `obmp-traffic-gen-ui`
+- `obmp-portal`
+
+Replace `build:` with `image:` in docker-compose.yml (keep build as override).
+
+### D4. Documentation
+
+- `docs/quickstart.md` — 5-minute setup guide
+- `docs/adding-a-lab.md` — how to add a CML lab environment
+- `docs/production-deployment.md` — production hardening checklist
+- `docs/architecture.md` — system diagram, data flow, port map
+
+---
+
+## Implementation Order
+
+| Priority | Step | Track | Description |
+|----------|------|-------|-------------|
+| 1 | A1 | Foundation | Create `inventory.yaml` |
+| 2 | A2 | Foundation | Create `config_loader.py` |
+| 3 | A3 | Foundation | Refactor hardcoded Python scripts |
+| 4 | A4 | Foundation | Parameterize `.env` + docker-compose |
+| 5 | A5-A6 | Foundation | Telegraf + InfluxDB datasource fixes |
+| 6 | B1 | CML Dev | Dynamic ExaBGP multi-peer |
+| 7 | B2-B4 | CML Dev | CML API client + deploy CLI |
+| 8 | C1 | Production | Multi-vendor NETCONF (Junos support) |
+| 9 | C3 | Production | Junos BMP config templates |
+| 10 | C5 | Production | Security hardening |
+| 11 | D1-D2 | Packaging | Config templates + bootstrap script |
+| 12 | D3 | Packaging | Publish Docker images to registry |
+| 13 | D4 | Packaging | Documentation |
+
+Steps 1-5 (Track A) unblock everything else. Steps 6-7 and 8-10 can proceed in parallel once the foundation is in place.
+
+---
+
+## Verification
+
+1. **Config centralization**: Change a router IP in `inventory.yaml`, verify all scripts pick it up
+2. **ExaBGP multi-peer**: Set 3+ peers, restart, verify BGP sessions establish
+3. **CML API**: `deploy.py --env cml-lab1 status` connects and lists nodes
+4. **BMP multi-source**: Router from lab 2 sends BMP, appears in `SELECT * FROM routers` and Grafana
+5. **Junos support**: NETCONF script connects to a Juniper router, pushes config
+6. **Production dry-run**: Point a test router from the ISP network at the collector, verify end-to-end
+7. **Clean deploy**: Clone repo on a fresh host, run `setup.sh`, `docker compose up`, confirm stack starts
+
+---
+
+## Risks
+
+- **Router name collisions**: Enforce unique hostnames across all environments
+- **Address space overlap**: Each environment needs distinct management subnets
+- **Juniper BMP differences**: Junos BMP implementation may differ in supported tables/TLVs — test early
+- **Production scale**: 500K-route labs are slow; production full tables will stress PostgreSQL more
+- **Credentials in inventory**: Must be gitignored; consider env var fallback for CI/CD

exabgp-ui/src/App.vue

@@ -41,6 +41,7 @@
           <AnnounceForm v-else-if="activeTab === 'inject'" @routes-changed="fetchRoutes" />
           <PeerStatus v-else-if="activeTab === 'peers'" :peers="peers" />
           <ChurnControl v-else-if="activeTab === 'churn'" />
+          <FullTable v-else-if="activeTab === 'full-table'" @routes-changed="fetchRoutes" />
         </div>
       </main>
     </div>
@@ -63,6 +64,7 @@ import RouteTable from './components/RouteTable.vue'
 import AnnounceForm from './components/AnnounceForm.vue'
 import PeerStatus from './components/PeerStatus.vue'
 import ChurnControl from './components/ChurnControl.vue'
+import FullTable from './components/FullTable.vue'
 
 const health = ref(null)
 const routes = ref([])
@@ -75,6 +77,7 @@ const tabs = [
   { id: 'inject', label: 'Inject' },
   { id: 'peers', label: 'Peers' },
   { id: 'churn', label: 'Churn' },
+  { id: 'full-table', label: 'Full Table' },
 ]
 
 async function fetchHealth() {

exabgp-ui/src/api.js

@@ -1,4 +1,4 @@
-const BASE = '/api'
+const BASE = '/exabgp/api'
 
 async function req(method, path, body) {
   const opts = { method, headers: { 'Content-Type': 'application/json' } }
@@ -18,4 +18,7 @@ export const api = {
   announce: payload => req('POST', '/announce', payload),
   withdraw: prefixes => req('POST', '/withdraw', { prefixes }),
   withdrawAll: () => req('POST', '/withdraw/all'),
+  fullTableStart: (count, batchSize) => req('POST', '/full-table/start', { count, batch_size: batchSize }),
+  fullTableStatus: () => req('GET', '/full-table/status'),
+  fullTableStop: () => req('POST', '/full-table/stop'),
 }

exabgp-ui/src/components/FullTable.vue

@@ -0,0 +1,477 @@
+<template>
+  <div class="full-table">
+    <h2 class="section-title">Full Table Injection</h2>
+    <p class="section-desc">
+      Inject a realistic IPv4 routing table into ExaBGP for stress testing.
+      Routes are generated with varied AS paths, prefix lengths, and communities matching real DFZ distribution.
+    </p>
+
+    <div class="config-card">
+      <!-- Level selector -->
+      <div class="form-group">
+        <label>Table Size</label>
+        <div class="level-grid">
+          <button
+            v-for="level in levels"
+            :key="level.count"
+            class="level-btn"
+            :class="{ selected: selectedCount === level.count }"
+            :disabled="injecting"
+            @click="selectedCount = level.count"
+          >
+            <span class="level-count">{{ level.label }}</span>
+            <span class="level-desc">{{ level.desc }}</span>
+          </button>
+        </div>
+      </div>
+
+      <!-- Custom count -->
+      <div class="form-group">
+        <label>Custom Count</label>
+        <input
+          v-model.number="selectedCount"
+          type="number"
+          min="100"
+          max="950000"
+          step="1000"
+          :disabled="injecting"
+          class="mono-input"
+        />
+      </div>
+
+      <!-- Action buttons -->
+      <div class="action-row">
+        <button v-if="!injecting" class="btn-start" @click="startInjection" :disabled="!selectedCount">
+          <span>&#9654;</span> Inject {{ formatNum(selectedCount) }} Routes
+        </button>
+        <button v-else class="btn-stop" @click="stopInjection">
+          <span>&#9632;</span> Stop Injection
+        </button>
+        <button
+          v-if="!injecting && lastCompleted"
+          class="btn-withdraw"
+          @click="withdrawAll"
+          :disabled="withdrawing"
+        >
+          {{ withdrawing ? 'Withdrawing...' : 'Withdraw All' }}
+        </button>
+      </div>
+    </div>
+
+    <!-- Status display -->
+    <div v-if="injecting || statusMsg" class="status-card">
+      <div class="status-header">
+        <span class="status-dot" :class="injecting ? 'dot-active' : 'dot-idle'"></span>
+        <span class="status-text">{{ statusMsg || 'Idle' }}</span>
+      </div>
+
+      <!-- Progress bar -->
+      <div v-if="state.total > 0" class="progress-section">
+        <div class="progress-labels">
+          <span>{{ formatNum(state.injected) }} / {{ formatNum(state.total) }}</span>
+          <span>{{ state.progress_pct || 0 }}%</span>
+        </div>
+        <div class="progress-track">
+          <div class="progress-fill" :style="{ width: (state.progress_pct || 0) + '%' }"></div>
+        </div>
+      </div>
+
+      <!-- Stats row -->
+      <div v-if="state.total > 0" class="stats-row">
+        <div class="stat-item">
+          <span class="stat-label">Rate</span>
+          <span class="stat-val">{{ formatNum(state.rate_pps || 0) }}/s</span>
+        </div>
+        <div class="stat-item">
+          <span class="stat-label">Elapsed</span>
+          <span class="stat-val">{{ state.elapsed_sec || 0 }}s</span>
+        </div>
+        <div class="stat-item">
+          <span class="stat-label">Active Routes</span>
+          <span class="stat-val">{{ formatNum(state.active_routes || 0) }}</span>
+        </div>
+      </div>
+
+      <!-- Error -->
+      <div v-if="state.error" class="inject-error">{{ state.error }}</div>
+    </div>
+  </div>
+</template>
+
+<script setup>
+import { ref, onUnmounted } from 'vue'
+import { api } from '../api.js'
+
+const emit = defineEmits(['routes-changed'])
+
+const levels = [
+  { count: 1000,   label: '1K',   desc: 'Quick test' },
+  { count: 10000,  label: '10K',  desc: 'Light load' },
+  { count: 50000,  label: '50K',  desc: 'Medium load' },
+  { count: 100000, label: '100K', desc: 'Stress test' },
+  { count: 500000, label: '500K', desc: 'Heavy load' },
+  { count: 900000, label: '900K', desc: 'Full DFZ' },
+]
+
+const selectedCount = ref(10000)
+const injecting = ref(false)
+const statusMsg = ref('')
+const lastCompleted = ref(false)
+const withdrawing = ref(false)
+const state = ref({})
+
+let pollTimer = null
+
+function formatNum(n) {
+  if (n == null) return '0'
+  return Number(n).toLocaleString()
+}
+
+async function startInjection() {
+  try {
+    statusMsg.value = 'Starting injection...'
+    injecting.value = true
+    lastCompleted.value = false
+    state.value = {}
+    await api.fullTableStart(selectedCount.value, 1000)
+    startPolling()
+  } catch (e) {
+    statusMsg.value = `Start failed: ${e.message}`
+    injecting.value = false
+  }
+}
+
+async function stopInjection() {
+  try {
+    await api.fullTableStop()
+    statusMsg.value = 'Stop requested...'
+  } catch (e) {
+    statusMsg.value = `Stop failed: ${e.message}`
+  }
+}
+
+async function withdrawAll() {
+  withdrawing.value = true
+  try {
+    const data = await api.withdrawAll()
+    statusMsg.value = `Withdrew ${data.count} routes`
+    lastCompleted.value = false
+    state.value = {}
+    emit('routes-changed')
+  } catch (e) {
+    statusMsg.value = `Withdraw failed: ${e.message}`
+  } finally {
+    withdrawing.value = false
+  }
+}
+
+function startPolling() {
+  stopPolling()
+  pollStatus()
+  pollTimer = setInterval(pollStatus, 2000)
+}
+
+function stopPolling() {
+  if (pollTimer) {
+    clearInterval(pollTimer)
+    pollTimer = null
+  }
+}
+
+async function pollStatus() {
+  try {
+    const data = await api.fullTableStatus()
+    state.value = data
+
+    if (data.active) {
+      statusMsg.value = `Injecting: ${formatNum(data.injected)} / ${formatNum(data.total)} (${data.rate_pps || 0}/s)`
+    } else if (data.error) {
+      statusMsg.value = `Error: ${data.error}`
+      injecting.value = false
+      stopPolling()
+    } else if (data.injected > 0) {
+      statusMsg.value = `Complete: ${formatNum(data.injected)} routes in ${data.elapsed_sec}s (${data.rate_pps}/s)`
+      injecting.value = false
+      lastCompleted.value = true
+      stopPolling()
+      emit('routes-changed')
+    }
+  } catch (e) {
+    // keep polling
+  }
+}
+
+onUnmounted(() => {
+  stopPolling()
+})
+</script>
+
+<style scoped>
+.full-table {
+  display: flex;
+  flex-direction: column;
+  gap: 18px;
+  max-width: 680px;
+}
+
+.section-title {
+  font-size: 14px;
+  font-weight: 600;
+  color: var(--muted);
+  text-transform: uppercase;
+  letter-spacing: 0.08em;
+}
+
+.section-desc {
+  color: var(--muted);
+  font-size: 13px;
+  line-height: 1.6;
+  margin-top: -8px;
+}
+
+.config-card {
+  background: var(--card-bg);
+  border: 1px solid var(--border);
+  border-radius: var(--radius);
+  padding: 20px;
+  display: flex;
+  flex-direction: column;
+  gap: 18px;
+}
+
+.form-group {
+  display: flex;
+  flex-direction: column;
+  gap: 6px;
+}
+
+label {
+  font-size: 12px;
+  font-weight: 600;
+  color: var(--muted);
+  text-transform: uppercase;
+  letter-spacing: 0.05em;
+}
+
+.level-grid {
+  display: grid;
+  grid-template-columns: repeat(3, 1fr);
+  gap: 8px;
+}
+
+.level-btn {
+  display: flex;
+  flex-direction: column;
+  align-items: center;
+  gap: 2px;
+  padding: 12px 8px;
+  background: var(--bg);
+  border: 1px solid var(--border);
+  border-radius: var(--radius);
+  color: var(--text);
+  transition: all 0.15s;
+}
+
+.level-btn:hover:not(:disabled) {
+  border-color: var(--accent);
+  background: rgba(79, 156, 249, 0.08);
+}
+
+.level-btn.selected {
+  border-color: var(--accent);
+  background: rgba(79, 156, 249, 0.15);
+  box-shadow: 0 0 0 1px var(--accent);
+}
+
+.level-count {
+  font-size: 18px;
+  font-weight: 700;
+  font-family: 'Cascadia Code', 'Fira Code', 'Consolas', monospace;
+  color: var(--accent);
+}
+
+.level-desc {
+  font-size: 11px;
+  color: var(--muted);
+}
+
+.mono-input {
+  font-family: 'Cascadia Code', 'Fira Code', 'Consolas', monospace;
+  background: var(--bg);
+  color: var(--text);
+  border: 1px solid var(--border);
+  border-radius: var(--radius);
+  padding: 7px 10px;
+  font-size: 13px;
+  outline: none;
+  max-width: 200px;
+}
+
+.mono-input:focus {
+  border-color: var(--accent);
+}
+
+.action-row {
+  display: flex;
+  gap: 10px;
+  padding-top: 4px;
+  border-top: 1px solid var(--border);
+}
+
+.btn-start {
+  padding: 9px 22px;
+  background: rgba(72, 187, 120, 0.15);
+  color: #48bb78;
+  border: 1px solid rgba(72, 187, 120, 0.3);
+  font-weight: 700;
+  font-size: 14px;
+  display: flex;
+  align-items: center;
+  gap: 7px;
+}
+
+.btn-start:hover:not(:disabled) {
+  background: rgba(72, 187, 120, 0.25);
+}
+
+.btn-stop {
+  padding: 9px 22px;
+  background: rgba(252, 129, 129, 0.15);
+  color: #fc8181;
+  border: 1px solid rgba(252, 129, 129, 0.3);
+  font-weight: 700;
+  font-size: 14px;
+  display: flex;
+  align-items: center;
+  gap: 7px;
+  animation: pulse-border 1.5s ease-in-out infinite;
+}
+
+.btn-stop:hover {
+  background: rgba(252, 129, 129, 0.25);
+}
+
+.btn-withdraw {
+  padding: 9px 18px;
+  background: rgba(246, 173, 85, 0.15);
+  color: #f6ad55;
+  border: 1px solid rgba(246, 173, 85, 0.3);
+  font-weight: 600;
+  font-size: 13px;
+}
+
+.btn-withdraw:hover:not(:disabled) {
+  background: rgba(246, 173, 85, 0.25);
+}
+
+.status-card {
+  background: var(--card-bg);
+  border: 1px solid var(--border);
+  border-radius: var(--radius);
+  padding: 16px 18px;
+  display: flex;
+  flex-direction: column;
+  gap: 12px;
+}
+
+.status-header {
+  display: flex;
+  align-items: center;
+  gap: 10px;
+}
+
+.status-dot {
+  width: 10px;
+  height: 10px;
+  border-radius: 50%;
+  flex-shrink: 0;
+}
+
+.dot-active {
+  background: #48bb78;
+  box-shadow: 0 0 8px #48bb78;
+  animation: pulse-dot 1s ease-in-out infinite;
+}
+
+.dot-idle {
+  background: var(--muted);
+}
+
+.status-text {
+  font-size: 14px;
+  font-weight: 600;
+  color: var(--text);
+  font-family: 'Cascadia Code', 'Fira Code', 'Consolas', monospace;
+}
+
+.progress-section {
+  display: flex;
+  flex-direction: column;
+  gap: 5px;
+}
+
+.progress-labels {
+  display: flex;
+  justify-content: space-between;
+  font-size: 11px;
+  color: var(--muted);
+}
+
+.progress-track {
+  height: 6px;
+  background: var(--border);
+  border-radius: 3px;
+  overflow: hidden;
+}
+
+.progress-fill {
+  height: 100%;
+  background: var(--accent);
+  border-radius: 3px;
+  transition: width 0.5s ease;
+}
+
+.stats-row {
+  display: flex;
+  gap: 24px;
+}
+
+.stat-item {
+  display: flex;
+  flex-direction: column;
+  gap: 2px;
+}
+
+.stat-label {
+  font-size: 10px;
+  color: var(--muted);
+  text-transform: uppercase;
+  letter-spacing: 0.05em;
+}
+
+.stat-val {
+  font-size: 15px;
+  font-weight: 700;
+  color: var(--text);
+  font-family: 'Cascadia Code', 'Fira Code', 'Consolas', monospace;
+}
+
+.inject-error {
+  font-size: 12px;
+  color: #fc8181;
+  padding: 6px 10px;
+  background: rgba(252, 129, 129, 0.08);
+  border-radius: 4px;
+  border: 1px solid rgba(252, 129, 129, 0.2);
+}
+
+@keyframes pulse-dot {
+  0%, 100% { opacity: 1; }
+  50% { opacity: 0.5; }
+}
+
+@keyframes pulse-border {
+  0%, 100% { border-color: rgba(252, 129, 129, 0.3); }
+  50% { border-color: rgba(252, 129, 129, 0.6); }
+}
+</style>

exabgp-ui/vite.config.js

@@ -2,6 +2,7 @@ import { defineConfig } from 'vite'
 import vue from '@vitejs/plugin-vue'
 
 export default defineConfig({
+  base: '/exabgp/',
   plugins: [vue()],
   server: {
     proxy: {

exabgp/api/server.py

@@ -48,11 +48,15 @@ peer_states = {}
 # ExaBGP command helpers
 # ---------------------------------------------------------------------------
 
+_quiet_mode = False
+
+
 def _send(cmd: str):
     """Write a command to ExaBGP via stdout."""
     with _stdout_lock:
         sys.stdout.write(cmd + '\n')
         sys.stdout.flush()
+    if not _quiet_mode:
         log.info('→ ExaBGP: %s', cmd)
 
 
@@ -162,7 +166,22 @@ def api_withdraw_all():
 # ---------------------------------------------------------------------------
 
 sys.path.insert(0, '/exabgp')
-from scenarios import SCENARIOS
+from scenarios import SCENARIOS, generate_full_internet
+
+# ---------------------------------------------------------------------------
+# Full-table background injection
+# ---------------------------------------------------------------------------
+
+_injection_state = {
+    'active': False,
+    'total': 0,
+    'injected': 0,
+    'elapsed_sec': 0,
+    'rate_pps': 0,
+    'error': None,
+    'stop_requested': False,
+}
+_injection_lock = threading.Lock()
 
 
 @app.route('/scenarios', methods=['GET'])
@@ -223,6 +242,131 @@ def get_peers():
     return jsonify({'peers': peer_states})
 
 
+# ---------------------------------------------------------------------------
+# Full-table injection endpoints
+# ---------------------------------------------------------------------------
+
+def _injection_worker(count, batch_size):
+    """Background thread: generate and inject full internet table."""
+    global _quiet_mode
+    try:
+        _quiet_mode = True  # suppress per-route logging
+        log.info('Generating %d full-table prefixes...', count)
+        routes = generate_full_internet(count)
+        with _injection_lock:
+            _injection_state['total'] = len(routes)
+        log.info('Generated %d routes, starting injection at batch_size=%d', len(routes), batch_size)
+
+        start_time = time.time()
+        for i, route in enumerate(routes):
+            with _injection_lock:
+                if _injection_state['stop_requested']:
+                    log.info('Injection stopped by user at %d/%d', i, len(routes))
+                    break
+
+            prefix = route['prefix']
+            announce_route(
+                prefix,
+                next_hop=route.get('next_hop', 'self'),
+                as_path=route.get('as_path', []),
+                communities=route.get('communities', []),
+                med=route.get('med'),
+                local_pref=route.get('local_pref'),
+            )
+
+            # Update progress periodically (every batch_size routes)
+            if (i + 1) % batch_size == 0:
+                elapsed = time.time() - start_time
+                with _injection_lock:
+                    _injection_state['injected'] = i + 1
+                    _injection_state['elapsed_sec'] = round(elapsed, 1)
+                    _injection_state['rate_pps'] = round((i + 1) / elapsed, 1) if elapsed > 0 else 0
+                log.info('Injection progress: %d/%d (%.0f/s)',
+                         i + 1, len(routes), (i + 1) / elapsed if elapsed > 0 else 0)
+
+        elapsed = time.time() - start_time
+        with _injection_lock:
+            _injection_state['injected'] = min(i + 1, len(routes))
+            _injection_state['elapsed_sec'] = round(elapsed, 1)
+            _injection_state['rate_pps'] = round(_injection_state['injected'] / elapsed, 1) if elapsed > 0 else 0
+            _injection_state['active'] = False
+        log.info('Injection complete: %d routes in %.1fs (%.0f/s)',
+                 _injection_state['injected'], elapsed,
+                 _injection_state['injected'] / elapsed if elapsed > 0 else 0)
+
+    except Exception as e:
+        log.error('Injection error: %s', e)
+        with _injection_lock:
+            _injection_state['error'] = str(e)
+            _injection_state['active'] = False
+    finally:
+        _quiet_mode = False
+
+
+@app.route('/full-table/start', methods=['POST'])
+def start_full_table():
+    """Start background injection of a full IPv4 routing table.
+
+    POST body (all optional):
+      count:      Number of prefixes (default 900000)
+      batch_size: Progress update interval (default 1000)
+    """
+    with _injection_lock:
+        if _injection_state['active']:
+            return jsonify({
+                'error': 'Injection already in progress',
+                'state': dict(_injection_state),
+            }), 409
+
+    data = request.get_json(force=True) if request.data else {}
+    count = int(data.get('count', 900000))
+    batch_size = int(data.get('batch_size', 1000))
+
+    with _injection_lock:
+        _injection_state.update({
+            'active': True,
+            'total': count,
+            'injected': 0,
+            'elapsed_sec': 0,
+            'rate_pps': 0,
+            'error': None,
+            'stop_requested': False,
+        })
+
+    t = threading.Thread(target=_injection_worker, args=(count, batch_size), daemon=True)
+    t.start()
+
+    log.info('Started full-table injection: %d prefixes', count)
+    return jsonify({
+        'status': 'started',
+        'count': count,
+        'message': f'Generating and injecting {count} prefixes in background. GET /full-table/status to track progress.',
+    })
+
+
+@app.route('/full-table/status', methods=['GET'])
+def full_table_status():
+    """Get current full-table injection progress."""
+    with _injection_lock:
+        state = dict(_injection_state)
+    if state['total'] > 0:
+        state['progress_pct'] = round(state['injected'] / state['total'] * 100, 1)
+    else:
+        state['progress_pct'] = 0
+    state['active_routes'] = len(active_routes)
+    return jsonify(state)
+
+
+@app.route('/full-table/stop', methods=['POST'])
+def stop_full_table():
+    """Stop an in-progress full-table injection."""
+    with _injection_lock:
+        if not _injection_state['active']:
+            return jsonify({'error': 'No injection in progress'}), 400
+        _injection_state['stop_requested'] = True
+    return jsonify({'status': 'stop_requested', 'injected_so_far': _injection_state['injected']})
+
+
 # ---------------------------------------------------------------------------
 # ExaBGP event loop (main thread)
 # ---------------------------------------------------------------------------

exabgp/inject.py

@@ -12,6 +12,9 @@ Usage:
   inject.py withdraw-all
   inject.py scenario <name>
   inject.py withdraw-scenario <name>
+  inject.py full-table [--count N] [--follow]     # inject full IPv4 table (background)
+  inject.py full-table-status                     # show injection progress
+  inject.py full-table-stop                       # stop injection
   inject.py churn [--count N] [--interval SEC]   # cycle announce/withdraw for ip_rib_log population
   inject.py monitor                               # live-refresh terminal view
 
@@ -29,8 +32,8 @@ import requests
 API = os.environ.get('EXABGP_API', 'http://localhost:5050')
 
 
-def _post(path, data=None):
+def _post(path, data=None, timeout=10):
-    r = requests.post(f'{API}{path}', json=data or {}, timeout=10)
+    r = requests.post(f'{API}{path}', json=data or {}, timeout=timeout)
     r.raise_for_status()
     return r.json()
 
@@ -174,6 +177,101 @@ def cmd_withdraw_scenario(args):
     print(f"Withdrew scenario '{args.name}': {data['count']} routes withdrawn")
 
 
+def cmd_full_table(args):
+    """Inject a full IPv4 routing table for stress testing."""
+    count = args.count
+    print(f"Starting full-table injection: {count} prefixes")
+    print("This generates routes in background. Use 'inject.py full-table-status' to track.\n")
+
+    data = _post('/full-table/start', {'count': count, 'batch_size': args.batch_size}, timeout=120)
+    print(data.get('message', 'Started'))
+
+    if args.follow:
+        print()
+        _follow_injection()
+
+
+def cmd_full_table_status(args):
+    """Show full-table injection progress."""
+    data = _get('/full-table/status')
+    active = data.get('active', False)
+    total = data.get('total', 0)
+    injected = data.get('injected', 0)
+    pct = data.get('progress_pct', 0)
+    rate = data.get('rate_pps', 0)
+    elapsed = data.get('elapsed_sec', 0)
+    error = data.get('error')
+    active_routes = data.get('active_routes', 0)
+
+    if error:
+        print(f"ERROR: {error}")
+    elif active:
+        bar_len = 40
+        filled = int(bar_len * pct / 100)
+        bar = '#' * filled + '-' * (bar_len - filled)
+        print(f"[{bar}] {pct:.1f}%")
+        print(f"  Injected: {injected:,} / {total:,}  ({rate:.0f} routes/s)")
+        print(f"  Elapsed:  {elapsed:.0f}s")
+        print(f"  Active routes in ExaBGP: {active_routes:,}")
+    elif total > 0:
+        print(f"Injection complete: {injected:,} / {total:,} routes in {elapsed:.0f}s ({rate:.0f}/s)")
+        print(f"Active routes in ExaBGP: {active_routes:,}")
+    else:
+        print("No injection running or completed.")
+        print(f"Active routes: {active_routes:,}")
+
+
+def cmd_full_table_stop(args):
+    """Stop an in-progress full-table injection."""
+    try:
+        data = _post('/full-table/stop')
+        print(f"Stop requested. Injected so far: {data.get('injected_so_far', '?'):,}")
+    except requests.exceptions.HTTPError as e:
+        if e.response.status_code == 400:
+            print("No injection in progress.")
+        else:
+            raise
+
+
+def _follow_injection():
+    """Poll injection status until complete."""
+    import shutil
+    lines_printed = 0
+    try:
+        while True:
+            data = _get('/full-table/status')
+            active = data.get('active', False)
+            total = data.get('total', 0)
+            injected = data.get('injected', 0)
+            pct = data.get('progress_pct', 0)
+            rate = data.get('rate_pps', 0)
+            elapsed = data.get('elapsed_sec', 0)
+            active_routes = data.get('active_routes', 0)
+
+            # Move cursor up to overwrite
+            if lines_printed > 0:
+                print(f"\033[{lines_printed}A", end='')
+
+            bar_len = 40
+            filled = int(bar_len * pct / 100)
+            bar = '#' * filled + '-' * (bar_len - filled)
+            output_lines = [
+                f"  [{bar}] {pct:.1f}%",
+                f"  Injected: {injected:,} / {total:,}  ({rate:.0f} routes/s)  elapsed: {elapsed:.0f}s",
+                f"  Active routes: {active_routes:,}",
+            ]
+            print('\n'.join(output_lines))
+            lines_printed = len(output_lines)
+
+            if not active:
+                print(f"\nDone! {injected:,} routes injected in {elapsed:.0f}s")
+                break
+
+            time.sleep(2)
+    except KeyboardInterrupt:
+        print("\n\nFollowing stopped (injection continues in background).")
+
+
 def cmd_churn(args):
     """
     Cycle announce/withdraw on the 'churn' scenario to generate ip_rib_log
@@ -236,6 +334,17 @@ def main():
     p = sub.add_parser('withdraw-scenario', help='Withdraw a named scenario')
     p.add_argument('name')
 
+    p = sub.add_parser('full-table', help='Inject full IPv4 routing table (background)')
+    p.add_argument('--count', type=int, default=900000, metavar='N',
+                   help='Number of prefixes to inject (default: 900000)')
+    p.add_argument('--batch-size', type=int, default=1000, metavar='N',
+                   help='Progress update interval (default: 1000)')
+    p.add_argument('--follow', '-f', action='store_true',
+                   help='Follow progress until complete')
+
+    sub.add_parser('full-table-status', help='Show full-table injection progress')
+    sub.add_parser('full-table-stop', help='Stop full-table injection')
+
     p = sub.add_parser('churn', help='Cycle announce/withdraw to populate ip_rib_log')
     p.add_argument('--count', type=int, default=0, metavar='N',
                    help='Number of cycles (0 = infinite)')
@@ -255,6 +364,9 @@ def main():
         'withdraw-all': cmd_withdraw_all,
         'scenario': cmd_scenario,
         'withdraw-scenario': cmd_withdraw_scenario,
+        'full-table': cmd_full_table,
+        'full-table-status': cmd_full_table_status,
+        'full-table-stop': cmd_full_table_stop,
         'churn': cmd_churn,
     }
 

exabgp/route_diversity_config.py

@@ -0,0 +1,658 @@
+#!/usr/bin/env python3
+"""
+Route Diversity Configuration Script
+=====================================
+Adds loopbacks, static routes, route-policies, and BGP redistribution
+to R9K-01 through R9K-07 to create locally-originated routes that
+produce meaningful RR Loc-RIB diffs between CORE-01 and CORE-02.
+
+IS-IS topology (natural asymmetry — no metric tuning needed):
+  CORE-01 → R9K-01, R9K-02, R9K-03, R9K-04, R9K-05
+  CORE-02 → R9K-05, R9K-06, R9K-07
+  R9K-04 ↔ R9K-06 (cross-link)
+  R9K-05 dual-homed to both COREs
+
+Overlapping prefixes from CORE-01-side and CORE-02-side routers produce
+next-hop diffs because each RR picks the client with lowest IGP cost.
+
+Address plan:
+  Loopbacks:     10.110.{router_id}.{1,2}/32  (unique per router)
+  Overlap LBs:   10.110.{100-103}.1/32        (shared across router pairs)
+  Static routes: 10.111.{router_id}.0/24      (unique per router)
+  Overlap statics: 10.111.{100-103}.0/24      (shared across router pairs)
+
+Usage:
+  python3 route_diversity_config.py              # apply all config
+  python3 route_diversity_config.py --verify-only # just check current state
+  python3 route_diversity_config.py --rollback    # remove all added config
+"""
+
+from ncclient import manager
+import xml.etree.ElementTree as ET
+import sys
+import argparse
+
+# YANG namespaces
+IFMGR_NS   = 'http://cisco.com/ns/yang/Cisco-IOS-XR-ifmgr-cfg'
+IPV4IO_NS  = 'http://cisco.com/ns/yang/Cisco-IOS-XR-ipv4-io-cfg'
+STATIC_NS  = 'http://cisco.com/ns/yang/Cisco-IOS-XR-ip-static-cfg'
+BGP_NS     = 'http://cisco.com/ns/yang/Cisco-IOS-XR-ipv4-bgp-cfg'
+ISIS_NS    = 'http://cisco.com/ns/yang/Cisco-IOS-XR-clns-isis-cfg'
+RPL_NS     = 'http://cisco.com/ns/yang/Cisco-IOS-XR-policy-repository-cfg'
+
+# ──────────────────────────────────────────────────────────────────────
+# Router definitions
+# ──────────────────────────────────────────────────────────────────────
+
+ROUTERS = {
+    'R9K-01': {
+        'mgmt': '10.100.0.1',
+        'loopbacks': [
+            ('Loopback10', '10.110.1.1', '255.255.255.255'),
+            ('Loopback11', '10.110.1.2', '255.255.255.255'),
+            ('Loopback100', '10.110.100.1', '255.255.255.255'),  # overlap with R9K-06
+            ('Loopback103', '10.110.103.1', '255.255.255.255'),  # overlap with R9K-04, R9K-07
+        ],
+        'statics': [
+            ('10.111.1.0', 24, 100),       # unique, tag=100 → LP=200
+            ('10.111.100.0', 24, 100),      # overlap with R9K-06 (tag=200)
+            ('10.111.103.0', 24, 100),      # overlap with R9K-04, R9K-07 (same tag)
+        ],
+    },
+    'R9K-02': {
+        'mgmt': '10.100.0.2',
+        'loopbacks': [
+            ('Loopback10', '10.110.2.1', '255.255.255.255'),
+            ('Loopback11', '10.110.2.2', '255.255.255.255'),
+            ('Loopback101', '10.110.101.1', '255.255.255.255'),  # overlap with R9K-07
+        ],
+        'statics': [
+            ('10.111.2.0', 24, 100),
+            ('10.111.101.0', 24, 100),      # overlap with R9K-07 (tag=300)
+        ],
+    },
+    'R9K-03': {
+        'mgmt': '10.100.0.3',
+        'loopbacks': [
+            ('Loopback10', '10.110.3.1', '255.255.255.255'),
+            ('Loopback11', '10.110.3.2', '255.255.255.255'),
+            ('Loopback102', '10.110.102.1', '255.255.255.255'),  # overlap with R9K-05
+        ],
+        'statics': [
+            ('10.111.3.0', 24, 100),
+            ('10.111.102.0', 24, 100),      # overlap with R9K-04 (tag=200)
+        ],
+    },
+    'R9K-04': {
+        'mgmt': '10.100.0.4',
+        'loopbacks': [
+            ('Loopback10', '10.110.4.1', '255.255.255.255'),
+            ('Loopback11', '10.110.4.2', '255.255.255.255'),
+            ('Loopback103', '10.110.103.1', '255.255.255.255'),  # overlap with R9K-01, R9K-07
+        ],
+        'statics': [
+            ('10.111.4.0', 24, 200),        # tag=200 → LP=150
+            ('10.111.102.0', 24, 200),       # overlap with R9K-03 (tag=100)
+            ('10.111.103.0', 24, 100),       # overlap with R9K-01, R9K-07 (same tag)
+        ],
+    },
+    'R9K-05': {
+        'mgmt': '10.100.0.5',
+        'loopbacks': [
+            ('Loopback10', '10.110.5.1', '255.255.255.255'),
+            ('Loopback11', '10.110.5.2', '255.255.255.255'),
+            ('Loopback102', '10.110.102.1', '255.255.255.255'),  # overlap with R9K-03
+        ],
+        'statics': [
+            ('10.111.5.0', 24, 100),
+        ],
+    },
+    'R9K-06': {
+        'mgmt': '10.100.0.6',
+        'loopbacks': [
+            ('Loopback10', '10.110.6.1', '255.255.255.255'),
+            ('Loopback11', '10.110.6.2', '255.255.255.255'),
+            ('Loopback100', '10.110.100.1', '255.255.255.255'),  # overlap with R9K-01
+        ],
+        'statics': [
+            ('10.111.6.0', 24, 200),        # tag=200 → LP=150
+            ('10.111.100.0', 24, 200),       # overlap with R9K-01 (tag=100)
+        ],
+    },
+    'R9K-07': {
+        'mgmt': '10.100.0.7',
+        'loopbacks': [
+            ('Loopback10', '10.110.7.1', '255.255.255.255'),
+            ('Loopback11', '10.110.7.2', '255.255.255.255'),
+            ('Loopback101', '10.110.101.1', '255.255.255.255'),  # overlap with R9K-02
+            ('Loopback103', '10.110.103.1', '255.255.255.255'),  # overlap with R9K-01, R9K-04
+        ],
+        'statics': [
+            ('10.111.7.0', 24, 300),         # tag=300 → LP=100
+            ('10.111.101.0', 24, 300),        # overlap with R9K-02 (tag=100)
+            ('10.111.103.0', 24, 100),        # overlap with R9K-01, R9K-04 (same tag)
+        ],
+    },
+}
+
+# ──────────────────────────────────────────────────────────────────────
+# Route-policy (RPL text blob)
+# ──────────────────────────────────────────────────────────────────────
+
+ROUTE_POLICY_NAME = 'REDIST-TO-BGP'
+ROUTE_POLICY_BODY = """\
+route-policy REDIST-TO-BGP
+  if tag is 100 then
+    set local-preference 200
+    set med 50
+    set community (65020:100) additive
+    pass
+  elseif tag is 200 then
+    set local-preference 150
+    set med 100
+    set community (65020:200) additive
+    pass
+  elseif tag is 300 then
+    set local-preference 100
+    set med 200
+    set community (65020:300) additive
+    pass
+  else
+    set local-preference 100
+    pass
+  endif
+end-policy
+"""
+
+# ──────────────────────────────────────────────────────────────────────
+# XML builders
+# ──────────────────────────────────────────────────────────────────────
+
+def loopback_xml(name, addr, mask):
+    """Create a loopback interface with an IPv4 address."""
+    return f"""
+<config>
+  <interface-configurations xmlns="{IFMGR_NS}">
+    <interface-configuration>
+      <active>act</active>
+      <interface-name>{name}</interface-name>
+      <interface-virtual/>
+      <ipv4-network xmlns="{IPV4IO_NS}">
+        <addresses>
+          <primary>
+            <address>{addr}</address>
+            <netmask>{mask}</netmask>
+          </primary>
+        </addresses>
+      </ipv4-network>
+    </interface-configuration>
+  </interface-configurations>
+</config>
+"""
+
+
+def static_route_xml(prefix, prefix_len, tag):
+    """Create a static route to Null0 with a tag."""
+    return f"""
+<config>
+  <router-static xmlns="{STATIC_NS}">
+    <default-vrf>
+      <address-family>
+        <vrfipv4>
+          <vrf-unicast>
+            <vrf-prefixes>
+              <vrf-prefix>
+                <prefix>{prefix}</prefix>
+                <prefix-length>{prefix_len}</prefix-length>
+                <vrf-route>
+                  <vrf-next-hop-table>
+                    <vrf-next-hop-interface-name>
+                      <interface-name>Null0</interface-name>
+                      <tag>{tag}</tag>
+                    </vrf-next-hop-interface-name>
+                  </vrf-next-hop-table>
+                </vrf-route>
+              </vrf-prefix>
+            </vrf-prefixes>
+          </vrf-unicast>
+        </vrfipv4>
+      </address-family>
+    </default-vrf>
+  </router-static>
+</config>
+"""
+
+
+def route_policy_xml(name, body):
+    """Create/replace a route-policy (RPL text blob)."""
+    return f"""
+<config>
+  <routing-policy xmlns="{RPL_NS}">
+    <route-policies>
+      <route-policy>
+        <route-policy-name>{name}</route-policy-name>
+        <rpl-route-policy>{body}</rpl-route-policy>
+      </route-policy>
+    </route-policies>
+  </routing-policy>
+</config>
+"""
+
+
+def isis_passive_xml(intf_name):
+    """Add a loopback to IS-IS instance 1 (passive by default for loopbacks)."""
+    return f"""
+<config>
+  <isis xmlns="{ISIS_NS}">
+    <instances>
+      <instance>
+        <instance-name>1</instance-name>
+        <interfaces>
+          <interface>
+            <interface-name>{intf_name}</interface-name>
+            <running/>
+            <interface-afs>
+              <interface-af>
+                <af-name>ipv4</af-name>
+                <saf-name>unicast</saf-name>
+                <interface-af-data/>
+              </interface-af>
+            </interface-afs>
+          </interface>
+        </interfaces>
+      </instance>
+    </instances>
+  </isis>
+</config>
+"""
+
+
+def bgp_redistribute_xml():
+    """Configure redistribute connected + static with REDIST-TO-BGP policy."""
+    return f"""
+<config>
+  <bgp xmlns="{BGP_NS}">
+    <instance>
+      <instance-name>default</instance-name>
+      <instance-as>
+        <as>0</as>
+        <four-byte-as>
+          <as>65020</as>
+          <bgp-running/>
+          <default-vrf>
+            <global>
+              <global-afs>
+                <global-af>
+                  <af-name>ipv4-unicast</af-name>
+                  <enable/>
+                  <connected-routes>
+                    <route-policy-name>{ROUTE_POLICY_NAME}</route-policy-name>
+                  </connected-routes>
+                  <static-routes>
+                    <route-policy-name>{ROUTE_POLICY_NAME}</route-policy-name>
+                  </static-routes>
+                </global-af>
+              </global-afs>
+            </global>
+          </default-vrf>
+        </four-byte-as>
+      </instance-as>
+    </instance>
+  </bgp>
+</config>
+"""
+
+
+# ──────────────────────────────────────────────────────────────────────
+# Rollback XML builders (delete operations)
+# ──────────────────────────────────────────────────────────────────────
+
+NC_NS = 'urn:ietf:params:xml:ns:netconf:base:1.0'
+
+
+def delete_loopback_xml(name):
+    return f"""
+<config>
+  <interface-configurations xmlns="{IFMGR_NS}">
+    <interface-configuration xmlns:nc="{NC_NS}" nc:operation="delete">
+      <active>act</active>
+      <interface-name>{name}</interface-name>
+    </interface-configuration>
+  </interface-configurations>
+</config>
+"""
+
+
+def delete_static_route_xml(prefix, prefix_len):
+    return f"""
+<config>
+  <router-static xmlns="{STATIC_NS}">
+    <default-vrf>
+      <address-family>
+        <vrfipv4>
+          <vrf-unicast>
+            <vrf-prefixes>
+              <vrf-prefix xmlns:nc="{NC_NS}" nc:operation="delete">
+                <prefix>{prefix}</prefix>
+                <prefix-length>{prefix_len}</prefix-length>
+              </vrf-prefix>
+            </vrf-prefixes>
+          </vrf-unicast>
+        </vrfipv4>
+      </address-family>
+    </default-vrf>
+  </router-static>
+</config>
+"""
+
+
+def delete_bgp_redistribute_xml():
+    return f"""
+<config>
+  <bgp xmlns="{BGP_NS}">
+    <instance>
+      <instance-name>default</instance-name>
+      <instance-as>
+        <as>0</as>
+        <four-byte-as>
+          <as>65020</as>
+          <bgp-running/>
+          <default-vrf>
+            <global>
+              <global-afs>
+                <global-af>
+                  <af-name>ipv4-unicast</af-name>
+                  <enable/>
+                  <connected-routes xmlns:nc="{NC_NS}" nc:operation="delete"/>
+                  <static-routes xmlns:nc="{NC_NS}" nc:operation="delete"/>
+                </global-af>
+              </global-afs>
+            </global>
+          </default-vrf>
+        </four-byte-as>
+      </instance-as>
+    </instance>
+  </bgp>
+</config>
+"""
+
+
+def delete_isis_interface_xml(intf_name):
+    return f"""
+<config>
+  <isis xmlns="{ISIS_NS}">
+    <instances>
+      <instance>
+        <instance-name>1</instance-name>
+        <interfaces>
+          <interface xmlns:nc="{NC_NS}" nc:operation="delete">
+            <interface-name>{intf_name}</interface-name>
+          </interface>
+        </interfaces>
+      </instance>
+    </instances>
+  </isis>
+</config>
+"""
+
+
+def delete_route_policy_xml(name):
+    return f"""
+<config>
+  <routing-policy xmlns="{RPL_NS}">
+    <route-policies>
+      <route-policy xmlns:nc="{NC_NS}" nc:operation="delete">
+        <route-policy-name>{name}</route-policy-name>
+      </route-policy>
+    </route-policies>
+  </routing-policy>
+</config>
+"""
+
+
+# ──────────────────────────────────────────────────────────────────────
+# Configuration functions
+# ──────────────────────────────────────────────────────────────────────
+
+def nc_connect(mgmt_ip):
+    """Open NETCONF session."""
+    return manager.connect(
+        host=mgmt_ip,
+        port=830,
+        username='webui',
+        password='cisco',
+        hostkey_verify=False,
+        device_params={'name': 'iosxr'},
+        timeout=30,
+    )
+
+
+def configure_router(label, cfg):
+    """Apply full route-diversity config to a single router."""
+    mgmt_ip = cfg['mgmt']
+    print(f"\n{'─'*60}")
+    print(f"  Configuring {label} ({mgmt_ip})")
+    print(f"{'─'*60}")
+
+    lb_names = [lb[0] for lb in cfg['loopbacks']]
+    static_prefixes = [f"{s[0]}/{s[1]}" for s in cfg['statics']]
+    print(f"  Loopbacks: {', '.join(lb_names)}")
+    print(f"  Statics:   {', '.join(static_prefixes)}")
+
+    try:
+        with nc_connect(mgmt_ip) as m:
+            # Phase 1: Route-policy (must exist before BGP references it)
+            print(f"  → Creating route-policy {ROUTE_POLICY_NAME}...")
+            m.edit_config(target='candidate', config=route_policy_xml(ROUTE_POLICY_NAME, ROUTE_POLICY_BODY))
+
+            # Phase 2: Loopback interfaces
+            for name, addr, mask in cfg['loopbacks']:
+                print(f"  → Creating {name} ({addr})...")
+                m.edit_config(target='candidate', config=loopback_xml(name, addr, mask))
+
+            # Phase 3: Static routes
+            for prefix, plen, tag in cfg['statics']:
+                print(f"  → Static {prefix}/{plen} → Null0 tag={tag}...")
+                m.edit_config(target='candidate', config=static_route_xml(prefix, plen, tag))
+
+            # Phase 4: IS-IS passive on new loopbacks
+            for name, _, _ in cfg['loopbacks']:
+                print(f"  → IS-IS passive: {name}...")
+                m.edit_config(target='candidate', config=isis_passive_xml(name))
+
+            # Phase 5: BGP redistribution
+            print(f"  → BGP redistribute connected + static...")
+            m.edit_config(target='candidate', config=bgp_redistribute_xml())
+
+            # Phase 6: Commit
+            print(f"  → Committing...")
+            m.commit()
+            print(f"  ✓ {label} done.")
+            return True
+
+    except Exception as e:
+        print(f"  ✗ ERROR on {label}: {e}")
+        return False
+
+
+def rollback_router(label, cfg):
+    """Remove all route-diversity config from a single router."""
+    mgmt_ip = cfg['mgmt']
+    print(f"\n{'─'*60}")
+    print(f"  Rolling back {label} ({mgmt_ip})")
+    print(f"{'─'*60}")
+
+    try:
+        with nc_connect(mgmt_ip) as m:
+            # Remove BGP redistribution first (references the policy)
+            print(f"  → Removing BGP redistribute...")
+            try:
+                m.edit_config(target='candidate', config=delete_bgp_redistribute_xml())
+            except Exception as e:
+                print(f"    (skip — may not exist: {e})")
+
+            # Remove IS-IS interfaces
+            for name, _, _ in cfg['loopbacks']:
+                print(f"  → Removing IS-IS interface {name}...")
+                try:
+                    m.edit_config(target='candidate', config=delete_isis_interface_xml(name))
+                except Exception as e:
+                    print(f"    (skip: {e})")
+
+            # Remove static routes
+            for prefix, plen, _ in cfg['statics']:
+                print(f"  → Removing static {prefix}/{plen}...")
+                try:
+                    m.edit_config(target='candidate', config=delete_static_route_xml(prefix, plen))
+                except Exception as e:
+                    print(f"    (skip: {e})")
+
+            # Remove loopbacks
+            for name, _, _ in cfg['loopbacks']:
+                print(f"  → Removing {name}...")
+                try:
+                    m.edit_config(target='candidate', config=delete_loopback_xml(name))
+                except Exception as e:
+                    print(f"    (skip: {e})")
+
+            # Remove route-policy
+            print(f"  → Removing route-policy {ROUTE_POLICY_NAME}...")
+            try:
+                m.edit_config(target='candidate', config=delete_route_policy_xml(ROUTE_POLICY_NAME))
+            except Exception as e:
+                print(f"    (skip: {e})")
+
+            print(f"  → Committing rollback...")
+            m.commit()
+            print(f"  ✓ {label} rolled back.")
+            return True
+
+    except Exception as e:
+        print(f"  ✗ ERROR rolling back {label}: {e}")
+        return False
+
+
+def verify_router(label, cfg):
+    """Check if route-diversity config is present on a router."""
+    mgmt_ip = cfg['mgmt']
+
+    try:
+        with nc_connect(mgmt_ip) as m:
+            # Check loopbacks
+            filt_intf = f"""<filter>
+              <interface-configurations xmlns="{IFMGR_NS}"/>
+            </filter>"""
+            r_intf = str(m.get_config(source='running', filter=filt_intf))
+
+            found_lbs = []
+            for name, _, _ in cfg['loopbacks']:
+                if name in r_intf:
+                    found_lbs.append(name)
+
+            # Check route-policy
+            filt_rpl = f"""<filter>
+              <routing-policy xmlns="{RPL_NS}"/>
+            </filter>"""
+            r_rpl = str(m.get_config(source='running', filter=filt_rpl))
+            has_policy = ROUTE_POLICY_NAME in r_rpl
+
+            # Check BGP redistribute
+            filt_bgp = f"""<filter>
+              <bgp xmlns="{BGP_NS}">
+                <instance><instance-name>default</instance-name></instance>
+              </bgp>
+            </filter>"""
+            r_bgp = str(m.get_config(source='running', filter=filt_bgp))
+            has_redist_connected = 'connected-routes' in r_bgp and ROUTE_POLICY_NAME in r_bgp
+            has_redist_static = 'static-routes' in r_bgp and ROUTE_POLICY_NAME in r_bgp
+
+            total_lbs = len(cfg['loopbacks'])
+            lb_str = f"{len(found_lbs)}/{total_lbs}"
+            pol = '✓' if has_policy else '✗'
+            rc = '✓' if has_redist_connected else '✗'
+            rs = '✓' if has_redist_static else '✗'
+            ok = len(found_lbs) == total_lbs and has_policy and has_redist_connected and has_redist_static
+            status = 'OK' if ok else 'INCOMPLETE'
+
+            print(f"  {label:8s}  LBs={lb_str:5s}  Policy={pol}  Redist-C={rc}  Redist-S={rs}  [{status}]")
+
+    except Exception as e:
+        print(f"  {label:8s}  verify error: {e}")
+
+
+# ──────────────────────────────────────────────────────────────────────
+# Main
+# ──────────────────────────────────────────────────────────────────────
+
+def main():
+    parser = argparse.ArgumentParser(description='Route Diversity Configuration for RR Diff Analysis')
+    parser.add_argument('--verify-only', action='store_true', help='Only verify current state')
+    parser.add_argument('--rollback', action='store_true', help='Remove all added config')
+    args = parser.parse_args()
+
+    print("Route Diversity Configuration Script")
+    print("=" * 60)
+    print(f"Targets: {len(ROUTERS)} routers ({', '.join(ROUTERS.keys())})")
+    print()
+
+    if args.verify_only:
+        print("Verify-only mode")
+        print('-' * 60)
+        print(f"  {'Router':8s}  {'LBs':5s}  {'Policy':6s}  {'Redist-C':8s}  {'Redist-S':8s}  Status")
+        for label, cfg in ROUTERS.items():
+            verify_router(label, cfg)
+        return
+
+    if args.rollback:
+        print("ROLLBACK mode — removing all route-diversity config")
+        print('-' * 60)
+        results = []
+        for label, cfg in ROUTERS.items():
+            ok = rollback_router(label, cfg)
+            results.append((label, ok))
+
+        failed = [l for l, ok in results if not ok]
+        print()
+        if failed:
+            print(f"FAILED rollback: {', '.join(failed)}")
+            sys.exit(1)
+        else:
+            print("All routers rolled back successfully.")
+        return
+
+    # Apply mode
+    results = []
+    for label, cfg in ROUTERS.items():
+        ok = configure_router(label, cfg)
+        results.append((label, ok))
+
+    # Post-apply verification
+    print(f"\n{'='*60}")
+    print("Post-apply verification")
+    print('=' * 60)
+    print(f"  {'Router':8s}  {'LBs':5s}  {'Policy':6s}  {'Redist-C':8s}  {'Redist-S':8s}  Status")
+    for label, cfg in ROUTERS.items():
+        verify_router(label, cfg)
+
+    failed = [l for l, ok in results if not ok]
+    print()
+    if failed:
+        print(f"FAILED: {', '.join(failed)}")
+        sys.exit(1)
+    else:
+        total_lbs = sum(len(c['loopbacks']) for c in ROUTERS.values())
+        total_statics = sum(len(c['statics']) for c in ROUTERS.values())
+        print(f"All routers configured successfully.")
+        print(f"  {total_lbs} loopbacks + {total_statics} static routes created")
+        print()
+        print("Wait ~60s for BGP convergence and BMP collection, then verify:")
+        print()
+        print("  # Check new prefixes in OpenBMP")
+        print("  docker exec -i obmp-psql psql -U openbmp -d openbmp -c \\")
+        print("    \"SELECT prefix::text, COUNT(*) FROM ip_rib")
+        print("     WHERE (prefix::text LIKE '10.110.%' OR prefix::text LIKE '10.111.%')")
+        print("     AND iswithdrawn = false GROUP BY prefix ORDER BY prefix;\"")
+
+
+if __name__ == '__main__':
+    main()

exabgp/scenarios/__init__.py

@@ -363,10 +363,178 @@ _HIJACK_ROUTES = [
 ]
 
 
+# ---------------------------------------------------------------------------
+# Scenario: te_community_steering
+# Routes tagged with TE communities representing different "colors" for
+# community-based TE policy steering.  Shows how communities drive path
+# selection when routers apply route-policy based on community values.
+# ---------------------------------------------------------------------------
+
+_TE_COMMUNITY_ROUTES = [
+    # Red paths (community 65020:100) — high-priority, low-latency
+    _r('10.210.0.0/24', [65100, 65020], communities=['65020:100'], med=10),
+    _r('10.210.1.0/24', [65100, 65020], communities=['65020:100'], med=10),
+    _r('10.210.2.0/24', [65100, 65020], communities=['65020:100'], med=10),
+    _r('10.210.3.0/24', [65100, 65020], communities=['65020:100'], med=10),
+    _r('10.210.4.0/24', [65100, 65020], communities=['65020:100'], med=10),
+    # Blue paths (community 65020:200) — bulk transfer, cost-optimized
+    _r('10.220.0.0/24', [65100, 65020, 3356], communities=['65020:200'], med=100),
+    _r('10.220.1.0/24', [65100, 65020, 3356], communities=['65020:200'], med=100),
+    _r('10.220.2.0/24', [65100, 65020, 3356], communities=['65020:200'], med=100),
+    _r('10.220.3.0/24', [65100, 65020, 3356], communities=['65020:200'], med=100),
+    _r('10.220.4.0/24', [65100, 65020, 3356], communities=['65020:200'], med=100),
+    # Green paths (community 65020:300) — backup/diverse paths
+    _r('10.230.0.0/24', [65100, 65020, 1299, 6762], communities=['65020:300'], med=200),
+    _r('10.230.1.0/24', [65100, 65020, 1299, 6762], communities=['65020:300'], med=200),
+    _r('10.230.2.0/24', [65100, 65020, 1299, 6762], communities=['65020:300'], med=200),
+    _r('10.230.3.0/24', [65100, 65020, 1299, 6762], communities=['65020:300'], med=200),
+    _r('10.230.4.0/24', [65100, 65020, 1299, 6762], communities=['65020:300'], med=200),
+]
+
+
+# ---------------------------------------------------------------------------
+# Scenario: origin_shift
+# Simulates an origin AS change: prefixes initially associated with
+# well-known origin ASNs are re-announced with a different origin.
+# Use: load internet_sample first, then load origin_shift to see the
+# origin_as column change in ip_rib_log (visible on Anomaly dashboard).
+# ---------------------------------------------------------------------------
+
+_ORIGIN_SHIFT_ROUTES = [
+    # These prefixes overlap with internet_sample but have different origin ASNs
+    _r('8.8.8.0/24',      [65100, 64999],        communities=['65100:origin-shift']),  # was 15169 (Google)
+    _r('1.1.1.0/24',      [65100, 64998],        communities=['65100:origin-shift']),  # was 13335 (Cloudflare)
+    _r('9.9.9.0/24',      [65100, 64997],        communities=['65100:origin-shift']),  # was 19281 (Quad9)
+    _r('208.67.222.0/24', [65100, 64996],        communities=['65100:origin-shift']),  # was 36692 (OpenDNS)
+    _r('156.154.70.0/24', [65100, 64995],        communities=['65100:origin-shift']),  # was 19318 (Neustar)
+]
+
+
+# ---------------------------------------------------------------------------
+# Scenario: path_diversity
+# Multiple announcements of the same prefix with different AS paths,
+# MEDs, and communities.  Demonstrates best-path selection:
+#   - Shorter AS path wins (unless local-pref overrides)
+#   - Lower MED preferred among paths from same neighbor AS
+#   - Communities tag paths for policy identification
+# ---------------------------------------------------------------------------
+
+_PATH_DIVERSITY_ROUTES = [
+    # Prefix 1: 3 paths with varying length and MED
+    _r('10.250.0.0/24', [65100, 174],                communities=['65100:path-a'], med=50),
+    _r('10.250.0.0/24', [65100, 174, 3356],          communities=['65100:path-b'], med=100),
+    _r('10.250.0.0/24', [65100, 174, 3356, 15169],   communities=['65100:path-c'], med=150),
+    # Prefix 2: paths with same length but different MED
+    _r('10.250.1.0/24', [65100, 1299, 15169], communities=['65100:low-med'],  med=10),
+    _r('10.250.1.0/24', [65100, 3356, 15169], communities=['65100:high-med'], med=500),
+    # Prefix 3: local-pref override (higher local-pref wins over shorter path)
+    _r('10.250.2.0/24', [65100, 2914],       communities=['65100:low-lp'],  local_pref=50),
+    _r('10.250.2.0/24', [65100, 2914, 7018], communities=['65100:high-lp'], local_pref=200),
+    # Prefix 4: transit diversity
+    _r('10.250.3.0/24', [65100, 174,  32934], communities=['65100:via-cogent']),
+    _r('10.250.3.0/24', [65100, 3356, 32934], communities=['65100:via-lumen']),
+    _r('10.250.3.0/24', [65100, 2914, 32934], communities=['65100:via-ntt']),
+]
+
+
 # ---------------------------------------------------------------------------
 # Registry
 # ---------------------------------------------------------------------------
 
+# ---------------------------------------------------------------------------
+# Full Internet Table Generator
+# Generates realistic-looking IPv4 prefixes across the routable address space
+# with varied AS paths, prefix lengths, origins, and communities.
+# Configurable count: 10K (quick test) to 900K+ (full table stress test).
+# ---------------------------------------------------------------------------
+
+# Well-known transit ASNs for realistic path construction
+_TRANSIT_ASNS = [174, 701, 1299, 2914, 3257, 3356, 6461, 6762, 7018, 3491, 5400, 1239]
+
+# Realistic origin ASNs (mix of large providers and small networks)
+_ORIGIN_POOL = [
+    13335, 15169, 16509, 8075, 20940, 32934, 714, 54113, 13414, 7922,
+    36459, 46489, 14618, 16276, 24940, 47541, 35916, 49981, 9808, 4134,
+    4837, 9121, 12322, 3320, 6830, 5511, 1273, 6939, 4766, 9318,
+    23693, 38001, 45102, 58453, 10026, 18881, 28573, 7738, 26599, 8151,
+    11888, 17676, 4713, 7545, 9299, 50304, 51167, 60068, 41095, 34984,
+]
+
+# IANA-allocated first octets for routable IPv4 (subset for realism)
+_ROUTABLE_FIRST_OCTETS = list(range(1, 56)) + list(range(57, 127)) + list(range(128, 224))
+
+
+def generate_full_internet(count=900000):
+    """Generate a realistic full IPv4 routing table.
+
+    Distributes prefixes across the IPv4 address space with realistic
+    prefix lengths (/8 through /24) and varied AS paths.
+
+    Args:
+        count: Number of prefixes to generate (default 900K).
+
+    Returns:
+        List of route dicts.
+    """
+    import random
+    rng = random.Random(42)  # deterministic for reproducibility
+
+    routes = []
+    generated = set()
+
+    # Prefix length distribution (approximates real DFZ):
+    # /24: ~55%, /23: ~8%, /22: ~7%, /21: ~5%, /20: ~5%,
+    # /19: ~4%, /18: ~3%, /17: ~2%, /16: ~5%, /15-/8: ~6%
+    prefix_len_weights = {
+        24: 55, 23: 8, 22: 7, 21: 5, 20: 5,
+        19: 4, 18: 3, 17: 2, 16: 5, 15: 2,
+        14: 1, 13: 1, 12: 1, 11: 0.5, 10: 0.3,
+        9: 0.1, 8: 0.1,
+    }
+    plen_choices = list(prefix_len_weights.keys())
+    plen_weights = list(prefix_len_weights.values())
+
+    # AS path length distribution: 1-hop: 5%, 2-hop: 30%, 3-hop: 40%, 4-hop: 20%, 5-hop: 5%
+    path_len_weights = [5, 30, 40, 20, 5]
+
+    while len(routes) < count:
+        # Pick a routable first octet weighted by allocation density
+        first = rng.choice(_ROUTABLE_FIRST_OCTETS)
+        plen = rng.choices(plen_choices, weights=plen_weights, k=1)[0]
+
+        # Generate random prefix within this /8
+        if plen <= 8:
+            prefix = f'{first}.0.0.0/{plen}'
+        elif plen <= 16:
+            second = rng.randint(0, 255) & (0xFF << (16 - plen))
+            prefix = f'{first}.{second}.0.0/{plen}'
+        elif plen <= 24:
+            second = rng.randint(0, 255)
+            third = rng.randint(0, 255) & (0xFF << (24 - plen))
+            prefix = f'{first}.{second}.{third}.0/{plen}'
+        else:
+            continue
+
+        if prefix in generated:
+            continue
+        generated.add(prefix)
+
+        # Build realistic AS path
+        path_len = rng.choices([1, 2, 3, 4, 5], weights=path_len_weights, k=1)[0]
+        origin = rng.choice(_ORIGIN_POOL) if rng.random() < 0.3 else (64512 + rng.randint(0, 65535 - 64512))
+        transits = rng.sample(_TRANSIT_ASNS, min(path_len - 1, len(_TRANSIT_ASNS)))
+        as_path = [65100] + transits[:path_len - 1] + [origin]
+
+        # Occasionally add communities (~20% of routes)
+        communities = []
+        if rng.random() < 0.2:
+            communities.append(f'65100:{rng.choice([100, 200, 300, 400, 500])}')
+
+        routes.append(_r(prefix, as_path, communities=communities or None))
+
+    return routes
+
+
 SCENARIOS = {
     'internet_sample': {
         'description': 'Partial internet table (~80 IPv4 + 14 IPv6 prefixes with realistic AS paths)',
@@ -404,4 +572,16 @@ SCENARIOS = {
         'description': '10 prefixes announced as if directly originated by AS 65100 — simulates a prefix hijack (community 65100:hijack)',
         'routes': _HIJACK_ROUTES,
     },
+    'te_community_steering': {
+        'description': 'Routes tagged with TE communities for color-based steering (65020:100=red, 65020:200=blue, 65020:300=green)',
+        'routes': _TE_COMMUNITY_ROUTES,
+    },
+    'origin_shift': {
+        'description': '5 prefixes with changed origin AS — simulates origin migration/hijack for anomaly detection',
+        'routes': _ORIGIN_SHIFT_ROUTES,
+    },
+    'path_diversity': {
+        'description': 'Same prefixes with different AS paths and MEDs — demonstrates best-path selection and path diversity',
+        'routes': _PATH_DIVERSITY_ROUTES,
+    },
 }

gnmi/gnmi_grpc_config.py

@@ -0,0 +1,158 @@
+#!/usr/bin/env python3
+"""
+gNMI gRPC Configuration Script
+===============================
+Enables gRPC dial-in telemetry on all 9 IOS-XR routers so that
+Telegraf (or any gNMI collector) can subscribe to streaming
+telemetry data.
+
+What this script applies per router:
+  - gRPC server on port 57400 with TLS disabled
+
+Uses SSH/CLI (paramiko) instead of NETCONF because IOS-XR 24.3.1
+rejects the NETCONF edit-config for gRPC with "Need to enable GRPC first".
+
+Router targets:
+  CORE-01 (10.100.0.100)
+  CORE-02 (10.100.0.200)
+  R9K-01  (10.100.0.1) through R9K-07 (10.100.0.7)
+"""
+
+import paramiko
+import time
+import sys
+
+ROUTERS = [
+    ('10.100.0.100', 'CORE-01'),
+    ('10.100.0.200', 'CORE-02'),
+    ('10.100.0.1',   'R9K-01'),
+    ('10.100.0.2',   'R9K-02'),
+    ('10.100.0.3',   'R9K-03'),
+    ('10.100.0.4',   'R9K-04'),
+    ('10.100.0.5',   'R9K-05'),
+    ('10.100.0.6',   'R9K-06'),
+    ('10.100.0.7',   'R9K-07'),
+]
+
+USERNAME = 'webui'
+PASSWORD = 'cisco'
+GRPC_PORT = 57400
+
+CONFIG_COMMANDS = [
+    'configure terminal',
+    'grpc',
+    f'port {GRPC_PORT}',
+    'no-tls',
+    'commit',
+    'end',
+]
+
+
+def configure_router(mgmt_ip, label):
+    """Apply gRPC configuration via SSH CLI."""
+    print(f"\n{'─'*60}")
+    print(f"  Configuring {label} ({mgmt_ip})")
+    print(f"{'─'*60}")
+    print(f"  Applying: gRPC port={GRPC_PORT} no-tls")
+
+    try:
+        client = paramiko.SSHClient()
+        client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
+        client.connect(mgmt_ip, username=USERNAME, password=PASSWORD, timeout=10)
+
+        shell = client.invoke_shell()
+        time.sleep(1)
+        shell.recv(65535)  # clear banner
+
+        for cmd in CONFIG_COMMANDS:
+            shell.send(cmd + '\n')
+            time.sleep(1.5)
+
+        output = shell.recv(65535).decode()
+        client.close()
+
+        if 'error' in output.lower() or 'fail' in output.lower():
+            print(f"  ✗ ERROR on {label}: {output.strip()}")
+            return False
+
+        print(f"  ✓ {label} done.")
+        return True
+
+    except Exception as e:
+        print(f"  ✗ ERROR on {label}: {e}")
+        return False
+
+
+def verify_router(mgmt_ip, label):
+    """Verify gRPC configuration via SSH."""
+    try:
+        client = paramiko.SSHClient()
+        client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
+        client.connect(mgmt_ip, username=USERNAME, password=PASSWORD, timeout=10)
+
+        shell = client.invoke_shell()
+        time.sleep(1)
+        shell.recv(65535)
+
+        shell.send('show running-config grpc\n')
+        time.sleep(3)
+        output = shell.recv(65535).decode()
+        client.close()
+
+        has_port = f'port {GRPC_PORT}' in output
+        has_notls = 'no-tls' in output
+
+        p = '✓' if has_port else '✗'
+        t = '✓' if has_notls else '✗'
+        status = 'OK' if (has_port and has_notls) else 'INCOMPLETE'
+        print(f"  {label:8s}  grpc-port={p}  no-tls={t}  [{status}]")
+        return has_port and has_notls
+
+    except Exception as e:
+        print(f"  {label:8s}  verify error: {e}")
+        return False
+
+
+def main():
+    print("gNMI gRPC Configuration Script")
+    print("================================")
+    print(f"Targets: all {len(ROUTERS)} routers")
+    print()
+
+    results = []
+    for mgmt_ip, label in ROUTERS:
+        ok = configure_router(mgmt_ip, label)
+        results.append((mgmt_ip, label, ok))
+
+    # Verification pass
+    print(f"\n{'='*60}")
+    print("Post-apply verification")
+    print('='*60)
+    print(f"  {'Router':8s}  {'gRPC Port':9s}  {'No-TLS':6s}  Status")
+
+    all_ok = True
+    for mgmt_ip, label, applied_ok in results:
+        if applied_ok:
+            if not verify_router(mgmt_ip, label):
+                all_ok = False
+        else:
+            print(f"  {label:8s}  skipped (apply failed)")
+            all_ok = False
+
+    failed = [label for _, label, ok in results if not ok]
+    print()
+    if failed:
+        print(f"FAILED: {', '.join(failed)}")
+        sys.exit(1)
+    elif all_ok:
+        print("All routers configured successfully.")
+        print()
+        print(f"gRPC is now listening on port {GRPC_PORT} (no TLS) on all routers.")
+        print("Next: start Telegraf with gNMI input plugin to begin collecting telemetry.")
+    else:
+        print("Some routers may have incomplete configuration. Check output above.")
+        sys.exit(1)
+
+
+if __name__ == '__main__':
+    main()

obmp-grafana/dashboards/Learning/db_schema_map.json

@@ -0,0 +1,131 @@
+{
+  "uid": "obmp-learn-07",
+  "title": "Database Schema Map",
+  "schemaVersion": 39,
+  "tags": ["obmp-learning"],
+  "editable": true,
+  "time": {
+    "from": "now-6h",
+    "to": "now"
+  },
+  "templating": {
+    "list": []
+  },
+  "panels": [
+    {
+      "id": 1,
+      "title": "Table Row Counts",
+      "type": "table",
+      "gridPos": { "h": 12, "w": 8, "x": 0, "y": 0 },
+      "datasource": { "type": "postgres", "uid": "obmp_postgres" },
+      "targets": [
+        {
+          "refId": "A",
+          "datasource": { "type": "postgres", "uid": "obmp_postgres" },
+          "rawSql": "SELECT 'routers' as table_name, count(*) as rows FROM routers\nUNION ALL SELECT 'collectors', count(*) FROM collectors\nUNION ALL SELECT 'bgp_peers', count(*) FROM bgp_peers\nUNION ALL SELECT 'peer_event_log', count(*) FROM peer_event_log\nUNION ALL SELECT 'base_attrs', count(*) FROM base_attrs\nUNION ALL SELECT 'ip_rib', count(*) FROM ip_rib\nUNION ALL SELECT 'ip_rib_log', count(*) FROM ip_rib_log\nUNION ALL SELECT 'l3vpn_rib', count(*) FROM l3vpn_rib\nUNION ALL SELECT 'global_ip_rib', count(*) FROM global_ip_rib\nUNION ALL SELECT 'ls_nodes', count(*) FROM ls_nodes\nUNION ALL SELECT 'ls_links', count(*) FROM ls_links\nUNION ALL SELECT 'ls_prefixes', count(*) FROM ls_prefixes\nUNION ALL SELECT 'ls_nodes_log', count(*) FROM ls_nodes_log\nUNION ALL SELECT 'ls_links_log', count(*) FROM ls_links_log\nUNION ALL SELECT 'ls_prefixes_log', count(*) FROM ls_prefixes_log\nUNION ALL SELECT 'rpki_validator', count(*) FROM rpki_validator\nUNION ALL SELECT 'info_asn', count(*) FROM info_asn\nUNION ALL SELECT 'info_route', count(*) FROM info_route\nUNION ALL SELECT 'stat_reports', count(*) FROM stat_reports\nUNION ALL SELECT 'geo_ip', count(*) FROM geo_ip\nORDER BY table_name",
+          "format": "table"
+        }
+      ]
+    },
+    {
+      "id": 2,
+      "title": "Table Relationships",
+      "type": "text",
+      "gridPos": { "h": 12, "w": 8, "x": 8, "y": 0 },
+      "options": {
+        "mode": "markdown",
+        "content": "## Entity Relationships\n\n### BMP Core Chain\n```\ncollectors\n  └── routers (collector_hash_id)\n        └── bgp_peers (router_hash_id)\n              ├── ip_rib (peer_hash_id)\n              ├── ip_rib_log (peer_hash_id)\n              ├── l3vpn_rib (peer_hash_id)\n              ├── ls_nodes (peer_hash_id)\n              ├── ls_links (peer_hash_id)\n              ├── ls_prefixes (peer_hash_id)\n              ├── peer_event_log (peer_hash_id)\n              └── stat_reports (peer_hash_id)\n```\n\n### Path Attributes\n```\nip_rib ──(base_attr_hash_id)──► base_attrs\n  │                                ├── as_path (bigint[])\n  │                                ├── origin_as\n  │                                ├── next_hop\n  │                                ├── med / local_pref\n  │                                ├── community_list[]\n  │                                ├── ext_community_list[]\n  │                                └── large_community_list[]\n  │\n  └──(prefix)──► global_ip_rib\n                    ├── rpki_origin_as\n                    ├── irr_origin_as\n                    └── num_peers\n```\n\n### Link-State Topology\n```\nls_nodes ◄── ls_links (local_node_hash_id, remote_node_hash_id)\nls_nodes ◄── ls_prefixes (local_node_hash_id)\n```\n\n### Reference Data\n```\nrpki_validator ──(prefix, origin_as)──► validates ip_rib\ninfo_asn ──(asn)──► enriches base_attrs.origin_as\ninfo_route ──(prefix)──► enriches ip_rib.prefix\ngeo_ip ──(ip)──► geolocates routers, peers\n```"
+      }
+    },
+    {
+      "id": 3,
+      "title": "BMP Core Tables",
+      "type": "text",
+      "gridPos": { "h": 8, "w": 8, "x": 16, "y": 0 },
+      "options": {
+        "mode": "markdown",
+        "content": "## BMP Core Tables\n\n| Table | Purpose | Key Columns |\n|-------|---------|-------------|\n| **routers** | BMP-monitored routers | hash_id, name, ip_address, router_as, state, bgp_id |\n| **collectors** | BMP collector instances | hash_id, admin_id, name, ip_address, router_count |\n| **bgp_peers** | BGP sessions per router | hash_id, router_hash_id, peer_addr, peer_as, state, isl3vpnpeer |\n| **peer_event_log** | Session state history (TimescaleDB) | peer_hash_id, state, timestamp, bmp_reason, bgp_err_code |\n| **stat_reports** | BMP statistics messages | peer_hash_id, prefixes_rejected, num_routes_adj_rib_in, num_routes_local_rib |\n| **users** | Access control | username, password, type (admin/oper) |"
+      }
+    },
+    {
+      "id": 4,
+      "title": "RIB & Path Attribute Tables",
+      "type": "text",
+      "gridPos": { "h": 8, "w": 8, "x": 16, "y": 8 },
+      "options": {
+        "mode": "markdown",
+        "content": "## RIB & Path Attribute Tables\n\n| Table | Purpose | Key Columns |\n|-------|---------|-------------|\n| **base_attrs** | BGP path attributes | hash_id, as_path[], as_path_count, origin_as, next_hop, med, local_pref, community_list[], ext_community_list[], large_community_list[], cluster_list, originator_id |\n| **ip_rib** | IPv4/IPv6 unicast RIB | hash_id, peer_hash_id, prefix, prefix_len, origin_as, iswithdrawn, labels, path_id |\n| **ip_rib_log** | RIB change history (TimescaleDB) | peer_hash_id, prefix, prefix_len, origin_as, iswithdrawn, timestamp |\n| **l3vpn_rib** | L3VPN/MPLS VPN routes | hash_id, peer_hash_id, rd, prefix, labels, ext_community_list[] |\n| **l3vpn_rib_log** | L3VPN change history (TimescaleDB) | peer_hash_id, rd, prefix, iswithdrawn, timestamp |\n| **global_ip_rib** | Aggregated prefix summary | prefix, recv_origin_as, rpki_origin_as, irr_origin_as, num_peers |"
+      }
+    },
+    {
+      "id": 5,
+      "title": "Link-State Tables",
+      "type": "text",
+      "gridPos": { "h": 8, "w": 12, "x": 0, "y": 12 },
+      "options": {
+        "mode": "markdown",
+        "content": "## Link-State Tables (BGP-LS / RFC 7752)\n\n| Table | Purpose | Key Columns |\n|-------|---------|-------------|\n| **ls_nodes** | IS-IS/OSPF nodes | hash_id, peer_hash_id, igp_router_id, name, protocol, asn, sr_capabilities, isis_area_id |\n| **ls_links** | IS-IS/OSPF links + TE/SR | hash_id, local/remote_node_hash_id, interface_addr, neighbor_addr, igp_metric, **te_def_metric**, **max_link_bw**, **max_resv_bw**, **unreserved_bw**, **admin_group**, **srlg**, **sr_adjacency_sids**, **peer_node_sid**, **protection_type**, **mpls_proto_mask** |\n| **ls_prefixes** | IS-IS/OSPF prefixes | hash_id, local_node_hash_id, prefix, metric, sr_prefix_sids, igp_flags |\n| **ls_nodes_log** | Node change history (TimescaleDB) | Same as ls_nodes + timestamp |\n| **ls_links_log** | Link change history (TimescaleDB) | Same as ls_links + timestamp |\n| **ls_prefixes_log** | Prefix change history (TimescaleDB) | Same as ls_prefixes + timestamp |\n\n**Bold columns** = TE/SR fields not used by any existing dashboard"
+      }
+    },
+    {
+      "id": 6,
+      "title": "Statistics Tables",
+      "type": "text",
+      "gridPos": { "h": 8, "w": 12, "x": 12, "y": 12 },
+      "options": {
+        "mode": "markdown",
+        "content": "## Statistics Tables (TimescaleDB Hypertables)\n\n| Table | Purpose | Key Columns |\n|-------|---------|-------------|\n| **stat_reports** | BMP stat messages | peer_hash_id, prefixes_rejected, known_dup_prefixes, num_routes_adj_rib_in |\n| **stats_chg_byprefix** | Per-prefix churn stats | interval_time, peer_hash_id, prefix, updates, withdraws |\n| **stats_chg_byasn** | Per-ASN churn stats | interval_time, peer_hash_id, origin_as, updates, withdraws |\n| **stats_chg_bypeer** | Per-peer churn stats | interval_time, peer_hash_id, updates, withdraws |\n| **stats_peer_rib** | Per-peer RIB size | interval_time, peer_hash_id, v4_prefixes, v6_prefixes |\n| **stats_peer_update_counts** | Update rate statistics | interval_time, peer_hash_id, advertise_avg/min/max, withdraw_avg/min/max |\n| **stats_ip_origins** | Per-ASN prefix counts | interval_time, asn, v4_prefixes, v6_prefixes, v4_with_rpki, v4_with_irr |"
+      }
+    },
+    {
+      "id": 7,
+      "title": "Reference & Enrichment Tables",
+      "type": "text",
+      "gridPos": { "h": 6, "w": 12, "x": 0, "y": 20 },
+      "options": {
+        "mode": "markdown",
+        "content": "## Reference & Enrichment Tables\n\n| Table | Purpose | Key Columns |\n|-------|---------|-------------|\n| **rpki_validator** | RPKI ROAs | prefix, prefix_len, prefix_len_max, origin_as |\n| **info_asn** | ASN WHOIS/IRR data | asn, as_name, org_name, country, source |\n| **info_route** | Route IRR data | prefix, prefix_len, origin_as, descr, source |\n| **geo_ip** | IP geolocation (DB-IP) | ip, country, city, latitude, longitude, isp_name |\n| **pdb_exchange_peers** | PeeringDB IXP data | ix_name, peer_name, peer_asn, speed, peer_ipv4/ipv6 |"
+      }
+    },
+    {
+      "id": 8,
+      "title": "Views Quick Reference",
+      "type": "text",
+      "gridPos": { "h": 6, "w": 12, "x": 12, "y": 20 },
+      "options": {
+        "mode": "markdown",
+        "content": "## Database Views\n\n| View | Joins | Purpose |\n|------|-------|---------|\n| **v_peers** | bgp_peers + routers + info_asn | Complete peer info with router name and ASN details |\n| **v_ip_routes** | ip_rib + bgp_peers + base_attrs + routers | Full route detail with path attributes |\n| **v_ip_routes_geo** | v_ip_routes + geo_ip | Routes with geolocation |\n| **v_ip_routes_history** | ip_rib_log + base_attrs + bgp_peers + routers | Historical route changes with attributes |\n| **v_l3vpn_routes** | l3vpn_rib + bgp_peers + base_attrs + routers | L3VPN routes with path attributes |\n| **v_l3vpn_routes_history** | l3vpn_rib_log + base_attrs + bgp_peers + routers | Historical L3VPN changes |\n| **v_ls_nodes** | ls_nodes + base_attrs + bgp_peers + routers | Link-state nodes with peer/router info |\n| **v_ls_links** | ls_links + ls_nodes(x2) + routers | Links with local/remote node names + TE fields |\n| **v_ls_prefixes** | ls_prefixes + ls_nodes + routers | LS prefixes with originating node info |\n\n### Enum Types\n- **opstate**: up, down\n- **ls_proto**: IS-IS_L1, IS-IS_L2, OSPFv2, OSPFv3, Direct, Static\n- **ospf_route_type**: Intra, Inter, Ext-1, Ext-2, NSSA-1, NSSA-2\n- **ls_mpls_proto_mask**: MPLS protocol bitmask"
+      }
+    },
+    {
+      "id": 9,
+      "title": "LinkState Column Details",
+      "type": "table",
+      "gridPos": { "h": 10, "w": 12, "x": 0, "y": 26 },
+      "datasource": { "type": "postgres", "uid": "obmp_postgres" },
+      "targets": [
+        {
+          "refId": "A",
+          "datasource": { "type": "postgres", "uid": "obmp_postgres" },
+          "rawSql": "SELECT column_name, data_type, \n       CASE \n         WHEN column_name IN ('admin_group','max_link_bw','max_resv_bw','unreserved_bw','te_def_metric','protection_type','srlg','sr_adjacency_sids','peer_node_sid','mpls_proto_mask') THEN 'TE/SR'\n         WHEN column_name IN ('hash_id','peer_hash_id','base_attr_hash_id','local_node_hash_id','remote_node_hash_id') THEN 'FK/Key'\n         ELSE 'Core'\n       END as category\nFROM information_schema.columns \nWHERE table_name = 'ls_links' AND table_schema = 'public'\nORDER BY ordinal_position",
+          "format": "table"
+        }
+      ]
+    },
+    {
+      "id": 10,
+      "title": "ip_rib Column Details",
+      "type": "table",
+      "gridPos": { "h": 10, "w": 12, "x": 12, "y": 26 },
+      "datasource": { "type": "postgres", "uid": "obmp_postgres" },
+      "targets": [
+        {
+          "refId": "A",
+          "datasource": { "type": "postgres", "uid": "obmp_postgres" },
+          "rawSql": "SELECT column_name, data_type,\n       CASE \n         WHEN column_name IN ('hash_id','peer_hash_id','base_attr_hash_id') THEN 'FK/Key'\n         ELSE 'Core'\n       END as category\nFROM information_schema.columns \nWHERE table_name = 'ip_rib' AND table_schema = 'public'\nORDER BY ordinal_position",
+          "format": "table"
+        }
+      ]
+    }
+  ]
+}

obmp-grafana/dashboards/Learning/link_utilization_te.json

@@ -0,0 +1,238 @@
+{
+  "annotations": {
+    "list": [
+      {
+        "builtIn": 1,
+        "datasource": { "type": "datasource", "uid": "grafana" },
+        "enable": true,
+        "hide": true,
+        "iconColor": "rgba(0, 211, 255, 1)",
+        "name": "Annotations & Alerts",
+        "type": "dashboard"
+      }
+    ]
+  },
+  "editable": true,
+  "fiscalYearStartMonth": 0,
+  "graphTooltip": 0,
+  "links": [],
+  "liveNow": false,
+  "panels": [
+    {
+      "datasource": { "type": "postgres", "uid": "obmp_postgres" },
+      "fieldConfig": {
+        "defaults": {
+          "custom": {
+            "align": "auto",
+            "cellOptions": { "type": "auto" },
+            "inspect": false
+          }
+        },
+        "overrides": [
+          {
+            "matcher": { "id": "byName", "options": "Max BW (B/s)" },
+            "properties": [{ "id": "unit", "value": "Bps" }]
+          },
+          {
+            "matcher": { "id": "byName", "options": "Max Reservable BW" },
+            "properties": [{ "id": "unit", "value": "Bps" }]
+          }
+        ]
+      },
+      "gridPos": { "h": 10, "w": 24, "x": 0, "y": 0 },
+      "id": 1,
+      "options": {
+        "showHeader": true,
+        "sortBy": [{ "desc": false, "displayName": "Local Router" }]
+      },
+      "targets": [
+        {
+          "datasource": { "type": "postgres", "uid": "obmp_postgres" },
+          "format": "table",
+          "rawQuery": true,
+          "rawSql": "SELECT local_router_name as \"Local Router\",\n       remote_router_name as \"Remote Router\",\n       interface_addr::text as \"Interface IP\",\n       neighbor_addr::text as \"Neighbor IP\",\n       max_link_bw as \"Max BW (B/s)\",\n       max_resv_bw as \"Max Reservable BW\",\n       unreserved_bw as \"Unreserved BW\",\n       igp_metric as \"IGP Metric\",\n       te_def_metric as \"TE Metric\"\nFROM v_ls_links\nWHERE peer_hash_id = '$peer_hash' AND iswithdrawn = false\nORDER BY local_router_name, remote_router_name",
+          "refId": "A"
+        }
+      ],
+      "title": "Link Capacity Inventory (from BGP-LS)",
+      "type": "table"
+    },
+    {
+      "datasource": { "type": "postgres", "uid": "obmp_postgres" },
+      "fieldConfig": {
+        "defaults": {
+          "color": { "mode": "palette-classic" },
+          "custom": {
+            "axisBorderShow": false,
+            "axisCenteredZero": false,
+            "axisLabel": "Bandwidth (B/s)",
+            "fillOpacity": 80,
+            "gradientMode": "none",
+            "lineWidth": 1,
+            "scaleDistribution": { "type": "linear" },
+            "showValue": "auto",
+            "stacking": { "group": "A", "mode": "none" }
+          },
+          "unit": "Bps"
+        },
+        "overrides": []
+      },
+      "gridPos": { "h": 10, "w": 12, "x": 0, "y": 10 },
+      "id": 2,
+      "options": {
+        "barRadius": 0.1,
+        "barWidth": 0.8,
+        "groupWidth": 0.7,
+        "legend": { "calcs": [], "displayMode": "list", "placement": "bottom" },
+        "orientation": "horizontal",
+        "tooltip": { "mode": "single", "sort": "none" },
+        "xTickLabelRotation": 0
+      },
+      "targets": [
+        {
+          "datasource": { "type": "postgres", "uid": "obmp_postgres" },
+          "format": "table",
+          "rawQuery": true,
+          "rawSql": "SELECT local_router_name || ' -> ' || remote_router_name as \"Link\",\n       COALESCE(max_link_bw, 0) as \"Max Bandwidth\",\n       COALESCE(max_resv_bw, 0) as \"Max Reservable\",\n       COALESCE(max_link_bw, 0) - COALESCE(max_resv_bw, 0) as \"Unreserved Gap\"\nFROM v_ls_links\nWHERE peer_hash_id = '$peer_hash' AND iswithdrawn = false\n      AND max_link_bw IS NOT NULL AND max_link_bw > 0\nORDER BY max_link_bw DESC",
+          "refId": "A"
+        }
+      ],
+      "title": "Capacity vs Reservable Bandwidth",
+      "type": "barchart"
+    },
+    {
+      "datasource": { "type": "postgres", "uid": "obmp_postgres" },
+      "fieldConfig": {
+        "defaults": {
+          "color": { "mode": "thresholds" },
+          "thresholds": {
+            "mode": "percentage",
+            "steps": [
+              { "color": "green", "value": null },
+              { "color": "yellow", "value": 50 },
+              { "color": "orange", "value": 75 },
+              { "color": "red", "value": 90 }
+            ]
+          },
+          "unit": "percentunit",
+          "max": 1,
+          "min": 0
+        },
+        "overrides": []
+      },
+      "gridPos": { "h": 10, "w": 12, "x": 12, "y": 10 },
+      "id": 3,
+      "options": {
+        "minVizHeight": 75,
+        "minVizWidth": 75,
+        "orientation": "auto",
+        "reduceOptions": {
+          "calcs": ["lastNotNull"],
+          "fields": "",
+          "values": true
+        },
+        "showThresholdLabels": false,
+        "showThresholdMarkers": true,
+        "sizing": "auto"
+      },
+      "targets": [
+        {
+          "datasource": { "type": "postgres", "uid": "obmp_postgres" },
+          "format": "table",
+          "rawQuery": true,
+          "rawSql": "SELECT local_router_name || ' -> ' || remote_router_name as \"Link\",\n       CASE WHEN max_link_bw > 0 \n            THEN 1.0 - (COALESCE(max_resv_bw, 0)::float / max_link_bw::float)\n            ELSE 0 END as \"Reservation Ratio\"\nFROM v_ls_links\nWHERE peer_hash_id = '$peer_hash' AND iswithdrawn = false\n      AND max_link_bw IS NOT NULL AND max_link_bw > 0\nORDER BY \"Reservation Ratio\" DESC\nLIMIT 10",
+          "refId": "A"
+        }
+      ],
+      "title": "Bandwidth Reservation Ratio (Higher = More Reserved)",
+      "type": "gauge"
+    },
+    {
+      "datasource": { "type": "postgres", "uid": "obmp_postgres" },
+      "fieldConfig": {
+        "defaults": {
+          "color": { "mode": "palette-classic" },
+          "custom": {
+            "axisBorderShow": false,
+            "fillOpacity": 80,
+            "gradientMode": "none",
+            "lineWidth": 1,
+            "showValue": "auto",
+            "stacking": { "group": "A", "mode": "none" }
+          }
+        },
+        "overrides": []
+      },
+      "gridPos": { "h": 8, "w": 12, "x": 0, "y": 20 },
+      "id": 4,
+      "options": {
+        "barRadius": 0.1,
+        "barWidth": 0.8,
+        "groupWidth": 0.7,
+        "legend": { "calcs": [], "displayMode": "list", "placement": "bottom" },
+        "orientation": "horizontal",
+        "tooltip": { "mode": "single", "sort": "none" }
+      },
+      "targets": [
+        {
+          "datasource": { "type": "postgres", "uid": "obmp_postgres" },
+          "format": "table",
+          "rawQuery": true,
+          "rawSql": "SELECT local_router_name || ' -> ' || remote_router_name as \"Link\",\n       igp_metric as \"IGP Metric\",\n       COALESCE(te_def_metric, 0) as \"TE Default Metric\"\nFROM v_ls_links\nWHERE peer_hash_id = '$peer_hash' AND iswithdrawn = false\nORDER BY igp_metric DESC\nLIMIT 20",
+          "refId": "A"
+        }
+      ],
+      "title": "IGP Metric vs TE Default Metric",
+      "type": "barchart"
+    },
+    {
+      "gridPos": { "h": 8, "w": 12, "x": 12, "y": 20 },
+      "id": 5,
+      "options": {
+        "mode": "markdown",
+        "content": "## What-If: CSPF Path Computation\n\nIn a real MPLS-TE or SR-TE deployment, the headend router runs **Constrained Shortest Path First (CSPF)** to find paths that satisfy:\n\n1. **Bandwidth constraint** - Enough unreserved BW at the required priority\n2. **Admin group (affinity)** - Link colors must match include/exclude masks\n3. **SRLG diversity** - Backup path avoids shared risk with primary\n4. **TE metric optimization** - Minimize TE metric (not IGP metric)\n\n### How BGP-LS Enables This\n\nBGP-LS distributes the complete IGP topology **with TE attributes** to an external controller (PCE, SDN controller). The controller can:\n\n- Build a Traffic Engineering Database (TED)\n- Run CSPF with arbitrary constraints\n- Program SR-TE policies via PCEP or gRPC\n\n### Data Available in This Lab\n\n| Attribute | Source | Available? |\n|-----------|--------|------------|\n| Topology (nodes/links) | BGP-LS | Yes |\n| IGP Metric | BGP-LS | Yes |\n| TE Default Metric | BGP-LS TLV 1092 | Check TE table |\n| Max Link BW | BGP-LS TLV 1089 | Check TE table |\n| Max Reservable BW | BGP-LS TLV 1090 | Check TE table |\n| Unreserved BW | BGP-LS TLV 1091 | Check TE table |\n| Admin Group | BGP-LS TLV 1088 | Check TE table |\n| SRLG | BGP-LS TLV 1096 | Check TE table |\n| SR Node SID | BGP-LS TLV 1034 | Check SR table |\n| SR Adj SID | BGP-LS TLV 1099 | Check SR table |"
+      },
+      "title": "CSPF & Traffic Engineering Concepts",
+      "type": "text"
+    },
+    {
+      "gridPos": { "h": 12, "w": 24, "x": 0, "y": 28 },
+      "id": 6,
+      "options": {
+        "mode": "markdown",
+        "content": "## Integration Guide: Adding Real-Time Link Utilization\n\nBMP/BGP-LS provides **capacity** data (max bandwidth, reservable bandwidth) but NOT real-time **utilization**. To complete the traffic engineering picture, you need streaming telemetry.\n\n### Architecture\n\n```\n                      +------------------+\n  IOS-XR Routers ---->| OpenBMP Collector|----> PostgreSQL (topology + capacity)\n       |              +------------------+\n       |              \n       +-- gNMI ----->| Telegraf         |----> InfluxDB/Prometheus (utilization)\n                      +------------------+\n                              |\n                      +------------------+\n                      |    Grafana       |  <-- Mixed datasource queries\n                      +------------------+\n```\n\n### Step 1: Enable Model-Driven Telemetry on IOS-XR\n\n```\ntelemetry model-driven\n  sensor-group INTF-COUNTERS\n    sensor-path Cisco-IOS-XR-infra-statsd-oper:infra-statistics/interfaces/interface/latest/generic-counters\n  !\n  subscription INTF-SUB\n    sensor-group-id INTF-COUNTERS sample-interval 30000\n    destination-id TELEGRAF\n  !\n  destination-group TELEGRAF\n    address-family ipv4 10.40.40.202 port 57000\n      encoding self-describing-gpb\n      protocol grpc no-tls\n    !\n  !\n```\n\n### Step 2: Telegraf Configuration\n\n```toml\n[[inputs.cisco_telemetry_mdt]]\n  transport = \"grpc\"\n  service_address = \":57000\"\n\n[[outputs.influxdb_v2]]\n  urls = [\"http://localhost:8086\"]\n  token = \"your-token\"\n  organization = \"openbmp\"\n  bucket = \"telemetry\"\n```\n\n### Step 3: Grafana Mixed Datasource Query\n\nCombine BGP-LS capacity from PostgreSQL with utilization from InfluxDB:\n\n```\n-- PostgreSQL: Get link capacity\nSELECT interface_addr::text as interface, max_link_bw\nFROM v_ls_links WHERE peer_hash_id = '$peer_hash'\n\n-- InfluxDB: Get interface utilization\nfrom(bucket: \"telemetry\")\n  |> range(start: -1h)\n  |> filter(fn: (r) => r._measurement == \"Cisco-IOS-XR-infra-statsd-oper\")\n  |> filter(fn: (r) => r._field == \"bytes-received\" or r._field == \"bytes-sent\")\n  |> derivative(unit: 1s, nonNegative: true)\n```\n\n### Step 4: Calculate Utilization %\n\nIn Grafana, use **Transformations** to:\n1. Join PostgreSQL capacity with InfluxDB utilization by interface IP\n2. Add calculated field: `utilization_pct = bytes_per_sec / max_link_bw * 100`\n3. Set threshold alerts: >80% = warning, >95% = critical\n\n### Key gNMI Sensor Paths for IOS-XR\n\n| Sensor Path | Data |\n|-------------|------|\n| `Cisco-IOS-XR-infra-statsd-oper:infra-statistics/interfaces/interface/latest/generic-counters` | Interface byte/packet counters |\n| `Cisco-IOS-XR-infra-statsd-oper:infra-statistics/interfaces/interface/latest/data-rate` | Current data rate (bits/sec) |\n| `Cisco-IOS-XR-mpls-te-oper:mpls-te/tunnels/summary` | MPLS-TE tunnel summary |\n| `Cisco-IOS-XR-ip-rsvp-oper:rsvp/interface-briefs` | RSVP interface reservations |\n| `Cisco-IOS-XR-segment-routing-ms-oper:srms/policy` | SR-MPLS policy state |\n\n### RFC 8571: Performance Metrics via BGP-LS\n\nIf routers support RFC 8571, these metrics flow through BGP-LS automatically:\n- **Unidirectional Link Delay** (TLV 1114) - microseconds\n- **Min/Max Link Delay** (TLV 1115)\n- **Delay Variation (jitter)** (TLV 1116)\n- **Link Loss** (TLV 1117) - percentage\n- **Residual Bandwidth** (TLV 1118)\n- **Available Bandwidth** (TLV 1119)\n- **Utilized Bandwidth** (TLV 1120)\n\nThese would appear in the `ls_links` table if the OpenBMP parser supports them."
+      },
+      "title": "Integration Guide: Streaming Telemetry for Link Utilization",
+      "type": "text"
+    }
+  ],
+  "schemaVersion": 39,
+  "tags": ["obmp-learning"],
+  "templating": {
+    "list": [
+      {
+        "current": {},
+        "datasource": { "type": "postgres", "uid": "obmp_postgres" },
+        "definition": "SELECT __text,__value FROM (\n    select peername as __text, peer_hash_id as __value, count(*) as count\n    from v_ls_nodes\n   group by peername,peer_hash_id) d\nwhere count > 0",
+        "hide": 0,
+        "includeAll": false,
+        "label": "BGP Peer",
+        "multi": false,
+        "name": "peer_hash",
+        "options": [],
+        "query": "SELECT __text,__value FROM (\n    select peername as __text, peer_hash_id as __value, count(*) as count\n    from v_ls_nodes\n   group by peername,peer_hash_id) d\nwhere count > 0",
+        "refresh": 1,
+        "regex": "",
+        "skipUrlSync": false,
+        "sort": 0,
+        "type": "query"
+      }
+    ]
+  },
+  "time": { "from": "now-6h", "to": "now" },
+  "timepicker": {},
+  "timezone": "",
+  "title": "Link Utilization & TE Thought Experiment",
+  "uid": "obmp-learn-10",
+  "version": 1
+}

obmp-grafana/dashboards/Learning/rr_locrib_diff.json

@@ -0,0 +1,418 @@
+{
+  "annotations": {"list": [{"builtIn": 1,"datasource": {"type": "datasource","uid": "grafana"},"enable": true,"hide": true,"iconColor": "rgba(0, 211, 255, 1)","name": "Annotations & Alerts","target": {"limit": 100,"matchAny": false,"tags": [],"type": "dashboard"},"type": "dashboard"}]},
+  "description": "Compare Route Reflector Loc-RIB tables via BMP Adj-RIB-In. Surfaces missing prefixes, attribute differences, and per-client consistency between two RRs.",
+  "editable": true,
+  "fiscalYearStartMonth": 0,
+  "graphTooltip": 1,
+  "id": null,
+  "links": [],
+  "panels": [
+    {
+      "datasource": {"type": "datasource","uid": "grafana"},
+      "gridPos": {"h": 5,"w": 24,"x": 0,"y": 0},
+      "id": 1,
+      "options": {
+        "content": "## Route Reflector Loc-RIB Diff\n\nCompares the RIB tables of two Route Reflectors via BMP. Select your two RRs using the **RR1** and **RR2** dropdowns above.\n\n**What this shows:**\n- **Summary stats** — prefix counts and diff totals\n- **Missing prefixes** — routes present on one RR but not the other (expected for each RR's own locally-originated routes)\n- **Attribute differences** — same prefix on both RRs but with different next-hop, AS path, or other attributes (indicates different best-path selection)\n- **Per-client consistency** — route counts each client sends to each RR (should be identical in a healthy RR cluster)",
+        "mode": "markdown"
+      },
+      "title": "RR Loc-RIB Diff — Overview",
+      "type": "text"
+    },
+    {
+      "datasource": {"type": "postgres","uid": "obmp_postgres"},
+      "description": "Total active (non-withdrawn) prefixes on RR1",
+      "fieldConfig": {
+        "defaults": {"color": {"fixedColor": "blue","mode": "fixed"},"thresholds": {"mode": "absolute","steps": [{"color": "blue","value": null}]}},
+        "overrides": []
+      },
+      "gridPos": {"h": 4,"w": 4,"x": 0,"y": 5},
+      "id": 10,
+      "options": {"colorMode": "value","graphMode": "none","justifyMode": "auto","orientation": "auto","reduceOptions": {"calcs": ["lastNotNull"],"fields": "","values": false},"textMode": "auto"},
+      "targets": [
+        {
+          "datasource": {"type": "postgres","uid": "obmp_postgres"},
+          "format": "table",
+          "rawSql": "SELECT COUNT(DISTINCT r.prefix || '/' || r.prefix_len) AS \"prefixes\"\nFROM ip_rib r\nJOIN bgp_peers p ON p.hash_id = r.peer_hash_id\nJOIN routers rt ON rt.hash_id = p.router_hash_id\nWHERE rt.name = '$rr1' AND r.iswithdrawn = false",
+          "refId": "A"
+        }
+      ],
+      "title": "$rr1 Prefixes",
+      "type": "stat"
+    },
+    {
+      "datasource": {"type": "postgres","uid": "obmp_postgres"},
+      "description": "Total active (non-withdrawn) prefixes on RR2",
+      "fieldConfig": {
+        "defaults": {"color": {"fixedColor": "green","mode": "fixed"},"thresholds": {"mode": "absolute","steps": [{"color": "green","value": null}]}},
+        "overrides": []
+      },
+      "gridPos": {"h": 4,"w": 4,"x": 4,"y": 5},
+      "id": 11,
+      "options": {"colorMode": "value","graphMode": "none","justifyMode": "auto","orientation": "auto","reduceOptions": {"calcs": ["lastNotNull"],"fields": "","values": false},"textMode": "auto"},
+      "targets": [
+        {
+          "datasource": {"type": "postgres","uid": "obmp_postgres"},
+          "format": "table",
+          "rawSql": "SELECT COUNT(DISTINCT r.prefix || '/' || r.prefix_len) AS \"prefixes\"\nFROM ip_rib r\nJOIN bgp_peers p ON p.hash_id = r.peer_hash_id\nJOIN routers rt ON rt.hash_id = p.router_hash_id\nWHERE rt.name = '$rr2' AND r.iswithdrawn = false",
+          "refId": "A"
+        }
+      ],
+      "title": "$rr2 Prefixes",
+      "type": "stat"
+    },
+    {
+      "datasource": {"type": "postgres","uid": "obmp_postgres"},
+      "description": "Prefixes present on RR1 but missing from RR2",
+      "fieldConfig": {
+        "defaults": {"color": {"fixedColor": "orange","mode": "fixed"},"thresholds": {"mode": "absolute","steps": [{"color": "orange","value": null},{"color": "red","value": 1}]}},
+        "overrides": []
+      },
+      "gridPos": {"h": 4,"w": 4,"x": 8,"y": 5},
+      "id": 12,
+      "options": {"colorMode": "value","graphMode": "none","justifyMode": "auto","orientation": "auto","reduceOptions": {"calcs": ["lastNotNull"],"fields": "","values": false},"textMode": "auto"},
+      "targets": [
+        {
+          "datasource": {"type": "postgres","uid": "obmp_postgres"},
+          "format": "table",
+          "rawSql": "SELECT COUNT(*) AS \"only_rr1\"\nFROM (\n  SELECT DISTINCT r1.prefix, r1.prefix_len\n  FROM ip_rib r1\n  JOIN bgp_peers p1 ON p1.hash_id = r1.peer_hash_id\n  JOIN routers rt1 ON rt1.hash_id = p1.router_hash_id\n  WHERE rt1.name = '$rr1' AND r1.iswithdrawn = false\n  EXCEPT\n  SELECT DISTINCT r2.prefix, r2.prefix_len\n  FROM ip_rib r2\n  JOIN bgp_peers p2 ON p2.hash_id = r2.peer_hash_id\n  JOIN routers rt2 ON rt2.hash_id = p2.router_hash_id\n  WHERE rt2.name = '$rr2' AND r2.iswithdrawn = false\n) sub",
+          "refId": "A"
+        }
+      ],
+      "title": "Only on $rr1",
+      "type": "stat"
+    },
+    {
+      "datasource": {"type": "postgres","uid": "obmp_postgres"},
+      "description": "Prefixes present on RR2 but missing from RR1",
+      "fieldConfig": {
+        "defaults": {"color": {"fixedColor": "orange","mode": "fixed"},"thresholds": {"mode": "absolute","steps": [{"color": "orange","value": null},{"color": "red","value": 1}]}},
+        "overrides": []
+      },
+      "gridPos": {"h": 4,"w": 4,"x": 12,"y": 5},
+      "id": 13,
+      "options": {"colorMode": "value","graphMode": "none","justifyMode": "auto","orientation": "auto","reduceOptions": {"calcs": ["lastNotNull"],"fields": "","values": false},"textMode": "auto"},
+      "targets": [
+        {
+          "datasource": {"type": "postgres","uid": "obmp_postgres"},
+          "format": "table",
+          "rawSql": "SELECT COUNT(*) AS \"only_rr2\"\nFROM (\n  SELECT DISTINCT r2.prefix, r2.prefix_len\n  FROM ip_rib r2\n  JOIN bgp_peers p2 ON p2.hash_id = r2.peer_hash_id\n  JOIN routers rt2 ON rt2.hash_id = p2.router_hash_id\n  WHERE rt2.name = '$rr2' AND r2.iswithdrawn = false\n  EXCEPT\n  SELECT DISTINCT r1.prefix, r1.prefix_len\n  FROM ip_rib r1\n  JOIN bgp_peers p1 ON p1.hash_id = r1.peer_hash_id\n  JOIN routers rt1 ON rt1.hash_id = p1.router_hash_id\n  WHERE rt1.name = '$rr1' AND r1.iswithdrawn = false\n) sub",
+          "refId": "A"
+        }
+      ],
+      "title": "Only on $rr2",
+      "type": "stat"
+    },
+    {
+      "datasource": {"type": "postgres","uid": "obmp_postgres"},
+      "description": "Prefixes present on both RRs but with different best-path attributes (next-hop, AS path, MED, or local-pref)",
+      "fieldConfig": {
+        "defaults": {"color": {"fixedColor": "yellow","mode": "fixed"},"thresholds": {"mode": "absolute","steps": [{"color": "green","value": null},{"color": "yellow","value": 1},{"color": "red","value": 10}]}},
+        "overrides": []
+      },
+      "gridPos": {"h": 4,"w": 4,"x": 16,"y": 5},
+      "id": 14,
+      "options": {"colorMode": "value","graphMode": "none","justifyMode": "auto","orientation": "auto","reduceOptions": {"calcs": ["lastNotNull"],"fields": "","values": false},"textMode": "auto"},
+      "targets": [
+        {
+          "datasource": {"type": "postgres","uid": "obmp_postgres"},
+          "format": "table",
+          "rawSql": "WITH c1 AS (\n  SELECT DISTINCT ON (r.prefix, r.prefix_len)\n    r.prefix, r.prefix_len, ba.next_hop, ba.as_path,\n    COALESCE(ba.local_pref, 0) AS lp, COALESCE(ba.med, 0) AS med\n  FROM ip_rib r\n  JOIN bgp_peers p ON p.hash_id = r.peer_hash_id\n  JOIN routers rt ON rt.hash_id = p.router_hash_id\n  JOIN base_attrs ba ON ba.hash_id = r.base_attr_hash_id\n  WHERE rt.name = '$rr1' AND r.iswithdrawn = false\n  ORDER BY r.prefix, r.prefix_len, ba.local_pref DESC NULLS LAST\n),\nc2 AS (\n  SELECT DISTINCT ON (r.prefix, r.prefix_len)\n    r.prefix, r.prefix_len, ba.next_hop, ba.as_path,\n    COALESCE(ba.local_pref, 0) AS lp, COALESCE(ba.med, 0) AS med\n  FROM ip_rib r\n  JOIN bgp_peers p ON p.hash_id = r.peer_hash_id\n  JOIN routers rt ON rt.hash_id = p.router_hash_id\n  JOIN base_attrs ba ON ba.hash_id = r.base_attr_hash_id\n  WHERE rt.name = '$rr2' AND r.iswithdrawn = false\n  ORDER BY r.prefix, r.prefix_len, ba.local_pref DESC NULLS LAST\n)\nSELECT COUNT(*) AS \"attr_diffs\"\nFROM c1 JOIN c2 ON c1.prefix = c2.prefix AND c1.prefix_len = c2.prefix_len\nWHERE c1.next_hop != c2.next_hop OR c1.as_path != c2.as_path\n   OR c1.lp != c2.lp OR c1.med != c2.med",
+          "refId": "A"
+        }
+      ],
+      "title": "Attribute Diffs",
+      "type": "stat"
+    },
+    {
+      "datasource": {"type": "postgres","uid": "obmp_postgres"},
+      "description": "Total RIB entries (including multiple paths per prefix) on each RR",
+      "fieldConfig": {
+        "defaults": {"color": {"fixedColor": "purple","mode": "fixed"},"thresholds": {"mode": "absolute","steps": [{"color": "purple","value": null}]}},
+        "overrides": []
+      },
+      "gridPos": {"h": 4,"w": 4,"x": 20,"y": 5},
+      "id": 15,
+      "options": {"colorMode": "value","graphMode": "none","justifyMode": "auto","orientation": "auto","reduceOptions": {"calcs": ["lastNotNull"],"fields": "","values": false},"textMode": "auto"},
+      "targets": [
+        {
+          "datasource": {"type": "postgres","uid": "obmp_postgres"},
+          "format": "table",
+          "rawSql": "SELECT\n  SUM(CASE WHEN rt.name = '$rr1' THEN 1 ELSE 0 END) AS \"RR1 Total Paths\",\n  SUM(CASE WHEN rt.name = '$rr2' THEN 1 ELSE 0 END) AS \"RR2 Total Paths\"\nFROM ip_rib r\nJOIN bgp_peers p ON p.hash_id = r.peer_hash_id\nJOIN routers rt ON rt.hash_id = p.router_hash_id\nWHERE rt.name IN ('$rr1', '$rr2') AND r.iswithdrawn = false",
+          "refId": "A"
+        }
+      ],
+      "title": "Total RIB Paths",
+      "type": "stat"
+    },
+    {
+      "collapsed": false,
+      "gridPos": {"h": 1,"w": 24,"x": 0,"y": 9},
+      "id": 20,
+      "title": "Missing Prefixes",
+      "type": "row"
+    },
+    {
+      "datasource": {"type": "postgres","uid": "obmp_postgres"},
+      "description": "Prefixes present on RR1 but NOT on RR2. Expected: RR2's own locally-originated routes won't appear here (they're on RR2 only). Unexpected entries indicate a convergence or session issue.",
+      "fieldConfig": {
+        "defaults": {"color": {"mode": "thresholds"},"custom": {"align": "auto","displayMode": "auto","filterable": true},"thresholds": {"mode": "absolute","steps": [{"color": "green","value": null}]}},
+        "overrides": [
+          {"matcher": {"id": "byName","options": "Learned From"},"properties": [{"id": "custom.width","value": 130}]},
+          {"matcher": {"id": "byName","options": "Origin AS"},"properties": [{"id": "custom.width","value": 90}]},
+          {"matcher": {"id": "byName","options": "Next Hop"},"properties": [{"id": "custom.width","value": 140}]}
+        ]
+      },
+      "gridPos": {"h": 10,"w": 12,"x": 0,"y": 10},
+      "id": 21,
+      "options": {"footer": {"fields": "","reducer": ["count"],"show": true},"showHeader": true,"sortBy": [{"desc": false,"displayName": "Prefix"}]},
+      "targets": [
+        {
+          "datasource": {"type": "postgres","uid": "obmp_postgres"},
+          "format": "table",
+          "rawSql": "SELECT DISTINCT ON (r1.prefix, r1.prefix_len)\n  r1.prefix::text AS \"Prefix\",\n  ba.origin_as AS \"Origin AS\",\n  ba.next_hop::text AS \"Next Hop\",\n  ba.as_path::text AS \"AS Path\",\n  p1.peer_addr::text AS \"Learned From\"\nFROM ip_rib r1\nJOIN bgp_peers p1 ON p1.hash_id = r1.peer_hash_id\nJOIN routers rt1 ON rt1.hash_id = p1.router_hash_id\nJOIN base_attrs ba ON ba.hash_id = r1.base_attr_hash_id\nWHERE rt1.name = '$rr1' AND r1.iswithdrawn = false\n  AND NOT EXISTS (\n    SELECT 1 FROM ip_rib r2\n    JOIN bgp_peers p2 ON p2.hash_id = r2.peer_hash_id\n    JOIN routers rt2 ON rt2.hash_id = p2.router_hash_id\n    WHERE rt2.name = '$rr2'\n      AND r2.prefix = r1.prefix AND r2.prefix_len = r1.prefix_len\n      AND r2.iswithdrawn = false\n  )\nORDER BY r1.prefix, r1.prefix_len, ba.local_pref DESC NULLS LAST",
+          "refId": "A"
+        }
+      ],
+      "title": "Prefixes Only on $rr1",
+      "type": "table"
+    },
+    {
+      "datasource": {"type": "postgres","uid": "obmp_postgres"},
+      "description": "Prefixes present on RR2 but NOT on RR1. Expected: RR1's own locally-originated routes won't appear here. Unexpected entries indicate a convergence or session issue.",
+      "fieldConfig": {
+        "defaults": {"color": {"mode": "thresholds"},"custom": {"align": "auto","displayMode": "auto","filterable": true},"thresholds": {"mode": "absolute","steps": [{"color": "green","value": null}]}},
+        "overrides": [
+          {"matcher": {"id": "byName","options": "Learned From"},"properties": [{"id": "custom.width","value": 130}]},
+          {"matcher": {"id": "byName","options": "Origin AS"},"properties": [{"id": "custom.width","value": 90}]},
+          {"matcher": {"id": "byName","options": "Next Hop"},"properties": [{"id": "custom.width","value": 140}]}
+        ]
+      },
+      "gridPos": {"h": 10,"w": 12,"x": 12,"y": 10},
+      "id": 22,
+      "options": {"footer": {"fields": "","reducer": ["count"],"show": true},"showHeader": true,"sortBy": [{"desc": false,"displayName": "Prefix"}]},
+      "targets": [
+        {
+          "datasource": {"type": "postgres","uid": "obmp_postgres"},
+          "format": "table",
+          "rawSql": "SELECT DISTINCT ON (r2.prefix, r2.prefix_len)\n  r2.prefix::text AS \"Prefix\",\n  ba.origin_as AS \"Origin AS\",\n  ba.next_hop::text AS \"Next Hop\",\n  ba.as_path::text AS \"AS Path\",\n  p2.peer_addr::text AS \"Learned From\"\nFROM ip_rib r2\nJOIN bgp_peers p2 ON p2.hash_id = r2.peer_hash_id\nJOIN routers rt2 ON rt2.hash_id = p2.router_hash_id\nJOIN base_attrs ba ON ba.hash_id = r2.base_attr_hash_id\nWHERE rt2.name = '$rr2' AND r2.iswithdrawn = false\n  AND NOT EXISTS (\n    SELECT 1 FROM ip_rib r1\n    JOIN bgp_peers p1 ON p1.hash_id = r1.peer_hash_id\n    JOIN routers rt1 ON rt1.hash_id = p1.router_hash_id\n    WHERE rt1.name = '$rr1'\n      AND r1.prefix = r2.prefix AND r1.prefix_len = r2.prefix_len\n      AND r1.iswithdrawn = false\n  )\nORDER BY r2.prefix, r2.prefix_len, ba.local_pref DESC NULLS LAST",
+          "refId": "A"
+        }
+      ],
+      "title": "Prefixes Only on $rr2",
+      "type": "table"
+    },
+    {
+      "collapsed": false,
+      "gridPos": {"h": 1,"w": 24,"x": 0,"y": 20},
+      "id": 30,
+      "title": "Attribute Differences (Same Prefix, Different Best Path)",
+      "type": "row"
+    },
+    {
+      "datasource": {"type": "postgres","uid": "obmp_postgres"},
+      "description": "Prefixes present on both RRs but where the selected best path differs in next-hop, AS path, local-pref, or MED. This is normal for multi-homed link subnets where each RR selects a different best path based on router-id tiebreaker. Filter by address family using the AFI dropdown.",
+      "fieldConfig": {
+        "defaults": {"color": {"mode": "thresholds"},"custom": {"align": "auto","displayMode": "auto","filterable": true},"thresholds": {"mode": "absolute","steps": [{"color": "green","value": null}]}},
+        "overrides": [
+          {"matcher": {"id": "byName","options": "RR1 Next Hop"},"properties": [{"id": "custom.displayMode","value": "color-text"},{"id": "color","value": {"mode": "fixed","fixedColor": "blue"}}]},
+          {"matcher": {"id": "byName","options": "RR2 Next Hop"},"properties": [{"id": "custom.displayMode","value": "color-text"},{"id": "color","value": {"mode": "fixed","fixedColor": "green"}}]},
+          {"matcher": {"id": "byName","options": "Diff Type"},"properties": [{"id": "custom.displayMode","value": "color-background"},{"id": "color","value": {"mode": "thresholds"}},{"id": "thresholds","value": {"mode": "absolute","steps": [{"color": "yellow","value": null}]}}]}
+        ]
+      },
+      "gridPos": {"h": 12,"w": 24,"x": 0,"y": 21},
+      "id": 31,
+      "options": {"footer": {"fields": "","reducer": ["count"],"show": true},"showHeader": true,"sortBy": [{"desc": false,"displayName": "Prefix"}]},
+      "targets": [
+        {
+          "datasource": {"type": "postgres","uid": "obmp_postgres"},
+          "format": "table",
+          "rawSql": "WITH c1 AS (\n  SELECT DISTINCT ON (r.prefix, r.prefix_len)\n    r.prefix, r.prefix_len, r.isipv4,\n    ba.next_hop, ba.as_path, ba.origin_as,\n    COALESCE(ba.local_pref, 0) AS lp, COALESCE(ba.med, 0) AS med,\n    ba.community_list, p.peer_addr AS learned_from\n  FROM ip_rib r\n  JOIN bgp_peers p ON p.hash_id = r.peer_hash_id\n  JOIN routers rt ON rt.hash_id = p.router_hash_id\n  JOIN base_attrs ba ON ba.hash_id = r.base_attr_hash_id\n  WHERE rt.name = '$rr1' AND r.iswithdrawn = false\n  ORDER BY r.prefix, r.prefix_len, ba.local_pref DESC NULLS LAST\n),\nc2 AS (\n  SELECT DISTINCT ON (r.prefix, r.prefix_len)\n    r.prefix, r.prefix_len, r.isipv4,\n    ba.next_hop, ba.as_path, ba.origin_as,\n    COALESCE(ba.local_pref, 0) AS lp, COALESCE(ba.med, 0) AS med,\n    ba.community_list, p.peer_addr AS learned_from\n  FROM ip_rib r\n  JOIN bgp_peers p ON p.hash_id = r.peer_hash_id\n  JOIN routers rt ON rt.hash_id = p.router_hash_id\n  JOIN base_attrs ba ON ba.hash_id = r.base_attr_hash_id\n  WHERE rt.name = '$rr2' AND r.iswithdrawn = false\n  ORDER BY r.prefix, r.prefix_len, ba.local_pref DESC NULLS LAST\n)\nSELECT\n  c1.prefix::text AS \"Prefix\",\n  CASE WHEN c1.isipv4 THEN 'IPv4' ELSE 'IPv6' END AS \"AFI\",\n  c1.next_hop::text AS \"RR1 Next Hop\",\n  c2.next_hop::text AS \"RR2 Next Hop\",\n  c1.as_path::text AS \"RR1 AS Path\",\n  c2.as_path::text AS \"RR2 AS Path\",\n  c1.lp AS \"RR1 LP\",\n  c2.lp AS \"RR2 LP\",\n  c1.med AS \"RR1 MED\",\n  c2.med AS \"RR2 MED\",\n  CASE\n    WHEN c1.next_hop != c2.next_hop AND c1.as_path != c2.as_path THEN 'NH+ASPath'\n    WHEN c1.next_hop != c2.next_hop THEN 'Next-Hop'\n    WHEN c1.as_path != c2.as_path THEN 'AS Path'\n    WHEN c1.lp != c2.lp THEN 'Local-Pref'\n    WHEN c1.med != c2.med THEN 'MED'\n    ELSE 'Other'\n  END AS \"Diff Type\"\nFROM c1 JOIN c2 ON c1.prefix = c2.prefix AND c1.prefix_len = c2.prefix_len\nWHERE (c1.next_hop != c2.next_hop OR c1.as_path != c2.as_path\n   OR c1.lp != c2.lp OR c1.med != c2.med)\n  AND ('$afi' = 'All' OR ('$afi' = 'IPv4' AND c1.isipv4 = true) OR ('$afi' = 'IPv6' AND c1.isipv4 = false))\nORDER BY c1.prefix",
+          "refId": "A"
+        }
+      ],
+      "title": "Attribute Differences — $rr1 vs $rr2",
+      "type": "table"
+    },
+    {
+      "collapsed": false,
+      "gridPos": {"h": 1,"w": 24,"x": 0,"y": 33},
+      "id": 40,
+      "title": "Per-Client Consistency",
+      "type": "row"
+    },
+    {
+      "datasource": {"type": "postgres","uid": "obmp_postgres"},
+      "description": "Route counts each RR client sends to each RR. In a healthy cluster, every client should send the same number of routes to both RRs. Mismatches indicate session issues, policy differences, or BMP reporting gaps.",
+      "fieldConfig": {
+        "defaults": {"color": {"mode": "thresholds"},"custom": {"align": "auto","displayMode": "auto","filterable": true},"thresholds": {"mode": "absolute","steps": [{"color": "green","value": null}]}},
+        "overrides": [
+          {"matcher": {"id": "byName","options": "Delta"},"properties": [{"id": "custom.displayMode","value": "color-background"},{"id": "color","value": {"mode": "thresholds"}},{"id": "thresholds","value": {"mode": "absolute","steps": [{"color": "green","value": null},{"color": "yellow","value": 1},{"color": "red","value": 5}]}}]},
+          {"matcher": {"id": "byName","options": "Status"},"properties": [{"id": "custom.displayMode","value": "color-background"},{"id": "mappings","value": [{"options": {"Match": {"color": "green","index": 0,"text": "Match"},"MISMATCH": {"color": "red","index": 1,"text": "MISMATCH"}},"type": "value"}]}]}
+        ]
+      },
+      "gridPos": {"h": 10,"w": 12,"x": 0,"y": 34},
+      "id": 41,
+      "options": {"footer": {"fields": "","reducer": ["sum"],"show": false},"showHeader": true,"sortBy": [{"desc": true,"displayName": "Delta"}]},
+      "targets": [
+        {
+          "datasource": {"type": "postgres","uid": "obmp_postgres"},
+          "format": "table",
+          "rawSql": "WITH rr1_peers AS (\n  SELECT p.peer_addr, p.hash_id\n  FROM bgp_peers p\n  JOIN routers rt ON rt.hash_id = p.router_hash_id\n  WHERE rt.name = '$rr1'\n),\nrr2_peers AS (\n  SELECT p.peer_addr, p.hash_id\n  FROM bgp_peers p\n  JOIN routers rt ON rt.hash_id = p.router_hash_id\n  WHERE rt.name = '$rr2'\n),\nrr1_counts AS (\n  SELECT pp.peer_addr, COUNT(*) AS cnt\n  FROM rr1_peers pp\n  JOIN ip_rib r ON r.peer_hash_id = pp.hash_id AND r.iswithdrawn = false\n  GROUP BY pp.peer_addr\n),\nrr2_counts AS (\n  SELECT pp.peer_addr, COUNT(*) AS cnt\n  FROM rr2_peers pp\n  JOIN ip_rib r ON r.peer_hash_id = pp.hash_id AND r.iswithdrawn = false\n  GROUP BY pp.peer_addr\n)\nSELECT\n  COALESCE(c1.peer_addr, c2.peer_addr)::text AS \"Client\",\n  COALESCE(c1.cnt, 0) AS \"Routes to RR1\",\n  COALESCE(c2.cnt, 0) AS \"Routes to RR2\",\n  ABS(COALESCE(c1.cnt, 0) - COALESCE(c2.cnt, 0)) AS \"Delta\",\n  CASE WHEN COALESCE(c1.cnt, 0) = COALESCE(c2.cnt, 0) THEN 'Match' ELSE 'MISMATCH' END AS \"Status\"\nFROM rr1_counts c1\nFULL OUTER JOIN rr2_counts c2 ON c1.peer_addr = c2.peer_addr\nWHERE c1.peer_addr IS NOT NULL AND c2.peer_addr IS NOT NULL\nORDER BY ABS(COALESCE(c1.cnt, 0) - COALESCE(c2.cnt, 0)) DESC, COALESCE(c1.peer_addr, c2.peer_addr)",
+          "refId": "A"
+        }
+      ],
+      "title": "Per-Client Route Consistency",
+      "type": "table"
+    },
+    {
+      "datasource": {"type": "postgres","uid": "obmp_postgres"},
+      "description": "For clients with route count mismatches, shows which specific prefixes differ. Select a client from the Client dropdown to drill down.",
+      "fieldConfig": {
+        "defaults": {"color": {"mode": "thresholds"},"custom": {"align": "auto","displayMode": "auto","filterable": true},"thresholds": {"mode": "absolute","steps": [{"color": "green","value": null}]}},
+        "overrides": [
+          {"matcher": {"id": "byName","options": "Present On"},"properties": [{"id": "custom.displayMode","value": "color-text"},{"id": "color","value": {"mode": "thresholds"}},{"id": "thresholds","value": {"mode": "absolute","steps": [{"color": "blue","value": null}]}}]}
+        ]
+      },
+      "gridPos": {"h": 10,"w": 12,"x": 12,"y": 34},
+      "id": 42,
+      "options": {"footer": {"fields": "","reducer": ["count"],"show": true},"showHeader": true,"sortBy": [{"desc": false,"displayName": "Prefix"}]},
+      "targets": [
+        {
+          "datasource": {"type": "postgres","uid": "obmp_postgres"},
+          "format": "table",
+          "rawSql": "WITH rr1_client_routes AS (\n  SELECT r.prefix, r.prefix_len\n  FROM ip_rib r\n  JOIN bgp_peers p ON p.hash_id = r.peer_hash_id\n  JOIN routers rt ON rt.hash_id = p.router_hash_id\n  WHERE rt.name = '$rr1' AND p.peer_addr::text = '$client'\n    AND r.iswithdrawn = false\n),\nrr2_client_routes AS (\n  SELECT r.prefix, r.prefix_len\n  FROM ip_rib r\n  JOIN bgp_peers p ON p.hash_id = r.peer_hash_id\n  JOIN routers rt ON rt.hash_id = p.router_hash_id\n  WHERE rt.name = '$rr2' AND p.peer_addr::text = '$client'\n    AND r.iswithdrawn = false\n)\nSELECT r1.prefix::text AS \"Prefix\", 'RR1 only' AS \"Present On\"\nFROM rr1_client_routes r1\nWHERE NOT EXISTS (\n  SELECT 1 FROM rr2_client_routes r2\n  WHERE r2.prefix = r1.prefix AND r2.prefix_len = r1.prefix_len\n)\nUNION ALL\nSELECT r2.prefix::text AS \"Prefix\", 'RR2 only' AS \"Present On\"\nFROM rr2_client_routes r2\nWHERE NOT EXISTS (\n  SELECT 1 FROM rr1_client_routes r1\n  WHERE r1.prefix = r2.prefix AND r1.prefix_len = r2.prefix_len\n)\nORDER BY \"Prefix\"",
+          "refId": "A"
+        }
+      ],
+      "title": "Client Prefix Diff — $client",
+      "type": "table"
+    },
+    {
+      "collapsed": false,
+      "gridPos": {"h": 1,"w": 24,"x": 0,"y": 44},
+      "id": 50,
+      "title": "Full RIB Comparison (All Paths)",
+      "type": "row"
+    },
+    {
+      "datasource": {"type": "postgres","uid": "obmp_postgres"},
+      "description": "Side-by-side view of all paths for a specific prefix on both RRs. Select a prefix from the Prefix dropdown to drill down.",
+      "fieldConfig": {
+        "defaults": {"color": {"mode": "thresholds"},"custom": {"align": "auto","displayMode": "auto","filterable": true},"thresholds": {"mode": "absolute","steps": [{"color": "green","value": null}]}},
+        "overrides": [
+          {"matcher": {"id": "byName","options": "Router"},"properties": [{"id": "custom.displayMode","value": "color-text"},{"id": "color","value": {"mode": "thresholds"}},{"id": "thresholds","value": {"mode": "absolute","steps": [{"color": "blue","value": null}]}}]}
+        ]
+      },
+      "gridPos": {"h": 10,"w": 24,"x": 0,"y": 45},
+      "id": 51,
+      "options": {"footer": {"fields": "","reducer": ["count"],"show": true},"showHeader": true,"sortBy": [{"desc": false,"displayName": "Router"}]},
+      "targets": [
+        {
+          "datasource": {"type": "postgres","uid": "obmp_postgres"},
+          "format": "table",
+          "rawSql": "SELECT\n  rt.name AS \"Router\",\n  p.peer_addr::text AS \"Learned From\",\n  ba.next_hop::text AS \"Next Hop\",\n  ba.as_path::text AS \"AS Path\",\n  ba.origin_as AS \"Origin AS\",\n  COALESCE(ba.local_pref, 0) AS \"Local Pref\",\n  COALESCE(ba.med, 0) AS \"MED\",\n  ba.community_list::text AS \"Communities\",\n  ba.cluster_list::text AS \"Cluster List\",\n  ba.originator_id::text AS \"Originator ID\",\n  r.labels AS \"Labels\",\n  r.timestamp AS \"Last Update\"\nFROM ip_rib r\nJOIN bgp_peers p ON p.hash_id = r.peer_hash_id\nJOIN routers rt ON rt.hash_id = p.router_hash_id\nJOIN base_attrs ba ON ba.hash_id = r.base_attr_hash_id\nWHERE rt.name IN ('$rr1', '$rr2')\n  AND r.iswithdrawn = false\n  AND r.prefix::text = '$prefix'\nORDER BY rt.name, p.peer_addr",
+          "refId": "A"
+        }
+      ],
+      "title": "All Paths for $prefix",
+      "type": "table"
+    }
+  ],
+  "refresh": "30s",
+  "schemaVersion": 38,
+  "tags": ["openbmp", "rr-diff", "bgp"],
+  "templating": {
+    "list": [
+      {
+        "current": {},
+        "datasource": {"type": "postgres","uid": "obmp_postgres"},
+        "definition": "SELECT name FROM routers WHERE state = 'up' ORDER BY name",
+        "hide": 0,
+        "includeAll": false,
+        "label": "RR1",
+        "multi": false,
+        "name": "rr1",
+        "options": [],
+        "query": "SELECT name FROM routers WHERE state = 'up' ORDER BY name",
+        "refresh": 1,
+        "regex": "",
+        "skipUrlSync": false,
+        "sort": 1,
+        "type": "query"
+      },
+      {
+        "current": {},
+        "datasource": {"type": "postgres","uid": "obmp_postgres"},
+        "definition": "SELECT name FROM routers WHERE state = 'up' ORDER BY name",
+        "hide": 0,
+        "includeAll": false,
+        "label": "RR2",
+        "multi": false,
+        "name": "rr2",
+        "options": [],
+        "query": "SELECT name FROM routers WHERE state = 'up' ORDER BY name",
+        "refresh": 1,
+        "regex": "",
+        "skipUrlSync": false,
+        "sort": 1,
+        "type": "query"
+      },
+      {
+        "current": {"selected": true,"text": "All","value": "All"},
+        "hide": 0,
+        "includeAll": false,
+        "label": "AFI",
+        "multi": false,
+        "name": "afi",
+        "options": [
+          {"selected": true,"text": "All","value": "All"},
+          {"selected": false,"text": "IPv4","value": "IPv4"},
+          {"selected": false,"text": "IPv6","value": "IPv6"}
+        ],
+        "query": "All,IPv4,IPv6",
+        "skipUrlSync": false,
+        "type": "custom"
+      },
+      {
+        "current": {},
+        "datasource": {"type": "postgres","uid": "obmp_postgres"},
+        "definition": "SELECT DISTINCT p.peer_addr::text FROM bgp_peers p JOIN routers rt ON rt.hash_id = p.router_hash_id WHERE rt.name IN ('$rr1', '$rr2') AND p.state = 'up' ORDER BY 1",
+        "hide": 0,
+        "includeAll": false,
+        "label": "Client",
+        "multi": false,
+        "name": "client",
+        "options": [],
+        "query": "SELECT DISTINCT p.peer_addr::text FROM bgp_peers p JOIN routers rt ON rt.hash_id = p.router_hash_id WHERE rt.name IN ('$rr1', '$rr2') AND p.state = 'up' ORDER BY 1",
+        "refresh": 1,
+        "regex": "",
+        "skipUrlSync": false,
+        "sort": 1,
+        "type": "query"
+      },
+      {
+        "current": {},
+        "datasource": {"type": "postgres","uid": "obmp_postgres"},
+        "definition": "SELECT DISTINCT r.prefix::text FROM ip_rib r JOIN bgp_peers p ON p.hash_id = r.peer_hash_id JOIN routers rt ON rt.hash_id = p.router_hash_id WHERE rt.name IN ('$rr1', '$rr2') AND r.iswithdrawn = false ORDER BY 1",
+        "hide": 0,
+        "includeAll": false,
+        "label": "Prefix",
+        "multi": false,
+        "name": "prefix",
+        "options": [],
+        "query": "SELECT DISTINCT r.prefix::text FROM ip_rib r JOIN bgp_peers p ON p.hash_id = r.peer_hash_id JOIN routers rt ON rt.hash_id = p.router_hash_id WHERE rt.name IN ('$rr1', '$rr2') AND r.iswithdrawn = false ORDER BY 1",
+        "refresh": 1,
+        "regex": "",
+        "skipUrlSync": false,
+        "sort": 1,
+        "type": "query"
+      }
+    ]
+  },
+  "time": {"from": "now-6h","to": "now"},
+  "timepicker": {},
+  "timezone": "",
+  "title": "RR Loc-RIB Diff",
+  "uid": "rr-locrib-diff",
+  "version": 1
+}

obmp-grafana/dashboards/Learning/te_sr_analytics.json

@@ -0,0 +1,345 @@
+{
+  "annotations": {"list": [{"builtIn": 1,"datasource": {"type": "datasource","uid": "grafana"},"enable": true,"hide": true,"iconColor": "rgba(0, 211, 255, 1)","name": "Annotations & Alerts","type": "dashboard"}]},
+  "editable": true,
+  "fiscalYearStartMonth": 0,
+  "graphTooltip": 1,
+  "id": null,
+  "links": [],
+  "panels": [
+    {
+      "datasource": {"type": "postgres","uid": "obmp_postgres"},
+      "fieldConfig": {
+        "defaults": {"custom": {"align": "auto","displayMode": "auto"}},
+        "overrides": []
+      },
+      "gridPos": {"h": 10,"w": 24,"x": 0,"y": 0},
+      "id": 1,
+      "options": {"footer": {"fields": "","reducer": ["sum"],"show": false},"showHeader": true},
+      "targets": [
+        {
+          "datasource": {"type": "postgres","uid": "obmp_postgres"},
+          "format": "table",
+          "rawSql": "SELECT local_router_name as \"Local Router\", \n       remote_router_name as \"Remote Router\",\n       igp_metric as \"IGP Metric\",\n       te_def_metric as \"TE Metric\",\n       max_link_bw as \"Max BW (B/s)\",\n       max_resv_bw as \"Max Reservable BW\",\n       unreserved_bw as \"Unreserved BW\",\n       admin_group as \"Admin Group\",\n       protection_type as \"Protection\",\n       srlg as \"SRLG\"\nFROM v_ls_links\nWHERE peer_hash_id = '$peer_hash' AND iswithdrawn = false\nORDER BY local_router_name, remote_router_name",
+          "refId": "A"
+        }
+      ],
+      "title": "TE Link Capacity Map",
+      "type": "table"
+    },
+    {
+      "datasource": {"type": "postgres","uid": "obmp_postgres"},
+      "fieldConfig": {
+        "defaults": {"color": {"mode": "palette-classic"}},
+        "overrides": []
+      },
+      "gridPos": {"h": 10,"w": 12,"x": 0,"y": 10},
+      "id": 2,
+      "options": {
+        "barRadius": 0,
+        "barWidth": 0.97,
+        "groupWidth": 0.7,
+        "legend": {"displayMode": "list","placement": "bottom"},
+        "orientation": "auto",
+        "showValue": "auto",
+        "stacking": "none",
+        "tooltip": {"mode": "single","sort": "none"},
+        "xTickLabelRotation": -45
+      },
+      "targets": [
+        {
+          "datasource": {"type": "postgres","uid": "obmp_postgres"},
+          "format": "table",
+          "rawSql": "SELECT local_router_name || ' -> ' || remote_router_name as \"Link\",\n       igp_metric as \"IGP Metric\",\n       COALESCE(te_def_metric, igp_metric) as \"TE Metric\"\nFROM v_ls_links\nWHERE peer_hash_id = '$peer_hash' AND iswithdrawn = false\nORDER BY igp_metric DESC",
+          "refId": "A"
+        }
+      ],
+      "title": "IGP Metric vs TE Metric Comparison",
+      "type": "barchart"
+    },
+    {
+      "datasource": {"type": "postgres","uid": "obmp_postgres"},
+      "fieldConfig": {
+        "defaults": {"color": {"mode": "palette-classic"}},
+        "overrides": []
+      },
+      "gridPos": {"h": 10,"w": 6,"x": 12,"y": 10},
+      "id": 3,
+      "options": {
+        "legend": {"displayMode": "list","placement": "bottom"},
+        "pieType": "pie",
+        "reduceOptions": {"calcs": ["lastNotNull"],"fields": "","values": true},
+        "tooltip": {"mode": "single","sort": "none"}
+      },
+      "targets": [
+        {
+          "datasource": {"type": "postgres","uid": "obmp_postgres"},
+          "format": "table",
+          "rawSql": "SELECT COALESCE(admin_group::text, 'None') as \"Admin Group\",\n       COUNT(*) as \"Link Count\"\nFROM v_ls_links\nWHERE peer_hash_id = '$peer_hash' AND iswithdrawn = false\nGROUP BY admin_group\nORDER BY \"Link Count\" DESC",
+          "refId": "A"
+        }
+      ],
+      "title": "Admin Group Distribution",
+      "type": "piechart"
+    },
+    {
+      "datasource": {"type": "postgres","uid": "obmp_postgres"},
+      "fieldConfig": {
+        "defaults": {"color": {"mode": "palette-classic"}},
+        "overrides": []
+      },
+      "gridPos": {"h": 10,"w": 6,"x": 18,"y": 10},
+      "id": 4,
+      "options": {
+        "legend": {"displayMode": "list","placement": "bottom"},
+        "pieType": "pie",
+        "reduceOptions": {"calcs": ["lastNotNull"],"fields": "","values": true},
+        "tooltip": {"mode": "single","sort": "none"}
+      },
+      "targets": [
+        {
+          "datasource": {"type": "postgres","uid": "obmp_postgres"},
+          "format": "table",
+          "rawSql": "SELECT COALESCE(protection_type, 'None') as \"Protection Type\",\n       COUNT(*) as \"Link Count\"\nFROM v_ls_links\nWHERE peer_hash_id = '$peer_hash' AND iswithdrawn = false\nGROUP BY protection_type\nORDER BY \"Link Count\" DESC",
+          "refId": "A"
+        }
+      ],
+      "title": "Link Protection Types",
+      "type": "piechart"
+    },
+    {
+      "datasource": {"type": "postgres","uid": "obmp_postgres"},
+      "fieldConfig": {
+        "defaults": {"custom": {"align": "auto","displayMode": "auto"}},
+        "overrides": []
+      },
+      "gridPos": {"h": 8,"w": 12,"x": 0,"y": 20},
+      "id": 5,
+      "options": {"footer": {"fields": "","reducer": ["sum"],"show": false},"showHeader": true},
+      "targets": [
+        {
+          "datasource": {"type": "postgres","uid": "obmp_postgres"},
+          "format": "table",
+          "rawSql": "SELECT nodename as \"Node\",\n       routerid as \"Router ID\",\n       protocol as \"Protocol\",\n       sr_capabilities as \"SR Capabilities (SRGB)\"\nFROM v_ls_nodes\nWHERE peer_hash_id = '$peer_hash' AND iswithdrawn = false\nORDER BY nodename",
+          "refId": "A"
+        }
+      ],
+      "title": "SR Node Capabilities",
+      "type": "table"
+    },
+    {
+      "datasource": {"type": "postgres","uid": "obmp_postgres"},
+      "fieldConfig": {
+        "defaults": {"custom": {"align": "auto","displayMode": "auto"}},
+        "overrides": []
+      },
+      "gridPos": {"h": 8,"w": 12,"x": 12,"y": 20},
+      "id": 6,
+      "options": {"footer": {"fields": "","reducer": ["sum"],"show": false},"showHeader": true},
+      "targets": [
+        {
+          "datasource": {"type": "postgres","uid": "obmp_postgres"},
+          "format": "table",
+          "rawSql": "SELECT n.nodename as \"Node\",\n       p.prefix::text as \"Prefix\",\n       p.prefix_len as \"Len\",\n       p.metric as \"Metric\",\n       p.sr_prefix_sids as \"Prefix SID\",\n       p.protocol::text as \"Protocol\"\nFROM ls_prefixes p\nJOIN ls_nodes n ON n.hash_id = p.local_node_hash_id \n     AND n.peer_hash_id = p.peer_hash_id\nWHERE p.peer_hash_id = '$peer_hash' AND p.iswithdrawn = false\nORDER BY n.nodename, p.prefix",
+          "refId": "A"
+        }
+      ],
+      "title": "SR Prefix SIDs",
+      "type": "table"
+    },
+    {
+      "datasource": {"type": "postgres","uid": "obmp_postgres"},
+      "fieldConfig": {
+        "defaults": {"custom": {"align": "auto","displayMode": "auto"}},
+        "overrides": []
+      },
+      "gridPos": {"h": 8,"w": 12,"x": 0,"y": 28},
+      "id": 7,
+      "options": {"footer": {"fields": "","reducer": ["sum"],"show": false},"showHeader": true},
+      "targets": [
+        {
+          "datasource": {"type": "postgres","uid": "obmp_postgres"},
+          "format": "table",
+          "rawSql": "SELECT local_router_name as \"Local\",\n       remote_router_name as \"Remote\",\n       sr_adjacency_sids as \"Adjacency SIDs\",\n       peer_node_sid as \"Peer Node SID\",\n       mpls_proto_mask::text as \"MPLS Proto\"\nFROM v_ls_links\nWHERE peer_hash_id = '$peer_hash' AND iswithdrawn = false\nORDER BY local_router_name, remote_router_name",
+          "refId": "A"
+        }
+      ],
+      "title": "SR Adjacency SIDs",
+      "type": "table"
+    },
+    {
+      "datasource": {"type": "postgres","uid": "obmp_postgres"},
+      "fieldConfig": {
+        "defaults": {"custom": {"align": "auto","displayMode": "auto"}},
+        "overrides": []
+      },
+      "gridPos": {"h": 8,"w": 12,"x": 12,"y": 28},
+      "id": 8,
+      "options": {"footer": {"fields": "","reducer": ["sum"],"show": false},"showHeader": true},
+      "targets": [
+        {
+          "datasource": {"type": "postgres","uid": "obmp_postgres"},
+          "format": "table",
+          "rawSql": "SELECT srlg as \"SRLG Value\",\n       COUNT(*) as \"Link Count\",\n       string_agg(DISTINCT local_router_name || ' -> ' || remote_router_name, ', ') as \"Links\"\nFROM v_ls_links\nWHERE peer_hash_id = '$peer_hash' AND iswithdrawn = false \n      AND srlg IS NOT NULL AND srlg != ''\nGROUP BY srlg\nORDER BY COUNT(*) DESC",
+          "refId": "A"
+        }
+      ],
+      "title": "SRLG Groups",
+      "type": "table"
+    },
+    {
+      "datasource": {"type": "postgres","uid": "obmp_postgres"},
+      "fieldConfig": {
+        "defaults": {
+          "color": {"mode": "thresholds"},
+          "thresholds": {"mode": "absolute","steps": [{"color": "green","value": null}]}
+        },
+        "overrides": []
+      },
+      "gridPos": {"h": 4,"w": 5,"x": 0,"y": 36},
+      "id": 9,
+      "options": {"colorMode": "value","graphMode": "area","justifyMode": "auto","orientation": "auto","reduceOptions": {"calcs": ["lastNotNull"],"fields": "","values": false},"textMode": "auto"},
+      "targets": [
+        {
+          "datasource": {"type": "postgres","uid": "obmp_postgres"},
+          "format": "table",
+          "rawSql": "SELECT COUNT(*) FROM v_ls_links WHERE peer_hash_id = '$peer_hash' AND iswithdrawn = false AND te_def_metric IS NOT NULL",
+          "refId": "A"
+        }
+      ],
+      "title": "Links with TE Metric",
+      "type": "stat"
+    },
+    {
+      "datasource": {"type": "postgres","uid": "obmp_postgres"},
+      "fieldConfig": {
+        "defaults": {
+          "color": {"mode": "thresholds"},
+          "thresholds": {"mode": "absolute","steps": [{"color": "green","value": null}]}
+        },
+        "overrides": []
+      },
+      "gridPos": {"h": 4,"w": 5,"x": 5,"y": 36},
+      "id": 10,
+      "options": {"colorMode": "value","graphMode": "area","justifyMode": "auto","orientation": "auto","reduceOptions": {"calcs": ["lastNotNull"],"fields": "","values": false},"textMode": "auto"},
+      "targets": [
+        {
+          "datasource": {"type": "postgres","uid": "obmp_postgres"},
+          "format": "table",
+          "rawSql": "SELECT COUNT(*) FROM v_ls_links WHERE peer_hash_id = '$peer_hash' AND iswithdrawn = false AND max_link_bw IS NOT NULL AND max_link_bw > 0",
+          "refId": "A"
+        }
+      ],
+      "title": "Links with Bandwidth",
+      "type": "stat"
+    },
+    {
+      "datasource": {"type": "postgres","uid": "obmp_postgres"},
+      "fieldConfig": {
+        "defaults": {
+          "color": {"mode": "thresholds"},
+          "thresholds": {"mode": "absolute","steps": [{"color": "green","value": null}]}
+        },
+        "overrides": []
+      },
+      "gridPos": {"h": 4,"w": 5,"x": 10,"y": 36},
+      "id": 11,
+      "options": {"colorMode": "value","graphMode": "area","justifyMode": "auto","orientation": "auto","reduceOptions": {"calcs": ["lastNotNull"],"fields": "","values": false},"textMode": "auto"},
+      "targets": [
+        {
+          "datasource": {"type": "postgres","uid": "obmp_postgres"},
+          "format": "table",
+          "rawSql": "SELECT COUNT(*) FROM v_ls_links WHERE peer_hash_id = '$peer_hash' AND iswithdrawn = false AND srlg IS NOT NULL AND srlg != ''",
+          "refId": "A"
+        }
+      ],
+      "title": "Links with SRLG",
+      "type": "stat"
+    },
+    {
+      "datasource": {"type": "postgres","uid": "obmp_postgres"},
+      "fieldConfig": {
+        "defaults": {
+          "color": {"mode": "thresholds"},
+          "thresholds": {"mode": "absolute","steps": [{"color": "green","value": null}]}
+        },
+        "overrides": []
+      },
+      "gridPos": {"h": 4,"w": 5,"x": 15,"y": 36},
+      "id": 12,
+      "options": {"colorMode": "value","graphMode": "area","justifyMode": "auto","orientation": "auto","reduceOptions": {"calcs": ["lastNotNull"],"fields": "","values": false},"textMode": "auto"},
+      "targets": [
+        {
+          "datasource": {"type": "postgres","uid": "obmp_postgres"},
+          "format": "table",
+          "rawSql": "SELECT COUNT(*) FROM v_ls_nodes WHERE peer_hash_id = '$peer_hash' AND iswithdrawn = false AND sr_capabilities IS NOT NULL AND sr_capabilities != ''",
+          "refId": "A"
+        }
+      ],
+      "title": "Nodes with SR",
+      "type": "stat"
+    },
+    {
+      "datasource": {"type": "postgres","uid": "obmp_postgres"},
+      "fieldConfig": {
+        "defaults": {
+          "color": {"mode": "thresholds"},
+          "thresholds": {"mode": "absolute","steps": [{"color": "green","value": null}]}
+        },
+        "overrides": []
+      },
+      "gridPos": {"h": 4,"w": 4,"x": 20,"y": 36},
+      "id": 13,
+      "options": {"colorMode": "value","graphMode": "area","justifyMode": "auto","orientation": "auto","reduceOptions": {"calcs": ["lastNotNull"],"fields": "","values": false},"textMode": "auto"},
+      "targets": [
+        {
+          "datasource": {"type": "postgres","uid": "obmp_postgres"},
+          "format": "table",
+          "rawSql": "SELECT COUNT(*) FROM v_ls_links WHERE peer_hash_id = '$peer_hash' AND iswithdrawn = false AND sr_adjacency_sids IS NOT NULL AND sr_adjacency_sids != ''",
+          "refId": "A"
+        }
+      ],
+      "title": "Links with Adj SID",
+      "type": "stat"
+    },
+    {
+      "gridPos": {"h": 10,"w": 24,"x": 0,"y": 40},
+      "id": 14,
+      "options": {
+        "code": {"language": "plaintext","showLineNumbers": false,"showMiniMap": false},
+        "content": "## Traffic Engineering & Segment Routing Analytics\n\nThis dashboard exposes TE and SR attributes from BGP-LS (RFC 7752) that OpenBMP collects but existing dashboards don't display.\n\n### TE Fields (from ls_links)\n- **admin_group**: Link color/affinity bitmap for RSVP-TE constraints\n- **max_link_bw / max_resv_bw**: Link capacity in bytes/sec\n- **unreserved_bw**: Available bandwidth per priority level\n- **te_def_metric**: TE metric (may differ from IGP metric)\n- **protection_type**: FRR protection (unprotected, shared, dedicated, etc.)\n- **srlg**: Shared Risk Link Group for diverse path computation\n\n### SR Fields\n- **sr_capabilities**: Node SRGB (Segment Routing Global Block) range\n- **sr_prefix_sids**: Prefix SID for SR-MPLS forwarding\n- **sr_adjacency_sids**: Adjacency SIDs for SR-TE path steering\n- **peer_node_sid**: BGP EPE SID (RFC 9086)\n\n### Notes\n- NULL values indicate the router is not advertising that TLV\n- To enable TE metrics on IOS-XR: `mpls traffic-eng` under IS-IS\n- To enable SR: `segment-routing mpls` under IS-IS with prefix-sid-map",
+        "mode": "markdown"
+      },
+      "title": "About This Dashboard",
+      "type": "text"
+    }
+  ],
+  "schemaVersion": 39,
+  "tags": ["obmp-learning"],
+  "templating": {
+    "list": [
+      {
+        "current": {},
+        "datasource": {"type": "postgres","uid": "obmp_postgres"},
+        "definition": "SELECT __text,__value FROM (\n    select peername as __text, peer_hash_id as __value, count(*) as count\n    from v_ls_nodes\n   group by peername,peer_hash_id) d\nwhere count > 0",
+        "hide": 0,
+        "includeAll": false,
+        "label": "BGP Peer",
+        "multi": false,
+        "name": "peer_hash",
+        "options": [],
+        "query": "SELECT __text,__value FROM (\n    select peername as __text, peer_hash_id as __value, count(*) as count\n    from v_ls_nodes\n   group by peername,peer_hash_id) d\nwhere count > 0",
+        "refresh": 1,
+        "regex": "",
+        "skipUrlSync": false,
+        "sort": 0,
+        "type": "query"
+      }
+    ]
+  },
+  "time": {"from": "now-6h","to": "now"},
+  "timepicker": {},
+  "timezone": "",
+  "title": "TE & Segment Routing Analytics",
+  "uid": "obmp-learn-08",
+  "version": 1
+}

obmp-grafana/dashboards/Learning/topology_anomaly.json

@@ -0,0 +1,235 @@
+{
+  "uid": "obmp-learn-09",
+  "title": "Topology Change & Anomaly Detection",
+  "tags": ["obmp-learning"],
+  "editable": true,
+  "schemaVersion": 39,
+  "time": {
+    "from": "now-6h",
+    "to": "now"
+  },
+  "templating": {
+    "list": [
+      {
+        "name": "peer_hash",
+        "label": "BGP Peer",
+        "type": "query",
+        "datasource": {
+          "type": "postgres",
+          "uid": "obmp_postgres"
+        },
+        "query": "SELECT __text,__value FROM (\n    select peername as __text, peer_hash_id as __value, count(*) as count\n    from v_ls_nodes\n   group by peername,peer_hash_id) d\nwhere count > 0",
+        "refresh": 1,
+        "multi": false
+      }
+    ]
+  },
+  "panels": [
+    {
+      "id": 1,
+      "title": "Link State Changes Over Time",
+      "type": "timeseries",
+      "gridPos": { "h": 8, "w": 12, "x": 0, "y": 0 },
+      "datasource": { "type": "postgres", "uid": "obmp_postgres" },
+      "targets": [
+        {
+          "rawSql": "SELECT $__timeGroupAlias(timestamp, '5m') as time,\n       SUM(CASE WHEN iswithdrawn = false THEN 1 ELSE 0 END) as \"Links Up\",\n       SUM(CASE WHEN iswithdrawn = true THEN 1 ELSE 0 END) as \"Links Down\"\nFROM ls_links_log\nWHERE $__timeFilter(timestamp) AND peer_hash_id = '$peer_hash'\nGROUP BY 1 ORDER BY 1",
+          "format": "time_series",
+          "refId": "A"
+        }
+      ]
+    },
+    {
+      "id": 2,
+      "title": "Node Changes Over Time",
+      "type": "timeseries",
+      "gridPos": { "h": 8, "w": 12, "x": 12, "y": 0 },
+      "datasource": { "type": "postgres", "uid": "obmp_postgres" },
+      "targets": [
+        {
+          "rawSql": "SELECT $__timeGroupAlias(timestamp, '5m') as time,\n       SUM(CASE WHEN iswithdrawn = false THEN 1 ELSE 0 END) as \"Nodes Appeared\",\n       SUM(CASE WHEN iswithdrawn = true THEN 1 ELSE 0 END) as \"Nodes Withdrawn\"\nFROM ls_nodes_log\nWHERE $__timeFilter(timestamp) AND peer_hash_id = '$peer_hash'\nGROUP BY 1 ORDER BY 1",
+          "format": "time_series",
+          "refId": "A"
+        }
+      ]
+    },
+    {
+      "id": 3,
+      "title": "BGP Peer Session Events",
+      "type": "timeseries",
+      "gridPos": { "h": 8, "w": 12, "x": 0, "y": 8 },
+      "datasource": { "type": "postgres", "uid": "obmp_postgres" },
+      "targets": [
+        {
+          "rawSql": "SELECT $__timeGroupAlias(pel.timestamp, '5m') as time,\n       SUM(CASE WHEN pel.state = 'up' THEN 1 ELSE 0 END) as \"Sessions Up\",\n       SUM(CASE WHEN pel.state = 'down' THEN 1 ELSE 0 END) as \"Sessions Down\"\nFROM peer_event_log pel\nWHERE $__timeFilter(pel.timestamp)\nGROUP BY 1 ORDER BY 1",
+          "format": "time_series",
+          "refId": "A"
+        }
+      ]
+    },
+    {
+      "id": 4,
+      "title": "RIB Update Rate",
+      "type": "timeseries",
+      "gridPos": { "h": 8, "w": 12, "x": 12, "y": 8 },
+      "datasource": { "type": "postgres", "uid": "obmp_postgres" },
+      "targets": [
+        {
+          "rawSql": "SELECT $__timeGroupAlias(timestamp, '5m') as time,\n       SUM(CASE WHEN iswithdrawn = false THEN 1 ELSE 0 END) as \"Advertisements\",\n       SUM(CASE WHEN iswithdrawn = true THEN 1 ELSE 0 END) as \"Withdrawals\"\nFROM ip_rib_log\nWHERE $__timeFilter(timestamp)\nGROUP BY 1 ORDER BY 1",
+          "format": "time_series",
+          "refId": "A"
+        }
+      ]
+    },
+    {
+      "id": 5,
+      "title": "Origin AS Changes (Potential Hijacks)",
+      "type": "table",
+      "gridPos": { "h": 10, "w": 12, "x": 0, "y": 16 },
+      "datasource": { "type": "postgres", "uid": "obmp_postgres" },
+      "targets": [
+        {
+          "rawSql": "SELECT DISTINCT ON (r1.prefix, r1.prefix_len)\n       r1.prefix::text as \"Prefix\",\n       r1.prefix_len as \"Len\",\n       r1.origin_as as \"Current Origin AS\",\n       r2.origin_as as \"Previous Origin AS\",\n       r1.timestamp as \"Changed At\"\nFROM ip_rib_log r1\nJOIN ip_rib_log r2 ON r1.prefix = r2.prefix \n     AND r1.prefix_len = r2.prefix_len\n     AND r1.timestamp > r2.timestamp\nWHERE r1.origin_as != r2.origin_as\n      AND $__timeFilter(r1.timestamp)\nORDER BY r1.prefix, r1.prefix_len, r1.timestamp DESC\nLIMIT 50",
+          "format": "table",
+          "refId": "A"
+        }
+      ]
+    },
+    {
+      "id": 6,
+      "title": "Most Churned Prefixes",
+      "type": "table",
+      "gridPos": { "h": 10, "w": 12, "x": 12, "y": 16 },
+      "datasource": { "type": "postgres", "uid": "obmp_postgres" },
+      "targets": [
+        {
+          "rawSql": "SELECT prefix::text as \"Prefix\",\n       prefix_len as \"Len\",\n       COUNT(*) as \"Total Updates\",\n       SUM(CASE WHEN iswithdrawn THEN 1 ELSE 0 END) as \"Withdrawals\",\n       MIN(timestamp) as \"First Seen\",\n       MAX(timestamp) as \"Last Change\",\n       CASE \n         WHEN COUNT(*) <= 2 THEN 'Stable'\n         WHEN COUNT(*) <= 10 THEN 'Moderate'\n         ELSE 'Unstable'\n       END as \"Stability\"\nFROM ip_rib_log\nWHERE $__timeFilter(timestamp)\nGROUP BY prefix, prefix_len\nHAVING COUNT(*) > 1\nORDER BY COUNT(*) DESC\nLIMIT 30",
+          "format": "table",
+          "refId": "A"
+        }
+      ]
+    },
+    {
+      "id": 7,
+      "title": "Recent Link State Changes",
+      "type": "table",
+      "gridPos": { "h": 10, "w": 24, "x": 0, "y": 26 },
+      "datasource": { "type": "postgres", "uid": "obmp_postgres" },
+      "targets": [
+        {
+          "rawSql": "SELECT l.timestamp as \"Time\",\n       CASE WHEN l.iswithdrawn THEN 'DOWN' ELSE 'UP' END as \"State\",\n       ln.name as \"Local Node\",\n       l.local_igp_router_id as \"Local IGP ID\",\n       rn.name as \"Remote Node\",\n       l.remote_igp_router_id as \"Remote IGP ID\",\n       l.igp_metric as \"IGP Metric\",\n       l.protocol::text as \"Protocol\"\nFROM ls_links_log l\nLEFT JOIN ls_nodes ln ON ln.hash_id = l.local_node_hash_id AND ln.peer_hash_id = l.peer_hash_id\nLEFT JOIN ls_nodes rn ON rn.hash_id = l.remote_node_hash_id AND rn.peer_hash_id = l.peer_hash_id\nWHERE $__timeFilter(l.timestamp) AND l.peer_hash_id = '$peer_hash'\nORDER BY l.timestamp DESC\nLIMIT 50",
+          "format": "table",
+          "refId": "A"
+        }
+      ]
+    },
+    {
+      "id": 8,
+      "title": "Multi-Peer Route Consistency",
+      "type": "table",
+      "gridPos": { "h": 8, "w": 12, "x": 0, "y": 36 },
+      "datasource": { "type": "postgres", "uid": "obmp_postgres" },
+      "targets": [
+        {
+          "rawSql": "SELECT r.prefix::text as \"Prefix\",\n       r.prefix_len as \"Len\",\n       COUNT(DISTINCT r.peer_hash_id) as \"Peer Count\",\n       COUNT(DISTINCT ba.origin_as) as \"Distinct Origins\",\n       COUNT(DISTINCT ba.as_path_count) as \"Distinct Path Lengths\",\n       string_agg(DISTINCT ba.origin_as::text, ', ') as \"Origin ASNs\"\nFROM ip_rib r\nJOIN base_attrs ba ON ba.hash_id = r.base_attr_hash_id\nWHERE r.iswithdrawn = false AND r.isipv4 = true\nGROUP BY r.prefix, r.prefix_len\nHAVING COUNT(DISTINCT ba.origin_as) > 1\nORDER BY COUNT(DISTINCT ba.origin_as) DESC\nLIMIT 30",
+          "format": "table",
+          "refId": "A"
+        }
+      ]
+    },
+    {
+      "id": 9,
+      "title": "Active Peers",
+      "type": "stat",
+      "gridPos": { "h": 4, "w": 4, "x": 0, "y": 44 },
+      "datasource": { "type": "postgres", "uid": "obmp_postgres" },
+      "targets": [
+        {
+          "rawSql": "SELECT COUNT(*) FROM bgp_peers WHERE state = 'up'",
+          "format": "table",
+          "refId": "A"
+        }
+      ]
+    },
+    {
+      "id": 10,
+      "title": "Total LS Links",
+      "type": "stat",
+      "gridPos": { "h": 4, "w": 4, "x": 4, "y": 44 },
+      "datasource": { "type": "postgres", "uid": "obmp_postgres" },
+      "targets": [
+        {
+          "rawSql": "SELECT COUNT(*) FROM ls_links WHERE peer_hash_id = '$peer_hash' AND iswithdrawn = false",
+          "format": "table",
+          "refId": "A"
+        }
+      ]
+    },
+    {
+      "id": 11,
+      "title": "Total LS Nodes",
+      "type": "stat",
+      "gridPos": { "h": 4, "w": 4, "x": 8, "y": 44 },
+      "datasource": { "type": "postgres", "uid": "obmp_postgres" },
+      "targets": [
+        {
+          "rawSql": "SELECT COUNT(*) FROM ls_nodes WHERE peer_hash_id = '$peer_hash' AND iswithdrawn = false",
+          "format": "table",
+          "refId": "A"
+        }
+      ]
+    },
+    {
+      "id": 12,
+      "title": "RIB Updates (24h)",
+      "type": "stat",
+      "gridPos": { "h": 4, "w": 4, "x": 12, "y": 44 },
+      "datasource": { "type": "postgres", "uid": "obmp_postgres" },
+      "targets": [
+        {
+          "rawSql": "SELECT COUNT(*) FROM ip_rib_log WHERE timestamp > NOW() - INTERVAL '24 hours'",
+          "format": "table",
+          "refId": "A"
+        }
+      ]
+    },
+    {
+      "id": 13,
+      "title": "Link Changes (24h)",
+      "type": "stat",
+      "gridPos": { "h": 4, "w": 4, "x": 16, "y": 44 },
+      "datasource": { "type": "postgres", "uid": "obmp_postgres" },
+      "targets": [
+        {
+          "rawSql": "SELECT COUNT(*) FROM ls_links_log WHERE timestamp > NOW() - INTERVAL '24 hours' AND peer_hash_id = '$peer_hash'",
+          "format": "table",
+          "refId": "A"
+        }
+      ]
+    },
+    {
+      "id": 14,
+      "title": "Origin Changes (24h)",
+      "type": "stat",
+      "gridPos": { "h": 4, "w": 4, "x": 20, "y": 44 },
+      "datasource": { "type": "postgres", "uid": "obmp_postgres" },
+      "targets": [
+        {
+          "rawSql": "SELECT COUNT(DISTINCT r1.prefix) FROM ip_rib_log r1\nJOIN ip_rib_log r2 ON r1.prefix = r2.prefix AND r1.prefix_len = r2.prefix_len AND r1.timestamp > r2.timestamp\nWHERE r1.origin_as != r2.origin_as AND r1.timestamp > NOW() - INTERVAL '24 hours'",
+          "format": "table",
+          "refId": "A"
+        }
+      ]
+    },
+    {
+      "id": 15,
+      "title": "About This Dashboard",
+      "type": "text",
+      "gridPos": { "h": 8, "w": 12, "x": 12, "y": 36 },
+      "options": {
+        "mode": "markdown",
+        "content": "## Topology Change & Anomaly Detection\n\nThis dashboard provides heuristic analysis of BMP data to detect network anomalies:\n\n### What to Watch For\n- **Link flaps**: Rapid up/down cycles in the Link State Changes panel indicate instability\n- **Origin AS changes**: Could indicate a route hijack or legitimate migration\n- **Multi-origin prefixes**: Same prefix seen from different origin ASNs across peers\n- **Correlated events**: Peer session drops followed by mass withdrawals indicate convergence events\n\n### Testing with ExaBGP Scenarios\n1. Load `origin_shift` scenario to simulate origin AS changes\n2. Load `hijack_simulation` to see how shorter paths override legitimate routes\n3. Load/unload `churn` scenario repeatedly to generate instability patterns\n\n### Data Sources\n- **ls_links_log / ls_nodes_log**: TimescaleDB hypertables tracking all BGP-LS topology changes\n- **ip_rib_log**: All BGP RIB updates and withdrawals with timestamps\n- **peer_event_log**: BGP session state changes (up/down events)"
+      }
+    }
+  ]
+}

obmp-grafana/dashboards/Telemetry-3001/combined_bmp_telemetry.json

@@ -0,0 +1,132 @@
+{
+  "annotations": {"list": [{"builtIn": 1,"datasource": {"type": "datasource","uid": "grafana"},"enable": true,"hide": true,"iconColor": "rgba(0, 211, 255, 1)","name": "Annotations & Alerts","type": "dashboard"}]},
+  "description": "Combined view of BMP control-plane data (from PostgreSQL) and gNMI data-plane telemetry (from InfluxDB). Correlate BGP peer state with interface traffic patterns.",
+  "editable": true,
+  "fiscalYearStartMonth": 0,
+  "graphTooltip": 1,
+  "id": null,
+  "links": [],
+  "templating": {
+    "list": [
+      {
+        "current": {},
+        "datasource": {"type": "influxdb","uid": "obmp_influxdb"},
+        "definition": "from(bucket: \"telemetry\")\n  |> range(start: -1h)\n  |> filter(fn: (r) => r._measurement == \"interface_counters\")\n  |> keep(columns: [\"source\"])\n  |> distinct(column: \"source\")\n  |> sort()",
+        "hide": 0,
+        "includeAll": true,
+        "label": "Router",
+        "multi": true,
+        "name": "router",
+        "options": [],
+        "query": "from(bucket: \"telemetry\")\n  |> range(start: -1h)\n  |> filter(fn: (r) => r._measurement == \"interface_counters\")\n  |> keep(columns: [\"source\"])\n  |> distinct(column: \"source\")\n  |> sort()",
+        "refresh": 2,
+        "regex": "",
+        "type": "query"
+      }
+    ]
+  },
+  "panels": [
+    {
+      "datasource": {"type": "postgres","uid": "obmp_postgres"},
+      "description": "Current BGP peer status from the OpenBMP PostgreSQL database. Shows peer address, name, and session state.",
+      "fieldConfig": {
+        "defaults": {
+          "color": {"mode": "thresholds"},
+          "custom": {"align": "auto","displayMode": "auto","filterable": true,"inspect": true},
+          "mappings": [],
+          "thresholds": {"mode": "absolute","steps": [{"color": "green","value": null}]}
+        },
+        "overrides": [
+          {"matcher": {"id": "byName","options": "state"},"properties": [{"id": "custom.displayMode","value": "color-background-solid"},{"id": "mappings","value": [{"options": {"down": {"color": "red","index": 1,"text": "DOWN"},"up": {"color": "green","index": 0,"text": "UP"}},"type": "value"}]}]},
+          {"matcher": {"id": "byName","options": "peer_addr"},"properties": [{"id": "custom.width","value": 160}]},
+          {"matcher": {"id": "byName","options": "name"},"properties": [{"id": "custom.width","value": 200}]}
+        ]
+      },
+      "gridPos": {"h": 10,"w": 24,"x": 0,"y": 0},
+      "id": 1,
+      "options": {"footer": {"fields": "","reducer": ["sum"],"show": false},"showHeader": true,"sortBy": [{"desc": false,"displayName": "state"}]},
+      "targets": [
+        {
+          "datasource": {"type": "postgres","uid": "obmp_postgres"},
+          "format": "table",
+          "rawSql": "SELECT\n  p.peer_addr,\n  COALESCE(p.name, p.peer_addr::text) AS name,\n  p.state,\n  p.peer_as AS \"AS\",\n  p.router_hash_id IS NOT NULL AS \"BMP Active\",\n  p.timestamp AS \"Last State Change\"\nFROM bgp_peers p\nWHERE p.isprepolicy = true\nORDER BY p.state, p.peer_addr",
+          "refId": "A"
+        }
+      ],
+      "title": "BGP Peer Status",
+      "type": "table"
+    },
+    {
+      "datasource": {"type": "influxdb","uid": "obmp_influxdb"},
+      "description": "Interface traffic rates from gNMI streaming telemetry. Shows bytes per second for each interface across selected routers.",
+      "fieldConfig": {
+        "defaults": {
+          "color": {"mode": "palette-classic"},
+          "custom": {"axisBorderShow": false,"axisCenteredZero": false,"axisLabel": "","axisPlacement": "auto","barAlignment": 0,"drawStyle": "line","fillOpacity": 10,"gradientMode": "none","hideFrom": {"legend": false,"tooltip": false,"viz": false},"lineInterpolation": "linear","lineWidth": 1,"pointSize": 5,"scaleDistribution": {"type": "linear"},"showPoints": "never","spanNulls": false,"stacking": {"group": "A","mode": "none"},"thresholdsStyle": {"mode": "off"}},
+          "mappings": [],
+          "thresholds": {"mode": "absolute","steps": [{"color": "green","value": null},{"color": "red","value": 80}]},
+          "unit": "Bps"
+        }
+      },
+      "gridPos": {"h": 10,"w": 24,"x": 0,"y": 10},
+      "id": 2,
+      "options": {"legend": {"calcs": ["mean","max"],"displayMode": "table","placement": "bottom"},"tooltip": {"mode": "multi","sort": "desc"}},
+      "targets": [
+        {
+          "datasource": {"type": "influxdb","uid": "obmp_influxdb"},
+          "query": "from(bucket: \"telemetry\")\n  |> range(start: v.timeRangeStart, stop: v.timeRangeStop)\n  |> filter(fn: (r) => r._measurement == \"interface_counters\")\n  |> filter(fn: (r) => r.source =~ /${router:regex}/)\n  |> filter(fn: (r) => r._field == \"in-octets\" or r._field == \"out-octets\")\n  |> toFloat()\n  |> derivative(unit: 1s, nonNegative: true)\n  |> map(fn: (r) => ({r with _value: if r._value < 0.0 then 0.0 else r._value}))",
+          "refId": "A"
+        }
+      ],
+      "title": "Interface Traffic",
+      "type": "timeseries"
+    },
+    {
+      "datasource": {"type": "postgres","uid": "obmp_postgres"},
+      "description": "BGP update activity over time from the OpenBMP PostgreSQL database. Shows peer event transitions and update counts for correlation with traffic patterns.",
+      "fieldConfig": {
+        "defaults": {
+          "color": {"mode": "palette-classic"},
+          "custom": {"axisBorderShow": false,"axisCenteredZero": false,"axisLabel": "","axisPlacement": "auto","barAlignment": 0,"drawStyle": "bars","fillOpacity": 50,"gradientMode": "none","hideFrom": {"legend": false,"tooltip": false,"viz": false},"lineInterpolation": "linear","lineWidth": 1,"pointSize": 5,"scaleDistribution": {"type": "linear"},"showPoints": "never","spanNulls": false,"stacking": {"group": "A","mode": "normal"},"thresholdsStyle": {"mode": "off"}},
+          "mappings": [],
+          "thresholds": {"mode": "absolute","steps": [{"color": "green","value": null}]},
+          "unit": "short"
+        }
+      },
+      "gridPos": {"h": 10,"w": 24,"x": 0,"y": 20},
+      "id": 3,
+      "options": {"legend": {"calcs": ["sum"],"displayMode": "table","placement": "bottom"},"tooltip": {"mode": "multi","sort": "desc"}},
+      "targets": [
+        {
+          "datasource": {"type": "postgres","uid": "obmp_postgres"},
+          "format": "time_series",
+          "rawSql": "SELECT\n  $__timeGroupAlias(e.timestamp, '1m'),\n  COALESCE(p.name, p.peer_addr::text) AS metric,\n  COUNT(*) AS \"value\"\nFROM peer_event_log e\nJOIN bgp_peers p ON p.hash_id = e.peer_hash_id\nWHERE $__timeFilter(e.timestamp)\nGROUP BY 1, 2\nORDER BY 1",
+          "refId": "A"
+        }
+      ],
+      "title": "BGP Update Activity",
+      "type": "timeseries"
+    },
+    {
+      "datasource": {"type": "datasource","uid": "grafana"},
+      "gridPos": {"h": 6,"w": 24,"x": 0,"y": 30},
+      "id": 4,
+      "options": {
+        "code": {"language": "plaintext","showLineNumbers": false,"showMiniMap": false},
+        "content": "## Combined BMP + Telemetry View\n\nThis dashboard integrates two complementary data sources to provide a unified network monitoring view:\n\n### Control Plane (BMP via PostgreSQL)\n- **BGP Peer Status** -- Real-time BGP session state from BMP (OpenBMP)\n- **BGP Update Activity** -- Session transitions and update events from `peer_event_log`\n\n### Data Plane (gNMI via InfluxDB)\n- **Interface Traffic** -- Streaming telemetry byte rates collected via gNMI at 10-second intervals\n\n### Correlation Use Cases\n- A BGP peer flap (control plane) should correlate with a traffic shift on affected interfaces (data plane)\n- Sustained high interface utilization (data plane) may precede BGP session resets due to congestion\n- Compare the number of active BGP peers with interface traffic to validate routing convergence",
+        "mode": "markdown"
+      },
+      "title": "About",
+      "type": "text"
+    }
+  ],
+  "schemaVersion": 39,
+  "style": "dark",
+  "tags": ["obmp-telemetry"],
+  "time": {"from": "now-1h","to": "now"},
+  "timepicker": {},
+  "timezone": "browser",
+  "title": "Combined BMP + Telemetry View",
+  "uid": "obmp-telem-03",
+  "version": 1
+}

obmp-grafana/dashboards/Telemetry-3001/interface_errors.json

@@ -0,0 +1,134 @@
+{
+  "annotations": {"list": [{"builtIn": 1,"datasource": {"type": "datasource","uid": "grafana"},"enable": true,"hide": true,"iconColor": "rgba(0, 211, 255, 1)","name": "Annotations & Alerts","type": "dashboard"}]},
+  "description": "Interface error and drop counters collected via gNMI streaming telemetry. Helps identify interfaces with packet loss or physical layer issues.",
+  "editable": true,
+  "fiscalYearStartMonth": 0,
+  "graphTooltip": 1,
+  "id": null,
+  "links": [],
+  "templating": {
+    "list": [
+      {
+        "current": {},
+        "datasource": {"type": "influxdb","uid": "obmp_influxdb"},
+        "definition": "from(bucket: \"telemetry\")\n  |> range(start: -1h)\n  |> filter(fn: (r) => r._measurement == \"interface_counters\")\n  |> keep(columns: [\"source\"])\n  |> distinct(column: \"source\")\n  |> sort()",
+        "hide": 0,
+        "includeAll": true,
+        "label": "Router",
+        "multi": true,
+        "name": "router",
+        "options": [],
+        "query": "from(bucket: \"telemetry\")\n  |> range(start: -1h)\n  |> filter(fn: (r) => r._measurement == \"interface_counters\")\n  |> keep(columns: [\"source\"])\n  |> distinct(column: \"source\")\n  |> sort()",
+        "refresh": 2,
+        "regex": "",
+        "type": "query"
+      },
+      {
+        "current": {},
+        "datasource": {"type": "influxdb","uid": "obmp_influxdb"},
+        "definition": "from(bucket: \"telemetry\")\n  |> range(start: -1h)\n  |> filter(fn: (r) => r._measurement == \"interface_counters\")\n  |> filter(fn: (r) => r.source =~ /${router:regex}/)\n  |> keep(columns: [\"name\"])\n  |> distinct(column: \"name\")\n  |> sort()",
+        "hide": 0,
+        "includeAll": true,
+        "label": "Interface",
+        "multi": true,
+        "name": "interface",
+        "options": [],
+        "query": "from(bucket: \"telemetry\")\n  |> range(start: -1h)\n  |> filter(fn: (r) => r._measurement == \"interface_counters\")\n  |> filter(fn: (r) => r.source =~ /${router:regex}/)\n  |> keep(columns: [\"name\"])\n  |> distinct(column: \"name\")\n  |> sort()",
+        "refresh": 2,
+        "regex": "",
+        "type": "query"
+      }
+    ]
+  },
+  "panels": [
+    {
+      "datasource": {"type": "influxdb","uid": "obmp_influxdb"},
+      "description": "Interface error counters over time: input errors, output errors, and CRC errors. A rising trend indicates physical or configuration issues.",
+      "fieldConfig": {
+        "defaults": {
+          "color": {"mode": "palette-classic"},
+          "custom": {"axisBorderShow": false,"axisCenteredZero": false,"axisLabel": "","axisPlacement": "auto","barAlignment": 0,"drawStyle": "line","fillOpacity": 10,"gradientMode": "none","hideFrom": {"legend": false,"tooltip": false,"viz": false},"lineInterpolation": "linear","lineWidth": 1,"pointSize": 5,"scaleDistribution": {"type": "linear"},"showPoints": "never","spanNulls": false,"stacking": {"group": "A","mode": "none"},"thresholdsStyle": {"mode": "off"}},
+          "mappings": [],
+          "thresholds": {"mode": "absolute","steps": [{"color": "green","value": null},{"color": "yellow","value": 1},{"color": "red","value": 100}]},
+          "unit": "short"
+        }
+      },
+      "gridPos": {"h": 10,"w": 24,"x": 0,"y": 0},
+      "id": 1,
+      "options": {"legend": {"calcs": ["mean","max","last"],"displayMode": "table","placement": "bottom"},"tooltip": {"mode": "multi","sort": "desc"}},
+      "targets": [
+        {
+          "datasource": {"type": "influxdb","uid": "obmp_influxdb"},
+          "query": "from(bucket: \"telemetry\")\n  |> range(start: v.timeRangeStart, stop: v.timeRangeStop)\n  |> filter(fn: (r) => r._measurement == \"interface_counters\")\n  |> filter(fn: (r) => r.source =~ /${router:regex}/)\n  |> filter(fn: (r) => r.name =~ /${interface:regex}/)\n  |> filter(fn: (r) => r._field == \"in-errors\" or r._field == \"out-errors\" or r._field == \"in-fcs-errors\")\n  |> toFloat()\n  |> derivative(unit: 1s, nonNegative: true)",
+          "refId": "A"
+        }
+      ],
+      "title": "Interface Errors",
+      "type": "timeseries"
+    },
+    {
+      "datasource": {"type": "influxdb","uid": "obmp_influxdb"},
+      "description": "Interface drop counters over time: input drops and output drops. Drops indicate congestion or queue overflow.",
+      "fieldConfig": {
+        "defaults": {
+          "color": {"mode": "palette-classic"},
+          "custom": {"axisBorderShow": false,"axisCenteredZero": false,"axisLabel": "","axisPlacement": "auto","barAlignment": 0,"drawStyle": "line","fillOpacity": 10,"gradientMode": "none","hideFrom": {"legend": false,"tooltip": false,"viz": false},"lineInterpolation": "linear","lineWidth": 1,"pointSize": 5,"scaleDistribution": {"type": "linear"},"showPoints": "never","spanNulls": false,"stacking": {"group": "A","mode": "none"},"thresholdsStyle": {"mode": "off"}},
+          "mappings": [],
+          "thresholds": {"mode": "absolute","steps": [{"color": "green","value": null},{"color": "yellow","value": 1},{"color": "red","value": 100}]},
+          "unit": "short"
+        }
+      },
+      "gridPos": {"h": 10,"w": 24,"x": 0,"y": 10},
+      "id": 2,
+      "options": {"legend": {"calcs": ["mean","max","last"],"displayMode": "table","placement": "bottom"},"tooltip": {"mode": "multi","sort": "desc"}},
+      "targets": [
+        {
+          "datasource": {"type": "influxdb","uid": "obmp_influxdb"},
+          "query": "from(bucket: \"telemetry\")\n  |> range(start: v.timeRangeStart, stop: v.timeRangeStop)\n  |> filter(fn: (r) => r._measurement == \"interface_counters\")\n  |> filter(fn: (r) => r.source =~ /${router:regex}/)\n  |> filter(fn: (r) => r.name =~ /${interface:regex}/)\n  |> filter(fn: (r) => r._field == \"in-discards\" or r._field == \"out-discards\")\n  |> toFloat()\n  |> derivative(unit: 1s, nonNegative: true)",
+          "refId": "A"
+        }
+      ],
+      "title": "Interface Drops",
+      "type": "timeseries"
+    },
+    {
+      "datasource": {"type": "influxdb","uid": "obmp_influxdb"},
+      "description": "Summary table showing the latest error and drop counter values per interface. Useful for quickly identifying problematic interfaces.",
+      "fieldConfig": {
+        "defaults": {
+          "color": {"mode": "thresholds"},
+          "custom": {"align": "auto","displayMode": "auto","filterable": true,"inspect": true},
+          "mappings": [],
+          "thresholds": {"mode": "absolute","steps": [{"color": "green","value": null},{"color": "yellow","value": 1},{"color": "red","value": 100}]}
+        },
+        "overrides": [
+          {"matcher": {"id": "byName","options": "in-errors"},"properties": [{"id": "custom.displayMode","value": "color-background-solid"},{"id": "thresholds","value": {"mode": "absolute","steps": [{"color": "green","value": null},{"color": "yellow","value": 1},{"color": "red","value": 100}]}}]},
+          {"matcher": {"id": "byName","options": "out-errors"},"properties": [{"id": "custom.displayMode","value": "color-background-solid"},{"id": "thresholds","value": {"mode": "absolute","steps": [{"color": "green","value": null},{"color": "yellow","value": 1},{"color": "red","value": 100}]}}]},
+          {"matcher": {"id": "byName","options": "in-discards"},"properties": [{"id": "custom.displayMode","value": "color-background-solid"},{"id": "thresholds","value": {"mode": "absolute","steps": [{"color": "green","value": null},{"color": "yellow","value": 1},{"color": "red","value": 100}]}}]},
+          {"matcher": {"id": "byName","options": "out-discards"},"properties": [{"id": "custom.displayMode","value": "color-background-solid"},{"id": "thresholds","value": {"mode": "absolute","steps": [{"color": "green","value": null},{"color": "yellow","value": 1},{"color": "red","value": 100}]}}]}
+        ]
+      },
+      "gridPos": {"h": 12,"w": 24,"x": 0,"y": 20},
+      "id": 3,
+      "options": {"footer": {"fields": "","reducer": ["sum"],"show": false},"showHeader": true,"sortBy": [{"desc": true,"displayName": "in-errors"}]},
+      "targets": [
+        {
+          "datasource": {"type": "influxdb","uid": "obmp_influxdb"},
+          "query": "from(bucket: \"telemetry\")\n  |> range(start: v.timeRangeStart, stop: v.timeRangeStop)\n  |> filter(fn: (r) => r._measurement == \"interface_counters\")\n  |> filter(fn: (r) => r.source =~ /${router:regex}/)\n  |> filter(fn: (r) => r.name =~ /${interface:regex}/)\n  |> filter(fn: (r) => r._field == \"in-errors\" or r._field == \"out-errors\" or r._field == \"in-fcs-errors\" or r._field == \"in-discards\" or r._field == \"out-discards\")\n  |> toFloat()\n  |> last()\n  |> pivot(rowKey: [\"_time\"], columnKey: [\"_field\"], valueColumn: \"_value\")\n  |> keep(columns: [\"source\", \"name\", \"in-errors\", \"out-errors\", \"in-fcs-errors\", \"in-discards\", \"out-discards\"])\n  |> sort(columns: [\"in-errors\"], desc: true)",
+          "refId": "A"
+        }
+      ],
+      "title": "Error Summary Table",
+      "type": "table"
+    }
+  ],
+  "schemaVersion": 39,
+  "style": "dark",
+  "tags": ["obmp-telemetry"],
+  "time": {"from": "now-1h","to": "now"},
+  "timepicker": {},
+  "timezone": "browser",
+  "title": "Interface Errors & Drops",
+  "uid": "obmp-telem-02",
+  "version": 1
+}

obmp-grafana/dashboards/Telemetry-3001/interface_utilization.json

@@ -0,0 +1,141 @@
+{
+  "annotations": {"list": [{"builtIn": 1,"datasource": {"type": "datasource","uid": "grafana"},"enable": true,"hide": true,"iconColor": "rgba(0, 211, 255, 1)","name": "Annotations & Alerts","type": "dashboard"}]},
+  "description": "Interface utilization metrics collected via gNMI streaming telemetry from IOS-XR routers. Shows byte rates, packet rates, and top interfaces by traffic volume.",
+  "editable": true,
+  "fiscalYearStartMonth": 0,
+  "graphTooltip": 1,
+  "id": null,
+  "links": [],
+  "templating": {
+    "list": [
+      {
+        "current": {},
+        "datasource": {"type": "influxdb","uid": "obmp_influxdb"},
+        "definition": "from(bucket: \"telemetry\")\n  |> range(start: -1h)\n  |> filter(fn: (r) => r._measurement == \"interface_counters\")\n  |> keep(columns: [\"source\"])\n  |> distinct(column: \"source\")\n  |> sort()",
+        "hide": 0,
+        "includeAll": true,
+        "label": "Router",
+        "multi": true,
+        "name": "router",
+        "options": [],
+        "query": "from(bucket: \"telemetry\")\n  |> range(start: -1h)\n  |> filter(fn: (r) => r._measurement == \"interface_counters\")\n  |> keep(columns: [\"source\"])\n  |> distinct(column: \"source\")\n  |> sort()",
+        "refresh": 2,
+        "regex": "",
+        "type": "query"
+      },
+      {
+        "current": {},
+        "datasource": {"type": "influxdb","uid": "obmp_influxdb"},
+        "definition": "from(bucket: \"telemetry\")\n  |> range(start: -1h)\n  |> filter(fn: (r) => r._measurement == \"interface_counters\")\n  |> filter(fn: (r) => r.source =~ /${router:regex}/)\n  |> keep(columns: [\"name\"])\n  |> distinct(column: \"name\")\n  |> sort()",
+        "hide": 0,
+        "includeAll": true,
+        "label": "Interface",
+        "multi": true,
+        "name": "interface",
+        "options": [],
+        "query": "from(bucket: \"telemetry\")\n  |> range(start: -1h)\n  |> filter(fn: (r) => r._measurement == \"interface_counters\")\n  |> filter(fn: (r) => r.source =~ /${router:regex}/)\n  |> keep(columns: [\"name\"])\n  |> distinct(column: \"name\")\n  |> sort()",
+        "refresh": 2,
+        "regex": "",
+        "type": "query"
+      }
+    ]
+  },
+  "panels": [
+    {
+      "datasource": {"type": "influxdb","uid": "obmp_influxdb"},
+      "description": "Rate of bytes received and sent per interface, computed as the derivative of cumulative counters. Unit: bytes per second.",
+      "fieldConfig": {
+        "defaults": {
+          "color": {"mode": "palette-classic"},
+          "custom": {"axisBorderShow": false,"axisCenteredZero": false,"axisLabel": "","axisPlacement": "auto","barAlignment": 0,"drawStyle": "line","fillOpacity": 10,"gradientMode": "none","hideFrom": {"legend": false,"tooltip": false,"viz": false},"lineInterpolation": "linear","lineWidth": 1,"pointSize": 5,"scaleDistribution": {"type": "linear"},"showPoints": "never","spanNulls": false,"stacking": {"group": "A","mode": "none"},"thresholdsStyle": {"mode": "off"}},
+          "mappings": [],
+          "thresholds": {"mode": "absolute","steps": [{"color": "green","value": null},{"color": "red","value": 80}]},
+          "unit": "Bps"
+        }
+      },
+      "gridPos": {"h": 10,"w": 24,"x": 0,"y": 0},
+      "id": 1,
+      "options": {"legend": {"calcs": ["mean","max"],"displayMode": "table","placement": "bottom"},"tooltip": {"mode": "multi","sort": "desc"}},
+      "targets": [
+        {
+          "datasource": {"type": "influxdb","uid": "obmp_influxdb"},
+          "query": "from(bucket: \"telemetry\")\n  |> range(start: v.timeRangeStart, stop: v.timeRangeStop)\n  |> filter(fn: (r) => r._measurement == \"interface_counters\")\n  |> filter(fn: (r) => r.source =~ /${router:regex}/)\n  |> filter(fn: (r) => r.name =~ /${interface:regex}/)\n  |> filter(fn: (r) => r._field == \"in-octets\" or r._field == \"out-octets\")\n  |> toFloat()\n  |> derivative(unit: 1s, nonNegative: true)\n  |> map(fn: (r) => ({r with _value: if r._value < 0.0 then 0.0 else r._value}))",
+          "refId": "A"
+        }
+      ],
+      "title": "Input/Output Bytes Rate",
+      "type": "timeseries"
+    },
+    {
+      "datasource": {"type": "influxdb","uid": "obmp_influxdb"},
+      "description": "Rate of packets received and sent per interface, computed as the derivative of cumulative counters. Unit: packets per second.",
+      "fieldConfig": {
+        "defaults": {
+          "color": {"mode": "palette-classic"},
+          "custom": {"axisBorderShow": false,"axisCenteredZero": false,"axisLabel": "","axisPlacement": "auto","barAlignment": 0,"drawStyle": "line","fillOpacity": 10,"gradientMode": "none","hideFrom": {"legend": false,"tooltip": false,"viz": false},"lineInterpolation": "linear","lineWidth": 1,"pointSize": 5,"scaleDistribution": {"type": "linear"},"showPoints": "never","spanNulls": false,"stacking": {"group": "A","mode": "none"},"thresholdsStyle": {"mode": "off"}},
+          "mappings": [],
+          "thresholds": {"mode": "absolute","steps": [{"color": "green","value": null},{"color": "red","value": 80}]},
+          "unit": "pps"
+        }
+      },
+      "gridPos": {"h": 10,"w": 24,"x": 0,"y": 10},
+      "id": 2,
+      "options": {"legend": {"calcs": ["mean","max"],"displayMode": "table","placement": "bottom"},"tooltip": {"mode": "multi","sort": "desc"}},
+      "targets": [
+        {
+          "datasource": {"type": "influxdb","uid": "obmp_influxdb"},
+          "query": "from(bucket: \"telemetry\")\n  |> range(start: v.timeRangeStart, stop: v.timeRangeStop)\n  |> filter(fn: (r) => r._measurement == \"interface_counters\")\n  |> filter(fn: (r) => r.source =~ /${router:regex}/)\n  |> filter(fn: (r) => r.name =~ /${interface:regex}/)\n  |> filter(fn: (r) => r._field == \"in-pkts\" or r._field == \"out-pkts\")\n  |> toFloat()\n  |> derivative(unit: 1s, nonNegative: true)\n  |> map(fn: (r) => ({r with _value: if r._value < 0.0 then 0.0 else r._value}))",
+          "refId": "A"
+        }
+      ],
+      "title": "Input/Output Packets Rate",
+      "type": "timeseries"
+    },
+    {
+      "datasource": {"type": "influxdb","uid": "obmp_influxdb"},
+      "description": "Top interfaces ranked by total bytes (received + sent) over the selected time range.",
+      "fieldConfig": {
+        "defaults": {
+          "color": {"mode": "palette-classic"},
+          "custom": {"axisBorderShow": false,"axisCenteredZero": false,"axisLabel": "","axisPlacement": "auto","fillOpacity": 80,"gradientMode": "none","hideFrom": {"legend": false,"tooltip": false,"viz": false},"lineWidth": 1,"scaleDistribution": {"type": "linear"},"thresholdsStyle": {"mode": "off"}},
+          "mappings": [],
+          "thresholds": {"mode": "absolute","steps": [{"color": "green","value": null}]},
+          "unit": "decbytes"
+        }
+      },
+      "gridPos": {"h": 10,"w": 24,"x": 0,"y": 20},
+      "id": 3,
+      "options": {"barRadius": 0,"barWidth": 0.6,"fullHighlight": false,"groupWidth": 0.7,"legend": {"calcs": [],"displayMode": "list","placement": "bottom"},"orientation": "horizontal","showValue": "auto","stacking": "none","tooltip": {"mode": "single","sort": "none"},"xTickLabelRotation": 0},
+      "targets": [
+        {
+          "datasource": {"type": "influxdb","uid": "obmp_influxdb"},
+          "query": "from(bucket: \"telemetry\")\n  |> range(start: v.timeRangeStart, stop: v.timeRangeStop)\n  |> filter(fn: (r) => r._measurement == \"interface_counters\")\n  |> filter(fn: (r) => r.source =~ /${router:regex}/)\n  |> filter(fn: (r) => r.name =~ /${interface:regex}/)\n  |> filter(fn: (r) => r._field == \"in-octets\" or r._field == \"out-octets\")\n  |> toFloat()\n  |> derivative(unit: 1s, nonNegative: true)\n  |> group(columns: [\"source\", \"name\", \"_field\"])\n  |> sum()\n  |> group(columns: [\"source\", \"name\"])\n  |> sum()\n  |> group()\n  |> sort(columns: [\"_value\"], desc: true)\n  |> limit(n: 20)",
+          "refId": "A"
+        }
+      ],
+      "title": "Top Interfaces by Traffic",
+      "type": "barchart"
+    },
+    {
+      "datasource": {"type": "datasource","uid": "grafana"},
+      "gridPos": {"h": 4,"w": 24,"x": 0,"y": 30},
+      "id": 4,
+      "options": {
+        "code": {"language": "plaintext","showLineNumbers": false,"showMiniMap": false},
+        "content": "## Interface Utilization Dashboard\n\nThis dashboard displays real-time interface utilization metrics collected via **gNMI streaming telemetry** from IOS-XR routers.\n\n- **Data source:** InfluxDB (Telegraf gNMI input plugin)\n- **YANG model:** OpenConfig (`openconfig-interfaces`)\n- **Subscription path:** `/interfaces/interface/state/counters`\n- **Sample interval:** 10 seconds\n\nUse the **Router** and **Interface** template variables at the top to filter the view.",
+        "mode": "markdown"
+      },
+      "title": "About",
+      "type": "text"
+    }
+  ],
+  "schemaVersion": 39,
+  "style": "dark",
+  "tags": ["obmp-telemetry"],
+  "time": {"from": "now-1h","to": "now"},
+  "timepicker": {},
+  "timezone": "browser",
+  "title": "Interface Utilization",
+  "uid": "obmp-telem-01",
+  "version": 1
+}

obmp-grafana/provisioning/dashboards/openbmp-dashboards.yml

@@ -134,3 +134,14 @@ providers:
     options:
       path: /var/lib/grafana/dashboards/Learning
       foldersFromFilesStructure: false
+  - name: 'OpenBMP-Telemetry'
+    orgId: 1
+    folder: 'OBMP-Telemetry'
+    folderUid: '3001'
+    type: file
+    disableDeletion: false
+    updateIntervalSeconds: 30
+    allowUiUpdates: true
+    options:
+      path: /var/lib/grafana/dashboards/Telemetry-3001
+      foldersFromFilesStructure: false

obmp-grafana/provisioning/datasources/influxdb-ds.yml

@@ -0,0 +1,16 @@
+apiVersion: 1
+
+datasources:
+  - name: InfluxDB-Telemetry
+    uid: obmp_influxdb
+    type: influxdb
+    access: proxy
+    url: http://10.40.40.202:8086
+    jsonData:
+      version: Flux
+      organization: openbmp
+      defaultBucket: telemetry
+    secureJsonData:
+      token: openbmp-telemetry-token
+    isDefault: false
+    editable: true

portal/index.html

@@ -0,0 +1,106 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+  <meta charset="UTF-8">
+  <meta name="viewport" content="width=device-width, initial-scale=1.0">
+  <title>OpenBMP Lab Portal</title>
+  <style>
+    * { margin: 0; padding: 0; box-sizing: border-box; }
+    body {
+      font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, sans-serif;
+      background: #111217;
+      color: #d8dee9;
+      min-height: 100vh;
+      display: flex;
+      flex-direction: column;
+      align-items: center;
+      padding: 2rem;
+    }
+    .header {
+      text-align: center;
+      margin-bottom: 2.5rem;
+    }
+    .header h1 {
+      font-size: 1.8rem;
+      color: #e2e8f0;
+      margin-bottom: 0.5rem;
+    }
+    .header p {
+      color: #7b8da0;
+      font-size: 0.95rem;
+    }
+    .grid {
+      display: grid;
+      grid-template-columns: repeat(auto-fit, minmax(260px, 1fr));
+      gap: 1.25rem;
+      max-width: 900px;
+      width: 100%;
+    }
+    .card {
+      background: #1a1d26;
+      border: 1px solid #2a2e3a;
+      border-radius: 10px;
+      padding: 1.5rem;
+      text-decoration: none;
+      color: inherit;
+      transition: border-color 0.2s, transform 0.15s;
+    }
+    .card:hover {
+      border-color: #3b82f6;
+      transform: translateY(-2px);
+    }
+    .card .icon {
+      font-size: 2rem;
+      margin-bottom: 0.75rem;
+      display: block;
+    }
+    .card h2 {
+      font-size: 1.1rem;
+      color: #e2e8f0;
+      margin-bottom: 0.4rem;
+    }
+    .card p {
+      font-size: 0.85rem;
+      color: #7b8da0;
+      line-height: 1.4;
+    }
+    .footer {
+      margin-top: 3rem;
+      color: #4a5568;
+      font-size: 0.8rem;
+      text-align: center;
+    }
+  </style>
+</head>
+<body>
+  <div class="header">
+    <h1>OpenBMP Lab</h1>
+    <p>BGP Monitoring Protocol &middot; Route Analysis &middot; Telemetry</p>
+  </div>
+
+  <div class="grid">
+    <a href="/grafana/" class="card">
+      <span class="icon">&#x1F4CA;</span>
+      <h2>Grafana Dashboards</h2>
+      <p>BGP analytics, RR Loc-RIB diff, IS-IS topology, telemetry, and 27+ dashboards.</p>
+    </a>
+
+    <a href="/exabgp/" class="card">
+      <span class="icon">&#x1F6E4;</span>
+      <h2>ExaBGP Route Injector</h2>
+      <p>Inject and withdraw BGP routes into the lab fabric via ExaBGP API.</p>
+    </a>
+
+    <a href="/traffic/" class="card">
+      <span class="icon">&#x1F680;</span>
+      <h2>Traffic Generator</h2>
+      <p>RFC 2544 throughput, latency, and loss testing across the network.</p>
+    </a>
+
+  </div>
+
+  <div class="footer">
+    OpenBMP Docker Stack &middot; 9 IOS-XR Routers &middot; CML Lab
+  </div>
+</body>
+</html>

telegraf/Dockerfile

@@ -0,0 +1,2 @@
+FROM telegraf:1.28-alpine
+COPY telegraf.conf /etc/telegraf/telegraf.conf

telegraf/telegraf.conf

@@ -0,0 +1,63 @@
+# Telegraf Configuration for gNMI Streaming Telemetry
+# Collects interface counters and data rates from IOS-XR routers
+
+[global_tags]
+
+[agent]
+  interval = "10s"
+  round_interval = true
+  metric_batch_size = 1000
+  metric_buffer_limit = 10000
+  collection_jitter = "0s"
+  flush_interval = "10s"
+  flush_jitter = "0s"
+  precision = "0s"
+
+###############################################################################
+#                            INPUT PLUGINS                                    #
+###############################################################################
+
+## CORE routers (directly reachable on port 57400 from host)
+## R9K routers (10.100.0.1-7) are blocked by CML management network filtering
+[[inputs.gnmi]]
+  addresses = [
+    "10.100.0.100:57400",
+    "10.100.0.200:57400"
+  ]
+  username = "webui"
+  password = "cisco"
+
+  ## No TLS (lab environment)
+  enable_tls = false
+
+  ## Use json_ietf encoding (supported by IOS-XR 24.3.1)
+  encoding = "json_ietf"
+
+  ## Redial in case of failures after
+  redial = "10s"
+
+  ## OpenConfig interface counters (bytes, packets, errors, discards)
+  [[inputs.gnmi.subscription]]
+    name = "interface_counters"
+    origin = "openconfig-interfaces"
+    path = "/interfaces/interface/state/counters"
+    subscription_mode = "sample"
+    sample_interval = "10s"
+
+  ## OpenConfig interface state (admin/oper status, description, type)
+  [[inputs.gnmi.subscription]]
+    name = "interface_state"
+    origin = "openconfig-interfaces"
+    path = "/interfaces/interface/state"
+    subscription_mode = "sample"
+    sample_interval = "30s"
+
+###############################################################################
+#                            OUTPUT PLUGINS                                   #
+###############################################################################
+
+[[outputs.influxdb_v2]]
+  urls = ["http://localhost:8086"]
+  token = "${INFLUXDB_TOKEN}"
+  organization = "openbmp"
+  bucket = "telemetry"

traffic-gen-ui/Dockerfile

@@ -0,0 +1,12 @@
+FROM node:20-alpine AS build
+WORKDIR /app
+COPY package.json ./
+RUN npm install
+COPY . .
+RUN npm run build
+
+FROM nginx:alpine
+COPY --from=build /app/dist /usr/share/nginx/html
+COPY nginx.conf /etc/nginx/conf.d/default.conf
+EXPOSE 5002
+CMD ["nginx", "-g", "daemon off;"]

traffic-gen-ui/index.html

@@ -0,0 +1,12 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+  <meta charset="UTF-8" />
+  <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+  <title>Traffic Generator</title>
+</head>
+<body>
+  <div id="app"></div>
+  <script type="module" src="/src/main.js"></script>
+</body>
+</html>

traffic-gen-ui/nginx.conf

@@ -0,0 +1,21 @@
+server {
+    listen 5002;
+    root /usr/share/nginx/html;
+    index index.html;
+
+    location /api/ {
+        proxy_pass http://localhost:5051/;
+        proxy_set_header Host $host;
+        proxy_set_header X-Real-IP $remote_addr;
+    }
+
+    location / {
+        try_files $uri $uri/ /index.html;
+        add_header Cache-Control "no-cache, no-store, must-revalidate";
+    }
+
+    location /assets/ {
+        expires 1y;
+        add_header Cache-Control "public, immutable";
+    }
+}

traffic-gen-ui/package.json

@@ -0,0 +1,17 @@
+{
+  "name": "traffic-gen-ui",
+  "version": "1.0.0",
+  "private": true,
+  "scripts": {
+    "dev": "vite",
+    "build": "vite build",
+    "preview": "vite preview"
+  },
+  "dependencies": {
+    "vue": "^3.3.0"
+  },
+  "devDependencies": {
+    "@vitejs/plugin-vue": "^4.2.0",
+    "vite": "^4.4.0"
+  }
+}

traffic-gen-ui/src/App.vue

@@ -0,0 +1,326 @@
+<template>
+  <div class="app-layout">
+    <!-- HEADER -->
+    <header class="app-header">
+      <div class="header-title">
+        <span class="logo-icon">&#9889;</span>
+        <h1>Traffic Generator</h1>
+      </div>
+      <StatusBar :health="health" :api-error="apiError" @modeChanged="fetchHealth(); fetchAll()" />
+    </header>
+
+    <!-- ERROR BANNER -->
+    <div v-if="apiError" class="error-banner">
+      <span class="error-icon">&#9888;</span>
+      API unreachable: {{ apiError }} &mdash; retrying every 5s
+    </div>
+
+    <!-- MAIN CONTENT -->
+    <div class="main-content">
+      <!-- LEFT COLUMN: Flow Builder -->
+      <aside class="left-col">
+        <FlowBuilder :key="editFlow ? editFlow.id : 'new'" :editFlow="editFlow"
+          @created="onFlowSaved" @updated="onFlowSaved" @cancel="editFlow = null" />
+      </aside>
+
+      <!-- RIGHT COLUMN: Tabs -->
+      <main class="right-col">
+        <div class="tabs">
+          <button
+            v-for="tab in tabs"
+            :key="tab.id"
+            class="tab-btn"
+            :class="{ active: activeTab === tab.id }"
+            @click="activeTab = tab.id"
+          >
+            {{ tab.label }}
+          </button>
+        </div>
+
+        <div class="tab-content">
+          <div v-if="activeTab === 'flows'">
+            <QuickPing />
+            <FlowTable :flows="flows" @refresh="fetchFlows" @edit="startEdit" />
+          </div>
+          <div v-else-if="activeTab === 'tests'">
+            <TestBuilder @created="fetchTests" @refresh="fetchAll" />
+            <div style="margin-top: 20px;">
+              <TestRunner :tests="tests" @refresh="fetchTests" />
+            </div>
+          </div>
+          <ResultsPanel v-else-if="activeTab === 'results'" :tests="tests" />
+          <StatsMonitor v-else-if="activeTab === 'monitor'" :flows="flows" />
+        </div>
+      </main>
+    </div>
+
+    <!-- FOOTER -->
+    <footer class="app-footer">
+      <span>Refreshing every 5s (health) / 3s (flows)</span>
+      <span class="footer-sep">|</span>
+      <a :href="baseUrl + ':3000'" target="_blank" class="footer-link">Grafana: :3000</a>
+      <span class="footer-sep">|</span>
+      <a :href="baseUrl + ':5001'" target="_blank" class="footer-link">Route Injector: :5001</a>
+    </footer>
+  </div>
+</template>
+
+<script setup>
+import { ref, computed, onMounted, onUnmounted } from 'vue'
+import { api } from './api.js'
+import StatusBar from './components/StatusBar.vue'
+import FlowBuilder from './components/FlowBuilder.vue'
+import FlowTable from './components/FlowTable.vue'
+import QuickPing from './components/QuickPing.vue'
+import TestBuilder from './components/TestBuilder.vue'
+import TestRunner from './components/TestRunner.vue'
+import ResultsPanel from './components/ResultsPanel.vue'
+import StatsMonitor from './components/StatsMonitor.vue'
+
+const health = ref(null)
+const flows = ref([])
+const tests = ref([])
+const apiError = ref(null)
+const activeTab = ref('flows')
+const editFlow = ref(null)
+const baseUrl = computed(() => `${window.location.protocol}//${window.location.hostname}`)
+
+function startEdit(flow) { editFlow.value = { ...flow } }
+function onFlowSaved() { editFlow.value = null; fetchFlows() }
+
+const tabs = [
+  { id: 'flows', label: 'Flows' },
+  { id: 'tests', label: 'Tests' },
+  { id: 'results', label: 'Results' },
+  { id: 'monitor', label: 'Monitor' },
+]
+
+async function fetchHealth() {
+  try {
+    health.value = await api.health()
+    apiError.value = null
+  } catch (e) {
+    apiError.value = e.message
+    health.value = null
+  }
+}
+
+async function fetchFlows() {
+  try {
+    const data = await api.flows()
+    flows.value = data.flows || []
+  } catch (_) {}
+}
+
+async function fetchTests() {
+  try {
+    const data = await api.tests()
+    tests.value = data.tests || []
+  } catch (_) {}
+}
+
+async function fetchAll() {
+  await Promise.all([fetchFlows(), fetchTests()])
+}
+
+let healthTimer = null
+let dataTimer = null
+
+onMounted(() => {
+  fetchHealth()
+  fetchAll()
+  healthTimer = setInterval(fetchHealth, 5000)
+  dataTimer = setInterval(fetchAll, 3000)
+})
+
+onUnmounted(() => {
+  clearInterval(healthTimer)
+  clearInterval(dataTimer)
+})
+</script>
+
+<style>
+:root {
+  --bg: #0f1117;
+  --card-bg: #1a1f2e;
+  --border: #2d3748;
+  --accent: #4f9cf9;
+  --success: #48bb78;
+  --danger: #fc8181;
+  --warning: #f6ad55;
+  --text: #e2e8f0;
+  --muted: #718096;
+  --radius: 8px;
+}
+
+* { box-sizing: border-box; margin: 0; padding: 0; }
+
+body {
+  background: var(--bg);
+  color: var(--text);
+  font-family: 'Segoe UI', system-ui, -apple-system, sans-serif;
+  font-size: 14px;
+  line-height: 1.5;
+}
+
+button {
+  cursor: pointer;
+  font-family: inherit;
+  font-size: 13px;
+  border: none;
+  border-radius: var(--radius);
+  transition: opacity 0.15s, background 0.15s;
+}
+
+button:disabled {
+  opacity: 0.5;
+  cursor: not-allowed;
+}
+
+input, select {
+  font-family: inherit;
+  font-size: 13px;
+  background: var(--bg);
+  color: var(--text);
+  border: 1px solid var(--border);
+  border-radius: var(--radius);
+  padding: 6px 10px;
+  outline: none;
+}
+
+input:focus {
+  border-color: var(--accent);
+}
+</style>
+
+<style scoped>
+.app-layout {
+  display: grid;
+  grid-template-rows: auto auto 1fr auto;
+  min-height: 100vh;
+}
+
+.app-header {
+  display: flex;
+  align-items: center;
+  justify-content: space-between;
+  padding: 12px 20px;
+  background: var(--card-bg);
+  border-bottom: 1px solid var(--border);
+  gap: 16px;
+  flex-wrap: wrap;
+}
+
+.header-title {
+  display: flex;
+  align-items: center;
+  gap: 10px;
+}
+
+.logo-icon {
+  font-size: 22px;
+  color: var(--warning);
+}
+
+.app-header h1 {
+  font-size: 18px;
+  font-weight: 600;
+  color: var(--text);
+  letter-spacing: 0.02em;
+}
+
+.error-banner {
+  background: rgba(252, 129, 129, 0.12);
+  border-bottom: 1px solid var(--danger);
+  color: var(--danger);
+  padding: 8px 20px;
+  font-size: 13px;
+  display: flex;
+  align-items: center;
+  gap: 8px;
+}
+
+.error-icon {
+  font-size: 16px;
+}
+
+.main-content {
+  display: grid;
+  grid-template-columns: 340px 1fr;
+  overflow: hidden;
+  height: calc(100vh - 110px);
+}
+
+.left-col {
+  border-right: 1px solid var(--border);
+  overflow-y: auto;
+  padding: 16px;
+}
+
+.right-col {
+  display: flex;
+  flex-direction: column;
+  overflow: hidden;
+}
+
+.tabs {
+  display: flex;
+  gap: 2px;
+  padding: 12px 16px 0;
+  background: var(--card-bg);
+  border-bottom: 1px solid var(--border);
+}
+
+.tab-btn {
+  background: transparent;
+  color: var(--muted);
+  padding: 8px 18px;
+  border-radius: var(--radius) var(--radius) 0 0;
+  border: 1px solid transparent;
+  border-bottom: none;
+  font-weight: 500;
+}
+
+.tab-btn:hover {
+  color: var(--text);
+  background: rgba(79, 156, 249, 0.08);
+}
+
+.tab-btn.active {
+  color: var(--accent);
+  background: var(--bg);
+  border-color: var(--border);
+  border-bottom: 1px solid var(--bg);
+  margin-bottom: -1px;
+}
+
+.tab-content {
+  flex: 1;
+  overflow-y: auto;
+  padding: 16px;
+}
+
+.app-footer {
+  padding: 8px 20px;
+  background: var(--card-bg);
+  border-top: 1px solid var(--border);
+  color: var(--muted);
+  font-size: 12px;
+  display: flex;
+  align-items: center;
+  gap: 10px;
+}
+
+.footer-sep {
+  color: var(--border);
+}
+
+.footer-link {
+  color: var(--accent);
+  text-decoration: none;
+}
+
+.footer-link:hover {
+  text-decoration: underline;
+}
+</style>

traffic-gen-ui/src/api.js

@@ -0,0 +1,48 @@
+const BASE = '/traffic/api'
+
+async function req(method, path, body) {
+  const opts = { method, headers: { 'Content-Type': 'application/json' } }
+  if (body) opts.body = JSON.stringify(body)
+  const r = await fetch(BASE + path, opts)
+  if (!r.ok) throw new Error(`${method} ${path} -> ${r.status}`)
+  return r.json()
+}
+
+export const api = {
+  health:       () => req('GET', '/healthz'),
+  interfaces:   () => req('GET', '/interfaces'),
+  mode:         () => req('GET', '/mode'),
+  setMode:      (mode) => req('POST', '/mode', { mode }),
+
+  // Flows
+  flows:        () => req('GET', '/flows'),
+  createFlow:   (f) => req('POST', '/flows', f),
+  getFlow:      (id) => req('GET', `/flows/${id}`),
+  updateFlow:   (id, f) => req('PUT', `/flows/${id}`, f),
+  deleteFlow:   (id) => req('DELETE', `/flows/${id}`),
+  startFlow:    (id) => req('POST', `/flows/${id}/start`),
+  stopFlow:     (id) => req('POST', `/flows/${id}/stop`),
+  flowStats:    (id) => req('GET', `/flows/${id}/stats`),
+
+  // Tests
+  tests:        () => req('GET', '/tests'),
+  createTest:   (t) => req('POST', '/tests', t),
+  getTest:      (id) => req('GET', `/tests/${id}`),
+  startTest:    (id) => req('POST', `/tests/${id}/start`),
+  stopTest:     (id) => req('POST', `/tests/${id}/stop`),
+  testResults:  (id) => req('GET', `/tests/${id}/results`),
+
+  // Presets
+  presets:      () => req('GET', '/presets'),
+  loadPreset:   (name, overrides) => req('POST', `/presets/${name}`, overrides),
+
+  // Stats
+  statsHistory: () => req('GET', '/stats/history'),
+
+  // Ping
+  ping:           (target, count) => req('POST', '/ping', { target, count: count || 5 }),
+
+  // Responder
+  responderStats: () => req('GET', '/responder/stats'),
+  responderReset: () => req('POST', '/responder/reset'),
+}

traffic-gen-ui/src/components/FlowBuilder.vue

@@ -0,0 +1,159 @@
+<template>
+  <div class="flow-builder">
+    <h3>{{ editing ? 'Edit Flow' : 'Create Flow' }}</h3>
+    <form @submit.prevent="submit">
+      <div class="form-row">
+        <label>Name</label>
+        <input v-model="form.name" placeholder="My Flow" />
+      </div>
+      <div class="form-row">
+        <label>Destination IP *</label>
+        <input v-model="form.dst_ip" placeholder="10.100.0.100" required />
+      </div>
+      <div class="form-row">
+        <label>Source IP</label>
+        <input v-model="form.src_ip" placeholder="auto (from interface)" />
+      </div>
+      <div class="form-row">
+        <label>Dst MAC</label>
+        <input v-model="form.dst_mac" placeholder="auto" />
+      </div>
+      <div class="form-row">
+        <label>Protocol</label>
+        <select v-model="form.protocol">
+          <option value="udp">UDP</option>
+          <option value="tcp">TCP</option>
+          <option value="icmp">ICMP</option>
+        </select>
+      </div>
+      <div v-if="form.protocol !== 'icmp'" class="form-row-pair">
+        <div class="form-row">
+          <label>Src Port</label>
+          <input v-model.number="form.src_port" type="number" min="1" max="65535" />
+        </div>
+        <div class="form-row">
+          <label>Dst Port</label>
+          <input v-model.number="form.dst_port" type="number" min="1" max="65535" />
+        </div>
+      </div>
+      <div class="form-row-pair">
+        <div class="form-row">
+          <label>Frame Size (bytes)</label>
+          <input v-model.number="form.frame_size" type="number" min="64" max="9000" />
+        </div>
+        <div class="form-row">
+          <label>Rate</label>
+          <input v-model.number="form.rate_val" type="number" min="1" step="any" />
+          <select v-model="form.rate_unit" class="rate-unit-standalone">
+            <option value="pps">pps</option>
+            <option value="kbps">Kbps</option>
+            <option value="mbps">Mbps</option>
+          </select>
+        </div>
+      </div>
+      <div class="form-row-pair">
+        <div class="form-row">
+          <label>Duration (sec)</label>
+          <input v-model.number="form.duration" type="number" min="0" :disabled="form.continuous" />
+          <label class="checkbox-inline">
+            <input type="checkbox" v-model="form.continuous" @change="onContinuousChange" />
+            Continuous
+          </label>
+        </div>
+        <div class="form-row">
+          <label>DSCP</label>
+          <input v-model.number="form.dscp" type="number" min="0" max="63" />
+        </div>
+      </div>
+      <div class="form-row">
+        <label>Responder URL (optional)</label>
+        <input v-model="form.responder_url" placeholder="http://host:5053" />
+      </div>
+      <div class="form-actions">
+        <button type="submit" class="btn btn-accent" :disabled="!form.dst_ip">
+          {{ editing ? 'Update' : 'Create Flow' }}
+        </button>
+        <button v-if="editing" type="button" class="btn btn-muted" @click="$emit('cancel')">Cancel</button>
+      </div>
+    </form>
+  </div>
+</template>
+
+<script setup>
+import { reactive, computed } from 'vue'
+import { api } from '../api.js'
+
+const props = defineProps({ editFlow: Object })
+const emit = defineEmits(['created', 'updated', 'cancel'])
+
+const editing = computed(() => !!props.editFlow)
+
+const defaults = {
+  name: '', dst_ip: '', src_ip: '', dst_mac: '',
+  protocol: 'udp', src_port: 50000, dst_port: 5001,
+  frame_size: 512, rate_val: 1000, rate_unit: 'pps', duration: 30,
+  dscp: 0, responder_url: '', continuous: false,
+}
+
+function ppsToDisplay(pps, frameSize) {
+  // Convert stored PPS to a friendlier unit if it was originally set that way
+  return { rate_val: pps, rate_unit: 'pps' }
+}
+
+const initData = props.editFlow
+  ? { ...props.editFlow, continuous: props.editFlow.duration === 0, ...ppsToDisplay(props.editFlow.rate_pps, props.editFlow.frame_size) }
+  : {}
+const form = reactive({ ...defaults, ...initData })
+
+function onContinuousChange() { if (form.continuous) form.duration = 0 }
+
+function computePps(val, unit, frameSize) {
+  if (unit === 'kbps') return Math.max(1, Math.round((val * 1000) / (frameSize * 8)))
+  if (unit === 'mbps') return Math.max(1, Math.round((val * 1_000_000) / (frameSize * 8)))
+  return Math.round(val)
+}
+
+async function submit() {
+  try {
+    const payload = { ...form }
+    payload.rate_pps = computePps(form.rate_val, form.rate_unit, form.frame_size)
+    delete payload.rate_val
+    delete payload.rate_unit
+    delete payload.continuous
+    if (form.continuous) payload.duration = 0
+    if (!payload.src_ip) delete payload.src_ip
+    if (!payload.dst_mac) delete payload.dst_mac
+    if (!payload.responder_url) delete payload.responder_url
+    if (!payload.name) payload.name = `${payload.protocol.toUpperCase()} -> ${payload.dst_ip}`
+
+    if (editing.value) {
+      await api.updateFlow(props.editFlow.id, payload)
+      emit('updated')
+    } else {
+      await api.createFlow(payload)
+      Object.assign(form, defaults)
+      emit('created')
+    }
+  } catch (e) {
+    alert('Error: ' + e.message)
+  }
+}
+</script>
+
+<style scoped>
+.flow-builder { padding: 0; }
+h3 { font-size: 15px; margin-bottom: 12px; color: var(--accent); }
+.form-row { margin-bottom: 10px; }
+.form-row label { display: block; font-size: 11px; color: var(--muted); margin-bottom: 3px; text-transform: uppercase; letter-spacing: 0.05em; }
+.form-row input, .form-row select { width: 100%; }
+.form-row-pair { display: grid; grid-template-columns: 1fr 1fr; gap: 8px; }
+.form-actions { display: flex; gap: 8px; margin-top: 14px; }
+.btn { padding: 8px 16px; font-weight: 600; font-size: 13px; }
+.btn-accent { background: var(--accent); color: #fff; }
+.btn-accent:hover { opacity: 0.9; }
+.btn-accent:disabled { opacity: 0.4; }
+.btn-muted { background: var(--border); color: var(--text); }
+.rate-unit-standalone { width: 100%; margin-top: 4px; }
+.checkbox-inline { display: inline-flex !important; align-items: center; gap: 4px; margin-top: 4px; font-size: 12px; cursor: pointer; }
+.checkbox-inline input { width: auto; }
+</style>

traffic-gen-ui/src/components/FlowTable.vue

@@ -0,0 +1,116 @@
+<template>
+  <div class="flow-table">
+    <div v-if="!flows.length" class="empty">No flows created yet. Use the builder to create one.</div>
+    <table v-else>
+      <thead>
+        <tr>
+          <th>Name</th>
+          <th>Dst IP</th>
+          <th>Proto</th>
+          <th>Size</th>
+          <th>Rate</th>
+          <th>State</th>
+          <th>TX Pkts</th>
+          <th>TX pps</th>
+          <th>RX Pkts</th>
+          <th>Actions</th>
+        </tr>
+      </thead>
+      <tbody>
+        <tr v-for="f in flows" :key="f.id" :class="{ running: f.state === 'running' }">
+          <td>{{ f.name || '-' }}</td>
+          <td class="mono">{{ f.dst_ip }}</td>
+          <td>{{ f.protocol.toUpperCase() }}</td>
+          <td>{{ f.frame_size }}B</td>
+          <td>{{ formatRate(f) }}</td>
+          <td>
+            <span class="state-badge" :class="'state-' + f.state">{{ f.state }}</span>
+          </td>
+          <td class="mono">{{ formatNum(f.stats?.tx_packets || 0) }}</td>
+          <td class="mono">{{ pps[f.id] || 0 }}</td>
+          <td class="mono">{{ formatNum(f.stats?.rx_packets || 0) }}</td>
+          <td class="actions">
+            <button v-if="f.state !== 'running'" class="btn-sm btn-go" @click="start(f.id)">Start</button>
+            <button v-else class="btn-sm btn-stop" @click="stop(f.id)">Stop</button>
+            <button class="btn-sm btn-edit" @click="emit('edit', f)" :disabled="f.state === 'running'">Edit</button>
+            <button class="btn-sm btn-del" @click="del(f.id)" :disabled="f.state === 'running'">Del</button>
+          </td>
+        </tr>
+      </tbody>
+    </table>
+  </div>
+</template>
+
+<script setup>
+import { ref, onMounted, onUnmounted } from 'vue'
+import { api } from '../api.js'
+
+const props = defineProps({ flows: Array })
+const emit = defineEmits(['refresh', 'edit'])
+const pps = ref({})
+const prevTx = ref({})
+
+let statsTimer = null
+
+function computePps() {
+  for (const f of (props.flows || [])) {
+    const txNow = f.stats?.tx_packets || 0
+    const prev = prevTx.value[f.id] || 0
+    if (f.state === 'running' && prev > 0) {
+      pps.value[f.id] = Math.max(0, txNow - prev)
+    } else if (f.state !== 'running') {
+      pps.value[f.id] = 0
+    }
+    prevTx.value[f.id] = txNow
+  }
+}
+
+function formatRate(f) {
+  const pps = f.rate_pps || 0
+  const mbps = (pps * (f.frame_size || 64) * 8) / 1_000_000
+  if (mbps >= 1) return mbps.toFixed(1) + ' Mbps'
+  if (mbps >= 0.001) return (mbps * 1000).toFixed(0) + ' Kbps'
+  return pps + ' pps'
+}
+
+function formatNum(n) {
+  if (n >= 1000000) return (n / 1000000).toFixed(1) + 'M'
+  if (n >= 1000) return (n / 1000).toFixed(1) + 'K'
+  return n
+}
+
+onMounted(() => { statsTimer = setInterval(computePps, 1000) })
+onUnmounted(() => { clearInterval(statsTimer) })
+
+async function start(id) {
+  try { await api.startFlow(id); emit('refresh') } catch (e) { alert(e.message) }
+}
+async function stop(id) {
+  try { await api.stopFlow(id); emit('refresh') } catch (e) { alert(e.message) }
+}
+async function del(id) {
+  try { await api.deleteFlow(id); emit('refresh') } catch (e) { alert(e.message) }
+}
+</script>
+
+<style scoped>
+.flow-table { overflow-x: auto; }
+.empty { color: var(--muted); padding: 20px; text-align: center; }
+table { width: 100%; border-collapse: collapse; }
+th { text-align: left; font-size: 11px; color: var(--muted); text-transform: uppercase; letter-spacing: 0.05em; padding: 6px 8px; border-bottom: 1px solid var(--border); }
+td { padding: 8px; border-bottom: 1px solid rgba(45,55,72,0.5); font-size: 13px; }
+tr.running { background: rgba(79,156,249,0.05); }
+.mono { font-family: 'Cascadia Code', 'Fira Code', monospace; font-size: 12px; }
+.state-badge { font-size: 11px; padding: 2px 8px; border-radius: 10px; font-weight: 600; }
+.state-idle { background: rgba(113,128,150,0.2); color: var(--muted); }
+.state-running { background: rgba(72,187,120,0.15); color: var(--success); }
+.state-stopped { background: rgba(246,173,85,0.15); color: var(--warning); }
+.actions { display: flex; gap: 4px; }
+.btn-sm { padding: 3px 10px; font-size: 11px; font-weight: 600; border-radius: 6px; }
+.btn-go { background: var(--success); color: #fff; }
+.btn-stop { background: var(--warning); color: #000; }
+.btn-del { background: rgba(252,129,129,0.15); color: var(--danger); }
+.btn-edit { background: rgba(79,156,249,0.15); color: var(--accent); }
+.btn-edit:disabled { opacity: 0.3; }
+.btn-del:disabled { opacity: 0.3; }
+</style>

traffic-gen-ui/src/components/QuickPing.vue

@@ -0,0 +1,76 @@
+<template>
+  <div class="quick-ping">
+    <div class="ping-row">
+      <input
+        v-model="target"
+        placeholder="IP address to ping..."
+        @keyup.enter="runPing"
+        :disabled="pinging"
+      />
+      <button class="btn-ping" @click="runPing" :disabled="!target || pinging">
+        {{ pinging ? 'Pinging...' : 'Ping' }}
+      </button>
+    </div>
+    <div v-if="result" class="ping-result" :class="result.reachable ? 'reachable' : 'unreachable'">
+      <div class="ping-summary">
+        <span class="ping-target">{{ result.target }}</span>
+        <span v-if="result.reachable" class="ping-status ok">Reachable</span>
+        <span v-else class="ping-status fail">Unreachable</span>
+      </div>
+      <div v-if="result.reachable && result.stats" class="ping-stats">
+        <span>{{ result.received }}/{{ result.sent }} replies</span>
+        <span>Min: {{ result.stats.min_ms }}ms</span>
+        <span>Avg: {{ result.stats.avg_ms }}ms</span>
+        <span>Max: {{ result.stats.max_ms }}ms</span>
+        <span>Loss: {{ result.loss_pct }}%</span>
+      </div>
+      <div v-if="result.error" class="ping-error">{{ result.error }}</div>
+    </div>
+  </div>
+</template>
+
+<script setup>
+import { ref } from 'vue'
+import { api } from '../api.js'
+
+const target = ref('')
+const pinging = ref(false)
+const result = ref(null)
+
+async function runPing() {
+  if (!target.value || pinging.value) return
+  pinging.value = true
+  result.value = null
+  try {
+    result.value = await api.ping(target.value, 5)
+  } catch (e) {
+    result.value = { target: target.value, reachable: false, error: e.message }
+  } finally {
+    pinging.value = false
+  }
+}
+</script>
+
+<style scoped>
+.quick-ping { margin-bottom: 16px; }
+.ping-row { display: flex; gap: 6px; }
+.ping-row input { flex: 1; }
+.btn-ping {
+  padding: 6px 16px; font-weight: 600; font-size: 13px;
+  background: var(--accent); color: #fff; white-space: nowrap;
+}
+.btn-ping:disabled { opacity: 0.4; }
+.ping-result {
+  margin-top: 8px; padding: 8px 12px;
+  border-radius: var(--radius); font-size: 13px;
+}
+.ping-result.reachable { background: rgba(72,187,120,0.1); border: 1px solid rgba(72,187,120,0.3); }
+.ping-result.unreachable { background: rgba(252,129,129,0.1); border: 1px solid rgba(252,129,129,0.3); }
+.ping-summary { display: flex; align-items: center; gap: 10px; }
+.ping-target { font-weight: 600; font-family: monospace; }
+.ping-status { font-size: 11px; padding: 2px 8px; border-radius: 10px; font-weight: 600; }
+.ping-status.ok { background: rgba(72,187,120,0.2); color: var(--success); }
+.ping-status.fail { background: rgba(252,129,129,0.2); color: var(--danger); }
+.ping-stats { display: flex; gap: 12px; margin-top: 6px; font-size: 12px; color: var(--muted); font-family: monospace; }
+.ping-error { margin-top: 4px; color: var(--danger); font-size: 12px; }
+</style>

traffic-gen-ui/src/components/ResultsPanel.vue

@@ -0,0 +1,157 @@
+<template>
+  <div class="results-panel">
+    <h3>Test Results</h3>
+    <div v-if="!completedTests.length" class="empty">No completed tests yet.</div>
+
+    <div v-for="t in completedTests" :key="t.id" class="result-card">
+      <div class="result-header">
+        <strong>{{ t.type }} Test</strong>
+        <span class="result-time">{{ t.completed_at }}</span>
+        <div class="export-btns">
+          <button class="btn-sm btn-export" @click="exportJSON(t)">Export JSON</button>
+          <button class="btn-sm btn-export" @click="exportCSV(t)">Export CSV</button>
+        </div>
+      </div>
+
+      <div v-if="t.error" class="error-msg">Error: {{ t.error }}</div>
+      <div v-if="t.results && Object.keys(t.results).length" class="result-table">
+        <!-- Frame Loss: array of rate steps per frame size -->
+        <template v-if="t.type === 'frame_loss'">
+          <div v-for="(rates, size) in t.results" :key="size" class="fl-section">
+            <div class="fl-title">Frame Size: {{ size }} B</div>
+            <table>
+              <thead>
+                <tr><th>Rate %</th><th>Rate (pps)</th><th>TX Packets</th><th>RX Packets</th><th>Loss %</th></tr>
+              </thead>
+              <tbody>
+                <tr v-for="r in rates" :key="r.rate_pct">
+                  <td class="mono">{{ r.rate_pct }}%</td>
+                  <td class="mono">{{ r.rate_pps }}</td>
+                  <td class="mono">{{ r.tx_packets }}</td>
+                  <td class="mono">{{ r.rx_packets }}</td>
+                  <td class="mono">{{ r.loss_pct }}%</td>
+                </tr>
+              </tbody>
+            </table>
+          </div>
+        </template>
+        <!-- Other test types -->
+        <table v-else>
+          <thead>
+            <tr>
+              <th>Frame Size (B)</th>
+              <th v-for="col in resultColumns(t)" :key="col">{{ col }}</th>
+            </tr>
+          </thead>
+          <tbody>
+            <tr v-for="(val, size) in t.results" :key="size">
+              <td>{{ size }}</td>
+              <td v-for="col in resultColumns(t)" :key="col" class="mono">
+                {{ formatVal(val, col) }}
+              </td>
+            </tr>
+          </tbody>
+        </table>
+      </div>
+
+      <div v-if="t.results && t.type !== 'frame_loss'" class="result-chart">
+        <div class="bar-chart">
+          <div v-for="(val, size) in t.results" :key="size" class="bar-item">
+            <div class="bar-fill" :style="{ height: barHeight(t, val) + '%' }"></div>
+            <span class="bar-label">{{ size }}B</span>
+          </div>
+        </div>
+      </div>
+    </div>
+  </div>
+</template>
+
+<script setup>
+import { computed } from 'vue'
+
+const props = defineProps({ tests: Array })
+
+const completedTests = computed(() =>
+  (props.tests || []).filter(t => (t.state === 'complete' || t.state === 'error') && (t.results || t.error))
+)
+
+function resultColumns(t) {
+  if (t.type === 'throughput') return ['Max Rate (pps)', 'Throughput (Mbps)']
+  if (t.type === 'latency') return ['Min (ms)', 'Avg (ms)', 'Max (ms)', 'Jitter (ms)']
+  if (t.type === 'frame_loss') return ['Loss %']
+  if (t.type === 'back_to_back') return ['Max Burst']
+  return ['Value']
+}
+
+function formatVal(val, col) {
+  if (typeof val === 'object') {
+    if (col.includes('Rate')) return val.max_throughput_pps ?? val.max_rate_pps ?? '-'
+    if (col.includes('Throughput')) {
+      const pps = val.max_throughput_pps ?? val.max_rate_pps ?? 0
+      const fs = val.frame_size ?? 64
+      return pps ? ((pps * fs * 8) / 1_000_000).toFixed(2) : '-'
+    }
+    if (col.includes('Min')) return val.min_ms != null ? val.min_ms.toFixed(2) : '-'
+    if (col.includes('Avg')) return val.avg_ms != null ? val.avg_ms.toFixed(2) : '-'
+    if (col.includes('Max') && col.includes('ms')) return val.max_ms != null ? val.max_ms.toFixed(2) : '-'
+    if (col.includes('Jitter')) return val.jitter_ms != null ? val.jitter_ms.toFixed(2) : '-'
+    if (col.includes('Loss')) return val.loss_pct ?? '-'
+    if (col.includes('Burst')) return val.max_burst_frames ?? val.max_burst ?? '-'
+    return JSON.stringify(val)
+  }
+  return val
+}
+
+function barHeight(t, val) {
+  const v = typeof val === 'object' ? (val.max_throughput_pps || val.max_rate_pps || val.avg_ms || val.loss_pct || val.max_burst_frames || 0) : val
+  const allVals = Object.values(t.results).map(r => typeof r === 'object' ? (r.max_throughput_pps || r.max_rate_pps || r.avg_ms || r.loss_pct || r.max_burst_frames || 0) : r)
+  const maxVal = Math.max(...allVals, 1)
+  return Math.min(100, Math.max(5, (v / maxVal) * 100))
+}
+
+function exportJSON(t) {
+  const blob = new Blob([JSON.stringify(t, null, 2)], { type: 'application/json' })
+  downloadBlob(blob, `test_${t.type}_${t.id}.json`)
+}
+
+function exportCSV(t) {
+  if (!t.results) return
+  const cols = resultColumns(t)
+  let csv = 'Frame Size,' + cols.join(',') + '\n'
+  for (const [size, val] of Object.entries(t.results)) {
+    csv += size + ',' + cols.map(c => formatVal(val, c)).join(',') + '\n'
+  }
+  downloadBlob(new Blob([csv], { type: 'text/csv' }), `test_${t.type}_${t.id}.csv`)
+}
+
+function downloadBlob(blob, name) {
+  const url = URL.createObjectURL(blob)
+  const a = document.createElement('a')
+  a.href = url; a.download = name; a.click()
+  URL.revokeObjectURL(url)
+}
+</script>
+
+<style scoped>
+h3 { font-size: 15px; margin-bottom: 12px; color: var(--accent); }
+.empty { color: var(--muted); padding: 20px; text-align: center; }
+.result-card { background: var(--card-bg); border: 1px solid var(--border); border-radius: var(--radius); padding: 12px; margin-bottom: 12px; }
+.result-header { display: flex; align-items: center; gap: 12px; margin-bottom: 10px; flex-wrap: wrap; }
+.result-header strong { font-size: 14px; text-transform: capitalize; }
+.result-time { font-size: 11px; color: var(--muted); }
+.export-btns { margin-left: auto; display: flex; gap: 4px; }
+.btn-sm { padding: 3px 10px; font-size: 11px; font-weight: 600; border-radius: 6px; }
+.btn-export { background: rgba(79,156,249,0.12); color: var(--accent); }
+.btn-export:hover { background: rgba(79,156,249,0.25); }
+table { width: 100%; border-collapse: collapse; }
+th { font-size: 11px; color: var(--muted); text-align: left; padding: 4px 8px; border-bottom: 1px solid var(--border); }
+td { font-size: 13px; padding: 4px 8px; }
+.mono { font-family: monospace; }
+.bar-chart { display: flex; align-items: flex-end; gap: 8px; height: 80px; margin-top: 12px; padding: 0 8px; }
+.bar-item { flex: 1; display: flex; flex-direction: column; align-items: center; height: 100%; }
+.bar-fill { width: 100%; background: var(--accent); border-radius: 3px 3px 0 0; min-height: 4px; transition: height 0.3s; margin-top: auto; }
+.bar-label { font-size: 10px; color: var(--muted); margin-top: 4px; }
+.fl-section { margin-bottom: 12px; }
+.fl-title { font-size: 12px; font-weight: 600; color: var(--accent); margin-bottom: 4px; }
+.error-msg { color: var(--danger); font-size: 13px; padding: 8px 0; }
+</style>

traffic-gen-ui/src/components/StatsMonitor.vue

@@ -0,0 +1,196 @@
+<template>
+  <div class="stats-monitor">
+    <h3>Live Statistics</h3>
+
+    <div class="flow-selector">
+      <label>Flow:</label>
+      <select v-model="selectedFlow">
+        <option value="">All Flows</option>
+        <option v-for="f in flows" :key="f.id" :value="f.id">{{ f.name || f.dst_ip }}</option>
+      </select>
+    </div>
+
+    <div class="stats-grid">
+      <div class="stat-card">
+        <div class="stat-value tx">{{ current.tx_pps || 0 }}</div>
+        <div class="stat-label">TX pps</div>
+      </div>
+      <div class="stat-card">
+        <div class="stat-value rx">{{ current.rx_pps || 0 }}</div>
+        <div class="stat-label">RX pps</div>
+      </div>
+      <div class="stat-card">
+        <div class="stat-value tx">{{ (current.tx_mbps || 0).toFixed(2) }}</div>
+        <div class="stat-label">TX Mbps</div>
+      </div>
+      <div class="stat-card">
+        <div class="stat-value rx">{{ (current.rx_mbps || 0).toFixed(2) }}</div>
+        <div class="stat-label">RX Mbps</div>
+      </div>
+      <div class="stat-card">
+        <div class="stat-value" :class="lossClass">{{ (current.loss_pct || 0).toFixed(1) }}%</div>
+        <div class="stat-label">Loss</div>
+      </div>
+      <div class="stat-card">
+        <div class="stat-value">{{ current.avg_latency_ms ? current.avg_latency_ms.toFixed(1) : '-' }}</div>
+        <div class="stat-label">Avg Latency (ms)</div>
+      </div>
+    </div>
+
+    <div class="totals">
+      <span>TX Packets: {{ current.tx_packets || 0 }}</span>
+      <span>RX Packets: {{ current.rx_packets || 0 }}</span>
+      <span>TX Bytes: {{ formatBytes(current.tx_bytes || 0) }}</span>
+      <span>RX Bytes: {{ formatBytes(current.rx_bytes || 0) }}</span>
+    </div>
+
+    <div class="history-chart">
+      <div class="chart-header">TX/RX Rate History (last 60s)</div>
+      <div class="sparkline">
+        <div v-for="(s, i) in history" :key="i" class="spark-bar">
+          <div class="spark-tx" :style="{ height: sparkHeight(s.tx_pps) + 'px' }"></div>
+          <div class="spark-rx" :style="{ height: sparkHeight(s.rx_pps) + 'px' }"></div>
+        </div>
+      </div>
+    </div>
+  </div>
+</template>
+
+<script setup>
+import { ref, computed, onMounted, onUnmounted } from 'vue'
+import { api } from '../api.js'
+
+const props = defineProps({ flows: Array })
+
+const selectedFlow = ref('')
+const current = ref({})
+const history = ref([])
+
+const lossClass = computed(() => {
+  const l = current.value.loss_pct || 0
+  if (l > 5) return 'loss-high'
+  if (l > 0) return 'loss-med'
+  return 'loss-ok'
+})
+
+let timer = null
+
+async function fetchStats() {
+  try {
+    if (selectedFlow.value) {
+      // Single flow: /flows/<id>/stats returns {flow_id, counters, rates}
+      // rates contains: tx_pps, rx_pps, tx_mbps, rx_mbps, loss_pct, tx_packets, tx_bytes, etc.
+      const s = await api.flowStats(selectedFlow.value)
+      const rates = s.rates || {}
+      const counters = s.counters || {}
+      current.value = {
+        tx_pps: Math.round(rates.tx_pps || 0),
+        rx_pps: Math.round(rates.rx_pps || 0),
+        tx_mbps: rates.tx_mbps || 0,
+        rx_mbps: rates.rx_mbps || 0,
+        loss_pct: rates.loss_pct || 0,
+        avg_latency_ms: rates.latency ? rates.latency.avg_ms : null,
+        tx_packets: counters.tx_packets || 0,
+        tx_bytes: counters.tx_bytes || 0,
+        rx_packets: counters.rx_packets || 0,
+        rx_bytes: counters.rx_bytes || 0,
+      }
+      // Append to history for sparkline
+      history.value.push({ tx_pps: current.value.tx_pps, rx_pps: current.value.rx_pps })
+      if (history.value.length > 60) history.value = history.value.slice(-60)
+    } else {
+      // All flows: /stats/history returns {history: {flow_id: [samples]}}
+      const h = await api.statsHistory()
+      const allHistory = h.history || {}
+      // Aggregate latest sample across all flows
+      let txPps = 0, rxPps = 0, txMbps = 0, rxMbps = 0
+      let txPkts = 0, txBytes = 0, rxPkts = 0, rxBytes = 0
+      let lossPcts = [], latencies = []
+
+      for (const [, samples] of Object.entries(allHistory)) {
+        if (!samples.length) continue
+        const latest = samples[samples.length - 1]
+        txPps += latest.tx_pps || 0
+        rxPps += latest.rx_pps || 0
+        txMbps += latest.tx_mbps || 0
+        rxMbps += latest.rx_mbps || 0
+        txPkts += latest.tx_packets || 0
+        txBytes += latest.tx_bytes || 0
+        rxPkts += latest.rx_packets || 0
+        rxBytes += latest.rx_bytes || 0
+        if (latest.loss_pct > 0) lossPcts.push(latest.loss_pct)
+        if (latest.latency && latest.latency.avg_ms) latencies.push(latest.latency.avg_ms)
+      }
+
+      current.value = {
+        tx_pps: Math.round(txPps),
+        rx_pps: Math.round(rxPps),
+        tx_mbps: txMbps,
+        rx_mbps: rxMbps,
+        loss_pct: txPkts > 0 ? Math.max(0, ((txPkts - rxPkts) / txPkts) * 100) : 0,
+        avg_latency_ms: latencies.length ? latencies.reduce((a, b) => a + b, 0) / latencies.length : null,
+        tx_packets: txPkts,
+        tx_bytes: txBytes,
+        rx_packets: rxPkts,
+        rx_bytes: rxBytes,
+      }
+
+      // Build aggregated sparkline from history samples
+      // Find max sample count across all flows
+      const flowIds = Object.keys(allHistory)
+      if (flowIds.length) {
+        const maxLen = Math.max(...flowIds.map(id => allHistory[id].length))
+        const sparkData = []
+        for (let i = Math.max(0, maxLen - 60); i < maxLen; i++) {
+          let sTx = 0, sRx = 0
+          for (const fid of flowIds) {
+            const s = allHistory[fid][i]
+            if (s) { sTx += s.tx_pps || 0; sRx += s.rx_pps || 0 }
+          }
+          sparkData.push({ tx_pps: Math.round(sTx), rx_pps: Math.round(sRx) })
+        }
+        history.value = sparkData
+      }
+    }
+  } catch (_) {}
+}
+
+function sparkHeight(val) {
+  if (!val) return 0
+  const max = Math.max(...history.value.map(s => Math.max(s.tx_pps || 0, s.rx_pps || 0)), 1)
+  return Math.max(1, (val / max) * 40)
+}
+
+function formatBytes(b) {
+  if (b < 1024) return b + ' B'
+  if (b < 1048576) return (b / 1024).toFixed(1) + ' KB'
+  if (b < 1073741824) return (b / 1048576).toFixed(1) + ' MB'
+  return (b / 1073741824).toFixed(2) + ' GB'
+}
+
+onMounted(() => { fetchStats(); timer = setInterval(fetchStats, 1000) })
+onUnmounted(() => { clearInterval(timer) })
+</script>
+
+<style scoped>
+h3 { font-size: 15px; margin-bottom: 12px; color: var(--accent); }
+.flow-selector { margin-bottom: 12px; display: flex; align-items: center; gap: 8px; }
+.flow-selector label { font-size: 12px; color: var(--muted); }
+.flow-selector select { flex: 1; }
+.stats-grid { display: grid; grid-template-columns: repeat(3, 1fr); gap: 10px; margin-bottom: 12px; }
+.stat-card { background: var(--card-bg); border: 1px solid var(--border); border-radius: var(--radius); padding: 10px; text-align: center; }
+.stat-value { font-size: 22px; font-weight: 700; font-family: monospace; }
+.stat-value.tx { color: var(--accent); }
+.stat-value.rx { color: var(--success); }
+.stat-label { font-size: 11px; color: var(--muted); margin-top: 2px; }
+.loss-ok { color: var(--success); }
+.loss-med { color: var(--warning); }
+.loss-high { color: var(--danger); }
+.totals { display: flex; gap: 16px; font-size: 12px; color: var(--muted); margin-bottom: 16px; flex-wrap: wrap; }
+.history-chart { background: var(--card-bg); border: 1px solid var(--border); border-radius: var(--radius); padding: 12px; }
+.chart-header { font-size: 12px; color: var(--muted); margin-bottom: 8px; }
+.sparkline { display: flex; align-items: flex-end; gap: 1px; height: 50px; }
+.spark-bar { flex: 1; display: flex; flex-direction: column; justify-content: flex-end; gap: 1px; }
+.spark-tx { background: var(--accent); border-radius: 1px; min-width: 2px; }
+.spark-rx { background: var(--success); border-radius: 1px; min-width: 2px; }
+</style>

traffic-gen-ui/src/components/StatusBar.vue

@@ -0,0 +1,61 @@
+<template>
+  <div class="status-bar">
+    <div class="status-badges">
+      <span class="badge" :class="connected ? 'badge-ok' : 'badge-err'">
+        {{ connected ? 'API Connected' : 'API Offline' }}
+      </span>
+      <span v-if="health" class="badge badge-mode" :class="'mode-' + (health.mode || 'sender')" @click="toggleMode">
+        {{ (health.mode || 'sender').toUpperCase() }}
+      </span>
+      <span v-if="health" class="badge badge-info">
+        Active Flows: {{ health.active_flows || 0 }}
+      </span>
+      <span v-if="health" class="badge badge-info">
+        Active Tests: {{ health.active_tests || 0 }}
+      </span>
+    </div>
+  </div>
+</template>
+
+<script setup>
+import { computed, ref } from 'vue'
+import { api } from '../api.js'
+
+const props = defineProps({ health: Object, apiError: String })
+const emit = defineEmits(['modeChanged'])
+const connected = computed(() => !props.apiError && props.health)
+const switching = ref(false)
+
+async function toggleMode() {
+  if (switching.value || !props.health) return
+  const current = props.health.mode || 'sender'
+  const next = current === 'sender' ? 'responder' : 'sender'
+  if (!confirm(`Switch to ${next.toUpperCase()} mode? This will stop all active flows/tests.`)) return
+  switching.value = true
+  try {
+    await api.setMode(next)
+    emit('modeChanged')
+  } catch (e) {
+    alert('Failed to switch mode: ' + e.message)
+  } finally {
+    switching.value = false
+  }
+}
+</script>
+
+<style scoped>
+.status-bar { display: flex; align-items: center; gap: 8px; }
+.status-badges { display: flex; gap: 6px; flex-wrap: wrap; }
+.badge {
+  font-size: 11px; padding: 3px 10px; border-radius: 12px;
+  font-weight: 600; letter-spacing: 0.03em;
+}
+.badge-ok { background: rgba(72,187,120,0.15); color: var(--success); }
+.badge-err { background: rgba(252,129,129,0.15); color: var(--danger); animation: pulse 1.5s infinite; }
+.badge-info { background: rgba(79,156,249,0.12); color: var(--accent); }
+.badge-mode { cursor: pointer; transition: background 0.2s; }
+.badge-mode:hover { opacity: 0.8; }
+.mode-sender { background: rgba(72,187,120,0.2); color: var(--success); }
+.mode-responder { background: rgba(246,173,85,0.2); color: var(--warning); }
+@keyframes pulse { 0%,100% { opacity: 1; } 50% { opacity: 0.5; } }
+</style>

traffic-gen-ui/src/components/TestBuilder.vue

@@ -0,0 +1,166 @@
+<template>
+  <div class="test-builder">
+    <h3>RFC 2544 Test</h3>
+
+    <div class="form-row">
+      <label>Test Type</label>
+      <select v-model="form.type">
+        <option value="throughput">Throughput (binary search for max rate)</option>
+        <option value="latency">Latency (measure RTT)</option>
+        <option value="frame_loss">Frame Loss (loss vs rate curve)</option>
+        <option value="back_to_back">Back-to-Back (max burst)</option>
+      </select>
+    </div>
+
+    <div class="form-row">
+      <label>Destination IP</label>
+      <input v-model="form.dst_ip" placeholder="10.100.0.1" />
+    </div>
+
+    <div class="form-row-pair">
+      <div class="form-row">
+        <label>Protocol</label>
+        <select v-model="form.protocol">
+          <option value="udp">UDP</option>
+          <option value="icmp">ICMP</option>
+          <option value="tcp">TCP</option>
+        </select>
+      </div>
+      <div class="form-row">
+        <label>Source IP</label>
+        <input v-model="form.src_ip" placeholder="auto" />
+      </div>
+    </div>
+
+    <div class="form-row">
+      <label>Frame Sizes</label>
+      <div class="frame-sizes">
+        <label v-for="s in standardSizes" :key="s" class="checkbox-label">
+          <input type="checkbox" :value="s" v-model="form.frame_sizes" /> {{ s }}
+        </label>
+      </div>
+    </div>
+
+    <div class="form-row-pair">
+      <div class="form-row">
+        <label>Trial Duration (sec)</label>
+        <input v-model.number="form.trial_duration" type="number" min="5" max="300" />
+      </div>
+      <div class="form-row">
+        <label>Max Rate</label>
+        <input v-model.number="form.max_rate_val" type="number" min="1" step="any" />
+        <select v-model="form.max_rate_unit" class="rate-unit-standalone">
+          <option value="pps">pps</option>
+          <option value="kbps">Kbps</option>
+          <option value="mbps">Mbps</option>
+        </select>
+      </div>
+    </div>
+
+    <div class="form-row">
+      <label>Acceptable Loss %</label>
+      <input v-model.number="form.acceptable_loss_pct" type="number" min="0" max="100" step="0.1" />
+    </div>
+
+    <button class="btn btn-accent" @click="create" :disabled="!form.dst_ip">
+      Create & Run Test
+    </button>
+
+    <div class="presets-section">
+      <h4>Quick Presets</h4>
+      <div class="preset-list">
+        <button v-for="(p, name) in presets" :key="name" class="btn-preset" @click="loadPreset(name)">
+          <strong>{{ name }}</strong>
+          <span>{{ p.description }}</span>
+        </button>
+      </div>
+    </div>
+  </div>
+</template>
+
+<script setup>
+import { reactive, ref, onMounted } from 'vue'
+import { api } from '../api.js'
+
+const emit = defineEmits(['created', 'refresh'])
+
+const standardSizes = [64, 128, 256, 512, 1024, 1280, 1518, 2048, 4096, 9000]
+const presets = ref({})
+
+const form = reactive({
+  type: 'throughput',
+  dst_ip: '',
+  src_ip: '',
+  protocol: 'udp',
+  frame_sizes: [64, 512, 1518],
+  trial_duration: 30,
+  max_rate_val: 10,
+  max_rate_unit: 'mbps',
+  acceptable_loss_pct: 0.0,
+})
+
+function computePps(val, unit) {
+  if (unit === 'kbps') return Math.max(1, Math.round((val * 1000) / (512 * 8)))
+  if (unit === 'mbps') return Math.max(1, Math.round((val * 1_000_000) / (512 * 8)))
+  return Math.round(val)
+}
+
+onMounted(async () => {
+  try { const r = await api.presets(); presets.value = r.presets || r } catch (_) {}
+})
+
+async function create() {
+  try {
+    const payload = {
+      type: form.type,
+      flow_config: {
+        dst_ip: form.dst_ip,
+        src_ip: form.src_ip || 'auto',
+        protocol: form.protocol,
+        src_port: 50000,
+        dst_port: 5001,
+      },
+      frame_sizes: form.frame_sizes,
+      trial_duration: form.trial_duration,
+      max_rate_pps: computePps(form.max_rate_val, form.max_rate_unit),
+      acceptable_loss_pct: form.acceptable_loss_pct,
+    }
+    const test = await api.createTest(payload)
+    await api.startTest(test.id)
+    emit('created')
+  } catch (e) { alert(e.message) }
+}
+
+async function loadPreset(name) {
+  const dstIp = prompt('Destination IP for this preset:', '10.100.0.100')
+  if (!dstIp) return
+  try {
+    await api.loadPreset(name, { dst_ip: dstIp })
+    emit('refresh')
+  } catch (e) { alert(e.message) }
+}
+</script>
+
+<style scoped>
+h3 { font-size: 15px; margin-bottom: 12px; color: var(--accent); }
+h4 { font-size: 13px; margin: 16px 0 8px; color: var(--muted); }
+.form-row { margin-bottom: 10px; }
+.form-row label { display: block; font-size: 11px; color: var(--muted); margin-bottom: 3px; text-transform: uppercase; letter-spacing: 0.05em; }
+.form-row input, .form-row select { width: 100%; }
+.form-row-pair { display: grid; grid-template-columns: 1fr 1fr; gap: 8px; }
+.rate-unit-standalone { width: 100%; margin-top: 4px; }
+.frame-sizes { display: flex; flex-wrap: wrap; gap: 8px; }
+.checkbox-label { font-size: 12px; display: flex; align-items: center; gap: 4px; color: var(--text); cursor: pointer; }
+.btn { padding: 8px 16px; font-weight: 600; font-size: 13px; width: 100%; margin-top: 8px; }
+.btn-accent { background: var(--accent); color: #fff; }
+.btn-accent:disabled { opacity: 0.4; }
+.preset-list { display: flex; flex-direction: column; gap: 6px; }
+.btn-preset {
+  display: flex; flex-direction: column; align-items: flex-start;
+  padding: 8px 12px; background: var(--card-bg); border: 1px solid var(--border);
+  border-radius: var(--radius); text-align: left;
+}
+.btn-preset:hover { border-color: var(--accent); }
+.btn-preset strong { font-size: 12px; color: var(--accent); }
+.btn-preset span { font-size: 11px; color: var(--muted); }
+</style>

traffic-gen-ui/src/components/TestRunner.vue

@@ -0,0 +1,195 @@
+<template>
+  <div class="test-runner">
+    <h3>Running Tests</h3>
+    <div v-if="!tests.length" class="empty">No tests yet. Create one above and click "Create & Run Test".</div>
+
+    <div v-for="t in sortedTests" :key="t.id" class="test-card" :class="'state-' + t.state">
+      <div class="test-header">
+        <div class="test-title">
+          <strong>{{ t.type }}</strong>
+          <span class="test-state" :class="'ts-' + t.state">{{ t.state }}</span>
+          <span v-if="t.frame_sizes" class="test-sizes">{{ t.frame_sizes.length }} frame sizes</span>
+        </div>
+        <div class="test-actions">
+          <button v-if="t.state === 'idle'" class="btn-sm btn-go" @click="start(t.id)">Start</button>
+          <button v-if="t.state === 'running'" class="btn-sm btn-stop" @click="stop(t.id)">Stop</button>
+          <button v-if="t.state === 'complete' || t.state === 'error'" class="btn-sm btn-del" @click="del(t.id)">Remove</button>
+        </div>
+      </div>
+
+      <!-- RUNNING: live progress -->
+      <div v-if="t.state === 'running'" class="progress-section">
+        <div class="progress-detail">
+          <span v-if="t.progress">{{ t.progress.message }}</span>
+          <span v-else>Starting...</span>
+          <span v-if="t.progress" class="progress-counter">
+            {{ (t.progress.completed_sizes || []).length }}/{{ t.progress.total_frames }} sizes done
+          </span>
+        </div>
+        <div class="progress-bar">
+          <div class="progress-fill" :style="{ width: progressPct(t) + '%' }"></div>
+        </div>
+        <!-- Show partial results as they come in -->
+        <div v-if="t.results && Object.keys(t.results).length" class="partial-results">
+          <div v-for="(val, size) in t.results" :key="size" class="partial-item">
+            <span class="partial-size">{{ size }}B</span>
+            <span class="partial-val" v-if="!Array.isArray(val) && val.max_throughput_pps != null">{{ val.max_throughput_pps }} pps</span>
+            <span class="partial-val" v-else-if="!Array.isArray(val) && val.avg_ms != null">{{ val.avg_ms }}ms avg</span>
+            <span class="partial-val" v-else-if="!Array.isArray(val) && val.max_burst_frames != null">{{ val.max_burst_frames }} frames</span>
+            <span class="partial-val" v-else-if="Array.isArray(val)">{{ val.length }} rate steps</span>
+          </div>
+        </div>
+      </div>
+
+      <!-- ERROR -->
+      <div v-if="t.state === 'error'" class="error-msg">{{ t.error || 'Test failed' }}</div>
+
+      <!-- COMPLETE: inline results summary -->
+      <div v-if="t.state === 'complete' && t.results && Object.keys(t.results).length" class="results-preview">
+        <!-- Frame Loss has a different structure (array per size) -->
+        <template v-if="t.type === 'frame_loss'">
+          <div v-for="(rates, size) in t.results" :key="size" class="fl-section">
+            <div class="fl-title">Frame Size: {{ size }} B</div>
+            <table>
+              <thead>
+                <tr>
+                  <th>Rate %</th>
+                  <th>Rate (pps)</th>
+                  <th>TX Packets</th>
+                  <th>RX Packets</th>
+                  <th>Loss %</th>
+                </tr>
+              </thead>
+              <tbody>
+                <tr v-for="r in rates" :key="r.rate_pct">
+                  <td class="mono">{{ r.rate_pct }}%</td>
+                  <td class="mono">{{ r.rate_pps }}</td>
+                  <td class="mono">{{ r.tx_packets }}</td>
+                  <td class="mono">{{ r.rx_packets }}</td>
+                  <td class="mono">{{ r.loss_pct }}%</td>
+                </tr>
+              </tbody>
+            </table>
+          </div>
+        </template>
+        <!-- Other test types: single value per size -->
+        <table v-else>
+          <thead>
+            <tr>
+              <th>Frame Size</th>
+              <th v-if="t.type === 'throughput'">Max Rate (pps)</th>
+              <th v-if="t.type === 'throughput'">Throughput</th>
+              <th v-if="t.type === 'latency'">Avg (ms)</th>
+              <th v-if="t.type === 'latency'">Min/Max (ms)</th>
+              <th v-if="t.type === 'back_to_back'">Max Burst</th>
+            </tr>
+          </thead>
+          <tbody>
+            <tr v-for="(val, size) in t.results" :key="size">
+              <td>{{ size }} B</td>
+              <td v-if="t.type === 'throughput'" class="mono">{{ val.max_throughput_pps || '-' }}</td>
+              <td v-if="t.type === 'throughput'" class="mono">{{ formatMbps(val) }}</td>
+              <td v-if="t.type === 'latency'" class="mono">{{ val.avg_ms != null ? val.avg_ms.toFixed(2) : '-' }}</td>
+              <td v-if="t.type === 'latency'" class="mono">{{ val.min_ms != null ? val.min_ms.toFixed(2) + ' / ' + val.max_ms.toFixed(2) : '-' }}</td>
+              <td v-if="t.type === 'back_to_back'" class="mono">{{ val.max_burst_frames ?? '-' }}</td>
+            </tr>
+          </tbody>
+        </table>
+      </div>
+
+      <div class="test-meta">
+        <span v-if="t.started_at">Started: {{ t.started_at }}</span>
+        <span v-if="t.completed_at">Completed: {{ t.completed_at }}</span>
+        <span v-if="t.state === 'running' && t.started_at">Elapsed: {{ elapsed(t) }}</span>
+      </div>
+    </div>
+  </div>
+</template>
+
+<script setup>
+import { computed } from 'vue'
+import { api } from '../api.js'
+
+const props = defineProps({ tests: Array })
+const emit = defineEmits(['refresh'])
+
+const sortedTests = computed(() => {
+  const order = { running: 0, idle: 1, complete: 2, error: 3 }
+  return [...(props.tests || [])].sort((a, b) => (order[a.state] ?? 9) - (order[b.state] ?? 9))
+})
+
+function progressPct(t) {
+  if (!t.progress || !t.progress.total_frames) return 10
+  const done = (t.progress.completed_sizes || []).length
+  const partial = t.progress.frame_idx > done ? 0.5 : 0
+  return Math.min(95, ((done + partial) / t.progress.total_frames) * 100)
+}
+
+function formatMbps(val) {
+  const pps = val.max_throughput_pps || 0
+  const fs = val.frame_size || 64
+  if (!pps) return '-'
+  const mbps = (pps * fs * 8) / 1_000_000
+  return mbps.toFixed(1) + ' Mbps'
+}
+
+function elapsed(t) {
+  if (!t.started_at) return ''
+  const start = new Date(t.started_at).getTime()
+  const secs = Math.round((Date.now() - start) / 1000)
+  const m = Math.floor(secs / 60)
+  const s = secs % 60
+  return m > 0 ? `${m}m ${s}s` : `${s}s`
+}
+
+async function start(id) {
+  try { await api.startTest(id); emit('refresh') } catch (e) { alert(e.message) }
+}
+async function stop(id) {
+  try { await api.stopTest(id); emit('refresh') } catch (e) { alert(e.message) }
+}
+async function del(id) {
+  emit('refresh')
+}
+</script>
+
+<style scoped>
+h3 { font-size: 15px; margin-bottom: 12px; color: var(--accent); }
+.empty { color: var(--muted); padding: 16px; text-align: center; font-size: 13px; }
+.test-card { background: var(--card-bg); border: 1px solid var(--border); border-radius: var(--radius); padding: 12px; margin-bottom: 10px; }
+.test-card.state-running { border-color: var(--accent); }
+.test-card.state-complete { border-color: var(--success); }
+.test-card.state-error { border-color: var(--danger); }
+.test-header { display: flex; justify-content: space-between; align-items: center; }
+.test-title { display: flex; align-items: center; gap: 8px; }
+.test-title strong { font-size: 14px; text-transform: capitalize; }
+.test-state { font-size: 11px; padding: 2px 8px; border-radius: 10px; font-weight: 600; }
+.ts-idle { background: rgba(113,128,150,0.2); color: var(--muted); }
+.ts-running { background: rgba(79,156,249,0.15); color: var(--accent); }
+.ts-complete { background: rgba(72,187,120,0.15); color: var(--success); }
+.ts-error { background: rgba(252,129,129,0.15); color: var(--danger); }
+.test-sizes { font-size: 11px; color: var(--muted); }
+.test-actions { display: flex; gap: 4px; }
+.btn-sm { padding: 3px 10px; font-size: 11px; font-weight: 600; border-radius: 6px; }
+.btn-go { background: var(--success); color: #fff; }
+.btn-stop { background: var(--warning); color: #000; }
+.btn-del { background: rgba(252,129,129,0.15); color: var(--danger); }
+.progress-section { margin: 10px 0; }
+.progress-detail { display: flex; justify-content: space-between; font-size: 12px; color: var(--muted); margin-bottom: 6px; font-family: monospace; }
+.progress-counter { color: var(--accent); font-weight: 600; }
+.progress-bar { height: 6px; background: var(--border); border-radius: 3px; overflow: hidden; }
+.progress-fill { height: 100%; background: var(--accent); border-radius: 3px; transition: width 0.5s; }
+.partial-results { display: flex; gap: 8px; flex-wrap: wrap; margin-top: 8px; }
+.partial-item { background: rgba(72,187,120,0.1); border: 1px solid rgba(72,187,120,0.2); padding: 2px 8px; border-radius: 6px; font-size: 11px; }
+.partial-size { font-weight: 600; color: var(--success); }
+.partial-val { color: var(--text); margin-left: 4px; font-family: monospace; }
+.error-msg { color: var(--danger); font-size: 13px; padding: 8px 0; }
+.results-preview { margin-top: 10px; }
+.results-preview table { width: 100%; border-collapse: collapse; }
+.results-preview th { font-size: 11px; color: var(--muted); text-align: left; padding: 4px 8px; border-bottom: 1px solid var(--border); }
+.results-preview td { font-size: 13px; padding: 4px 8px; }
+.mono { font-family: monospace; }
+.fl-section { margin-bottom: 12px; }
+.fl-title { font-size: 12px; font-weight: 600; color: var(--accent); margin-bottom: 4px; }
+.test-meta { display: flex; gap: 12px; margin-top: 8px; font-size: 11px; color: var(--muted); }
+</style>

traffic-gen-ui/src/main.js

@@ -0,0 +1,3 @@
+import { createApp } from 'vue'
+import App from './App.vue'
+createApp(App).mount('#app')

traffic-gen-ui/vite.config.js

@@ -0,0 +1,15 @@
+import { defineConfig } from 'vite'
+import vue from '@vitejs/plugin-vue'
+
+export default defineConfig({
+  base: '/traffic/',
+  plugins: [vue()],
+  server: {
+    proxy: {
+      '/api': {
+        target: 'http://localhost:5051',
+        rewrite: path => path.replace(/^\/api/, '')
+      }
+    }
+  }
+})

traffic-gen/Dockerfile

@@ -0,0 +1,8 @@
+FROM python:3.11-slim
+RUN apt-get update && apt-get install -y --no-install-recommends \
+    tcpreplay libpcap-dev procps iputils-ping && rm -rf /var/lib/apt/lists/*
+RUN pip install --no-cache-dir flask scapy psutil
+COPY . /traffic-gen/
+WORKDIR /traffic-gen
+EXPOSE 5051
+CMD ["python3", "server.py"]

traffic-gen/engine/__init__.py

@@ -0,0 +1 @@
+# traffic-gen engine package

traffic-gen/engine/packet_builder.py

@@ -0,0 +1,120 @@
+"""
+Packet Builder - constructs Scapy packets from flow configuration.
+
+Each generated packet embeds:
+  - Magic bytes b'TGEN' (4 bytes)
+  - Sequence number (4 bytes, big-endian)
+  - Sender timestamp in nanoseconds (8 bytes, big-endian)
+  - Padding to reach requested frame_size
+"""
+
+import struct
+import time
+
+from scapy.all import (
+    Ether, IP, UDP, TCP, ICMP, Dot1Q, Raw, conf,
+)
+
+MAGIC = b'TGEN'
+HEADER_LEN = 4 + 4 + 8  # magic + seq + timestamp_ns
+
+
+def _build_payload(seq: int, frame_size: int, header_overhead: int) -> Raw:
+    """Build payload with magic bytes, sequence number, timestamp placeholder,
+    and padding to reach the desired frame_size."""
+    timestamp_ns = time.time_ns()
+    header = MAGIC + struct.pack('!I', seq) + struct.pack('!Q', timestamp_ns)
+    # frame_size includes Ethernet header (14) + FCS (4) in standard accounting,
+    # but Scapy doesn't add FCS, so we target frame_size - 4 total bytes on wire.
+    # header_overhead accounts for Ether + IP + L4 headers already present.
+    pad_len = max(0, frame_size - 4 - header_overhead - HEADER_LEN)
+    return Raw(load=header + (b'\x00' * pad_len))
+
+
+def stamp_payload(payload_bytes: bytes, seq: int) -> bytes:
+    """Re-stamp an existing payload with a new sequence number and fresh timestamp."""
+    timestamp_ns = time.time_ns()
+    return (
+        MAGIC
+        + struct.pack('!I', seq)
+        + struct.pack('!Q', timestamp_ns)
+        + payload_bytes[HEADER_LEN:]
+    )
+
+
+def parse_payload(payload_bytes: bytes):
+    """Extract (seq, timestamp_ns) from a TGEN payload, or None if invalid."""
+    if len(payload_bytes) < HEADER_LEN:
+        return None
+    if payload_bytes[:4] != MAGIC:
+        return None
+    seq = struct.unpack('!I', payload_bytes[4:8])[0]
+    timestamp_ns = struct.unpack('!Q', payload_bytes[8:16])[0]
+    return seq, timestamp_ns
+
+
+def build_packet(flow_config: dict, seq: int = 0):
+    """Build a Scapy packet from a flow configuration dict.
+
+    Required keys:
+        dst_ip, protocol
+
+    Optional keys:
+        src_mac, dst_mac, src_ip, src_port, dst_port, dscp, vlan_id, frame_size
+    """
+    protocol = flow_config.get('protocol', 'udp').lower()
+    frame_size = flow_config.get('frame_size', 512)
+
+    # --- Layer 2 ---
+    src_mac = flow_config.get('src_mac', 'auto')
+    dst_mac = flow_config.get('dst_mac')
+
+    ether_kwargs = {}
+    if src_mac and src_mac != 'auto':
+        ether_kwargs['src'] = src_mac
+    if dst_mac:
+        ether_kwargs['dst'] = dst_mac
+
+    pkt = Ether(**ether_kwargs)
+    header_overhead = 14  # Ethernet
+
+    # --- VLAN ---
+    vlan_id = flow_config.get('vlan_id')
+    if vlan_id is not None:
+        pkt = pkt / Dot1Q(vlan=int(vlan_id))
+        header_overhead += 4
+
+    # --- Layer 3 ---
+    ip_kwargs = {'dst': flow_config['dst_ip']}
+    src_ip = flow_config.get('src_ip')
+    if src_ip and src_ip != 'auto':
+        ip_kwargs['src'] = src_ip
+
+    dscp = flow_config.get('dscp', 0)
+    if dscp:
+        ip_kwargs['tos'] = int(dscp) << 2
+
+    pkt = pkt / IP(**ip_kwargs)
+    header_overhead += 20  # IP (no options)
+
+    # --- Layer 4 ---
+    if protocol == 'udp':
+        src_port = flow_config.get('src_port') or 12000
+        dst_port = flow_config.get('dst_port') or 5001
+        pkt = pkt / UDP(sport=int(src_port), dport=int(dst_port))
+        header_overhead += 8
+    elif protocol == 'tcp':
+        src_port = flow_config.get('src_port') or 12000
+        dst_port = flow_config.get('dst_port') or 80
+        pkt = pkt / TCP(sport=int(src_port), dport=int(dst_port), flags='S')
+        header_overhead += 20
+    elif protocol == 'icmp':
+        pkt = pkt / ICMP()
+        header_overhead += 8
+    else:
+        raise ValueError(f'Unsupported protocol: {protocol}')
+
+    # --- Payload ---
+    pkt = pkt / _build_payload(seq, frame_size, header_overhead)
+
+    return pkt

traffic-gen/engine/responder.py

@@ -0,0 +1,204 @@
+"""
+Responder - high-performance UDP packet receiver for TGEN traffic.
+
+Uses multiple receiver threads on SO_REUSEPORT UDP sockets for parallel
+packet processing. Each thread has its own socket and stats counters
+to avoid contention.
+"""
+
+import logging
+import os
+import socket
+import struct
+import threading
+import time
+
+from engine.packet_builder import MAGIC, HEADER_LEN
+
+log = logging.getLogger(__name__)
+
+DEFAULT_LISTEN_PORT = 5001
+RECV_BUF_SIZE = 16 * 1024 * 1024
+NUM_WORKERS = int(os.environ.get('RESPONDER_WORKERS', 4))
+
+
+class _WorkerStats:
+    """Per-worker stats — no sharing, no locks."""
+    __slots__ = ('rx_packets', 'rx_bytes', 'out_of_order', 'duplicates',
+                 'last_seq', 'lat_buf', 'lat_idx', 'lat_count')
+
+    def __init__(self):
+        self.rx_packets = 0
+        self.rx_bytes = 0
+        self.out_of_order = 0
+        self.duplicates = 0
+        self.last_seq = -1
+        self.lat_buf = [0.0] * 4096
+        self.lat_idx = 0
+        self.lat_count = 0
+
+
+class Responder:
+    def __init__(self, mode: str = 'log', listen_port: int = DEFAULT_LISTEN_PORT):
+        self._mode = mode
+        self._listen_port = listen_port
+        self._sockets = []
+        self._threads = []
+        self._workers = []
+        self._running = False
+        self._stop_event = threading.Event()
+
+    def start(self, interface: str = None):
+        if self._running:
+            return
+
+        self._stop_event.clear()
+        n = NUM_WORKERS
+
+        for i in range(n):
+            sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
+            sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
+            sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEPORT, 1)
+            try:
+                sock.setsockopt(socket.SOL_SOCKET, socket.SO_RCVBUF, RECV_BUF_SIZE)
+            except OSError:
+                pass
+            sock.settimeout(0.5)
+            sock.bind(('0.0.0.0', self._listen_port))
+            self._sockets.append(sock)
+
+            ws = _WorkerStats()
+            self._workers.append(ws)
+
+            t = threading.Thread(target=self._recv_loop, args=(sock, ws),
+                                 daemon=True, name=f'responder-rx-{i}')
+            self._threads.append(t)
+            t.start()
+
+        actual_buf = self._sockets[0].getsockopt(socket.SOL_SOCKET, socket.SO_RCVBUF)
+        self._running = True
+        log.info('Responder started on port=%d mode=%s workers=%d rcvbuf=%d',
+                 self._listen_port, self._mode, n, actual_buf)
+
+    def stop(self):
+        self._stop_event.set()
+        for t in self._threads:
+            if t.is_alive():
+                t.join(timeout=3)
+        for s in self._sockets:
+            try:
+                s.close()
+            except Exception:
+                pass
+        self._sockets.clear()
+        self._threads.clear()
+        self._workers.clear()
+        self._running = False
+        log.info('Responder stopped')
+
+    def is_running(self) -> bool:
+        return self._running
+
+    def get_stats(self) -> dict:
+        rx_packets = 0
+        rx_bytes = 0
+        out_of_order = 0
+        duplicates = 0
+        all_lat = []
+
+        for ws in self._workers:
+            rx_packets += ws.rx_packets
+            rx_bytes += ws.rx_bytes
+            out_of_order += ws.out_of_order
+            duplicates += ws.duplicates
+            n = min(ws.lat_count, len(ws.lat_buf))
+            if n > 0:
+                all_lat.extend(ws.lat_buf[:n] if ws.lat_count <= len(ws.lat_buf) else ws.lat_buf[:])
+
+        latency = {}
+        if all_lat:
+            avg = sum(all_lat) / len(all_lat)
+            mn = min(all_lat)
+            mx = max(all_lat)
+            jitter = 0.0
+            if len(all_lat) > 1:
+                jitter = sum(abs(all_lat[i] - all_lat[i-1]) for i in range(1, len(all_lat))) / (len(all_lat) - 1)
+            latency = {
+                'min_ms': round(mn, 3),
+                'max_ms': round(mx, 3),
+                'avg_ms': round(avg, 3),
+                'jitter_ms': round(jitter, 3),
+                'samples': len(all_lat),
+            }
+
+        return {
+            'rx_packets': rx_packets,
+            'rx_bytes': rx_bytes,
+            'out_of_order': out_of_order,
+            'duplicates': duplicates,
+            'latency': latency,
+            'running': self._running,
+        }
+
+    def reset_stats(self):
+        for ws in self._workers:
+            ws.rx_packets = 0
+            ws.rx_bytes = 0
+            ws.last_seq = -1
+            ws.out_of_order = 0
+            ws.duplicates = 0
+            ws.lat_idx = 0
+            ws.lat_count = 0
+        log.info('Responder stats reset')
+
+    def _recv_loop(self, sock, ws):
+        """Per-worker receive loop."""
+        echo = self._mode == 'echo'
+        recvfrom = sock.recvfrom
+        time_ns = time.time_ns
+        stop_is_set = self._stop_event.is_set
+        lat_buf = ws.lat_buf
+        lat_buf_len = len(lat_buf)
+        magic = MAGIC
+
+        while not stop_is_set():
+            try:
+                data, addr = recvfrom(65535)
+            except socket.timeout:
+                continue
+            except OSError:
+                if stop_is_set():
+                    break
+                raise
+
+            rx_ns = time_ns()
+            dlen = len(data)
+
+            if dlen < HEADER_LEN or data[:4] != magic:
+                continue
+
+            seq = int.from_bytes(data[4:8], 'big')
+            sender_ns = int.from_bytes(data[8:16], 'big')
+
+            ws.rx_packets += 1
+            ws.rx_bytes += dlen
+
+            last = ws.last_seq
+            if seq == last:
+                ws.duplicates += 1
+            elif seq < last:
+                ws.out_of_order += 1
+            ws.last_seq = seq
+
+            lat_ms = (rx_ns - sender_ns) / 1_000_000
+            if 0 < lat_ms < 60000:
+                idx = ws.lat_idx
+                lat_buf[idx] = lat_ms
+                ws.lat_idx = (idx + 1) % lat_buf_len
+                ws.lat_count += 1
+
+            if echo:
+                try:
+                    sock.sendto(data + struct.pack('!Q', rx_ns), addr)
+                except Exception:
+                    pass

traffic-gen/engine/rfc2544.py

@@ -0,0 +1,435 @@
+"""
+RFC 2544 test implementations:
+  - ThroughputTest: binary search for max zero-loss throughput
+  - LatencyTest: measure latency at a given rate
+  - FrameLossTest: measure loss at decreasing rates
+  - BackToBackTest: find max burst length with zero loss
+"""
+
+import logging
+import socket
+import struct
+import threading
+import time
+import urllib.request
+import json
+from typing import Dict, List, Optional
+
+from scapy.all import send, sr, conf, IP, ICMP
+
+from engine.packet_builder import build_packet, parse_payload, MAGIC
+
+log = logging.getLogger(__name__)
+conf.verb = 0
+
+
+class _BaseTest:
+    """Base class for RFC 2544 tests."""
+
+    def __init__(self, test_id: str, flow_config: dict, frame_sizes: List[int],
+                 trial_duration: float = 60, max_rate_pps: int = 10000,
+                 acceptable_loss_pct: float = 0.0, responder_url: str = None):
+        self.test_id = test_id
+        self.flow_config = dict(flow_config)
+        self.frame_sizes = frame_sizes
+        self.trial_duration = trial_duration
+        self.max_rate_pps = max_rate_pps
+        self.acceptable_loss_pct = acceptable_loss_pct
+        self.responder_url = responder_url  # e.g. "http://172.30.0.10:5053"
+
+        self.state = 'idle'      # idle -> running -> complete/error
+        self.results = {}
+        self.error = None
+        self.started_at = None
+        self.completed_at = None
+
+        # Progress tracking
+        self._progress_msg = ''
+        self._current_frame_idx = 0
+        self._current_trial_tx = 0
+
+        self._thread: Optional[threading.Thread] = None
+        self._stop_event = threading.Event()
+        self._lock = threading.Lock()
+
+    def start(self):
+        if self.state == 'running':
+            return
+        self._stop_event.clear()
+        self.state = 'running'
+        self.started_at = time.strftime('%Y-%m-%dT%H:%M:%SZ', time.gmtime())
+        self.results = {}
+        self.error = None
+        self._thread = threading.Thread(target=self._run_safe, daemon=True,
+                                        name=f'test-{self.test_id[:8]}')
+        self._thread.start()
+
+    def stop(self):
+        self._stop_event.set()
+        if self._thread and self._thread.is_alive():
+            self._thread.join(timeout=self.trial_duration + 5)
+        if self.state == 'running':
+            self.state = 'error'
+            self.error = 'Cancelled by user'
+
+    def _run_safe(self):
+        try:
+            self._run()
+            if self.state == 'running':
+                self.state = 'complete'
+        except Exception as e:
+            log.error('Test %s error: %s', self.test_id[:8], e)
+            self.state = 'error'
+            self.error = str(e)
+        finally:
+            self.completed_at = time.strftime('%Y-%m-%dT%H:%M:%SZ', time.gmtime())
+
+    def _run(self):
+        raise NotImplementedError
+
+    def _is_stopped(self) -> bool:
+        return self._stop_event.is_set()
+
+    def _responder_reset(self):
+        """Reset responder stats before a trial."""
+        if not self.responder_url:
+            return
+        try:
+            req = urllib.request.Request(
+                f'{self.responder_url}/responder/reset', method='POST',
+                data=b'{}', headers={'Content-Type': 'application/json'})
+            urllib.request.urlopen(req, timeout=3)
+        except Exception as e:
+            log.warning('Responder reset failed: %s', e)
+
+    def _responder_stats(self) -> Optional[dict]:
+        """Query responder for rx stats after a trial."""
+        if not self.responder_url:
+            return None
+        try:
+            req = urllib.request.Request(f'{self.responder_url}/responder/stats')
+            resp = urllib.request.urlopen(req, timeout=5)
+            return json.loads(resp.read())
+        except Exception as e:
+            log.warning('Responder stats query failed: %s', e)
+            return None
+
+    def _send_trial(self, frame_size: int, rate_pps: int, duration: float):
+        """Send packets at a given rate for a duration. Returns (tx_count, rx_count, latencies)."""
+        flow = dict(self.flow_config)
+        flow['frame_size'] = frame_size
+        protocol = flow.get('protocol', 'udp').lower()
+
+        # Reset responder counters before trial
+        self._responder_reset()
+
+        tx_count = 0
+        rx_count = 0
+        latencies = []
+        start = time.time()
+
+        if protocol == 'icmp':
+            # ICMP: use sr() to measure latency from responses
+            seq = 0
+            while (time.time() - start) < duration and not self._is_stopped():
+                pkt = build_packet(flow, seq=seq)
+                seq += 1
+                answered, _ = sr(pkt[IP], timeout=1, verbose=0)
+                tx_count += 1
+                for sent_pkt, recv_pkt in answered:
+                    rx_count += 1
+                    rtt_ms = (recv_pkt.time - sent_pkt.sent_time) * 1000
+                    latencies.append(rtt_ms)
+                elapsed = time.time() - start
+                expected = elapsed * rate_pps
+                if tx_count > expected:
+                    sleep_time = (tx_count - expected) / rate_pps
+                    if sleep_time > 0:
+                        self._stop_event.wait(min(sleep_time, 0.1))
+        else:
+            # UDP/TCP: high-performance raw socket path
+            dst_ip = flow['dst_ip']
+            pkt_template = build_packet(flow, seq=0)
+            ip_template = bytes(pkt_template[pkt_template.firstlayer().payload.__class__])
+            magic_offset = ip_template.find(MAGIC)
+
+            # Find and zero UDP checksum in template so receivers accept packets
+            # IP header length from IHL field (byte 0, low nibble) * 4
+            ip_ihl = (ip_template[0] & 0x0F) * 4
+            ip_proto = ip_template[9]  # protocol field
+            udp_csum_offset = ip_ihl + 6 if ip_proto == 17 else -1  # 17 = UDP
+
+            raw_sock = socket.socket(socket.AF_INET, socket.SOCK_RAW, socket.IPPROTO_RAW)
+            raw_sock.setsockopt(socket.IPPROTO_IP, socket.IP_HDRINCL, 1)
+
+            batch_size = max(1, min(rate_pps // 5, 500))
+            interval = batch_size / rate_pps if rate_pps > 0 else 1.0
+            seq = 0
+
+            try:
+                while (time.time() - start) < duration and not self._is_stopped():
+                    batch_start = time.time()
+                    for _ in range(batch_size):
+                        pkt_bytes = bytearray(ip_template)
+                        if magic_offset >= 0:
+                            struct.pack_into('!I', pkt_bytes, magic_offset + 4, seq)
+                            struct.pack_into('!Q', pkt_bytes, magic_offset + 8, time.time_ns())
+                        pkt_bytes[10:12] = b'\x00\x00'  # zero IP checksum
+                        if udp_csum_offset > 0:
+                            pkt_bytes[udp_csum_offset:udp_csum_offset + 2] = b'\x00\x00'
+                        try:
+                            raw_sock.sendto(bytes(pkt_bytes), (dst_ip, 0))
+                            tx_count += 1
+                        except Exception:
+                            pass
+                        seq += 1
+                    batch_elapsed = time.time() - batch_start
+                    sleep_time = interval - batch_elapsed
+                    if sleep_time > 0:
+                        self._stop_event.wait(sleep_time)
+            finally:
+                raw_sock.close()
+
+        # Query responder for actual rx stats (UDP/TCP path)
+        if protocol != 'icmp':
+            resp_stats = self._responder_stats()
+            if resp_stats and resp_stats.get('rx_packets', 0) > 0:
+                rx_count = resp_stats['rx_packets']
+                lat = resp_stats.get('latency', {})
+                if lat.get('samples', 0) > 0:
+                    latencies = [lat['avg_ms']]  # Use avg as representative
+
+        return tx_count, rx_count, latencies
+
+    def get_info(self) -> dict:
+        # Reverse-lookup the slug for this test class
+        type_slug = next((k for k, v in TEST_TYPES.items() if v is self.__class__), self.__class__.__name__)
+        info = {
+            'id': self.test_id,
+            'test_id': self.test_id,
+            'type': type_slug,
+            'state': self.state,
+            'results': self.results,
+            'error': self.error,
+            'frame_sizes': self.frame_sizes,
+            'started_at': self.started_at,
+            'completed_at': self.completed_at,
+        }
+        if self.state == 'running':
+            info['progress'] = {
+                'frame_idx': self._current_frame_idx,
+                'total_frames': len(self.frame_sizes),
+                'message': self._progress_msg,
+                'completed_sizes': list(self.results.keys()),
+            }
+        return info
+
+
+class ThroughputTest(_BaseTest):
+    """Binary search for maximum throughput with acceptable loss."""
+
+    def _run(self):
+        for idx, fs in enumerate(self.frame_sizes):
+            if self._is_stopped():
+                break
+
+            self._current_frame_idx = idx
+            low = 0
+            high = self.max_rate_pps
+            best_rate = 0
+            convergence_threshold = max(1, int(self.max_rate_pps * 0.01))
+            step = 0
+
+            log.info('Throughput test: frame_size=%d, searching [%d, %d] pps', fs, low, high)
+
+            while (high - low) > convergence_threshold and not self._is_stopped():
+                mid = (low + high) // 2
+                if mid == 0:
+                    break
+                step += 1
+
+                self._progress_msg = f'Frame {fs}B: trial {step}, testing {mid} pps [{low}-{high}]'
+                tx, rx, _ = self._send_trial(fs, mid, self.trial_duration)
+
+                if tx == 0:
+                    loss_pct = 100.0
+                elif rx > 0:
+                    loss_pct = ((tx - rx) / tx) * 100
+                else:
+                    loss_pct = 0.0  # No responder/ICMP — assume success
+
+                log.info('  frame=%d rate=%d tx=%d rx=%d loss=%.2f%%',
+                         fs, mid, tx, rx, loss_pct)
+
+                if loss_pct <= self.acceptable_loss_pct:
+                    best_rate = mid
+                    low = mid + 1
+                else:
+                    high = mid - 1
+
+            self.results[str(fs)] = {
+                'max_throughput_pps': best_rate,
+                'frame_size': fs,
+            }
+            log.info('Throughput result: frame_size=%d -> %d pps', fs, best_rate)
+
+
+class LatencyTest(_BaseTest):
+    """Measure latency at a specified rate."""
+
+    def _run(self):
+        rate = self.flow_config.get('rate_pps', 100)
+
+        for idx, fs in enumerate(self.frame_sizes):
+            if self._is_stopped():
+                break
+
+            self._current_frame_idx = idx
+            self._progress_msg = f'Frame {fs}B: sending at {rate} pps for {self.trial_duration}s'
+            log.info('Latency test: frame_size=%d at %d pps for %ds', fs, rate, self.trial_duration)
+            _, _, latencies = self._send_trial(fs, rate, self.trial_duration)
+
+            if latencies:
+                avg_ms = sum(latencies) / len(latencies)
+                min_ms = min(latencies)
+                max_ms = max(latencies)
+                jitter_ms = (
+                    sum(abs(latencies[i] - latencies[i - 1]) for i in range(1, len(latencies)))
+                    / max(1, len(latencies) - 1)
+                ) if len(latencies) > 1 else 0.0
+
+                self.results[str(fs)] = {
+                    'frame_size': fs,
+                    'min_ms': round(min_ms, 3),
+                    'avg_ms': round(avg_ms, 3),
+                    'max_ms': round(max_ms, 3),
+                    'jitter_ms': round(jitter_ms, 3),
+                    'samples': len(latencies),
+                }
+            else:
+                self.results[str(fs)] = {
+                    'frame_size': fs,
+                    'min_ms': None, 'avg_ms': None, 'max_ms': None,
+                    'jitter_ms': None, 'samples': 0,
+                    'note': 'No responses received (use ICMP or configure responder)',
+                }
+            log.info('Latency result: frame_size=%d -> %s', fs, self.results[str(fs)])
+
+
+class FrameLossTest(_BaseTest):
+    """Measure frame loss at decreasing rates (100%, 90%, 80%, ...)."""
+
+    def _run(self):
+        for idx, fs in enumerate(self.frame_sizes):
+            if self._is_stopped():
+                break
+
+            self._current_frame_idx = idx
+            results_for_size = []
+            for pct in range(100, 0, -10):
+                if self._is_stopped():
+                    break
+
+                rate = int(self.max_rate_pps * pct / 100)
+                if rate == 0:
+                    continue
+
+                self._progress_msg = f'Frame {fs}B: testing at {pct}% rate ({rate} pps)'
+                log.info('FrameLoss test: frame_size=%d rate=%d (%d%%)', fs, rate, pct)
+                tx, rx, _ = self._send_trial(fs, rate, self.trial_duration)
+
+                if tx > 0 and rx > 0:
+                    loss_pct = ((tx - rx) / tx) * 100
+                elif tx > 0 and rx == 0:
+                    loss_pct = 0.0  # No responder — cannot measure
+                else:
+                    loss_pct = 100.0
+
+                results_for_size.append({
+                    'rate_pct': pct,
+                    'rate_pps': rate,
+                    'tx_packets': tx,
+                    'rx_packets': rx,
+                    'loss_pct': round(loss_pct, 3),
+                })
+
+            self.results[str(fs)] = results_for_size
+
+
+class BackToBackTest(_BaseTest):
+    """Find maximum burst length with zero loss."""
+
+    def _run(self):
+        for idx, fs in enumerate(self.frame_sizes):
+            if self._is_stopped():
+                break
+
+            low = 1
+            high = self.max_rate_pps  # Use max_rate_pps as max burst length
+            best_burst = 0
+            convergence = max(1, high // 100)
+
+            self._current_frame_idx = idx
+            log.info('BackToBack test: frame_size=%d searching burst [%d, %d]', fs, low, high)
+
+            while (high - low) > convergence and not self._is_stopped():
+                mid = (low + high) // 2
+                if mid == 0:
+                    break
+
+                flow = dict(self.flow_config)
+                flow['frame_size'] = fs
+                protocol = flow.get('protocol', 'udp').lower()
+
+                # Send burst of 'mid' packets as fast as possible
+                tx_count = 0
+                rx_count = 0
+                for seq in range(mid):
+                    if self._is_stopped():
+                        break
+                    pkt = build_packet(flow, seq=seq)
+                    if protocol == 'icmp':
+                        answered, _ = sr(pkt[IP], timeout=0.5, verbose=0)
+                        tx_count += 1
+                        rx_count += len(answered)
+                    else:
+                        send(pkt[IP], verbose=0)
+                        tx_count += 1
+
+                if tx_count > 0 and rx_count > 0:
+                    loss_pct = ((tx_count - rx_count) / tx_count) * 100
+                elif tx_count > 0:
+                    loss_pct = 0.0  # No responder — cannot measure
+                else:
+                    loss_pct = 100.0
+
+                log.info('  burst=%d tx=%d rx=%d loss=%.2f%%', mid, tx_count, rx_count, loss_pct)
+
+                if loss_pct <= self.acceptable_loss_pct:
+                    best_burst = mid
+                    low = mid + 1
+                else:
+                    high = mid - 1
+
+            self.results[str(fs)] = {
+                'frame_size': fs,
+                'max_burst_frames': best_burst,
+            }
+            log.info('BackToBack result: frame_size=%d -> %d frames', fs, best_burst)
+
+
+# Factory function
+TEST_TYPES = {
+    'throughput': ThroughputTest,
+    'latency': LatencyTest,
+    'frame_loss': FrameLossTest,
+    'back_to_back': BackToBackTest,
+}
+
+
+def create_test(test_id: str, test_type: str, flow_config: dict, **kwargs):
+    """Create an RFC 2544 test instance by type name."""
+    cls = TEST_TYPES.get(test_type)
+    if cls is None:
+        raise ValueError(f'Unknown test type: {test_type}. Available: {list(TEST_TYPES)}')
+    return cls(test_id=test_id, flow_config=flow_config, **kwargs)

traffic-gen/engine/sender.py

@@ -0,0 +1,318 @@
+"""
+FlowSender - manages traffic generation with background threads per flow.
+"""
+
+import logging
+import shutil
+import socket
+import struct
+import threading
+import time
+import urllib.request
+import json
+
+from scapy.all import send, sendpfast, sr, conf
+
+from engine.packet_builder import build_packet, stamp_payload, MAGIC, HEADER_LEN
+
+log = logging.getLogger(__name__)
+
+# Suppress Scapy verbosity globally
+conf.verb = 0
+
+HAS_TCPREPLAY = shutil.which('tcpreplay') is not None
+
+
+class FlowSender:
+    """Manages sending threads for multiple flows."""
+
+    def __init__(self):
+        self._lock = threading.Lock()
+        self._flows = {}          # flow_id -> flow_config dict
+        self._threads = {}        # flow_id -> Thread
+        self._stop_events = {}    # flow_id -> Event
+        self._stats = {}          # flow_id -> {tx_packets, tx_bytes, ...}
+
+    # ------------------------------------------------------------------
+    # Flow CRUD
+    # ------------------------------------------------------------------
+
+    def add_flow(self, flow_id: str, flow_config: dict):
+        with self._lock:
+            self._flows[flow_id] = flow_config
+            self._stats[flow_id] = {
+                'tx_packets': 0, 'tx_bytes': 0,
+                'rx_packets': 0, 'rx_bytes': 0,
+                'latency_samples': [],
+            }
+
+    def get_flow(self, flow_id: str):
+        with self._lock:
+            return self._flows.get(flow_id)
+
+    def get_all_flows(self):
+        with self._lock:
+            return dict(self._flows)
+
+    def update_flow(self, flow_id: str, updates: dict):
+        with self._lock:
+            if flow_id not in self._flows:
+                return False
+            self._flows[flow_id].update(updates)
+            return True
+
+    def remove_flow(self, flow_id: str):
+        self.stop(flow_id)
+        with self._lock:
+            self._flows.pop(flow_id, None)
+            self._stats.pop(flow_id, None)
+
+    # ------------------------------------------------------------------
+    # Start / Stop
+    # ------------------------------------------------------------------
+
+    def start(self, flow_id: str):
+        with self._lock:
+            if flow_id not in self._flows:
+                raise KeyError(f'Flow {flow_id} not found')
+            if flow_id in self._threads and self._threads[flow_id].is_alive():
+                return  # already running
+            self._flows[flow_id]['state'] = 'running'
+            self._stats[flow_id] = {
+                'tx_packets': 0, 'tx_bytes': 0,
+                'rx_packets': 0, 'rx_bytes': 0,
+                'latency_samples': [],
+            }
+            stop_event = threading.Event()
+            self._stop_events[flow_id] = stop_event
+            t = threading.Thread(
+                target=self._send_loop,
+                args=(flow_id, stop_event),
+                daemon=True,
+                name=f'sender-{flow_id[:8]}',
+            )
+            self._threads[flow_id] = t
+            t.start()
+
+    def stop(self, flow_id: str):
+        with self._lock:
+            ev = self._stop_events.pop(flow_id, None)
+            if ev:
+                ev.set()
+            t = self._threads.pop(flow_id, None)
+            if flow_id in self._flows:
+                self._flows[flow_id]['state'] = 'stopped'
+        if t and t.is_alive():
+            t.join(timeout=5)
+
+    def is_running(self, flow_id: str) -> bool:
+        with self._lock:
+            t = self._threads.get(flow_id)
+            return t is not None and t.is_alive()
+
+    # ------------------------------------------------------------------
+    # Stats
+    # ------------------------------------------------------------------
+
+    def get_stats(self, flow_id: str) -> dict:
+        with self._lock:
+            s = self._stats.get(flow_id, {})
+            return dict(s)
+
+    def get_all_stats(self) -> dict:
+        with self._lock:
+            return {fid: dict(s) for fid, s in self._stats.items()}
+
+    # ------------------------------------------------------------------
+    # Internal send loop
+    # ------------------------------------------------------------------
+
+    def _send_loop(self, flow_id: str, stop_event: threading.Event):
+        with self._lock:
+            flow = dict(self._flows[flow_id])
+
+        rate_pps = flow.get('rate_pps', 1000)
+        duration = flow.get('duration', 30)
+        protocol = flow.get('protocol', 'udp').lower()
+        responder_url = flow.get('responder_url')
+        use_icmp_sr = (protocol == 'icmp' and not responder_url)
+
+        # Build template packet
+        pkt_template = build_packet(flow, seq=0)
+        pkt_bytes_len = len(bytes(pkt_template))
+
+        seq = 0
+        start_time = time.time()
+        last_responder_poll = 0
+        log.info('Flow %s: starting send loop at %d pps for %ds',
+                 flow_id[:8], rate_pps, duration)
+
+        # Capture responder baseline so we report deltas, not cumulative totals
+        responder_baseline_rx = 0
+        responder_baseline_bytes = 0
+        if responder_url:
+            try:
+                base = self._fetch_responder(responder_url)
+                responder_baseline_rx = base.get('rx_packets', 0)
+                responder_baseline_bytes = base.get('rx_bytes', 0)
+                # Also reset responder so baseline is clean
+                self._reset_responder(responder_url)
+                responder_baseline_rx = 0
+                responder_baseline_bytes = 0
+            except Exception:
+                pass
+
+        raw_sock = None
+        try:
+            if use_icmp_sr:
+                self._send_loop_icmp(flow_id, flow, stop_event, rate_pps, duration)
+                return
+
+            # --- High-performance path: raw socket ---
+            dst_ip = flow['dst_ip']
+            # Build template as raw IP bytes (strip Ethernet layer)
+            ip_template = bytes(pkt_template[pkt_template.firstlayer().payload.__class__])
+            # Find where TGEN magic starts in the IP-layer bytes
+            magic_offset = ip_template.find(MAGIC)
+
+            # Find and zero UDP checksum offset in template
+            ip_ihl = (ip_template[0] & 0x0F) * 4
+            ip_proto = ip_template[9]
+            udp_csum_offset = ip_ihl + 6 if ip_proto == 17 else -1  # 17 = UDP
+
+            raw_sock = socket.socket(socket.AF_INET, socket.SOCK_RAW, socket.IPPROTO_RAW)
+            raw_sock.setsockopt(socket.IPPROTO_IP, socket.IP_HDRINCL, 1)
+
+            # Adaptive batching: send bursts then sleep to hit target rate
+            batch_size = max(1, min(rate_pps // 5, 500))
+            interval = batch_size / rate_pps if rate_pps > 0 else 1.0
+
+            while not stop_event.is_set():
+                elapsed = time.time() - start_time
+                if duration and elapsed >= duration:
+                    break
+
+                batch_start = time.time()
+                sent_this_batch = 0
+
+                for _ in range(batch_size):
+                    pkt_bytes = bytearray(ip_template)
+                    if magic_offset >= 0:
+                        struct.pack_into('!I', pkt_bytes, magic_offset + 4, seq)
+                        struct.pack_into('!Q', pkt_bytes, magic_offset + 8, time.time_ns())
+                    pkt_bytes[10:12] = b'\x00\x00'  # zero IP checksum
+                    if udp_csum_offset > 0:
+                        pkt_bytes[udp_csum_offset:udp_csum_offset + 2] = b'\x00\x00'
+                    try:
+                        raw_sock.sendto(bytes(pkt_bytes), (dst_ip, 0))
+                        sent_this_batch += 1
+                    except Exception:
+                        pass
+                    seq += 1
+
+                with self._lock:
+                    stats = self._stats.get(flow_id)
+                    if stats:
+                        stats['tx_packets'] += sent_this_batch
+                        stats['tx_bytes'] += pkt_bytes_len * sent_this_batch
+
+                # Poll responder for rx stats periodically
+                if responder_url and (time.time() - last_responder_poll) >= 1.0:
+                    self._poll_responder(flow_id, responder_url,
+                                         responder_baseline_rx, responder_baseline_bytes)
+                    last_responder_poll = time.time()
+
+                # Precise rate limiting: sleep remaining time for this batch
+                batch_elapsed = time.time() - batch_start
+                sleep_time = interval - batch_elapsed
+                if sleep_time > 0:
+                    stop_event.wait(sleep_time)
+
+        except Exception as e:
+            log.error('Flow %s: send loop error: %s', flow_id[:8], e)
+        finally:
+            if raw_sock:
+                raw_sock.close()
+            with self._lock:
+                if flow_id in self._flows:
+                    self._flows[flow_id]['state'] = 'stopped'
+            # Final responder poll
+            if responder_url:
+                self._poll_responder(flow_id, responder_url,
+                                     responder_baseline_rx, responder_baseline_bytes)
+            log.info('Flow %s: send loop finished. seq=%d', flow_id[:8], seq)
+
+    def _send_loop_icmp(self, flow_id, flow, stop_event, rate_pps, duration):
+        """ICMP mode: use sr() to measure latency from router responses."""
+        pkt_template = build_packet(flow, seq=0)
+        pkt_bytes_len = len(bytes(pkt_template))
+        seq = 0
+        start_time = time.time()
+
+        try:
+            while not stop_event.is_set():
+                elapsed = time.time() - start_time
+                if duration and elapsed >= duration:
+                    break
+
+                pkt = build_packet(flow, seq=seq)
+                answered, _ = sr(pkt[pkt.firstlayer().payload.__class__],
+                                 timeout=1, verbose=0)
+                with self._lock:
+                    stats = self._stats.get(flow_id)
+                    if stats:
+                        stats['tx_packets'] += 1
+                        stats['tx_bytes'] += pkt_bytes_len
+                        for sent_pkt, recv_pkt in answered:
+                            rtt_ms = (recv_pkt.time - sent_pkt.sent_time) * 1000
+                            stats['rx_packets'] += 1
+                            stats['rx_bytes'] += len(bytes(recv_pkt))
+                            stats['latency_samples'].append(rtt_ms)
+                            if len(stats['latency_samples']) > 1000:
+                                stats['latency_samples'] = stats['latency_samples'][-1000:]
+                seq += 1
+                sleep_time = (1.0 / rate_pps) - (time.time() - start_time - elapsed)
+                if sleep_time > 0:
+                    stop_event.wait(sleep_time)
+        except Exception as e:
+            log.error('Flow %s: ICMP send error: %s', flow_id[:8], e)
+        finally:
+            with self._lock:
+                if flow_id in self._flows:
+                    self._flows[flow_id]['state'] = 'stopped'
+
+    def _fetch_responder(self, responder_url: str) -> dict:
+        """Fetch raw stats from the responder."""
+        url = responder_url.rstrip('/') + '/responder/stats'
+        req = urllib.request.Request(url, method='GET')
+        req.add_header('Accept', 'application/json')
+        with urllib.request.urlopen(req, timeout=2) as resp:
+            return json.loads(resp.read().decode())
+
+    def _reset_responder(self, responder_url: str):
+        """Reset responder counters."""
+        url = responder_url.rstrip('/') + '/responder/reset'
+        req = urllib.request.Request(url, method='POST')
+        req.add_header('Content-Type', 'application/json')
+        with urllib.request.urlopen(req, timeout=2) as resp:
+            resp.read()
+
+    def _poll_responder(self, flow_id: str, responder_url: str,
+                        baseline_rx: int = 0, baseline_bytes: int = 0):
+        """Poll a responder's /responder/stats endpoint for rx metrics."""
+        try:
+            data = self._fetch_responder(responder_url)
+            rx_pkts = data.get('rx_packets', 0) - baseline_rx
+            rx_bytes = data.get('rx_bytes', 0) - baseline_bytes
+            with self._lock:
+                stats = self._stats.get(flow_id)
+                if stats:
+                    stats['rx_packets'] = max(0, rx_pkts)
+                    stats['rx_bytes'] = max(0, rx_bytes)
+                    lat = data.get('latency', {})
+                    if lat.get('avg_ms') is not None:
+                        stats['latency_samples'].append(lat['avg_ms'])
+                        if len(stats['latency_samples']) > 1000:
+                            stats['latency_samples'] = stats['latency_samples'][-1000:]
+        except Exception as e:
+            log.debug('Responder poll error for flow %s: %s', flow_id[:8], e)

traffic-gen/engine/stats.py

@@ -0,0 +1,124 @@
+"""
+StatsCollector - ring buffer of per-flow traffic statistics.
+
+Stores the last 300 samples (5 minutes at 1-second intervals) per flow,
+with derived rates and loss calculations.
+"""
+
+import threading
+import time
+from collections import defaultdict, deque
+
+
+class StatsCollector:
+    """Collects and stores per-flow traffic statistics in a ring buffer."""
+
+    def __init__(self, max_samples: int = 300):
+        self._lock = threading.Lock()
+        self._max_samples = max_samples
+        # flow_id -> deque of sample dicts
+        self._history = defaultdict(lambda: deque(maxlen=self._max_samples))
+        # flow_id -> previous counters for rate calculation
+        self._prev = {}
+
+    def record(self, flow_id: str, tx_packets: int, tx_bytes: int,
+               rx_packets: int = 0, rx_bytes: int = 0,
+               latency: dict = None):
+        """Record a stats sample for a flow.
+
+        Args:
+            flow_id: Flow identifier.
+            tx_packets: Cumulative transmitted packets.
+            tx_bytes: Cumulative transmitted bytes.
+            rx_packets: Cumulative received packets.
+            rx_bytes: Cumulative received bytes.
+            latency: Optional dict with min_ms, avg_ms, max_ms, jitter_ms.
+        """
+        now = time.time()
+
+        with self._lock:
+            prev = self._prev.get(flow_id)
+
+            if prev is not None:
+                dt = now - prev['time']
+                if dt > 0:
+                    d_tx_pkts = tx_packets - prev['tx_packets']
+                    d_tx_bytes = tx_bytes - prev['tx_bytes']
+                    d_rx_pkts = rx_packets - prev['rx_packets']
+                    d_rx_bytes = rx_bytes - prev['rx_bytes']
+
+                    tx_pps = d_tx_pkts / dt
+                    rx_pps = d_rx_pkts / dt
+                    tx_mbps = (d_tx_bytes * 8) / (dt * 1_000_000)
+                    rx_mbps = (d_rx_bytes * 8) / (dt * 1_000_000)
+                else:
+                    tx_pps = rx_pps = tx_mbps = rx_mbps = 0.0
+            else:
+                tx_pps = rx_pps = tx_mbps = rx_mbps = 0.0
+
+            # Loss calculation: use rate-based when actively sending (avoids
+            # poll-lag artifacts), cumulative when flow has stopped
+            loss_pct = 0.0
+            if prev is not None and tx_pps > 0 and rx_pps > 0:
+                # Rate-based: compare instantaneous tx/rx rates
+                loss_pct = max(0.0, ((tx_pps - rx_pps) / tx_pps) * 100)
+            elif tx_packets > 0 and rx_packets > 0 and tx_pps == 0:
+                # Flow stopped: use cumulative counters (final accurate value)
+                loss_pct = max(0.0, ((tx_packets - rx_packets) / tx_packets) * 100)
+
+            sample = {
+                'timestamp': now,
+                'tx_packets': tx_packets,
+                'tx_bytes': tx_bytes,
+                'rx_packets': rx_packets,
+                'rx_bytes': rx_bytes,
+                'tx_pps': round(tx_pps, 2),
+                'rx_pps': round(rx_pps, 2),
+                'tx_mbps': round(tx_mbps, 4),
+                'rx_mbps': round(rx_mbps, 4),
+                'loss_pct': round(loss_pct, 3),
+            }
+
+            if latency:
+                sample['latency'] = latency
+
+            self._history[flow_id].append(sample)
+            self._prev[flow_id] = {
+                'time': now,
+                'tx_packets': tx_packets,
+                'tx_bytes': tx_bytes,
+                'rx_packets': rx_packets,
+                'rx_bytes': rx_bytes,
+            }
+
+    def get_history(self, flow_id: str, count: int = 60) -> list:
+        """Get the last N samples for a flow."""
+        with self._lock:
+            history = self._history.get(flow_id)
+            if not history:
+                return []
+            samples = list(history)
+            return samples[-count:]
+
+    def get_latest(self, flow_id: str) -> dict:
+        """Get the most recent sample for a flow."""
+        with self._lock:
+            history = self._history.get(flow_id)
+            if not history:
+                return {}
+            return dict(history[-1])
+
+    def get_all_latest(self) -> dict:
+        """Get the most recent sample for every flow."""
+        with self._lock:
+            result = {}
+            for flow_id, history in self._history.items():
+                if history:
+                    result[flow_id] = dict(history[-1])
+            return result
+
+    def remove_flow(self, flow_id: str):
+        """Remove all data for a flow."""
+        with self._lock:
+            self._history.pop(flow_id, None)
+            self._prev.pop(flow_id, None)

traffic-gen/presets/__init__.py

@@ -0,0 +1,75 @@
+"""
+Built-in test presets for the traffic generator.
+"""
+
+PRESETS = {
+    'quick_icmp': {
+        'description': 'Quick ICMP ping test to verify connectivity',
+        'flow': {
+            'protocol': 'icmp',
+            'frame_size': 64,
+            'rate_pps': 10,
+            'duration': 10,
+        },
+    },
+    'udp_flood_small': {
+        'description': 'UDP flood with 64-byte frames at 1000 pps',
+        'flow': {
+            'protocol': 'udp',
+            'dst_port': 5001,
+            'frame_size': 64,
+            'rate_pps': 1000,
+            'duration': 30,
+        },
+    },
+    'udp_flood_large': {
+        'description': 'UDP flood with 1518-byte frames at 500 pps',
+        'flow': {
+            'protocol': 'udp',
+            'dst_port': 5001,
+            'frame_size': 1518,
+            'rate_pps': 500,
+            'duration': 30,
+        },
+    },
+    'rfc2544_throughput': {
+        'description': 'RFC 2544 throughput test across standard frame sizes',
+        'flow': {
+            'protocol': 'udp',
+            'dst_port': 5001,
+            'frame_size': 64,
+            'rate_pps': 10000,
+            'duration': 60,
+        },
+        'test': {
+            'type': 'throughput',
+            'frame_sizes': [64, 128, 256, 512, 1024, 1280, 1518],
+            'trial_duration': 60,
+            'acceptable_loss_pct': 0.0,
+        },
+    },
+    'rfc2544_latency': {
+        'description': 'RFC 2544 latency test at moderate rate',
+        'flow': {
+            'protocol': 'icmp',
+            'frame_size': 64,
+            'rate_pps': 100,
+            'duration': 30,
+        },
+        'test': {
+            'type': 'latency',
+            'frame_sizes': [64, 512, 1518],
+            'trial_duration': 30,
+        },
+    },
+    'tcp_session': {
+        'description': 'TCP SYN flood at 100 pps (for testing ACL/firewall)',
+        'flow': {
+            'protocol': 'tcp',
+            'dst_port': 80,
+            'frame_size': 64,
+            'rate_pps': 100,
+            'duration': 30,
+        },
+    },
+}

traffic-gen/requirements.txt

@@ -0,0 +1,3 @@
+flask
+scapy
+psutil

traffic-gen/server.py

@@ -0,0 +1,790 @@
+#!/usr/bin/env python3
+"""
+Traffic Generator API Server
+
+Scapy-based traffic generator with a Flask API. Runs in two modes
+controlled by TRAFFIC_GEN_MODE env var:
+  - sender (default): generates traffic, runs RFC 2544 tests
+  - responder: listens for test packets, echoes/timestamps, reports rx stats
+
+API endpoints:
+  GET  /healthz                - health + active flows/tests count
+  GET  /interfaces             - list available network interfaces
+  GET  /mode                   - current mode
+
+  Sender mode:
+    GET    /flows              - list all flows
+    POST   /flows              - create flow
+    GET    /flows/<id>         - get flow details + stats
+    PUT    /flows/<id>         - update flow (only if idle)
+    DELETE /flows/<id>         - delete flow
+    POST   /flows/<id>/start   - start sending
+    POST   /flows/<id>/stop    - stop sending
+    GET    /flows/<id>/stats   - real-time stats
+
+    GET    /tests              - list all tests
+    POST   /tests              - create RFC 2544 test
+    GET    /tests/<id>         - test details + results
+    POST   /tests/<id>/start   - start test
+    POST   /tests/<id>/stop    - abort test
+    GET    /tests/<id>/results - exportable results
+
+    GET    /presets             - list presets
+    POST   /presets/<name>     - create flow/test from preset
+
+    GET    /stats/history      - historical stats for all flows
+
+  Responder mode:
+    GET    /responder/stats    - receive statistics
+    POST   /responder/reset    - reset receive counters
+"""
+
+import sys
+import os
+import json
+import threading
+import time
+import logging
+import uuid
+
+import psutil
+from flask import Flask, request, jsonify
+
+# --- logging to stderr so stdout stays clean ---
+logging.basicConfig(
+    stream=sys.stderr,
+    level=logging.INFO,
+    format='%(asctime)s [TGEN] %(levelname)s %(message)s',
+)
+log = logging.getLogger(__name__)
+
+app = Flask(__name__)
+
+# ---------------------------------------------------------------------------
+# Global state (set in main())
+# ---------------------------------------------------------------------------
+MODE = 'sender'  # or 'responder'
+
+# Sender-mode globals
+_sender = None        # FlowSender instance
+_stats_collector = None  # StatsCollector instance
+_flows_meta = {}      # flow_id -> metadata dict (includes config)
+_flows_lock = threading.Lock()
+
+_tests = {}           # test_id -> RFC 2544 test instance
+_tests_lock = threading.Lock()
+
+# Responder-mode globals
+_responder = None     # Responder instance
+
+
+
+# ---------------------------------------------------------------------------
+# Helper
+# ---------------------------------------------------------------------------
+
+def _now_iso():
+    return time.strftime('%Y-%m-%dT%H:%M:%SZ', time.gmtime())
+
+
+
+def _flow_response(flow_id: str) -> dict:
+    """Build a serializable flow dict."""
+    with _flows_lock:
+        meta = _flows_meta.get(flow_id)
+        if meta is None:
+            return None
+    result = dict(meta)
+    if _sender:
+        running = _sender.is_running(flow_id)
+        result['is_running'] = running
+        result['stats'] = _sender.get_stats(flow_id)
+        # Sync state: if sender thread finished but meta still says running
+        if not running and result.get('state') == 'running':
+            with _flows_lock:
+                if flow_id in _flows_meta:
+                    _flows_meta[flow_id]['state'] = 'stopped'
+            result['state'] = 'stopped'
+    return result
+
+
+# ---------------------------------------------------------------------------
+# Common endpoints
+# ---------------------------------------------------------------------------
+
+@app.route('/healthz')
+def health():
+    with _flows_lock:
+        active_flows = sum(
+            1 for fid in _flows_meta
+            if _sender and _sender.is_running(fid)
+        ) if MODE == 'sender' else 0
+    with _tests_lock:
+        active_tests = sum(
+            1 for t in _tests.values() if t.state == 'running'
+        ) if MODE == 'sender' else 0
+    resp = {
+        'status': 'ok',
+        'mode': MODE,
+        'active_flows': active_flows,
+        'active_tests': active_tests,
+    }
+    if MODE == 'responder' and _responder:
+        resp['responder_running'] = _responder.is_running()
+    return jsonify(resp)
+
+
+@app.route('/interfaces')
+def list_interfaces():
+    ifaces = []
+    addrs = psutil.net_if_addrs()
+    stats = psutil.net_if_stats()
+    for name, addr_list in addrs.items():
+        info = {'name': name, 'ip': None, 'mac': None, 'is_up': False}
+        for addr in addr_list:
+            if addr.family.name == 'AF_INET':
+                info['ip'] = addr.address
+            elif addr.family.name == 'AF_PACKET':
+                info['mac'] = addr.address
+        if name in stats:
+            info['is_up'] = stats[name].isup
+        ifaces.append(info)
+    return jsonify({'interfaces': ifaces})
+
+
+@app.route('/mode')
+def get_mode():
+    return jsonify({'mode': MODE})
+
+
+@app.route('/mode', methods=['POST'])
+def set_mode():
+    global MODE, _sender, _stats_collector, _responder
+
+    data = request.get_json(force=True)
+    new_mode = data.get('mode', '').lower()
+    if new_mode not in ('sender', 'responder'):
+        return jsonify({'error': 'mode must be "sender" or "responder"'}), 400
+    if new_mode == MODE:
+        return jsonify({'mode': MODE, 'changed': False})
+
+    listen_iface = os.environ.get('TRAFFIC_GEN_INTERFACE', None)
+    responder_sub_mode = os.environ.get('TRAFFIC_GEN_RESPONDER_MODE', 'log')
+
+    # Tear down current mode
+    if MODE == 'sender':
+        # Stop all running flows
+        if _sender:
+            for fid in list(_sender.get_all_flows().keys()):
+                _sender.stop(fid)
+        with _flows_lock:
+            _flows_meta.clear()
+        with _tests_lock:
+            for t in _tests.values():
+                if t.state == 'running':
+                    t.stop()
+            _tests.clear()
+    elif MODE == 'responder':
+        if _responder:
+            _responder.stop()
+            _responder = None
+
+    # Start new mode
+    if new_mode == 'sender':
+        from engine.sender import FlowSender
+        from engine.stats import StatsCollector
+        _sender = FlowSender()
+        _stats_collector = StatsCollector()
+        _responder = None
+    elif new_mode == 'responder':
+        from engine.responder import Responder
+        _sender = None
+        _stats_collector = None
+        _responder = Responder(mode=responder_sub_mode)
+        _responder.start(interface=listen_iface)
+
+    MODE = new_mode
+    log.info('Mode switched to %s', MODE)
+    return jsonify({'mode': MODE, 'changed': True})
+
+
+# ---------------------------------------------------------------------------
+# Quick Ping (works in any mode)
+# ---------------------------------------------------------------------------
+
+@app.route('/ping', methods=['POST'])
+def quick_ping():
+    """Send ICMP pings to a target and return results."""
+    import subprocess
+
+    data = request.get_json(force=True)
+    target = data.get('target', '').strip()
+    count = min(int(data.get('count', 5)), 20)
+
+    if not target:
+        return jsonify({'error': 'target is required'}), 400
+
+    try:
+        result = subprocess.run(
+            ['ping', '-c', str(count), '-W', '2', target],
+            capture_output=True, text=True, timeout=count * 3 + 5
+        )
+        output = result.stdout + result.stderr
+
+        # Parse ping output
+        lines = output.strip().split('\n')
+        replies = []
+        for line in lines:
+            if 'time=' in line:
+                try:
+                    time_ms = float(line.split('time=')[1].split(' ')[0])
+                    replies.append(time_ms)
+                except (IndexError, ValueError):
+                    pass
+
+        stats = {}
+        if replies:
+            stats = {
+                'min_ms': round(min(replies), 2),
+                'avg_ms': round(sum(replies) / len(replies), 2),
+                'max_ms': round(max(replies), 2),
+            }
+
+        return jsonify({
+            'target': target,
+            'sent': count,
+            'received': len(replies),
+            'loss_pct': round(((count - len(replies)) / count) * 100, 1),
+            'replies': replies,
+            'stats': stats,
+            'reachable': len(replies) > 0,
+        })
+    except subprocess.TimeoutExpired:
+        return jsonify({'target': target, 'error': 'Ping timed out', 'reachable': False})
+    except Exception as e:
+        return jsonify({'target': target, 'error': str(e), 'reachable': False})
+
+
+# ---------------------------------------------------------------------------
+# Sender-mode: Flow endpoints
+# ---------------------------------------------------------------------------
+
+@app.route('/flows', methods=['GET'])
+def list_flows():
+    if MODE != 'sender':
+        return jsonify({'error': 'Not in sender mode'}), 400
+    with _flows_lock:
+        flow_ids = list(_flows_meta.keys())
+    flows = [_flow_response(fid) for fid in flow_ids]
+    return jsonify({'count': len(flows), 'flows': flows})
+
+
+@app.route('/flows', methods=['POST'])
+def create_flow():
+    if MODE != 'sender':
+        return jsonify({'error': 'Not in sender mode'}), 400
+    data = request.get_json(force=True)
+
+    # Validate required fields
+    if 'dst_ip' not in data:
+        return jsonify({'error': 'dst_ip is required'}), 400
+    if 'protocol' not in data:
+        return jsonify({'error': 'protocol is required'}), 400
+    if data['protocol'].lower() not in ('udp', 'tcp', 'icmp'):
+        return jsonify({'error': 'protocol must be udp, tcp, or icmp'}), 400
+
+    flow_id = str(uuid.uuid4())
+    flow_config = {
+        'id': flow_id,
+        'name': data.get('name', f'flow-{flow_id[:8]}'),
+        'src_mac': data.get('src_mac', 'auto'),
+        'dst_mac': data.get('dst_mac'),
+        'src_ip': data.get('src_ip'),
+        'dst_ip': data['dst_ip'],
+        'protocol': data['protocol'].lower(),
+        'src_port': data.get('src_port'),
+        'dst_port': data.get('dst_port'),
+        'frame_size': int(data.get('frame_size', 512)),
+        'rate_pps': int(data.get('rate_pps', 1000)),
+        'duration': int(data.get('duration', 30)),
+        'dscp': int(data.get('dscp', 0)),
+        'vlan_id': data.get('vlan_id'),
+        'state': 'idle',
+        'responder_url': data.get('responder_url') or os.environ.get('RESPONDER_URL') or None,
+        'created_at': _now_iso(),
+    }
+
+    with _flows_lock:
+        _flows_meta[flow_id] = flow_config
+    _sender.add_flow(flow_id, dict(flow_config))
+
+    log.info('Created flow %s (%s -> %s, %s)',
+             flow_id[:8], flow_config.get('src_ip', 'auto'),
+             flow_config['dst_ip'], flow_config['protocol'])
+    return jsonify(_flow_response(flow_id)), 201
+
+
+@app.route('/flows/<flow_id>', methods=['GET'])
+def get_flow(flow_id):
+    if MODE != 'sender':
+        return jsonify({'error': 'Not in sender mode'}), 400
+    result = _flow_response(flow_id)
+    if result is None:
+        return jsonify({'error': 'Flow not found'}), 404
+    return jsonify(result)
+
+
+@app.route('/flows/<flow_id>', methods=['PUT'])
+def update_flow(flow_id):
+    if MODE != 'sender':
+        return jsonify({'error': 'Not in sender mode'}), 400
+
+    with _flows_lock:
+        meta = _flows_meta.get(flow_id)
+        if meta is None:
+            return jsonify({'error': 'Flow not found'}), 404
+        if meta.get('state') == 'running':
+            return jsonify({'error': 'Cannot update a running flow'}), 409
+
+    data = request.get_json(force=True)
+    updatable = [
+        'name', 'src_mac', 'dst_mac', 'src_ip', 'dst_ip', 'protocol',
+        'src_port', 'dst_port', 'frame_size', 'rate_pps', 'duration',
+        'dscp', 'vlan_id', 'responder_url',
+    ]
+
+    with _flows_lock:
+        for key in updatable:
+            if key in data:
+                _flows_meta[flow_id][key] = data[key]
+        _sender.update_flow(flow_id, dict(_flows_meta[flow_id]))
+
+    return jsonify(_flow_response(flow_id))
+
+
+@app.route('/flows/<flow_id>', methods=['DELETE'])
+def delete_flow(flow_id):
+    if MODE != 'sender':
+        return jsonify({'error': 'Not in sender mode'}), 400
+
+    with _flows_lock:
+        if flow_id not in _flows_meta:
+            return jsonify({'error': 'Flow not found'}), 404
+
+    _sender.remove_flow(flow_id)
+    if _stats_collector:
+        _stats_collector.remove_flow(flow_id)
+
+    with _flows_lock:
+        _flows_meta.pop(flow_id, None)
+
+    log.info('Deleted flow %s', flow_id[:8])
+    return jsonify({'deleted': flow_id})
+
+
+@app.route('/flows/<flow_id>/start', methods=['POST'])
+def start_flow(flow_id):
+    if MODE != 'sender':
+        return jsonify({'error': 'Not in sender mode'}), 400
+
+    with _flows_lock:
+        if flow_id not in _flows_meta:
+            return jsonify({'error': 'Flow not found'}), 404
+        _flows_meta[flow_id]['state'] = 'running'
+
+    try:
+        _sender.start(flow_id)
+    except KeyError:
+        return jsonify({'error': 'Flow not found in sender'}), 404
+
+    log.info('Started flow %s', flow_id[:8])
+    return jsonify(_flow_response(flow_id))
+
+
+@app.route('/flows/<flow_id>/stop', methods=['POST'])
+def stop_flow(flow_id):
+    if MODE != 'sender':
+        return jsonify({'error': 'Not in sender mode'}), 400
+
+    with _flows_lock:
+        if flow_id not in _flows_meta:
+            return jsonify({'error': 'Flow not found'}), 404
+        _flows_meta[flow_id]['state'] = 'stopped'
+
+    _sender.stop(flow_id)
+
+    log.info('Stopped flow %s', flow_id[:8])
+    return jsonify(_flow_response(flow_id))
+
+
+@app.route('/flows/<flow_id>/stats', methods=['GET'])
+def flow_stats(flow_id):
+    if MODE != 'sender':
+        return jsonify({'error': 'Not in sender mode'}), 400
+
+    with _flows_lock:
+        if flow_id not in _flows_meta:
+            return jsonify({'error': 'Flow not found'}), 404
+
+    stats = _sender.get_stats(flow_id)
+    latest = _stats_collector.get_latest(flow_id) if _stats_collector else {}
+    return jsonify({'flow_id': flow_id, 'counters': stats, 'rates': latest})
+
+
+# ---------------------------------------------------------------------------
+# Sender-mode: Test endpoints
+# ---------------------------------------------------------------------------
+
+@app.route('/tests', methods=['GET'])
+def list_tests():
+    if MODE != 'sender':
+        return jsonify({'error': 'Not in sender mode'}), 400
+    with _tests_lock:
+        tests = [t.get_info() for t in _tests.values()]
+    return jsonify({'count': len(tests), 'tests': tests})
+
+
+@app.route('/tests', methods=['POST'])
+def create_test():
+    if MODE != 'sender':
+        return jsonify({'error': 'Not in sender mode'}), 400
+
+    from engine.rfc2544 import create_test as _create_test
+
+    data = request.get_json(force=True)
+    test_type = data.get('type')
+
+    if not test_type:
+        return jsonify({'error': 'type is required'}), 400
+
+    # Accept either flow_config directly or flow_id to look up
+    flow_config = data.get('flow_config')
+    flow_id = data.get('flow_id')
+
+    if flow_config:
+        # Direct flow config provided — no flow_id needed
+        if not flow_config.get('dst_ip'):
+            return jsonify({'error': 'flow_config.dst_ip is required'}), 400
+        flow_config.setdefault('src_ip', 'auto')
+        flow_config.setdefault('protocol', 'udp')
+        flow_config.setdefault('src_port', 50000)
+        flow_config.setdefault('dst_port', 5001)
+    elif flow_id:
+        with _flows_lock:
+            flow_meta = _flows_meta.get(flow_id)
+            if flow_meta is None:
+                return jsonify({'error': 'Flow not found'}), 404
+        flow_config = dict(flow_meta)
+    else:
+        return jsonify({'error': 'flow_config or flow_id is required'}), 400
+
+    test_id = str(uuid.uuid4())
+    kwargs = {
+        'frame_sizes': data.get('frame_sizes', [64, 512, 1518]),
+        'trial_duration': float(data.get('trial_duration', 60)),
+        'max_rate_pps': int(data.get('max_rate_pps', flow_config.get('rate_pps', 10000))),
+        'acceptable_loss_pct': float(data.get('acceptable_loss_pct', 0.0)),
+        'responder_url': data.get('responder_url') or os.environ.get('RESPONDER_URL') or None,
+    }
+
+    try:
+        test = _create_test(test_id, test_type, flow_config, **kwargs)
+    except ValueError as e:
+        return jsonify({'error': str(e)}), 400
+
+    with _tests_lock:
+        _tests[test_id] = test
+
+    log.info('Created test %s (type=%s, dst=%s)', test_id[:8], test_type,
+             flow_config.get('dst_ip', '?'))
+    return jsonify(test.get_info()), 201
+
+
+@app.route('/tests/<test_id>', methods=['GET'])
+def get_test(test_id):
+    if MODE != 'sender':
+        return jsonify({'error': 'Not in sender mode'}), 400
+    with _tests_lock:
+        test = _tests.get(test_id)
+    if test is None:
+        return jsonify({'error': 'Test not found'}), 404
+    return jsonify(test.get_info())
+
+
+@app.route('/tests/<test_id>/start', methods=['POST'])
+def start_test(test_id):
+    if MODE != 'sender':
+        return jsonify({'error': 'Not in sender mode'}), 400
+    with _tests_lock:
+        test = _tests.get(test_id)
+    if test is None:
+        return jsonify({'error': 'Test not found'}), 404
+    test.start()
+    log.info('Started test %s', test_id[:8])
+    return jsonify(test.get_info())
+
+
+@app.route('/tests/<test_id>/stop', methods=['POST'])
+def stop_test(test_id):
+    if MODE != 'sender':
+        return jsonify({'error': 'Not in sender mode'}), 400
+    with _tests_lock:
+        test = _tests.get(test_id)
+    if test is None:
+        return jsonify({'error': 'Test not found'}), 404
+    test.stop()
+    log.info('Stopped test %s', test_id[:8])
+    return jsonify(test.get_info())
+
+
+@app.route('/tests/<test_id>/results', methods=['GET'])
+def test_results(test_id):
+    if MODE != 'sender':
+        return jsonify({'error': 'Not in sender mode'}), 400
+    with _tests_lock:
+        test = _tests.get(test_id)
+    if test is None:
+        return jsonify({'error': 'Test not found'}), 404
+    return jsonify({
+        'test_id': test_id,
+        'type': test.__class__.__name__,
+        'state': test.state,
+        'results': test.results,
+        'started_at': test.started_at,
+        'completed_at': test.completed_at,
+    })
+
+
+# ---------------------------------------------------------------------------
+# Sender-mode: Presets
+# ---------------------------------------------------------------------------
+
+@app.route('/presets', methods=['GET'])
+def list_presets():
+    if MODE != 'sender':
+        return jsonify({'error': 'Not in sender mode'}), 400
+
+    from presets import PRESETS
+    out = {}
+    for name, preset in PRESETS.items():
+        out[name] = {
+            'description': preset.get('description', ''),
+            'has_test': 'test' in preset,
+        }
+    return jsonify({'presets': out})
+
+
+@app.route('/presets/<name>', methods=['POST'])
+def load_preset(name):
+    if MODE != 'sender':
+        return jsonify({'error': 'Not in sender mode'}), 400
+
+    from presets import PRESETS
+    from engine.rfc2544 import create_test as _create_test
+
+    if name not in PRESETS:
+        return jsonify({'error': f'Unknown preset: {name}. Available: {list(PRESETS)}'}), 404
+
+    preset = PRESETS[name]
+    data = request.get_json(silent=True) or {}
+
+    # The preset flow needs a dst_ip; caller can override
+    flow_data = dict(preset['flow'])
+    flow_data['dst_ip'] = data.get('dst_ip', flow_data.get('dst_ip', '10.0.0.1'))
+    if 'src_ip' in data:
+        flow_data['src_ip'] = data['src_ip']
+    if 'responder_url' in data:
+        flow_data['responder_url'] = data['responder_url']
+
+    # Validate protocol present
+    if 'protocol' not in flow_data:
+        flow_data['protocol'] = 'udp'
+
+    flow_id = str(uuid.uuid4())
+    flow_config = {
+        'id': flow_id,
+        'name': data.get('name', f'{name}-{flow_id[:8]}'),
+        'src_mac': flow_data.get('src_mac', 'auto'),
+        'dst_mac': flow_data.get('dst_mac'),
+        'src_ip': flow_data.get('src_ip'),
+        'dst_ip': flow_data['dst_ip'],
+        'protocol': flow_data['protocol'],
+        'src_port': flow_data.get('src_port'),
+        'dst_port': flow_data.get('dst_port'),
+        'frame_size': int(flow_data.get('frame_size', 512)),
+        'rate_pps': int(flow_data.get('rate_pps', 1000)),
+        'duration': int(flow_data.get('duration', 30)),
+        'dscp': int(flow_data.get('dscp', 0)),
+        'vlan_id': flow_data.get('vlan_id'),
+        'state': 'idle',
+        'responder_url': flow_data.get('responder_url') or os.environ.get('RESPONDER_URL') or None,
+        'created_at': _now_iso(),
+    }
+
+    with _flows_lock:
+        _flows_meta[flow_id] = flow_config
+    _sender.add_flow(flow_id, dict(flow_config))
+
+    result = {'flow': _flow_response(flow_id)}
+
+    # Optionally create a test
+    if 'test' in preset:
+        test_cfg = preset['test']
+        test_id = str(uuid.uuid4())
+        kwargs = {
+            'frame_sizes': test_cfg.get('frame_sizes', [64, 512, 1518]),
+            'trial_duration': float(test_cfg.get('trial_duration', 60)),
+            'max_rate_pps': int(test_cfg.get('max_rate_pps', flow_config.get('rate_pps', 10000))),
+            'acceptable_loss_pct': float(test_cfg.get('acceptable_loss_pct', 0.0)),
+        }
+        test = _create_test(test_id, test_cfg['type'], dict(flow_config), **kwargs)
+        with _tests_lock:
+            _tests[test_id] = test
+        result['test'] = test.get_info()
+
+    log.info('Loaded preset %s -> flow %s', name, flow_id[:8])
+    return jsonify(result), 201
+
+
+# ---------------------------------------------------------------------------
+# Sender-mode: Stats history
+# ---------------------------------------------------------------------------
+
+@app.route('/stats/history', methods=['GET'])
+def stats_history():
+    if MODE != 'sender':
+        return jsonify({'error': 'Not in sender mode'}), 400
+
+    count = int(request.args.get('count', 60))
+    with _flows_lock:
+        flow_ids = list(_flows_meta.keys())
+
+    history = {}
+    for fid in flow_ids:
+        history[fid] = _stats_collector.get_history(fid, count)
+    return jsonify({'history': history})
+
+
+# ---------------------------------------------------------------------------
+# Responder-mode endpoints
+# ---------------------------------------------------------------------------
+
+@app.route('/responder/stats', methods=['GET'])
+def responder_stats():
+    if MODE != 'responder' or _responder is None:
+        return jsonify({'error': 'Not in responder mode'}), 400
+    return jsonify(_responder.get_stats())
+
+
+@app.route('/responder/reset', methods=['POST'])
+def responder_reset():
+    if MODE != 'responder' or _responder is None:
+        return jsonify({'error': 'Not in responder mode'}), 400
+    _responder.reset_stats()
+    return jsonify({'status': 'reset'})
+
+
+# ---------------------------------------------------------------------------
+# Stats collection loop (sender mode)
+# ---------------------------------------------------------------------------
+
+def _stats_loop(stop_event: threading.Event):
+    """Runs every 1s, recording per-flow stats into the StatsCollector."""
+    while not stop_event.is_set():
+        try:
+            all_stats = _sender.get_all_stats()
+            for flow_id, s in all_stats.items():
+                latency = None
+                samples = s.get('latency_samples', [])
+                if samples:
+                    latency = {
+                        'min_ms': round(min(samples), 3),
+                        'max_ms': round(max(samples), 3),
+                        'avg_ms': round(sum(samples) / len(samples), 3),
+                    }
+                _stats_collector.record(
+                    flow_id,
+                    tx_packets=s.get('tx_packets', 0),
+                    tx_bytes=s.get('tx_bytes', 0),
+                    rx_packets=s.get('rx_packets', 0),
+                    rx_bytes=s.get('rx_bytes', 0),
+                    latency=latency,
+                )
+        except Exception as e:
+            log.debug('Stats loop error: %s', e)
+        stop_event.wait(1.0)
+
+
+# ---------------------------------------------------------------------------
+# Main
+# ---------------------------------------------------------------------------
+
+def main():
+    global MODE, _sender, _stats_collector, _responder
+
+    MODE = os.environ.get('TRAFFIC_GEN_MODE', 'sender').lower()
+    api_port = int(os.environ.get('TRAFFIC_GEN_PORT', 5051))
+    listen_iface = os.environ.get('TRAFFIC_GEN_INTERFACE', None)
+    responder_sub_mode = os.environ.get('TRAFFIC_GEN_RESPONDER_MODE', 'log')  # echo or log
+
+    log.info('Traffic Generator starting in %s mode', MODE)
+
+    if MODE == 'sender':
+        from engine.sender import FlowSender
+        from engine.stats import StatsCollector
+
+        _sender = FlowSender()
+        _stats_collector = StatsCollector()
+
+        # Start stats collection loop
+        stats_stop = threading.Event()
+        stats_thread = threading.Thread(
+            target=_stats_loop, args=(stats_stop,), daemon=True, name='stats-loop'
+        )
+        stats_thread.start()
+
+        # Start Flask in background thread
+        flask_thread = threading.Thread(
+            target=lambda: app.run(host='0.0.0.0', port=api_port, threaded=True),
+            daemon=True,
+        )
+        flask_thread.start()
+        log.info('Flask API listening on port %d', api_port)
+
+        # Main thread: keep alive
+        try:
+            while True:
+                time.sleep(1)
+        except KeyboardInterrupt:
+            log.info('Shutting down sender...')
+            stats_stop.set()
+
+    elif MODE == 'responder':
+        from engine.responder import Responder
+
+        _responder = Responder(mode=responder_sub_mode)
+        _responder.start(interface=listen_iface)
+
+        # Start Flask in background thread
+        flask_thread = threading.Thread(
+            target=lambda: app.run(host='0.0.0.0', port=api_port, threaded=True),
+            daemon=True,
+        )
+        flask_thread.start()
+        log.info('Flask API listening on port %d', api_port)
+
+        # Main thread: keep alive
+        try:
+            while True:
+                time.sleep(1)
+        except KeyboardInterrupt:
+            log.info('Shutting down responder...')
+            _responder.stop()
+
+    else:
+        log.error('Unknown mode: %s. Use "sender" or "responder".', MODE)
+        sys.exit(1)
+
+
+if __name__ == '__main__':
+    main()