sam/obmp-docker
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Compare commits

base: sam:obmp-expanded-features
Branches Tags
sam:main
sam:portability-cleanup
sam:network-visualization
sam:platform-enhancements
sam:production-readiness
sam:platform-production
sam:rr-diff-pipeline
sam:obmp-expanded-features
..
compare: sam:main
Branches Tags
sam:portability-cleanup
sam:network-visualization
sam:platform-enhancements
sam:production-readiness
sam:platform-production
sam:rr-diff-pipeline
sam:obmp-expanded-features
sam:main

No commits in common. "obmp-expanded-features" and "main" have entirely different histories.
obmp-expan ... main

42 changed files with 3 additions and 5309 deletions
Show Stats Expand all files Collapse all files

2
.gitignore vendored
View File

@ -2,6 +2,4 @@
*.log
.env
.claude/
__pycache__/
*.pyc

387
DB_SCHEMA.md
View File

@ -1,387 +0,0 @@
# OpenBMP Database Schema Reference
PostgreSQL database `openbmp` with TimescaleDB extension for time-series data.
## Entity Relationship Diagram
```
collectors
└── routers (collector_hash_id)
└── bgp_peers (router_hash_id)
├── ip_rib (peer_hash_id) ──► base_attrs (base_attr_hash_id)
├── ip_rib_log (peer_hash_id)
├── l3vpn_rib (peer_hash_id) ──► base_attrs
├── ls_nodes (peer_hash_id)
├── ls_links (peer_hash_id) ──► ls_nodes (local/remote_node_hash_id)
├── ls_prefixes (peer_hash_id) ──► ls_nodes (local_node_hash_id)
├── peer_event_log (peer_hash_id)
├── stat_reports (peer_hash_id)
└── stats_* tables (peer_hash_id)
ip_rib.prefix ◄──► global_ip_rib.prefix (aggregated view)
├── rpki_origin_as ◄── rpki_validator
└── irr_origin_as ◄── info_route
base_attrs.origin_as ──► info_asn.asn (ASN enrichment)
routers.geo_ip_start ──► geo_ip.ip (geolocation)
```
---
## BMP Core Tables
### routers
BMP-monitored routers (one row per monitored device).
| Column | Type | Description |
|--------|------|-------------|
| hash_id | uuid | Primary key |
| name | varchar(200) | Router hostname |
| ip_address | inet | Router management IP |
| router_as | bigint | Router ASN |
| bgp_id | inet | BGP router-id |
| collector_hash_id | uuid | FK to collectors |
| state | opstate | up / down |
| timestamp | timestamp | Last update time |
| description | varchar(255) | Router description |
| init_data | text | BMP init message data |
| term_reason_code | int | BMP termination reason |
### collectors
BMP collector instances.
| Column | Type | Description |
|--------|------|-------------|
| hash_id | uuid | Primary key |
| admin_id | varchar(64) | Admin identifier |
| name | varchar(200) | Collector name |
| ip_address | varchar(40) | Collector IP |
| state | opstate | up / down |
| router_count | smallint | Number of monitored routers |
### bgp_peers
BGP sessions per router (one row per peer per router).
| Column | Type | Description |
|--------|------|-------------|
| hash_id | uuid | Primary key (composite with router_hash_id) |
| router_hash_id | uuid | FK to routers |
| peer_addr | inet | Peer IP address |
| peer_as | bigint | Peer ASN |
| peer_bgp_id | inet | Peer BGP router-id |
| name | varchar(200) | Peer name |
| state | opstate | up / down |
| isl3vpnpeer | boolean | L3VPN peer flag |
| isipv4 | boolean | IPv4 peer |
| isprepolicy | boolean | Pre-policy RIB |
| islocrib | boolean | Local RIB |
| local_ip | inet | Local IP |
| local_asn | bigint | Local ASN |
| local_hold_time | smallint | Local hold time |
| remote_hold_time | smallint | Remote hold time |
| sent_capabilities | varchar(4096) | BGP capabilities sent |
| recv_capabilities | varchar(4096) | BGP capabilities received |
| table_name | varchar(255) | VRF/table name |
### peer_event_log (TimescaleDB)
Historical BGP session state changes.
| Column | Type | Description |
|--------|------|-------------|
| id | bigint | Event sequence |
| peer_hash_id | uuid | FK to bgp_peers |
| state | opstate | up / down |
| timestamp | timestamp | Event time (partition key) |
| bmp_reason | smallint | BMP reason code |
| bgp_err_code | smallint | BGP error code |
| bgp_err_subcode | smallint | BGP error subcode |
| error_text | varchar(255) | Error description |
---
## BGP Path Attributes
### base_attrs
BGP path attributes shared across routes.
| Column | Type | Description |
|--------|------|-------------|
| hash_id | uuid | Primary key |
| peer_hash_id | uuid | FK to bgp_peers |
| origin | varchar(16) | IGP / EGP / Incomplete |
| as_path | bigint[] | AS path array |
| as_path_count | smallint | AS path length |
| origin_as | bigint | Origin ASN |
| next_hop | inet | BGP next-hop |
| med | bigint | Multi-Exit Discriminator |
| local_pref | bigint | Local preference |
| community_list | varchar(15)[] | Standard communities |
| ext_community_list | varchar(50)[] | Extended communities (RT, etc.) |
| large_community_list | varchar(40)[] | Large communities (RFC 8092) |
| cluster_list | varchar(40)[] | Route reflector cluster list |
| isatomicagg | boolean | Atomic aggregate flag |
| originator_id | inet | RR originator ID |
| aggregator | varchar(64) | Aggregator |
**Indexes**: GIN on as_path, community_list, ext_community_list, large_community_list
---
## IP RIB Tables
### ip_rib
Current IPv4/IPv6 unicast routing table.
| Column | Type | Description |
|--------|------|-------------|
| hash_id | uuid | Route hash |
| peer_hash_id | uuid | FK to bgp_peers (composite PK) |
| base_attr_hash_id | uuid | FK to base_attrs |
| prefix | inet | IP prefix |
| prefix_len | smallint | Prefix length |
| origin_as | bigint | Origin ASN |
| isipv4 | boolean | IPv4 flag |
| iswithdrawn | boolean | Withdrawn flag |
| labels | varchar(255) | MPLS labels |
| path_id | bigint | Add-Path ID |
| isprepolicy | boolean | Pre-policy flag |
| isadjribin | boolean | Adj-RIB-In flag |
| timestamp | timestamp | Last update |
| first_added_timestamp | timestamp | First seen |
### ip_rib_log (TimescaleDB)
Historical RIB changes — every advertisement and withdrawal.
| Column | Type | Description |
|--------|------|-------------|
| id | bigint | Change event ID |
| peer_hash_id | uuid | FK to bgp_peers |
| base_attr_hash_id | uuid | FK to base_attrs |
| prefix | inet | IP prefix |
| prefix_len | smallint | Prefix length |
| origin_as | bigint | Origin ASN |
| iswithdrawn | boolean | Withdrawal flag |
| timestamp | timestamp | Event time (partition key) |
### global_ip_rib
Aggregated prefix summary across all peers.
| Column | Type | Description |
|--------|------|-------------|
| prefix | inet | IP prefix (composite PK) |
| prefix_len | smallint | Prefix length |
| recv_origin_as | bigint | Received origin AS |
| rpki_origin_as | bigint | RPKI-validated origin AS |
| irr_origin_as | bigint | IRR-registered origin AS |
| irr_source | varchar(32) | IRR source (RADB, RIPE, etc.) |
| num_peers | int | Total advertising peers |
| iswithdrawn | boolean | Withdrawn flag |
---
## L3VPN Tables
### l3vpn_rib
L3VPN (RFC 4364) routes with Route Distinguisher.
| Column | Type | Description |
|--------|------|-------------|
| hash_id | uuid | Route hash |
| peer_hash_id | uuid | FK to bgp_peers |
| base_attr_hash_id | uuid | FK to base_attrs |
| rd | varchar(128) | Route Distinguisher |
| prefix | inet | VPN prefix |
| prefix_len | smallint | Prefix length |
| origin_as | bigint | Origin ASN |
| labels | varchar(255) | MPLS VPN labels |
| ext_community_list | varchar(50)[] | Route Targets |
| path_id | bigint | Add-Path ID |
| iswithdrawn | boolean | Withdrawn flag |
### l3vpn_rib_log (TimescaleDB)
Historical L3VPN route changes.
---
## Link-State Tables (BGP-LS / RFC 7752)
### ls_nodes
IS-IS / OSPF node information from BGP-LS.
| Column | Type | Description |
|--------|------|-------------|
| hash_id | uuid | Node hash |
| peer_hash_id | uuid | FK to bgp_peers (composite PK) |
| base_attr_hash_id | uuid | FK to base_attrs |
| asn | bigint | Node ASN |
| bgp_ls_id | bigint | BGP-LS Identifier |
| igp_router_id | varchar(46) | IGP Router ID |
| router_id | varchar(46) | BGP Router ID |
| protocol | ls_proto | IS-IS_L1, IS-IS_L2, OSPFv2, OSPFv3 |
| isis_area_id | varchar(46) | IS-IS area |
| ospf_area_id | varchar(16) | OSPF area |
| name | varchar(255) | Node hostname |
| flags | varchar(20) | Node flags |
| mt_ids | varchar(128) | Multi-Topology IDs |
| **sr_capabilities** | **varchar(255)** | **SR Global Block (SRGB) ranges** |
| iswithdrawn | boolean | Withdrawn flag |
### ls_links
IS-IS / OSPF links with full TE and SR attributes.
| Column | Type | Description |
|--------|------|-------------|
| hash_id | uuid | Link hash |
| peer_hash_id | uuid | FK to bgp_peers (composite PK) |
| local_node_hash_id | uuid | FK to ls_nodes (local end) |
| remote_node_hash_id | uuid | FK to ls_nodes (remote end) |
| local_router_id | varchar(46) | Local BGP Router ID |
| remote_router_id | varchar(46) | Remote BGP Router ID |
| local_igp_router_id | varchar(46) | Local IGP Router ID |
| remote_igp_router_id | varchar(46) | Remote IGP Router ID |
| interface_addr | inet | Local interface IP |
| neighbor_addr | inet | Remote interface IP |
| igp_metric | bigint | IGP metric |
| protocol | ls_proto | IGP protocol |
| mt_id | int | Multi-Topology ID |
| local_link_id | bigint | Local link identifier |
| remote_link_id | bigint | Remote link identifier |
| name | varchar(255) | Link name |
| **admin_group** | **bigint** | **TE admin group / link color bitmap** |
| **max_link_bw** | **bigint** | **Maximum link bandwidth (bytes/sec)** |
| **max_resv_bw** | **bigint** | **Maximum reservable bandwidth** |
| **unreserved_bw** | **varchar(128)** | **Unreserved BW per priority (8 values)** |
| **te_def_metric** | **bigint** | **TE default metric (for CSPF)** |
| **protection_type** | **varchar(60)** | **Link protection (FRR type)** |
| **mpls_proto_mask** | **ls_mpls_proto_mask** | **MPLS protocol support flags** |
| **srlg** | **varchar(128)** | **Shared Risk Link Group** |
| **peer_node_sid** | **varchar(128)** | **SR Peer Node SID (EPE, RFC 9086)** |
| **sr_adjacency_sids** | **varchar(255)** | **SR Adjacency SIDs** |
| iswithdrawn | boolean | Withdrawn flag |
**Bold** = TE/SR fields available via BGP-LS but not used by default dashboards.
### ls_prefixes
IS-IS / OSPF prefix information.
| Column | Type | Description |
|--------|------|-------------|
| hash_id | uuid | Prefix hash |
| peer_hash_id | uuid | FK to bgp_peers (composite PK) |
| local_node_hash_id | uuid | FK to ls_nodes |
| prefix | inet | Advertised prefix |
| prefix_len | smallint | Prefix length |
| protocol | ls_proto | IGP protocol |
| metric | bigint | Prefix metric |
| mt_id | int | Multi-Topology ID |
| ospf_route_type | ospf_route_type | Intra/Inter/Ext-1/Ext-2/NSSA |
| igp_flags | varchar(20) | IGP flags |
| route_tag | bigint | Route tag |
| **sr_prefix_sids** | **varchar(255)** | **SR Prefix SIDs (node SIDs)** |
| iswithdrawn | boolean | Withdrawn flag |
### ls_nodes_log, ls_links_log, ls_prefixes_log (TimescaleDB)
Historical link-state changes. Same columns as parent tables plus `id` (bigint) and timestamp as partition key.
---
## Statistics Tables (TimescaleDB)
| Table | Purpose | Key Columns |
|-------|---------|-------------|
| **stat_reports** | BMP stat messages per peer | prefixes_rejected, known_dup_prefixes, num_routes_adj_rib_in, num_routes_local_rib |
| **stats_chg_byprefix** | Per-prefix update/withdrawal counts | interval_time, prefix, updates, withdraws |
| **stats_chg_byasn** | Per-ASN update/withdrawal counts | interval_time, origin_as, updates, withdraws |
| **stats_chg_bypeer** | Per-peer update/withdrawal counts | interval_time, updates, withdraws |
| **stats_peer_rib** | Per-peer RIB size over time | interval_time, v4_prefixes, v6_prefixes |
| **stats_peer_update_counts** | Update rate statistics | interval_time, advertise_avg/min/max, withdraw_avg/min/max |
| **stats_ip_origins** | Per-ASN IP prefix counts | interval_time, asn, v4_prefixes, v6_prefixes, v4_with_rpki, v4_with_irr |
| **stats_l3vpn_chg_byprefix** | L3VPN per-prefix stats | interval_time, rd, prefix, updates, withdraws |
| **stats_l3vpn_chg_bypeer** | L3VPN per-peer stats | interval_time, updates, withdraws |
| **stats_l3vpn_chg_byrd** | L3VPN per-RD stats | interval_time, rd, updates, withdraws |
---
## Reference & Enrichment Tables
| Table | Purpose | Key Columns |
|-------|---------|-------------|
| **rpki_validator** | RPKI ROAs | prefix, prefix_len, prefix_len_max, origin_as |
| **info_asn** | ASN WHOIS/IRR data | asn, as_name, org_name, country, source |
| **info_route** | Route IRR data | prefix, origin_as, descr, source |
| **geo_ip** | IP geolocation (DB-IP) | ip, country, city, latitude, longitude, isp_name |
| **pdb_exchange_peers** | PeeringDB IXP peering | ix_name, peer_name, peer_asn, speed, peer_ipv4/ipv6 |
---
## Views
| View | Joins | Purpose |
|------|-------|---------|
| **v_peers** | bgp_peers + routers + info_asn | Complete peer info with router name and ASN details |
| **v_ip_routes** | ip_rib + bgp_peers + base_attrs + routers | Full route detail with path attributes |
| **v_ip_routes_geo** | v_ip_routes + geo_ip | Routes with geolocation |
| **v_ip_routes_history** | ip_rib_log + base_attrs + bgp_peers + routers | Historical route changes with attributes |
| **v_l3vpn_routes** | l3vpn_rib + bgp_peers + base_attrs + routers | L3VPN routes with path attributes |
| **v_l3vpn_routes_history** | l3vpn_rib_log + base_attrs + bgp_peers + routers | Historical L3VPN changes |
| **v_ls_nodes** | ls_nodes + base_attrs + bgp_peers + routers | Link-state nodes with peer/router info |
| **v_ls_links** | ls_links + ls_nodes(x2) + routers | Links with local/remote node names + all TE/SR fields |
| **v_ls_prefixes** | ls_prefixes + ls_nodes + routers | LS prefixes with originating node info |
---
## Custom Enum Types
| Type | Values |
|------|--------|
| **opstate** | up, down |
| **ls_proto** | IS-IS_L1, IS-IS_L2, OSPFv2, OSPFv3, Direct, Static |
| **ospf_route_type** | Intra, Inter, Ext-1, Ext-2, NSSA-1, NSSA-2 |
| **ls_mpls_proto_mask** | MPLS protocol bitmask |
| **user_role** | admin, oper |
---
## Key Query Patterns
### Get all active routes with full attributes
```sql
SELECT r.prefix, r.prefix_len, ba.origin_as, ba.as_path,
ba.med, ba.local_pref, ba.community_list, ba.next_hop
FROM ip_rib r
JOIN base_attrs ba ON ba.hash_id = r.base_attr_hash_id
WHERE r.iswithdrawn = false AND r.isipv4 = true
```
### Get link-state topology with TE attributes
```sql
SELECT local_router_name, remote_router_name,
igp_metric, te_def_metric, max_link_bw, admin_group, srlg,
sr_adjacency_sids
FROM v_ls_links
WHERE peer_hash_id = '<peer_hash>' AND iswithdrawn = false
```
### Time-series RIB changes
```sql
SELECT date_trunc('minute', timestamp) as time,
SUM(CASE WHEN iswithdrawn = false THEN 1 ELSE 0 END) as ads,
SUM(CASE WHEN iswithdrawn = true THEN 1 ELSE 0 END) as withdrawals
FROM ip_rib_log
WHERE timestamp > NOW() - INTERVAL '24 hours'
GROUP BY 1 ORDER BY 1
```
### RPKI validation status
```sql
SELECT CASE
WHEN rv.origin_as IS NOT NULL AND rv.origin_as = r.origin_as THEN 'Valid'
WHEN rv.origin_as IS NOT NULL THEN 'Invalid'
ELSE 'NotFound'
END as status,
COUNT(*)
FROM ip_rib r
LEFT JOIN rpki_validator rv ON rv.prefix = r.prefix AND rv.prefix_len = r.prefix_len
WHERE r.iswithdrawn = false
GROUP BY 1
```

262
DOCS.md
View File

@ -16,8 +16,6 @@
12. [Troubleshooting](#12-troubleshooting)
13. [Data Retention](#13-data-retention)
14. [Environment Variables Reference](#14-environment-variables-reference)
15. [gNMI Streaming Telemetry (Phase 4)](#15-gnmi-streaming-telemetry-phase-4)
16. [Traffic Generator (Phase 4)](#16-traffic-generator-phase-4)
---
@ -30,7 +28,7 @@ This is a **BGP Monitoring Platform (BMP) lab stack** deployed via Docker Compos
- Receives BMP (BGP Monitoring Protocol, RFC 7854) telemetry from routers on TCP port 5000
- Streams BMP data through Kafka into a TimescaleDB/PostgreSQL database
- Provides **30 Grafana dashboards** (17 operational + 6 learning + 4 advanced analytics + 3 streaming telemetry) for real-time and historical BGP analysis
- Provides **23 Grafana dashboards** (17 operational + 6 learning-focused) for real-time and historical BGP analysis
- Includes an **ExaBGP route injector** that peers with the two CORE routers and injects synthetic BGP routes, enabling testing of BGP policy, route propagation, and Grafana dashboards without needing internet connectivity
- Provides a **Vue 3 web UI** at `:5001` for point-and-click scenario management, live route tables, and peer monitoring
@ -66,7 +64,7 @@ IOS-XR Routers (9x, AS 65020)
PostgreSQL 14 + TimescaleDB
|
+---------> obmp-grafana (grafana/grafana:9.1.7) :3000
| 30 dashboards, PostgreSQL + InfluxDB datasources
| 23 dashboards, PostgreSQL datasource
+---------> obmp-whois (openbmp/whois:2.2.0) :4300
WHOIS query server backed by the DB
@ -75,24 +73,6 @@ ExaBGP (obmp-exabgp, built locally)
Peers eBGP to CORE-01 and CORE-02 (AS 65100 -> AS 65020)
HTTP API on :5050 — inject/withdraw routes on demand
Routes propagate via iBGP mesh to all 9 routers -> BMP -> DB -> Grafana
gNMI Streaming Telemetry (Phase 4):
IOS-XR Routers (gRPC :57400)
|
v
obmp-telegraf (telegraf:1.28 + gnmi plugin)
|
v
obmp-influxdb (influxdb:2.7) :8086
|
v
obmp-grafana (InfluxDB datasource -> Telemetry dashboards)
Traffic Generator (Phase 4):
obmp-traffic-gen (python:3.11 + Scapy + Flask) :5051
Dual-mode: sender (generate traffic) / responder (echo/log)
RFC 2544 testing, custom packet flows
obmp-traffic-gen-ui (Vue 3 + NGINX) :5002
```
### Container Summary
@ -107,11 +87,7 @@ Traffic Generator (Phase 4):
| obmp-grafana | grafana/grafana:9.1.7 | 3000 | Visualization |
| obmp-whois | openbmp/whois:2.2.0 | 4300 | WHOIS query server |
| obmp-exabgp | local build | 5050 (host net) | BGP route injector |
| obmp-exabgp-ui | local build | 5001 (host net) | Route injector web UI |
| obmp-influxdb | influxdb:2.7 | 8086 | Time-series DB for telemetry |
| obmp-telegraf | local build | - (host net) | gNMI telemetry collector |
| obmp-traffic-gen | local build | 5051 (host net) | Scapy traffic generator |
| obmp-traffic-gen-ui | local build | 5002 (host net) | Traffic generator web UI |
| obmp-exabgp-ui | local build | 5001 (host net) | Vue 3 web control panel |
---
@ -336,9 +312,6 @@ python3 inject.py scenarios
| `convergence_test` | 10 | Prefixes for timing BGP convergence — announce then check ip_rib_log timestamps |
| `route_leak` | 10 | Real prefixes re-announced with short AS paths — simulates a route leak (community 65100:999) |
| `hijack_simulation` | 10 | Prefixes claimed directly by AS 65100 — simulates a prefix hijack (community 65100:hijack) |
| `te_community_steering` | 15 | Routes tagged with TE communities for color-based steering (65020:100=red, 65020:200=blue, 65020:300=green) |
| `origin_shift` | 5 | Prefixes with changed origin AS — simulates origin migration for anomaly detection |
| `path_diversity` | 10 | Same prefixes with different AS paths/MEDs — demonstrates best-path selection |
### 7.4 Load a scenario
@ -522,23 +495,6 @@ Six learning-focused dashboards in a separate folder, designed to teach BGP conc
> **RPKI note:** The `rpki_validator` table is populated by a cron job in `psql-app` every 2 hours. Dashboard `obmp-learn-04` will show zero counts until the cron runs — check `ENABLE_RPKI=1` in `docker-compose.yml`.
### Advanced Analytics Dashboards (folder: `OBMP-Learning`)
Four advanced dashboards that go beyond basic BMP monitoring, unlocking TE/SR data and providing heuristic analysis.
| Dashboard | UID | What it provides |
|-----------|-----|-----------------|
| Database Schema Map | `obmp-learn-07` | Interactive schema reference — live table row counts, entity relationships, column details for all 33 tables and 11 views |
| TE & Segment Routing Analytics | `obmp-learn-08` | Exposes TE/SR fields from BGP-LS: link bandwidth, admin groups, SRLG, SR SIDs, adjacency SIDs, protection types |
| Topology Change & Anomaly Detection | `obmp-learn-09` | Heuristic analysis: link state changes over time, origin AS hijack detection, convergence timeline, route consistency |
| Link Utilization & TE Thought Experiment | `obmp-learn-10` | BGP-LS capacity data (bandwidth, TE metrics) + integration guide for streaming telemetry (gNMI/MDT) |
> **TE/SR data note:** Some TE fields (admin_group, max_link_bw, srlg, sr_adjacency_sids) may be NULL if routers don't advertise those TLVs. Enable `mpls traffic-eng` under IS-IS and `segment-routing mpls` for full data.
### Database Schema Reference
A standalone database schema reference is also available at `DB_SCHEMA.md` in the repo root. It documents all 33 tables, 11 views, TE/SR columns, enum types, and common query patterns.
---
## 10. Sanity Checks
@ -854,215 +810,3 @@ Adjust in `docker-compose.yml` under the `psql-app` service environment block.
| Variable | Default | Description |
|----------|---------|-------------|
| `EXABGP_API` | `http://localhost:5050` | ExaBGP API base URL |
---
## 15. gNMI Streaming Telemetry (Phase 4)
### Overview
gNMI (gRPC Network Management Interface) adds **data-plane visibility** alongside BMP's control-plane monitoring. Telegraf collects real-time interface counters from all 9 IOS-XR routers via gNMI subscriptions and stores them in InfluxDB. Grafana queries InfluxDB for telemetry dashboards.
### Architecture
```
IOS-XR Routers (9x, gRPC port 57400)
|
gNMI subscriptions (10s sample)
|
v
obmp-telegraf (telegraf:1.28 + gnmi input plugin)
host networking → reaches routers on 10.100.0.x
|
v
obmp-influxdb (influxdb:2.7, port 8086)
bucket: "telemetry", org: "openbmp"
|
v
obmp-grafana (InfluxDB datasource, Flux queries)
3 dashboards in OBMP-Telemetry folder
```
### Enabling gRPC on Routers
The routers need gRPC enabled before Telegraf can collect telemetry. A NETCONF script is provided:
```bash
# From the host (requires ncclient: pip install ncclient)
cd /home/user/obmp-docker/gnmi
python3 gnmi_grpc_config.py
```
This connects to all 9 routers via NETCONF (port 830, credentials webui/cisco) and pushes:
```
grpc
port 57400
no-tls
```
**Verify on router:**
```
show grpc status
```
Expected: gRPC listening on port 57400.
### Telemetry Data Collected
Telegraf subscribes to two IOS-XR YANG paths at 10-second intervals:
| Subscription | YANG Path | Data |
|-------------|-----------|------|
| interface_counters | `Cisco-IOS-XR-infra-statsd-oper:infra-statistics/interfaces/interface/latest/generic-counters` | bytes/packets in/out, errors, drops, CRC |
| interface_rates | `Cisco-IOS-XR-infra-statsd-oper:infra-statistics/interfaces/interface/latest/data-rate` | bits/sec in/out, packet rate |
### InfluxDB Access
- **URL:** `http://localhost:8086`
- **Org:** `openbmp`
- **Bucket:** `telemetry`
- **Token:** `openbmp-telemetry-token`
- **Retention:** 30 days
### Grafana Telemetry Dashboards
Three dashboards in the **OBMP-Telemetry** folder:
| Dashboard | UID | Description |
|-----------|-----|-------------|
| Interface Utilization | obmp-telem-01 | Input/output bytes rate, packets rate, top interfaces by throughput |
| Interface Errors | obmp-telem-02 | CRC errors, input/output errors, drops, overruns |
| Combined BMP + Telemetry | obmp-telem-03 | Mixed datasource — BGP peer status (PostgreSQL) alongside interface counters (InfluxDB) |
All dashboards have `$router` and `$interface` template variables for filtering.
### Troubleshooting gNMI
```bash
# Check Telegraf logs for gNMI connection status
docker logs obmp-telegraf --tail 50
# Verify InfluxDB has data
curl -s -H "Authorization: Token openbmp-telemetry-token" \
"http://localhost:8086/api/v2/query?org=openbmp" \
--data-urlencode 'q=from(bucket:"telemetry") |> range(start: -5m) |> limit(n:5)'
# Check InfluxDB health
curl http://localhost:8086/health
```
---
## 16. Traffic Generator (Phase 4)
### Overview
A portable, containerized traffic generator with a web UI for RFC 2544 testing and custom packet flows. Built with Scapy + Flask (backend) and Vue 3 + NGINX (frontend). The container supports **dual-mode operation**: sender (generate traffic) or responder (receive/echo packets).
### Accessing the UI
- **Web UI:** `http://localhost:5002`
- **API:** `http://localhost:5051`
### Dual-Mode Operation
Set via `TRAFFIC_GEN_MODE` environment variable in `docker-compose.yml`:
| Mode | Description |
|------|-------------|
| `sender` (default) | Generates traffic, runs RFC 2544 tests, sends custom flows |
| `responder` | Listens for incoming test packets, echoes/timestamps them, reports receive stats |
**Typical deployment:** One instance as `sender` on the host, optionally a second instance as `responder` on another endpoint. Without a responder, the sender uses ICMP echo for latency measurement (routers respond natively).
### Creating Flows
Use the **Flow Builder** panel (left sidebar) in the UI:
| Field | Default | Description |
|-------|---------|-------------|
| Name | - | Human-readable flow name |
| Destination IP | `10.100.0.100` | Target router IP |
| Source IP | `10.40.40.202` | Host IP |
| Protocol | UDP | UDP, TCP, or ICMP |
| Source Port | 50000 | (UDP/TCP only) |
| Destination Port | 5001 | (UDP/TCP only) |
| Frame Size | 512 | Packet size in bytes |
| Rate (pps) | 1000 | Packets per second |
| Duration | 30 | Seconds (0 = infinite) |
| DSCP | 0 | Differentiated Services Code Point |
After creating a flow, use the **Flows** tab to Start/Stop/Delete flows.
### RFC 2544 Testing
Use the **Tests** tab to configure and run RFC 2544 tests:
| Test Type | Description |
|-----------|-------------|
| **Throughput** | Binary search for maximum zero-loss forwarding rate |
| **Latency** | Measure round-trip time at determined throughput rate |
| **Frame Loss** | Loss percentage vs. offered load curve |
| **Back-to-Back** | Maximum burst length at line rate with zero loss |
**Parameters:**
- **Base Flow:** Select a previously created flow as the test template
- **Frame Sizes:** Standard sizes: 64, 128, 256, 512, 1024, 1280, 1518 bytes
- **Trial Duration:** Per-frame-size test duration (5–300 sec)
- **Max Rate (pps):** Upper bound for binary search
- **Acceptable Loss %:** Threshold for pass/fail
### Quick Presets
Six built-in presets are available in the **Tests** tab:
| Preset | Description |
|--------|-------------|
| quick_icmp | ICMP ping to CORE-01 at 10 pps |
| udp_flood_small | 64-byte UDP at 5000 pps |
| udp_flood_large | 1518-byte UDP at 1000 pps |
| rfc2544_throughput | Full throughput test with standard frame sizes |
| rfc2544_latency | Latency measurement with standard frame sizes |
| tcp_session | TCP flow at 500 pps |
### API Reference
| Method | Path | Description |
|--------|------|-------------|
| GET | `/healthz` | Health check + engine status |
| GET | `/interfaces` | Available network interfaces |
| GET | `/mode` | Current mode (sender/responder) |
| GET/POST | `/flows` | List / create flows |
| GET/PUT/DELETE | `/flows/<id>` | Get / update / delete flow |
| POST | `/flows/<id>/start` | Start sending |
| POST | `/flows/<id>/stop` | Stop sending |
| GET | `/flows/<id>/stats` | Real-time stats for a flow |
| GET/POST | `/tests` | List / create RFC 2544 tests |
| GET | `/tests/<id>` | Test details + results |
| POST | `/tests/<id>/start` | Start test execution |
| POST | `/tests/<id>/stop` | Abort test |
| GET | `/tests/<id>/results` | Exportable results |
| GET | `/presets` | Available test presets |
| POST | `/presets/<name>` | Create flow + test from preset |
| GET | `/stats/history` | Stats ring buffer (300 samples) |
| GET | `/responder/stats` | Responder-mode receive stats |
| POST | `/responder/reset` | Reset responder counters |
### Integration with gNMI Telemetry
The key value of combining the traffic generator with gNMI: **send traffic while watching real-time interface counters**.
1. Create a UDP flow targeting a router (e.g., R9K-01 at 10.100.0.1)
2. Open the Grafana **Interface Utilization** dashboard, select that router
3. Start the flow — gNMI counters show traffic appearing on the interface
4. Run an RFC 2544 throughput test — Grafana shows the stepped traffic pattern from binary search iterations
5. Compare Scapy-reported stats with gNMI-reported counters for cross-validation
The **Combined BMP + Telemetry** dashboard shows both control-plane (BMP BGP updates) and data-plane (gNMI interface counters) side by side, enabling correlation of BGP changes with traffic impact.
### Environment Variables
| Variable | Default | Description |
|----------|---------|-------------|
| `TRAFFIC_GEN_API_PORT` | `5051` | Flask API listen port |
| `TRAFFIC_GEN_MODE` | `sender` | Operating mode: `sender` or `responder` |
| `INFLUXDB_TOKEN` | `openbmp-telemetry-token` | InfluxDB auth token (Telegraf) |

56
docker-compose.yml
View File

@ -231,62 +231,6 @@ services:
network_mode: host
# Serves on port 5001 (host network, defined in nginx.conf)
# --- Phase 4: gNMI Streaming Telemetry ---
influxdb:
restart: unless-stopped
container_name: obmp-influxdb
image: influxdb:2.7
ports:
- "8086:8086"
volumes:
- ${OBMP_DATA_ROOT}/influxdb:/var/lib/influxdb2
environment:
- DOCKER_INFLUXDB_INIT_MODE=setup
- DOCKER_INFLUXDB_INIT_USERNAME=openbmp
- DOCKER_INFLUXDB_INIT_PASSWORD=openbmp123
- DOCKER_INFLUXDB_INIT_ORG=openbmp
- DOCKER_INFLUXDB_INIT_BUCKET=telemetry
- DOCKER_INFLUXDB_INIT_ADMIN_TOKEN=openbmp-telemetry-token
- DOCKER_INFLUXDB_INIT_RETENTION=30d
telegraf:
restart: unless-stopped
container_name: obmp-telegraf
build:
context: ./telegraf
dockerfile: Dockerfile
network_mode: host
depends_on:
- influxdb
environment:
- INFLUXDB_TOKEN=openbmp-telemetry-token
# --- Phase 4: Traffic Generator ---
traffic-gen:
restart: unless-stopped
container_name: obmp-traffic-gen
build:
context: ./traffic-gen
dockerfile: Dockerfile
network_mode: host
cap_add:
- NET_RAW
- NET_ADMIN
environment:
- TRAFFIC_GEN_API_PORT=5051
- TRAFFIC_GEN_MODE=sender
traffic-gen-ui:
restart: unless-stopped
container_name: obmp-traffic-gen-ui
build:
context: ./traffic-gen-ui
dockerfile: Dockerfile
network_mode: host
# Serves on port 5002 (host network, defined in nginx.conf)
whois:
restart: unless-stopped
container_name: obmp-whois

86
exabgp/scenarios/__init__.py
View File

@ -363,80 +363,6 @@ _HIJACK_ROUTES = [
]
# ---------------------------------------------------------------------------
# Scenario: te_community_steering
# Routes tagged with TE communities representing different "colors" for
# community-based TE policy steering. Shows how communities drive path
# selection when routers apply route-policy based on community values.
# ---------------------------------------------------------------------------
_TE_COMMUNITY_ROUTES = [
# Red paths (community 65020:100) — high-priority, low-latency
_r('10.210.0.0/24', [65100, 65020], communities=['65020:100'], med=10),
_r('10.210.1.0/24', [65100, 65020], communities=['65020:100'], med=10),
_r('10.210.2.0/24', [65100, 65020], communities=['65020:100'], med=10),
_r('10.210.3.0/24', [65100, 65020], communities=['65020:100'], med=10),
_r('10.210.4.0/24', [65100, 65020], communities=['65020:100'], med=10),
# Blue paths (community 65020:200) — bulk transfer, cost-optimized
_r('10.220.0.0/24', [65100, 65020, 3356], communities=['65020:200'], med=100),
_r('10.220.1.0/24', [65100, 65020, 3356], communities=['65020:200'], med=100),
_r('10.220.2.0/24', [65100, 65020, 3356], communities=['65020:200'], med=100),
_r('10.220.3.0/24', [65100, 65020, 3356], communities=['65020:200'], med=100),
_r('10.220.4.0/24', [65100, 65020, 3356], communities=['65020:200'], med=100),
# Green paths (community 65020:300) — backup/diverse paths
_r('10.230.0.0/24', [65100, 65020, 1299, 6762], communities=['65020:300'], med=200),
_r('10.230.1.0/24', [65100, 65020, 1299, 6762], communities=['65020:300'], med=200),
_r('10.230.2.0/24', [65100, 65020, 1299, 6762], communities=['65020:300'], med=200),
_r('10.230.3.0/24', [65100, 65020, 1299, 6762], communities=['65020:300'], med=200),
_r('10.230.4.0/24', [65100, 65020, 1299, 6762], communities=['65020:300'], med=200),
]
# ---------------------------------------------------------------------------
# Scenario: origin_shift
# Simulates an origin AS change: prefixes initially associated with
# well-known origin ASNs are re-announced with a different origin.
# Use: load internet_sample first, then load origin_shift to see the
# origin_as column change in ip_rib_log (visible on Anomaly dashboard).
# ---------------------------------------------------------------------------
_ORIGIN_SHIFT_ROUTES = [
# These prefixes overlap with internet_sample but have different origin ASNs
_r('8.8.8.0/24', [65100, 64999], communities=['65100:origin-shift']), # was 15169 (Google)
_r('1.1.1.0/24', [65100, 64998], communities=['65100:origin-shift']), # was 13335 (Cloudflare)
_r('9.9.9.0/24', [65100, 64997], communities=['65100:origin-shift']), # was 19281 (Quad9)
_r('208.67.222.0/24', [65100, 64996], communities=['65100:origin-shift']), # was 36692 (OpenDNS)
_r('156.154.70.0/24', [65100, 64995], communities=['65100:origin-shift']), # was 19318 (Neustar)
]
# ---------------------------------------------------------------------------
# Scenario: path_diversity
# Multiple announcements of the same prefix with different AS paths,
# MEDs, and communities. Demonstrates best-path selection:
# - Shorter AS path wins (unless local-pref overrides)
# - Lower MED preferred among paths from same neighbor AS
# - Communities tag paths for policy identification
# ---------------------------------------------------------------------------
_PATH_DIVERSITY_ROUTES = [
# Prefix 1: 3 paths with varying length and MED
_r('10.250.0.0/24', [65100, 174], communities=['65100:path-a'], med=50),
_r('10.250.0.0/24', [65100, 174, 3356], communities=['65100:path-b'], med=100),
_r('10.250.0.0/24', [65100, 174, 3356, 15169], communities=['65100:path-c'], med=150),
# Prefix 2: paths with same length but different MED
_r('10.250.1.0/24', [65100, 1299, 15169], communities=['65100:low-med'], med=10),
_r('10.250.1.0/24', [65100, 3356, 15169], communities=['65100:high-med'], med=500),
# Prefix 3: local-pref override (higher local-pref wins over shorter path)
_r('10.250.2.0/24', [65100, 2914], communities=['65100:low-lp'], local_pref=50),
_r('10.250.2.0/24', [65100, 2914, 7018], communities=['65100:high-lp'], local_pref=200),
# Prefix 4: transit diversity
_r('10.250.3.0/24', [65100, 174, 32934], communities=['65100:via-cogent']),
_r('10.250.3.0/24', [65100, 3356, 32934], communities=['65100:via-lumen']),
_r('10.250.3.0/24', [65100, 2914, 32934], communities=['65100:via-ntt']),
]
# ---------------------------------------------------------------------------
# Registry
# ---------------------------------------------------------------------------
@ -478,16 +404,4 @@ SCENARIOS = {
'description': '10 prefixes announced as if directly originated by AS 65100 — simulates a prefix hijack (community 65100:hijack)',
'routes': _HIJACK_ROUTES,
},
'te_community_steering': {
'description': 'Routes tagged with TE communities for color-based steering (65020:100=red, 65020:200=blue, 65020:300=green)',
'routes': _TE_COMMUNITY_ROUTES,
},
'origin_shift': {
'description': '5 prefixes with changed origin AS — simulates origin migration/hijack for anomaly detection',
'routes': _ORIGIN_SHIFT_ROUTES,
},
'path_diversity': {
'description': 'Same prefixes with different AS paths and MEDs — demonstrates best-path selection and path diversity',
'routes': _PATH_DIVERSITY_ROUTES,
},
}

158
gnmi/gnmi_grpc_config.py
View File

@ -1,158 +0,0 @@
#!/usr/bin/env python3
"""
gNMI gRPC Configuration Script
===============================
Enables gRPC dial-in telemetry on all 9 IOS-XR routers so that
Telegraf (or any gNMI collector) can subscribe to streaming
telemetry data.
What this script applies per router:
- gRPC server on port 57400 with TLS disabled
Uses SSH/CLI (paramiko) instead of NETCONF because IOS-XR 24.3.1
rejects the NETCONF edit-config for gRPC with "Need to enable GRPC first".
Router targets:
CORE-01 (10.100.0.100)
CORE-02 (10.100.0.200)
R9K-01 (10.100.0.1) through R9K-07 (10.100.0.7)
"""
import paramiko
import time
import sys
ROUTERS = [
('10.100.0.100', 'CORE-01'),
('10.100.0.200', 'CORE-02'),
('10.100.0.1', 'R9K-01'),
('10.100.0.2', 'R9K-02'),
('10.100.0.3', 'R9K-03'),
('10.100.0.4', 'R9K-04'),
('10.100.0.5', 'R9K-05'),
('10.100.0.6', 'R9K-06'),
('10.100.0.7', 'R9K-07'),
]
USERNAME = 'webui'
PASSWORD = 'cisco'
GRPC_PORT = 57400
CONFIG_COMMANDS = [
'configure terminal',
'grpc',
f'port {GRPC_PORT}',
'no-tls',
'commit',
'end',
]
def configure_router(mgmt_ip, label):
"""Apply gRPC configuration via SSH CLI."""
print(f"\n{'─'*60}")
print(f" Configuring {label} ({mgmt_ip})")
print(f"{'─'*60}")
print(f" Applying: gRPC port={GRPC_PORT} no-tls")
try:
client = paramiko.SSHClient()
client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
client.connect(mgmt_ip, username=USERNAME, password=PASSWORD, timeout=10)
shell = client.invoke_shell()
time.sleep(1)
shell.recv(65535) # clear banner
for cmd in CONFIG_COMMANDS:
shell.send(cmd + '\n')
time.sleep(1.5)
output = shell.recv(65535).decode()
client.close()
if 'error' in output.lower() or 'fail' in output.lower():
print(f" ✗ ERROR on {label}: {output.strip()}")
return False
print(f" ✓ {label} done.")
return True
except Exception as e:
print(f" ✗ ERROR on {label}: {e}")
return False
def verify_router(mgmt_ip, label):
"""Verify gRPC configuration via SSH."""
try:
client = paramiko.SSHClient()
client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
client.connect(mgmt_ip, username=USERNAME, password=PASSWORD, timeout=10)
shell = client.invoke_shell()
time.sleep(1)
shell.recv(65535)
shell.send('show running-config grpc\n')
time.sleep(3)
output = shell.recv(65535).decode()
client.close()
has_port = f'port {GRPC_PORT}' in output
has_notls = 'no-tls' in output
p = '✓' if has_port else '✗'
t = '✓' if has_notls else '✗'
status = 'OK' if (has_port and has_notls) else 'INCOMPLETE'
print(f" {label:8s} grpc-port={p} no-tls={t} [{status}]")
return has_port and has_notls
except Exception as e:
print(f" {label:8s} verify error: {e}")
return False
def main():
print("gNMI gRPC Configuration Script")
print("================================")
print(f"Targets: all {len(ROUTERS)} routers")
print()
results = []
for mgmt_ip, label in ROUTERS:
ok = configure_router(mgmt_ip, label)
results.append((mgmt_ip, label, ok))
# Verification pass
print(f"\n{'='*60}")
print("Post-apply verification")
print('='*60)
print(f" {'Router':8s} {'gRPC Port':9s} {'No-TLS':6s} Status")
all_ok = True
for mgmt_ip, label, applied_ok in results:
if applied_ok:
if not verify_router(mgmt_ip, label):
all_ok = False
else:
print(f" {label:8s} skipped (apply failed)")
all_ok = False
failed = [label for _, label, ok in results if not ok]
print()
if failed:
print(f"FAILED: {', '.join(failed)}")
sys.exit(1)
elif all_ok:
print("All routers configured successfully.")
print()
print(f"gRPC is now listening on port {GRPC_PORT} (no TLS) on all routers.")
print("Next: start Telegraf with gNMI input plugin to begin collecting telemetry.")
else:
print("Some routers may have incomplete configuration. Check output above.")
sys.exit(1)
if __name__ == '__main__':
main()

131
obmp-grafana/dashboards/Learning/db_schema_map.json
View File

@ -1,131 +0,0 @@
{
"uid": "obmp-learn-07",
"title": "Database Schema Map",
"schemaVersion": 39,
"tags": ["obmp-learning"],
"editable": true,
"time": {
"from": "now-6h",
"to": "now"
},
"templating": {
"list": []
},
"panels": [
{
"id": 1,
"title": "Table Row Counts",
"type": "table",
"gridPos": { "h": 12, "w": 8, "x": 0, "y": 0 },
"datasource": { "type": "postgres", "uid": "obmp_postgres" },
"targets": [
{
"refId": "A",
"datasource": { "type": "postgres", "uid": "obmp_postgres" },
"rawSql": "SELECT 'routers' as table_name, count(*) as rows FROM routers\nUNION ALL SELECT 'collectors', count(*) FROM collectors\nUNION ALL SELECT 'bgp_peers', count(*) FROM bgp_peers\nUNION ALL SELECT 'peer_event_log', count(*) FROM peer_event_log\nUNION ALL SELECT 'base_attrs', count(*) FROM base_attrs\nUNION ALL SELECT 'ip_rib', count(*) FROM ip_rib\nUNION ALL SELECT 'ip_rib_log', count(*) FROM ip_rib_log\nUNION ALL SELECT 'l3vpn_rib', count(*) FROM l3vpn_rib\nUNION ALL SELECT 'global_ip_rib', count(*) FROM global_ip_rib\nUNION ALL SELECT 'ls_nodes', count(*) FROM ls_nodes\nUNION ALL SELECT 'ls_links', count(*) FROM ls_links\nUNION ALL SELECT 'ls_prefixes', count(*) FROM ls_prefixes\nUNION ALL SELECT 'ls_nodes_log', count(*) FROM ls_nodes_log\nUNION ALL SELECT 'ls_links_log', count(*) FROM ls_links_log\nUNION ALL SELECT 'ls_prefixes_log', count(*) FROM ls_prefixes_log\nUNION ALL SELECT 'rpki_validator', count(*) FROM rpki_validator\nUNION ALL SELECT 'info_asn', count(*) FROM info_asn\nUNION ALL SELECT 'info_route', count(*) FROM info_route\nUNION ALL SELECT 'stat_reports', count(*) FROM stat_reports\nUNION ALL SELECT 'geo_ip', count(*) FROM geo_ip\nORDER BY table_name",
"format": "table"
}
]
},
{
"id": 2,
"title": "Table Relationships",
"type": "text",
"gridPos": { "h": 12, "w": 8, "x": 8, "y": 0 },
"options": {
"mode": "markdown",
"content": "## Entity Relationships\n\n### BMP Core Chain\n```\ncollectors\n └── routers (collector_hash_id)\n └── bgp_peers (router_hash_id)\n ├── ip_rib (peer_hash_id)\n ├── ip_rib_log (peer_hash_id)\n ├── l3vpn_rib (peer_hash_id)\n ├── ls_nodes (peer_hash_id)\n ├── ls_links (peer_hash_id)\n ├── ls_prefixes (peer_hash_id)\n ├── peer_event_log (peer_hash_id)\n └── stat_reports (peer_hash_id)\n```\n\n### Path Attributes\n```\nip_rib ──(base_attr_hash_id)──► base_attrs\n │ ├── as_path (bigint[])\n │ ├── origin_as\n │ ├── next_hop\n │ ├── med / local_pref\n │ ├── community_list[]\n │ ├── ext_community_list[]\n │ └── large_community_list[]\n │\n └──(prefix)──► global_ip_rib\n ├── rpki_origin_as\n ├── irr_origin_as\n └── num_peers\n```\n\n### Link-State Topology\n```\nls_nodes ◄── ls_links (local_node_hash_id, remote_node_hash_id)\nls_nodes ◄── ls_prefixes (local_node_hash_id)\n```\n\n### Reference Data\n```\nrpki_validator ──(prefix, origin_as)──► validates ip_rib\ninfo_asn ──(asn)──► enriches base_attrs.origin_as\ninfo_route ──(prefix)──► enriches ip_rib.prefix\ngeo_ip ──(ip)──► geolocates routers, peers\n```"
}
},
{
"id": 3,
"title": "BMP Core Tables",
"type": "text",
"gridPos": { "h": 8, "w": 8, "x": 16, "y": 0 },
"options": {
"mode": "markdown",
"content": "## BMP Core Tables\n\n| Table | Purpose | Key Columns |\n|-------|---------|-------------|\n| **routers** | BMP-monitored routers | hash_id, name, ip_address, router_as, state, bgp_id |\n| **collectors** | BMP collector instances | hash_id, admin_id, name, ip_address, router_count |\n| **bgp_peers** | BGP sessions per router | hash_id, router_hash_id, peer_addr, peer_as, state, isl3vpnpeer |\n| **peer_event_log** | Session state history (TimescaleDB) | peer_hash_id, state, timestamp, bmp_reason, bgp_err_code |\n| **stat_reports** | BMP statistics messages | peer_hash_id, prefixes_rejected, num_routes_adj_rib_in, num_routes_local_rib |\n| **users** | Access control | username, password, type (admin/oper) |"
}
},
{
"id": 4,
"title": "RIB & Path Attribute Tables",
"type": "text",
"gridPos": { "h": 8, "w": 8, "x": 16, "y": 8 },
"options": {
"mode": "markdown",
"content": "## RIB & Path Attribute Tables\n\n| Table | Purpose | Key Columns |\n|-------|---------|-------------|\n| **base_attrs** | BGP path attributes | hash_id, as_path[], as_path_count, origin_as, next_hop, med, local_pref, community_list[], ext_community_list[], large_community_list[], cluster_list, originator_id |\n| **ip_rib** | IPv4/IPv6 unicast RIB | hash_id, peer_hash_id, prefix, prefix_len, origin_as, iswithdrawn, labels, path_id |\n| **ip_rib_log** | RIB change history (TimescaleDB) | peer_hash_id, prefix, prefix_len, origin_as, iswithdrawn, timestamp |\n| **l3vpn_rib** | L3VPN/MPLS VPN routes | hash_id, peer_hash_id, rd, prefix, labels, ext_community_list[] |\n| **l3vpn_rib_log** | L3VPN change history (TimescaleDB) | peer_hash_id, rd, prefix, iswithdrawn, timestamp |\n| **global_ip_rib** | Aggregated prefix summary | prefix, recv_origin_as, rpki_origin_as, irr_origin_as, num_peers |"
}
},
{
"id": 5,
"title": "Link-State Tables",
"type": "text",
"gridPos": { "h": 8, "w": 12, "x": 0, "y": 12 },
"options": {
"mode": "markdown",
"content": "## Link-State Tables (BGP-LS / RFC 7752)\n\n| Table | Purpose | Key Columns |\n|-------|---------|-------------|\n| **ls_nodes** | IS-IS/OSPF nodes | hash_id, peer_hash_id, igp_router_id, name, protocol, asn, sr_capabilities, isis_area_id |\n| **ls_links** | IS-IS/OSPF links + TE/SR | hash_id, local/remote_node_hash_id, interface_addr, neighbor_addr, igp_metric, **te_def_metric**, **max_link_bw**, **max_resv_bw**, **unreserved_bw**, **admin_group**, **srlg**, **sr_adjacency_sids**, **peer_node_sid**, **protection_type**, **mpls_proto_mask** |\n| **ls_prefixes** | IS-IS/OSPF prefixes | hash_id, local_node_hash_id, prefix, metric, sr_prefix_sids, igp_flags |\n| **ls_nodes_log** | Node change history (TimescaleDB) | Same as ls_nodes + timestamp |\n| **ls_links_log** | Link change history (TimescaleDB) | Same as ls_links + timestamp |\n| **ls_prefixes_log** | Prefix change history (TimescaleDB) | Same as ls_prefixes + timestamp |\n\n**Bold columns** = TE/SR fields not used by any existing dashboard"
}
},
{
"id": 6,
"title": "Statistics Tables",
"type": "text",
"gridPos": { "h": 8, "w": 12, "x": 12, "y": 12 },
"options": {
"mode": "markdown",
"content": "## Statistics Tables (TimescaleDB Hypertables)\n\n| Table | Purpose | Key Columns |\n|-------|---------|-------------|\n| **stat_reports** | BMP stat messages | peer_hash_id, prefixes_rejected, known_dup_prefixes, num_routes_adj_rib_in |\n| **stats_chg_byprefix** | Per-prefix churn stats | interval_time, peer_hash_id, prefix, updates, withdraws |\n| **stats_chg_byasn** | Per-ASN churn stats | interval_time, peer_hash_id, origin_as, updates, withdraws |\n| **stats_chg_bypeer** | Per-peer churn stats | interval_time, peer_hash_id, updates, withdraws |\n| **stats_peer_rib** | Per-peer RIB size | interval_time, peer_hash_id, v4_prefixes, v6_prefixes |\n| **stats_peer_update_counts** | Update rate statistics | interval_time, peer_hash_id, advertise_avg/min/max, withdraw_avg/min/max |\n| **stats_ip_origins** | Per-ASN prefix counts | interval_time, asn, v4_prefixes, v6_prefixes, v4_with_rpki, v4_with_irr |"
}
},
{
"id": 7,
"title": "Reference & Enrichment Tables",
"type": "text",
"gridPos": { "h": 6, "w": 12, "x": 0, "y": 20 },
"options": {
"mode": "markdown",
"content": "## Reference & Enrichment Tables\n\n| Table | Purpose | Key Columns |\n|-------|---------|-------------|\n| **rpki_validator** | RPKI ROAs | prefix, prefix_len, prefix_len_max, origin_as |\n| **info_asn** | ASN WHOIS/IRR data | asn, as_name, org_name, country, source |\n| **info_route** | Route IRR data | prefix, prefix_len, origin_as, descr, source |\n| **geo_ip** | IP geolocation (DB-IP) | ip, country, city, latitude, longitude, isp_name |\n| **pdb_exchange_peers** | PeeringDB IXP data | ix_name, peer_name, peer_asn, speed, peer_ipv4/ipv6 |"
}
},
{
"id": 8,
"title": "Views Quick Reference",
"type": "text",
"gridPos": { "h": 6, "w": 12, "x": 12, "y": 20 },
"options": {
"mode": "markdown",
"content": "## Database Views\n\n| View | Joins | Purpose |\n|------|-------|---------|\n| **v_peers** | bgp_peers + routers + info_asn | Complete peer info with router name and ASN details |\n| **v_ip_routes** | ip_rib + bgp_peers + base_attrs + routers | Full route detail with path attributes |\n| **v_ip_routes_geo** | v_ip_routes + geo_ip | Routes with geolocation |\n| **v_ip_routes_history** | ip_rib_log + base_attrs + bgp_peers + routers | Historical route changes with attributes |\n| **v_l3vpn_routes** | l3vpn_rib + bgp_peers + base_attrs + routers | L3VPN routes with path attributes |\n| **v_l3vpn_routes_history** | l3vpn_rib_log + base_attrs + bgp_peers + routers | Historical L3VPN changes |\n| **v_ls_nodes** | ls_nodes + base_attrs + bgp_peers + routers | Link-state nodes with peer/router info |\n| **v_ls_links** | ls_links + ls_nodes(x2) + routers | Links with local/remote node names + TE fields |\n| **v_ls_prefixes** | ls_prefixes + ls_nodes + routers | LS prefixes with originating node info |\n\n### Enum Types\n- **opstate**: up, down\n- **ls_proto**: IS-IS_L1, IS-IS_L2, OSPFv2, OSPFv3, Direct, Static\n- **ospf_route_type**: Intra, Inter, Ext-1, Ext-2, NSSA-1, NSSA-2\n- **ls_mpls_proto_mask**: MPLS protocol bitmask"
}
},
{
"id": 9,
"title": "LinkState Column Details",
"type": "table",
"gridPos": { "h": 10, "w": 12, "x": 0, "y": 26 },
"datasource": { "type": "postgres", "uid": "obmp_postgres" },
"targets": [
{
"refId": "A",
"datasource": { "type": "postgres", "uid": "obmp_postgres" },
"rawSql": "SELECT column_name, data_type, \n CASE \n WHEN column_name IN ('admin_group','max_link_bw','max_resv_bw','unreserved_bw','te_def_metric','protection_type','srlg','sr_adjacency_sids','peer_node_sid','mpls_proto_mask') THEN 'TE/SR'\n WHEN column_name IN ('hash_id','peer_hash_id','base_attr_hash_id','local_node_hash_id','remote_node_hash_id') THEN 'FK/Key'\n ELSE 'Core'\n END as category\nFROM information_schema.columns \nWHERE table_name = 'ls_links' AND table_schema = 'public'\nORDER BY ordinal_position",
"format": "table"
}
]
},
{
"id": 10,
"title": "ip_rib Column Details",
"type": "table",
"gridPos": { "h": 10, "w": 12, "x": 12, "y": 26 },
"datasource": { "type": "postgres", "uid": "obmp_postgres" },
"targets": [
{
"refId": "A",
"datasource": { "type": "postgres", "uid": "obmp_postgres" },
"rawSql": "SELECT column_name, data_type,\n CASE \n WHEN column_name IN ('hash_id','peer_hash_id','base_attr_hash_id') THEN 'FK/Key'\n ELSE 'Core'\n END as category\nFROM information_schema.columns \nWHERE table_name = 'ip_rib' AND table_schema = 'public'\nORDER BY ordinal_position",
"format": "table"
}
]
}
]
}

238
obmp-grafana/dashboards/Learning/link_utilization_te.json
View File

@ -1,238 +0,0 @@
{
"annotations": {
"list": [
{
"builtIn": 1,
"datasource": { "type": "datasource", "uid": "grafana" },
"enable": true,
"hide": true,
"iconColor": "rgba(0, 211, 255, 1)",
"name": "Annotations & Alerts",
"type": "dashboard"
}
]
},
"editable": true,
"fiscalYearStartMonth": 0,
"graphTooltip": 0,
"links": [],
"liveNow": false,
"panels": [
{
"datasource": { "type": "postgres", "uid": "obmp_postgres" },
"fieldConfig": {
"defaults": {
"custom": {
"align": "auto",
"cellOptions": { "type": "auto" },
"inspect": false
}
},
"overrides": [
{
"matcher": { "id": "byName", "options": "Max BW (B/s)" },
"properties": [{ "id": "unit", "value": "Bps" }]
},
{
"matcher": { "id": "byName", "options": "Max Reservable BW" },
"properties": [{ "id": "unit", "value": "Bps" }]
}
]
},
"gridPos": { "h": 10, "w": 24, "x": 0, "y": 0 },
"id": 1,
"options": {
"showHeader": true,
"sortBy": [{ "desc": false, "displayName": "Local Router" }]
},
"targets": [
{
"datasource": { "type": "postgres", "uid": "obmp_postgres" },
"format": "table",
"rawQuery": true,
"rawSql": "SELECT local_router_name as \"Local Router\",\n remote_router_name as \"Remote Router\",\n interface_addr::text as \"Interface IP\",\n neighbor_addr::text as \"Neighbor IP\",\n max_link_bw as \"Max BW (B/s)\",\n max_resv_bw as \"Max Reservable BW\",\n unreserved_bw as \"Unreserved BW\",\n igp_metric as \"IGP Metric\",\n te_def_metric as \"TE Metric\"\nFROM v_ls_links\nWHERE peer_hash_id = '$peer_hash' AND iswithdrawn = false\nORDER BY local_router_name, remote_router_name",
"refId": "A"
}
],
"title": "Link Capacity Inventory (from BGP-LS)",
"type": "table"
},
{
"datasource": { "type": "postgres", "uid": "obmp_postgres" },
"fieldConfig": {
"defaults": {
"color": { "mode": "palette-classic" },
"custom": {
"axisBorderShow": false,
"axisCenteredZero": false,
"axisLabel": "Bandwidth (B/s)",
"fillOpacity": 80,
"gradientMode": "none",
"lineWidth": 1,
"scaleDistribution": { "type": "linear" },
"showValue": "auto",
"stacking": { "group": "A", "mode": "none" }
},
"unit": "Bps"
},
"overrides": []
},
"gridPos": { "h": 10, "w": 12, "x": 0, "y": 10 },
"id": 2,
"options": {
"barRadius": 0.1,
"barWidth": 0.8,
"groupWidth": 0.7,
"legend": { "calcs": [], "displayMode": "list", "placement": "bottom" },
"orientation": "horizontal",
"tooltip": { "mode": "single", "sort": "none" },
"xTickLabelRotation": 0
},
"targets": [
{
"datasource": { "type": "postgres", "uid": "obmp_postgres" },
"format": "table",
"rawQuery": true,
"rawSql": "SELECT local_router_name || ' -> ' || remote_router_name as \"Link\",\n COALESCE(max_link_bw, 0) as \"Max Bandwidth\",\n COALESCE(max_resv_bw, 0) as \"Max Reservable\",\n COALESCE(max_link_bw, 0) - COALESCE(max_resv_bw, 0) as \"Unreserved Gap\"\nFROM v_ls_links\nWHERE peer_hash_id = '$peer_hash' AND iswithdrawn = false\n AND max_link_bw IS NOT NULL AND max_link_bw > 0\nORDER BY max_link_bw DESC",
"refId": "A"
}
],
"title": "Capacity vs Reservable Bandwidth",
"type": "barchart"
},
{
"datasource": { "type": "postgres", "uid": "obmp_postgres" },
"fieldConfig": {
"defaults": {
"color": { "mode": "thresholds" },
"thresholds": {
"mode": "percentage",
"steps": [
{ "color": "green", "value": null },
{ "color": "yellow", "value": 50 },
{ "color": "orange", "value": 75 },
{ "color": "red", "value": 90 }
]
},
"unit": "percentunit",
"max": 1,
"min": 0
},
"overrides": []
},
"gridPos": { "h": 10, "w": 12, "x": 12, "y": 10 },
"id": 3,
"options": {
"minVizHeight": 75,
"minVizWidth": 75,
"orientation": "auto",
"reduceOptions": {
"calcs": ["lastNotNull"],
"fields": "",
"values": true
},
"showThresholdLabels": false,
"showThresholdMarkers": true,
"sizing": "auto"
},
"targets": [
{
"datasource": { "type": "postgres", "uid": "obmp_postgres" },
"format": "table",
"rawQuery": true,
"rawSql": "SELECT local_router_name || ' -> ' || remote_router_name as \"Link\",\n CASE WHEN max_link_bw > 0 \n THEN 1.0 - (COALESCE(max_resv_bw, 0)::float / max_link_bw::float)\n ELSE 0 END as \"Reservation Ratio\"\nFROM v_ls_links\nWHERE peer_hash_id = '$peer_hash' AND iswithdrawn = false\n AND max_link_bw IS NOT NULL AND max_link_bw > 0\nORDER BY \"Reservation Ratio\" DESC\nLIMIT 10",
"refId": "A"
}
],
"title": "Bandwidth Reservation Ratio (Higher = More Reserved)",
"type": "gauge"
},
{
"datasource": { "type": "postgres", "uid": "obmp_postgres" },
"fieldConfig": {
"defaults": {
"color": { "mode": "palette-classic" },
"custom": {
"axisBorderShow": false,
"fillOpacity": 80,
"gradientMode": "none",
"lineWidth": 1,
"showValue": "auto",
"stacking": { "group": "A", "mode": "none" }
}
},
"overrides": []
},
"gridPos": { "h": 8, "w": 12, "x": 0, "y": 20 },
"id": 4,
"options": {
"barRadius": 0.1,
"barWidth": 0.8,
"groupWidth": 0.7,
"legend": { "calcs": [], "displayMode": "list", "placement": "bottom" },
"orientation": "horizontal",
"tooltip": { "mode": "single", "sort": "none" }
},
"targets": [
{
"datasource": { "type": "postgres", "uid": "obmp_postgres" },
"format": "table",
"rawQuery": true,
"rawSql": "SELECT local_router_name || ' -> ' || remote_router_name as \"Link\",\n igp_metric as \"IGP Metric\",\n COALESCE(te_def_metric, 0) as \"TE Default Metric\"\nFROM v_ls_links\nWHERE peer_hash_id = '$peer_hash' AND iswithdrawn = false\nORDER BY igp_metric DESC\nLIMIT 20",
"refId": "A"
}
],
"title": "IGP Metric vs TE Default Metric",
"type": "barchart"
},
{
"gridPos": { "h": 8, "w": 12, "x": 12, "y": 20 },
"id": 5,
"options": {
"mode": "markdown",
"content": "## What-If: CSPF Path Computation\n\nIn a real MPLS-TE or SR-TE deployment, the headend router runs **Constrained Shortest Path First (CSPF)** to find paths that satisfy:\n\n1. **Bandwidth constraint** - Enough unreserved BW at the required priority\n2. **Admin group (affinity)** - Link colors must match include/exclude masks\n3. **SRLG diversity** - Backup path avoids shared risk with primary\n4. **TE metric optimization** - Minimize TE metric (not IGP metric)\n\n### How BGP-LS Enables This\n\nBGP-LS distributes the complete IGP topology **with TE attributes** to an external controller (PCE, SDN controller). The controller can:\n\n- Build a Traffic Engineering Database (TED)\n- Run CSPF with arbitrary constraints\n- Program SR-TE policies via PCEP or gRPC\n\n### Data Available in This Lab\n\n| Attribute | Source | Available? |\n|-----------|--------|------------|\n| Topology (nodes/links) | BGP-LS | Yes |\n| IGP Metric | BGP-LS | Yes |\n| TE Default Metric | BGP-LS TLV 1092 | Check TE table |\n| Max Link BW | BGP-LS TLV 1089 | Check TE table |\n| Max Reservable BW | BGP-LS TLV 1090 | Check TE table |\n| Unreserved BW | BGP-LS TLV 1091 | Check TE table |\n| Admin Group | BGP-LS TLV 1088 | Check TE table |\n| SRLG | BGP-LS TLV 1096 | Check TE table |\n| SR Node SID | BGP-LS TLV 1034 | Check SR table |\n| SR Adj SID | BGP-LS TLV 1099 | Check SR table |"
},
"title": "CSPF & Traffic Engineering Concepts",
"type": "text"
},
{
"gridPos": { "h": 12, "w": 24, "x": 0, "y": 28 },
"id": 6,
"options": {
"mode": "markdown",
"content": "## Integration Guide: Adding Real-Time Link Utilization\n\nBMP/BGP-LS provides **capacity** data (max bandwidth, reservable bandwidth) but NOT real-time **utilization**. To complete the traffic engineering picture, you need streaming telemetry.\n\n### Architecture\n\n```\n +------------------+\n IOS-XR Routers ---->| OpenBMP Collector|----> PostgreSQL (topology + capacity)\n | +------------------+\n | \n +-- gNMI ----->| Telegraf |----> InfluxDB/Prometheus (utilization)\n +------------------+\n |\n +------------------+\n | Grafana | <-- Mixed datasource queries\n +------------------+\n```\n\n### Step 1: Enable Model-Driven Telemetry on IOS-XR\n\n```\ntelemetry model-driven\n sensor-group INTF-COUNTERS\n sensor-path Cisco-IOS-XR-infra-statsd-oper:infra-statistics/interfaces/interface/latest/generic-counters\n !\n subscription INTF-SUB\n sensor-group-id INTF-COUNTERS sample-interval 30000\n destination-id TELEGRAF\n !\n destination-group TELEGRAF\n address-family ipv4 10.40.40.202 port 57000\n encoding self-describing-gpb\n protocol grpc no-tls\n !\n !\n```\n\n### Step 2: Telegraf Configuration\n\n```toml\n[[inputs.cisco_telemetry_mdt]]\n transport = \"grpc\"\n service_address = \":57000\"\n\n[[outputs.influxdb_v2]]\n urls = [\"http://localhost:8086\"]\n token = \"your-token\"\n organization = \"openbmp\"\n bucket = \"telemetry\"\n```\n\n### Step 3: Grafana Mixed Datasource Query\n\nCombine BGP-LS capacity from PostgreSQL with utilization from InfluxDB:\n\n```\n-- PostgreSQL: Get link capacity\nSELECT interface_addr::text as interface, max_link_bw\nFROM v_ls_links WHERE peer_hash_id = '$peer_hash'\n\n-- InfluxDB: Get interface utilization\nfrom(bucket: \"telemetry\")\n |> range(start: -1h)\n |> filter(fn: (r) => r._measurement == \"Cisco-IOS-XR-infra-statsd-oper\")\n |> filter(fn: (r) => r._field == \"bytes-received\" or r._field == \"bytes-sent\")\n |> derivative(unit: 1s, nonNegative: true)\n```\n\n### Step 4: Calculate Utilization %\n\nIn Grafana, use **Transformations** to:\n1. Join PostgreSQL capacity with InfluxDB utilization by interface IP\n2. Add calculated field: `utilization_pct = bytes_per_sec / max_link_bw * 100`\n3. Set threshold alerts: >80% = warning, >95% = critical\n\n### Key gNMI Sensor Paths for IOS-XR\n\n| Sensor Path | Data |\n|-------------|------|\n| `Cisco-IOS-XR-infra-statsd-oper:infra-statistics/interfaces/interface/latest/generic-counters` | Interface byte/packet counters |\n| `Cisco-IOS-XR-infra-statsd-oper:infra-statistics/interfaces/interface/latest/data-rate` | Current data rate (bits/sec) |\n| `Cisco-IOS-XR-mpls-te-oper:mpls-te/tunnels/summary` | MPLS-TE tunnel summary |\n| `Cisco-IOS-XR-ip-rsvp-oper:rsvp/interface-briefs` | RSVP interface reservations |\n| `Cisco-IOS-XR-segment-routing-ms-oper:srms/policy` | SR-MPLS policy state |\n\n### RFC 8571: Performance Metrics via BGP-LS\n\nIf routers support RFC 8571, these metrics flow through BGP-LS automatically:\n- **Unidirectional Link Delay** (TLV 1114) - microseconds\n- **Min/Max Link Delay** (TLV 1115)\n- **Delay Variation (jitter)** (TLV 1116)\n- **Link Loss** (TLV 1117) - percentage\n- **Residual Bandwidth** (TLV 1118)\n- **Available Bandwidth** (TLV 1119)\n- **Utilized Bandwidth** (TLV 1120)\n\nThese would appear in the `ls_links` table if the OpenBMP parser supports them."
},
"title": "Integration Guide: Streaming Telemetry for Link Utilization",
"type": "text"
}
],
"schemaVersion": 39,
"tags": ["obmp-learning"],
"templating": {
"list": [
{
"current": {},
"datasource": { "type": "postgres", "uid": "obmp_postgres" },
"definition": "SELECT __text,__value FROM (\n select peername as __text, peer_hash_id as __value, count(*) as count\n from v_ls_nodes\n group by peername,peer_hash_id) d\nwhere count > 0",
"hide": 0,
"includeAll": false,
"label": "BGP Peer",
"multi": false,
"name": "peer_hash",
"options": [],
"query": "SELECT __text,__value FROM (\n select peername as __text, peer_hash_id as __value, count(*) as count\n from v_ls_nodes\n group by peername,peer_hash_id) d\nwhere count > 0",
"refresh": 1,
"regex": "",
"skipUrlSync": false,
"sort": 0,
"type": "query"
}
]
},
"time": { "from": "now-6h", "to": "now" },
"timepicker": {},
"timezone": "",
"title": "Link Utilization & TE Thought Experiment",
"uid": "obmp-learn-10",
"version": 1
}

345
obmp-grafana/dashboards/Learning/te_sr_analytics.json
View File

@ -1,345 +0,0 @@
{
"annotations": {"list": [{"builtIn": 1,"datasource": {"type": "datasource","uid": "grafana"},"enable": true,"hide": true,"iconColor": "rgba(0, 211, 255, 1)","name": "Annotations & Alerts","type": "dashboard"}]},
"editable": true,
"fiscalYearStartMonth": 0,
"graphTooltip": 1,
"id": null,
"links": [],
"panels": [
{
"datasource": {"type": "postgres","uid": "obmp_postgres"},
"fieldConfig": {
"defaults": {"custom": {"align": "auto","displayMode": "auto"}},
"overrides": []
},
"gridPos": {"h": 10,"w": 24,"x": 0,"y": 0},
"id": 1,
"options": {"footer": {"fields": "","reducer": ["sum"],"show": false},"showHeader": true},
"targets": [
{
"datasource": {"type": "postgres","uid": "obmp_postgres"},
"format": "table",
"rawSql": "SELECT local_router_name as \"Local Router\", \n remote_router_name as \"Remote Router\",\n igp_metric as \"IGP Metric\",\n te_def_metric as \"TE Metric\",\n max_link_bw as \"Max BW (B/s)\",\n max_resv_bw as \"Max Reservable BW\",\n unreserved_bw as \"Unreserved BW\",\n admin_group as \"Admin Group\",\n protection_type as \"Protection\",\n srlg as \"SRLG\"\nFROM v_ls_links\nWHERE peer_hash_id = '$peer_hash' AND iswithdrawn = false\nORDER BY local_router_name, remote_router_name",
"refId": "A"
}
],
"title": "TE Link Capacity Map",
"type": "table"
},
{
"datasource": {"type": "postgres","uid": "obmp_postgres"},
"fieldConfig": {
"defaults": {"color": {"mode": "palette-classic"}},
"overrides": []
},
"gridPos": {"h": 10,"w": 12,"x": 0,"y": 10},
"id": 2,
"options": {
"barRadius": 0,
"barWidth": 0.97,
"groupWidth": 0.7,
"legend": {"displayMode": "list","placement": "bottom"},
"orientation": "auto",
"showValue": "auto",
"stacking": "none",
"tooltip": {"mode": "single","sort": "none"},
"xTickLabelRotation": -45
},
"targets": [
{
"datasource": {"type": "postgres","uid": "obmp_postgres"},
"format": "table",
"rawSql": "SELECT local_router_name || ' -> ' || remote_router_name as \"Link\",\n igp_metric as \"IGP Metric\",\n COALESCE(te_def_metric, igp_metric) as \"TE Metric\"\nFROM v_ls_links\nWHERE peer_hash_id = '$peer_hash' AND iswithdrawn = false\nORDER BY igp_metric DESC",
"refId": "A"
}
],
"title": "IGP Metric vs TE Metric Comparison",
"type": "barchart"
},
{
"datasource": {"type": "postgres","uid": "obmp_postgres"},
"fieldConfig": {
"defaults": {"color": {"mode": "palette-classic"}},
"overrides": []
},
"gridPos": {"h": 10,"w": 6,"x": 12,"y": 10},
"id": 3,
"options": {
"legend": {"displayMode": "list","placement": "bottom"},
"pieType": "pie",
"reduceOptions": {"calcs": ["lastNotNull"],"fields": "","values": true},
"tooltip": {"mode": "single","sort": "none"}
},
"targets": [
{
"datasource": {"type": "postgres","uid": "obmp_postgres"},
"format": "table",
"rawSql": "SELECT COALESCE(admin_group::text, 'None') as \"Admin Group\",\n COUNT(*) as \"Link Count\"\nFROM v_ls_links\nWHERE peer_hash_id = '$peer_hash' AND iswithdrawn = false\nGROUP BY admin_group\nORDER BY \"Link Count\" DESC",
"refId": "A"
}
],
"title": "Admin Group Distribution",
"type": "piechart"
},
{
"datasource": {"type": "postgres","uid": "obmp_postgres"},
"fieldConfig": {
"defaults": {"color": {"mode": "palette-classic"}},
"overrides": []
},
"gridPos": {"h": 10,"w": 6,"x": 18,"y": 10},
"id": 4,
"options": {
"legend": {"displayMode": "list","placement": "bottom"},
"pieType": "pie",
"reduceOptions": {"calcs": ["lastNotNull"],"fields": "","values": true},
"tooltip": {"mode": "single","sort": "none"}
},
"targets": [
{
"datasource": {"type": "postgres","uid": "obmp_postgres"},
"format": "table",
"rawSql": "SELECT COALESCE(protection_type, 'None') as \"Protection Type\",\n COUNT(*) as \"Link Count\"\nFROM v_ls_links\nWHERE peer_hash_id = '$peer_hash' AND iswithdrawn = false\nGROUP BY protection_type\nORDER BY \"Link Count\" DESC",
"refId": "A"
}
],
"title": "Link Protection Types",
"type": "piechart"
},
{
"datasource": {"type": "postgres","uid": "obmp_postgres"},
"fieldConfig": {
"defaults": {"custom": {"align": "auto","displayMode": "auto"}},
"overrides": []
},
"gridPos": {"h": 8,"w": 12,"x": 0,"y": 20},
"id": 5,
"options": {"footer": {"fields": "","reducer": ["sum"],"show": false},"showHeader": true},
"targets": [
{
"datasource": {"type": "postgres","uid": "obmp_postgres"},
"format": "table",
"rawSql": "SELECT nodename as \"Node\",\n routerid as \"Router ID\",\n protocol as \"Protocol\",\n sr_capabilities as \"SR Capabilities (SRGB)\"\nFROM v_ls_nodes\nWHERE peer_hash_id = '$peer_hash' AND iswithdrawn = false\nORDER BY nodename",
"refId": "A"
}
],
"title": "SR Node Capabilities",
"type": "table"
},
{
"datasource": {"type": "postgres","uid": "obmp_postgres"},
"fieldConfig": {
"defaults": {"custom": {"align": "auto","displayMode": "auto"}},
"overrides": []
},
"gridPos": {"h": 8,"w": 12,"x": 12,"y": 20},
"id": 6,
"options": {"footer": {"fields": "","reducer": ["sum"],"show": false},"showHeader": true},
"targets": [
{
"datasource": {"type": "postgres","uid": "obmp_postgres"},
"format": "table",
"rawSql": "SELECT n.nodename as \"Node\",\n p.prefix::text as \"Prefix\",\n p.prefix_len as \"Len\",\n p.metric as \"Metric\",\n p.sr_prefix_sids as \"Prefix SID\",\n p.protocol::text as \"Protocol\"\nFROM ls_prefixes p\nJOIN ls_nodes n ON n.hash_id = p.local_node_hash_id \n AND n.peer_hash_id = p.peer_hash_id\nWHERE p.peer_hash_id = '$peer_hash' AND p.iswithdrawn = false\nORDER BY n.nodename, p.prefix",
"refId": "A"
}
],
"title": "SR Prefix SIDs",
"type": "table"
},
{
"datasource": {"type": "postgres","uid": "obmp_postgres"},
"fieldConfig": {
"defaults": {"custom": {"align": "auto","displayMode": "auto"}},
"overrides": []
},
"gridPos": {"h": 8,"w": 12,"x": 0,"y": 28},
"id": 7,
"options": {"footer": {"fields": "","reducer": ["sum"],"show": false},"showHeader": true},
"targets": [
{
"datasource": {"type": "postgres","uid": "obmp_postgres"},
"format": "table",
"rawSql": "SELECT local_router_name as \"Local\",\n remote_router_name as \"Remote\",\n sr_adjacency_sids as \"Adjacency SIDs\",\n peer_node_sid as \"Peer Node SID\",\n mpls_proto_mask::text as \"MPLS Proto\"\nFROM v_ls_links\nWHERE peer_hash_id = '$peer_hash' AND iswithdrawn = false\nORDER BY local_router_name, remote_router_name",
"refId": "A"
}
],
"title": "SR Adjacency SIDs",
"type": "table"
},
{
"datasource": {"type": "postgres","uid": "obmp_postgres"},
"fieldConfig": {
"defaults": {"custom": {"align": "auto","displayMode": "auto"}},
"overrides": []
},
"gridPos": {"h": 8,"w": 12,"x": 12,"y": 28},
"id": 8,
"options": {"footer": {"fields": "","reducer": ["sum"],"show": false},"showHeader": true},
"targets": [
{
"datasource": {"type": "postgres","uid": "obmp_postgres"},
"format": "table",
"rawSql": "SELECT srlg as \"SRLG Value\",\n COUNT(*) as \"Link Count\",\n string_agg(DISTINCT local_router_name || ' -> ' || remote_router_name, ', ') as \"Links\"\nFROM v_ls_links\nWHERE peer_hash_id = '$peer_hash' AND iswithdrawn = false \n AND srlg IS NOT NULL AND srlg != ''\nGROUP BY srlg\nORDER BY COUNT(*) DESC",
"refId": "A"
}
],
"title": "SRLG Groups",
"type": "table"
},
{
"datasource": {"type": "postgres","uid": "obmp_postgres"},
"fieldConfig": {
"defaults": {
"color": {"mode": "thresholds"},
"thresholds": {"mode": "absolute","steps": [{"color": "green","value": null}]}
},
"overrides": []
},
"gridPos": {"h": 4,"w": 5,"x": 0,"y": 36},
"id": 9,
"options": {"colorMode": "value","graphMode": "area","justifyMode": "auto","orientation": "auto","reduceOptions": {"calcs": ["lastNotNull"],"fields": "","values": false},"textMode": "auto"},
"targets": [
{
"datasource": {"type": "postgres","uid": "obmp_postgres"},
"format": "table",
"rawSql": "SELECT COUNT(*) FROM v_ls_links WHERE peer_hash_id = '$peer_hash' AND iswithdrawn = false AND te_def_metric IS NOT NULL",
"refId": "A"
}
],
"title": "Links with TE Metric",
"type": "stat"
},
{
"datasource": {"type": "postgres","uid": "obmp_postgres"},
"fieldConfig": {
"defaults": {
"color": {"mode": "thresholds"},
"thresholds": {"mode": "absolute","steps": [{"color": "green","value": null}]}
},
"overrides": []
},
"gridPos": {"h": 4,"w": 5,"x": 5,"y": 36},
"id": 10,
"options": {"colorMode": "value","graphMode": "area","justifyMode": "auto","orientation": "auto","reduceOptions": {"calcs": ["lastNotNull"],"fields": "","values": false},"textMode": "auto"},
"targets": [
{
"datasource": {"type": "postgres","uid": "obmp_postgres"},
"format": "table",
"rawSql": "SELECT COUNT(*) FROM v_ls_links WHERE peer_hash_id = '$peer_hash' AND iswithdrawn = false AND max_link_bw IS NOT NULL AND max_link_bw > 0",
"refId": "A"
}
],
"title": "Links with Bandwidth",
"type": "stat"
},
{
"datasource": {"type": "postgres","uid": "obmp_postgres"},
"fieldConfig": {
"defaults": {
"color": {"mode": "thresholds"},
"thresholds": {"mode": "absolute","steps": [{"color": "green","value": null}]}
},
"overrides": []
},
"gridPos": {"h": 4,"w": 5,"x": 10,"y": 36},
"id": 11,
"options": {"colorMode": "value","graphMode": "area","justifyMode": "auto","orientation": "auto","reduceOptions": {"calcs": ["lastNotNull"],"fields": "","values": false},"textMode": "auto"},
"targets": [
{
"datasource": {"type": "postgres","uid": "obmp_postgres"},
"format": "table",
"rawSql": "SELECT COUNT(*) FROM v_ls_links WHERE peer_hash_id = '$peer_hash' AND iswithdrawn = false AND srlg IS NOT NULL AND srlg != ''",
"refId": "A"
}
],
"title": "Links with SRLG",
"type": "stat"
},
{
"datasource": {"type": "postgres","uid": "obmp_postgres"},
"fieldConfig": {
"defaults": {
"color": {"mode": "thresholds"},
"thresholds": {"mode": "absolute","steps": [{"color": "green","value": null}]}
},
"overrides": []
},
"gridPos": {"h": 4,"w": 5,"x": 15,"y": 36},
"id": 12,
"options": {"colorMode": "value","graphMode": "area","justifyMode": "auto","orientation": "auto","reduceOptions": {"calcs": ["lastNotNull"],"fields": "","values": false},"textMode": "auto"},
"targets": [
{
"datasource": {"type": "postgres","uid": "obmp_postgres"},
"format": "table",
"rawSql": "SELECT COUNT(*) FROM v_ls_nodes WHERE peer_hash_id = '$peer_hash' AND iswithdrawn = false AND sr_capabilities IS NOT NULL AND sr_capabilities != ''",
"refId": "A"
}
],
"title": "Nodes with SR",
"type": "stat"
},
{
"datasource": {"type": "postgres","uid": "obmp_postgres"},
"fieldConfig": {
"defaults": {
"color": {"mode": "thresholds"},
"thresholds": {"mode": "absolute","steps": [{"color": "green","value": null}]}
},
"overrides": []
},
"gridPos": {"h": 4,"w": 4,"x": 20,"y": 36},
"id": 13,
"options": {"colorMode": "value","graphMode": "area","justifyMode": "auto","orientation": "auto","reduceOptions": {"calcs": ["lastNotNull"],"fields": "","values": false},"textMode": "auto"},
"targets": [
{
"datasource": {"type": "postgres","uid": "obmp_postgres"},
"format": "table",
"rawSql": "SELECT COUNT(*) FROM v_ls_links WHERE peer_hash_id = '$peer_hash' AND iswithdrawn = false AND sr_adjacency_sids IS NOT NULL AND sr_adjacency_sids != ''",
"refId": "A"
}
],
"title": "Links with Adj SID",
"type": "stat"
},
{
"gridPos": {"h": 10,"w": 24,"x": 0,"y": 40},
"id": 14,
"options": {
"code": {"language": "plaintext","showLineNumbers": false,"showMiniMap": false},
"content": "## Traffic Engineering & Segment Routing Analytics\n\nThis dashboard exposes TE and SR attributes from BGP-LS (RFC 7752) that OpenBMP collects but existing dashboards don't display.\n\n### TE Fields (from ls_links)\n- **admin_group**: Link color/affinity bitmap for RSVP-TE constraints\n- **max_link_bw / max_resv_bw**: Link capacity in bytes/sec\n- **unreserved_bw**: Available bandwidth per priority level\n- **te_def_metric**: TE metric (may differ from IGP metric)\n- **protection_type**: FRR protection (unprotected, shared, dedicated, etc.)\n- **srlg**: Shared Risk Link Group for diverse path computation\n\n### SR Fields\n- **sr_capabilities**: Node SRGB (Segment Routing Global Block) range\n- **sr_prefix_sids**: Prefix SID for SR-MPLS forwarding\n- **sr_adjacency_sids**: Adjacency SIDs for SR-TE path steering\n- **peer_node_sid**: BGP EPE SID (RFC 9086)\n\n### Notes\n- NULL values indicate the router is not advertising that TLV\n- To enable TE metrics on IOS-XR: `mpls traffic-eng` under IS-IS\n- To enable SR: `segment-routing mpls` under IS-IS with prefix-sid-map",
"mode": "markdown"
},
"title": "About This Dashboard",
"type": "text"
}
],
"schemaVersion": 39,
"tags": ["obmp-learning"],
"templating": {
"list": [
{
"current": {},
"datasource": {"type": "postgres","uid": "obmp_postgres"},
"definition": "SELECT __text,__value FROM (\n select peername as __text, peer_hash_id as __value, count(*) as count\n from v_ls_nodes\n group by peername,peer_hash_id) d\nwhere count > 0",
"hide": 0,
"includeAll": false,
"label": "BGP Peer",
"multi": false,
"name": "peer_hash",
"options": [],
"query": "SELECT __text,__value FROM (\n select peername as __text, peer_hash_id as __value, count(*) as count\n from v_ls_nodes\n group by peername,peer_hash_id) d\nwhere count > 0",
"refresh": 1,
"regex": "",
"skipUrlSync": false,
"sort": 0,
"type": "query"
}
]
},
"time": {"from": "now-6h","to": "now"},
"timepicker": {},
"timezone": "",
"title": "TE & Segment Routing Analytics",
"uid": "obmp-learn-08",
"version": 1
}

235
obmp-grafana/dashboards/Learning/topology_anomaly.json
View File

@ -1,235 +0,0 @@
{
"uid": "obmp-learn-09",
"title": "Topology Change & Anomaly Detection",
"tags": ["obmp-learning"],
"editable": true,
"schemaVersion": 39,
"time": {
"from": "now-6h",
"to": "now"
},
"templating": {
"list": [
{
"name": "peer_hash",
"label": "BGP Peer",
"type": "query",
"datasource": {
"type": "postgres",
"uid": "obmp_postgres"
},
"query": "SELECT __text,__value FROM (\n select peername as __text, peer_hash_id as __value, count(*) as count\n from v_ls_nodes\n group by peername,peer_hash_id) d\nwhere count > 0",
"refresh": 1,
"multi": false
}
]
},
"panels": [
{
"id": 1,
"title": "Link State Changes Over Time",
"type": "timeseries",
"gridPos": { "h": 8, "w": 12, "x": 0, "y": 0 },
"datasource": { "type": "postgres", "uid": "obmp_postgres" },
"targets": [
{
"rawSql": "SELECT $__timeGroupAlias(timestamp, '5m') as time,\n SUM(CASE WHEN iswithdrawn = false THEN 1 ELSE 0 END) as \"Links Up\",\n SUM(CASE WHEN iswithdrawn = true THEN 1 ELSE 0 END) as \"Links Down\"\nFROM ls_links_log\nWHERE $__timeFilter(timestamp) AND peer_hash_id = '$peer_hash'\nGROUP BY 1 ORDER BY 1",
"format": "time_series",
"refId": "A"
}
]
},
{
"id": 2,
"title": "Node Changes Over Time",
"type": "timeseries",
"gridPos": { "h": 8, "w": 12, "x": 12, "y": 0 },
"datasource": { "type": "postgres", "uid": "obmp_postgres" },
"targets": [
{
"rawSql": "SELECT $__timeGroupAlias(timestamp, '5m') as time,\n SUM(CASE WHEN iswithdrawn = false THEN 1 ELSE 0 END) as \"Nodes Appeared\",\n SUM(CASE WHEN iswithdrawn = true THEN 1 ELSE 0 END) as \"Nodes Withdrawn\"\nFROM ls_nodes_log\nWHERE $__timeFilter(timestamp) AND peer_hash_id = '$peer_hash'\nGROUP BY 1 ORDER BY 1",
"format": "time_series",
"refId": "A"
}
]
},
{
"id": 3,
"title": "BGP Peer Session Events",
"type": "timeseries",
"gridPos": { "h": 8, "w": 12, "x": 0, "y": 8 },
"datasource": { "type": "postgres", "uid": "obmp_postgres" },
"targets": [
{
"rawSql": "SELECT $__timeGroupAlias(pel.timestamp, '5m') as time,\n SUM(CASE WHEN pel.state = 'up' THEN 1 ELSE 0 END) as \"Sessions Up\",\n SUM(CASE WHEN pel.state = 'down' THEN 1 ELSE 0 END) as \"Sessions Down\"\nFROM peer_event_log pel\nWHERE $__timeFilter(pel.timestamp)\nGROUP BY 1 ORDER BY 1",
"format": "time_series",
"refId": "A"
}
]
},
{
"id": 4,
"title": "RIB Update Rate",
"type": "timeseries",
"gridPos": { "h": 8, "w": 12, "x": 12, "y": 8 },
"datasource": { "type": "postgres", "uid": "obmp_postgres" },
"targets": [
{
"rawSql": "SELECT $__timeGroupAlias(timestamp, '5m') as time,\n SUM(CASE WHEN iswithdrawn = false THEN 1 ELSE 0 END) as \"Advertisements\",\n SUM(CASE WHEN iswithdrawn = true THEN 1 ELSE 0 END) as \"Withdrawals\"\nFROM ip_rib_log\nWHERE $__timeFilter(timestamp)\nGROUP BY 1 ORDER BY 1",
"format": "time_series",
"refId": "A"
}
]
},
{
"id": 5,
"title": "Origin AS Changes (Potential Hijacks)",
"type": "table",
"gridPos": { "h": 10, "w": 12, "x": 0, "y": 16 },
"datasource": { "type": "postgres", "uid": "obmp_postgres" },
"targets": [
{
"rawSql": "SELECT DISTINCT ON (r1.prefix, r1.prefix_len)\n r1.prefix::text as \"Prefix\",\n r1.prefix_len as \"Len\",\n r1.origin_as as \"Current Origin AS\",\n r2.origin_as as \"Previous Origin AS\",\n r1.timestamp as \"Changed At\"\nFROM ip_rib_log r1\nJOIN ip_rib_log r2 ON r1.prefix = r2.prefix \n AND r1.prefix_len = r2.prefix_len\n AND r1.timestamp > r2.timestamp\nWHERE r1.origin_as != r2.origin_as\n AND $__timeFilter(r1.timestamp)\nORDER BY r1.prefix, r1.prefix_len, r1.timestamp DESC\nLIMIT 50",
"format": "table",
"refId": "A"
}
]
},
{
"id": 6,
"title": "Most Churned Prefixes",
"type": "table",
"gridPos": { "h": 10, "w": 12, "x": 12, "y": 16 },
"datasource": { "type": "postgres", "uid": "obmp_postgres" },
"targets": [
{
"rawSql": "SELECT prefix::text as \"Prefix\",\n prefix_len as \"Len\",\n COUNT(*) as \"Total Updates\",\n SUM(CASE WHEN iswithdrawn THEN 1 ELSE 0 END) as \"Withdrawals\",\n MIN(timestamp) as \"First Seen\",\n MAX(timestamp) as \"Last Change\",\n CASE \n WHEN COUNT(*) <= 2 THEN 'Stable'\n WHEN COUNT(*) <= 10 THEN 'Moderate'\n ELSE 'Unstable'\n END as \"Stability\"\nFROM ip_rib_log\nWHERE $__timeFilter(timestamp)\nGROUP BY prefix, prefix_len\nHAVING COUNT(*) > 1\nORDER BY COUNT(*) DESC\nLIMIT 30",
"format": "table",
"refId": "A"
}
]
},
{
"id": 7,
"title": "Recent Link State Changes",
"type": "table",
"gridPos": { "h": 10, "w": 24, "x": 0, "y": 26 },
"datasource": { "type": "postgres", "uid": "obmp_postgres" },
"targets": [
{
"rawSql": "SELECT l.timestamp as \"Time\",\n CASE WHEN l.iswithdrawn THEN 'DOWN' ELSE 'UP' END as \"State\",\n ln.name as \"Local Node\",\n l.local_igp_router_id as \"Local IGP ID\",\n rn.name as \"Remote Node\",\n l.remote_igp_router_id as \"Remote IGP ID\",\n l.igp_metric as \"IGP Metric\",\n l.protocol::text as \"Protocol\"\nFROM ls_links_log l\nLEFT JOIN ls_nodes ln ON ln.hash_id = l.local_node_hash_id AND ln.peer_hash_id = l.peer_hash_id\nLEFT JOIN ls_nodes rn ON rn.hash_id = l.remote_node_hash_id AND rn.peer_hash_id = l.peer_hash_id\nWHERE $__timeFilter(l.timestamp) AND l.peer_hash_id = '$peer_hash'\nORDER BY l.timestamp DESC\nLIMIT 50",
"format": "table",
"refId": "A"
}
]
},
{
"id": 8,
"title": "Multi-Peer Route Consistency",
"type": "table",
"gridPos": { "h": 8, "w": 12, "x": 0, "y": 36 },
"datasource": { "type": "postgres", "uid": "obmp_postgres" },
"targets": [
{
"rawSql": "SELECT r.prefix::text as \"Prefix\",\n r.prefix_len as \"Len\",\n COUNT(DISTINCT r.peer_hash_id) as \"Peer Count\",\n COUNT(DISTINCT ba.origin_as) as \"Distinct Origins\",\n COUNT(DISTINCT ba.as_path_count) as \"Distinct Path Lengths\",\n string_agg(DISTINCT ba.origin_as::text, ', ') as \"Origin ASNs\"\nFROM ip_rib r\nJOIN base_attrs ba ON ba.hash_id = r.base_attr_hash_id\nWHERE r.iswithdrawn = false AND r.isipv4 = true\nGROUP BY r.prefix, r.prefix_len\nHAVING COUNT(DISTINCT ba.origin_as) > 1\nORDER BY COUNT(DISTINCT ba.origin_as) DESC\nLIMIT 30",
"format": "table",
"refId": "A"
}
]
},
{
"id": 9,
"title": "Active Peers",
"type": "stat",
"gridPos": { "h": 4, "w": 4, "x": 0, "y": 44 },
"datasource": { "type": "postgres", "uid": "obmp_postgres" },
"targets": [
{
"rawSql": "SELECT COUNT(*) FROM bgp_peers WHERE state = 'up'",
"format": "table",
"refId": "A"
}
]
},
{
"id": 10,
"title": "Total LS Links",
"type": "stat",
"gridPos": { "h": 4, "w": 4, "x": 4, "y": 44 },
"datasource": { "type": "postgres", "uid": "obmp_postgres" },
"targets": [
{
"rawSql": "SELECT COUNT(*) FROM ls_links WHERE peer_hash_id = '$peer_hash' AND iswithdrawn = false",
"format": "table",
"refId": "A"
}
]
},
{
"id": 11,
"title": "Total LS Nodes",
"type": "stat",
"gridPos": { "h": 4, "w": 4, "x": 8, "y": 44 },
"datasource": { "type": "postgres", "uid": "obmp_postgres" },
"targets": [
{
"rawSql": "SELECT COUNT(*) FROM ls_nodes WHERE peer_hash_id = '$peer_hash' AND iswithdrawn = false",
"format": "table",
"refId": "A"
}
]
},
{
"id": 12,
"title": "RIB Updates (24h)",
"type": "stat",
"gridPos": { "h": 4, "w": 4, "x": 12, "y": 44 },
"datasource": { "type": "postgres", "uid": "obmp_postgres" },
"targets": [
{
"rawSql": "SELECT COUNT(*) FROM ip_rib_log WHERE timestamp > NOW() - INTERVAL '24 hours'",
"format": "table",
"refId": "A"
}
]
},
{
"id": 13,
"title": "Link Changes (24h)",
"type": "stat",
"gridPos": { "h": 4, "w": 4, "x": 16, "y": 44 },
"datasource": { "type": "postgres", "uid": "obmp_postgres" },
"targets": [
{
"rawSql": "SELECT COUNT(*) FROM ls_links_log WHERE timestamp > NOW() - INTERVAL '24 hours' AND peer_hash_id = '$peer_hash'",
"format": "table",
"refId": "A"
}
]
},
{
"id": 14,
"title": "Origin Changes (24h)",
"type": "stat",
"gridPos": { "h": 4, "w": 4, "x": 20, "y": 44 },
"datasource": { "type": "postgres", "uid": "obmp_postgres" },
"targets": [
{
"rawSql": "SELECT COUNT(DISTINCT r1.prefix) FROM ip_rib_log r1\nJOIN ip_rib_log r2 ON r1.prefix = r2.prefix AND r1.prefix_len = r2.prefix_len AND r1.timestamp > r2.timestamp\nWHERE r1.origin_as != r2.origin_as AND r1.timestamp > NOW() - INTERVAL '24 hours'",
"format": "table",
"refId": "A"
}
]
},
{
"id": 15,
"title": "About This Dashboard",
"type": "text",
"gridPos": { "h": 8, "w": 12, "x": 12, "y": 36 },
"options": {
"mode": "markdown",
"content": "## Topology Change & Anomaly Detection\n\nThis dashboard provides heuristic analysis of BMP data to detect network anomalies:\n\n### What to Watch For\n- **Link flaps**: Rapid up/down cycles in the Link State Changes panel indicate instability\n- **Origin AS changes**: Could indicate a route hijack or legitimate migration\n- **Multi-origin prefixes**: Same prefix seen from different origin ASNs across peers\n- **Correlated events**: Peer session drops followed by mass withdrawals indicate convergence events\n\n### Testing with ExaBGP Scenarios\n1. Load `origin_shift` scenario to simulate origin AS changes\n2. Load `hijack_simulation` to see how shorter paths override legitimate routes\n3. Load/unload `churn` scenario repeatedly to generate instability patterns\n\n### Data Sources\n- **ls_links_log / ls_nodes_log**: TimescaleDB hypertables tracking all BGP-LS topology changes\n- **ip_rib_log**: All BGP RIB updates and withdrawals with timestamps\n- **peer_event_log**: BGP session state changes (up/down events)"
}
}
]
}

132
obmp-grafana/dashboards/Telemetry-3001/combined_bmp_telemetry.json
View File

@ -1,132 +0,0 @@
{
"annotations": {"list": [{"builtIn": 1,"datasource": {"type": "datasource","uid": "grafana"},"enable": true,"hide": true,"iconColor": "rgba(0, 211, 255, 1)","name": "Annotations & Alerts","type": "dashboard"}]},
"description": "Combined view of BMP control-plane data (from PostgreSQL) and gNMI data-plane telemetry (from InfluxDB). Correlate BGP peer state with interface traffic patterns.",
"editable": true,
"fiscalYearStartMonth": 0,
"graphTooltip": 1,
"id": null,
"links": [],
"templating": {
"list": [
{
"current": {},
"datasource": {"type": "influxdb","uid": "obmp_influxdb"},
"definition": "from(bucket: \"telemetry\")\n |> range(start: -1h)\n |> filter(fn: (r) => r._measurement == \"interface_counters\")\n |> keep(columns: [\"source\"])\n |> distinct(column: \"source\")\n |> sort()",
"hide": 0,
"includeAll": true,
"label": "Router",
"multi": true,
"name": "router",
"options": [],
"query": "from(bucket: \"telemetry\")\n |> range(start: -1h)\n |> filter(fn: (r) => r._measurement == \"interface_counters\")\n |> keep(columns: [\"source\"])\n |> distinct(column: \"source\")\n |> sort()",
"refresh": 2,
"regex": "",
"type": "query"
}
]
},
"panels": [
{
"datasource": {"type": "postgres","uid": "obmp_postgres"},
"description": "Current BGP peer status from the OpenBMP PostgreSQL database. Shows peer address, name, and session state.",
"fieldConfig": {
"defaults": {
"color": {"mode": "thresholds"},
"custom": {"align": "auto","displayMode": "auto","filterable": true,"inspect": true},
"mappings": [],
"thresholds": {"mode": "absolute","steps": [{"color": "green","value": null}]}
},
"overrides": [
{"matcher": {"id": "byName","options": "state"},"properties": [{"id": "custom.displayMode","value": "color-background-solid"},{"id": "mappings","value": [{"options": {"down": {"color": "red","index": 1,"text": "DOWN"},"up": {"color": "green","index": 0,"text": "UP"}},"type": "value"}]}]},
{"matcher": {"id": "byName","options": "peer_addr"},"properties": [{"id": "custom.width","value": 160}]},
{"matcher": {"id": "byName","options": "name"},"properties": [{"id": "custom.width","value": 200}]}
]
},
"gridPos": {"h": 10,"w": 24,"x": 0,"y": 0},
"id": 1,
"options": {"footer": {"fields": "","reducer": ["sum"],"show": false},"showHeader": true,"sortBy": [{"desc": false,"displayName": "state"}]},
"targets": [
{
"datasource": {"type": "postgres","uid": "obmp_postgres"},
"format": "table",
"rawSql": "SELECT\n p.peer_addr,\n COALESCE(p.name, p.peer_addr::text) AS name,\n p.state,\n p.peer_as AS \"AS\",\n p.router_hash_id IS NOT NULL AS \"BMP Active\",\n p.timestamp AS \"Last State Change\"\nFROM bgp_peers p\nWHERE p.isprepolicy = true\nORDER BY p.state, p.peer_addr",
"refId": "A"
}
],
"title": "BGP Peer Status",
"type": "table"
},
{
"datasource": {"type": "influxdb","uid": "obmp_influxdb"},
"description": "Interface traffic rates from gNMI streaming telemetry. Shows bytes per second for each interface across selected routers.",
"fieldConfig": {
"defaults": {
"color": {"mode": "palette-classic"},
"custom": {"axisBorderShow": false,"axisCenteredZero": false,"axisLabel": "","axisPlacement": "auto","barAlignment": 0,"drawStyle": "line","fillOpacity": 10,"gradientMode": "none","hideFrom": {"legend": false,"tooltip": false,"viz": false},"lineInterpolation": "linear","lineWidth": 1,"pointSize": 5,"scaleDistribution": {"type": "linear"},"showPoints": "never","spanNulls": false,"stacking": {"group": "A","mode": "none"},"thresholdsStyle": {"mode": "off"}},
"mappings": [],
"thresholds": {"mode": "absolute","steps": [{"color": "green","value": null},{"color": "red","value": 80}]},
"unit": "Bps"
}
},
"gridPos": {"h": 10,"w": 24,"x": 0,"y": 10},
"id": 2,
"options": {"legend": {"calcs": ["mean","max"],"displayMode": "table","placement": "bottom"},"tooltip": {"mode": "multi","sort": "desc"}},
"targets": [
{
"datasource": {"type": "influxdb","uid": "obmp_influxdb"},
"query": "from(bucket: \"telemetry\")\n |> range(start: v.timeRangeStart, stop: v.timeRangeStop)\n |> filter(fn: (r) => r._measurement == \"interface_counters\")\n |> filter(fn: (r) => r.source =~ /${router:regex}/)\n |> filter(fn: (r) => r._field == \"in-octets\" or r._field == \"out-octets\")\n |> derivative(unit: 1s, nonNegative: true)\n |> map(fn: (r) => ({r with _value: if r._value < 0.0 then 0.0 else r._value}))",
"refId": "A"
}
],
"title": "Interface Traffic",
"type": "timeseries"
},
{
"datasource": {"type": "postgres","uid": "obmp_postgres"},
"description": "BGP update activity over time from the OpenBMP PostgreSQL database. Shows peer event transitions and update counts for correlation with traffic patterns.",
"fieldConfig": {
"defaults": {
"color": {"mode": "palette-classic"},
"custom": {"axisBorderShow": false,"axisCenteredZero": false,"axisLabel": "","axisPlacement": "auto","barAlignment": 0,"drawStyle": "bars","fillOpacity": 50,"gradientMode": "none","hideFrom": {"legend": false,"tooltip": false,"viz": false},"lineInterpolation": "linear","lineWidth": 1,"pointSize": 5,"scaleDistribution": {"type": "linear"},"showPoints": "never","spanNulls": false,"stacking": {"group": "A","mode": "normal"},"thresholdsStyle": {"mode": "off"}},
"mappings": [],
"thresholds": {"mode": "absolute","steps": [{"color": "green","value": null}]},
"unit": "short"
}
},
"gridPos": {"h": 10,"w": 24,"x": 0,"y": 20},
"id": 3,
"options": {"legend": {"calcs": ["sum"],"displayMode": "table","placement": "bottom"},"tooltip": {"mode": "multi","sort": "desc"}},
"targets": [
{
"datasource": {"type": "postgres","uid": "obmp_postgres"},
"format": "time_series",
"rawSql": "SELECT\n $__timeGroupAlias(e.timestamp, '1m'),\n COALESCE(p.name, p.peer_addr::text) AS metric,\n COUNT(*) AS \"value\"\nFROM peer_event_log e\nJOIN bgp_peers p ON p.hash_id = e.peer_hash_id\nWHERE $__timeFilter(e.timestamp)\nGROUP BY 1, 2\nORDER BY 1",
"refId": "A"
}
],
"title": "BGP Update Activity",
"type": "timeseries"
},
{
"datasource": {"type": "datasource","uid": "grafana"},
"gridPos": {"h": 6,"w": 24,"x": 0,"y": 30},
"id": 4,
"options": {
"code": {"language": "plaintext","showLineNumbers": false,"showMiniMap": false},
"content": "## Combined BMP + Telemetry View\n\nThis dashboard integrates two complementary data sources to provide a unified network monitoring view:\n\n### Control Plane (BMP via PostgreSQL)\n- **BGP Peer Status** -- Real-time BGP session state from BMP (OpenBMP)\n- **BGP Update Activity** -- Session transitions and update events from `peer_event_log`\n\n### Data Plane (gNMI via InfluxDB)\n- **Interface Traffic** -- Streaming telemetry byte rates collected via gNMI at 10-second intervals\n\n### Correlation Use Cases\n- A BGP peer flap (control plane) should correlate with a traffic shift on affected interfaces (data plane)\n- Sustained high interface utilization (data plane) may precede BGP session resets due to congestion\n- Compare the number of active BGP peers with interface traffic to validate routing convergence",
"mode": "markdown"
},
"title": "About",
"type": "text"
}
],
"schemaVersion": 39,
"style": "dark",
"tags": ["obmp-telemetry"],
"time": {"from": "now-1h","to": "now"},
"timepicker": {},
"timezone": "browser",
"title": "Combined BMP + Telemetry View",
"uid": "obmp-telem-03",
"version": 1
}

134
obmp-grafana/dashboards/Telemetry-3001/interface_errors.json
View File

@ -1,134 +0,0 @@
{
"annotations": {"list": [{"builtIn": 1,"datasource": {"type": "datasource","uid": "grafana"},"enable": true,"hide": true,"iconColor": "rgba(0, 211, 255, 1)","name": "Annotations & Alerts","type": "dashboard"}]},
"description": "Interface error and drop counters collected via gNMI streaming telemetry. Helps identify interfaces with packet loss or physical layer issues.",
"editable": true,
"fiscalYearStartMonth": 0,
"graphTooltip": 1,
"id": null,
"links": [],
"templating": {
"list": [
{
"current": {},
"datasource": {"type": "influxdb","uid": "obmp_influxdb"},
"definition": "from(bucket: \"telemetry\")\n |> range(start: -1h)\n |> filter(fn: (r) => r._measurement == \"interface_counters\")\n |> keep(columns: [\"source\"])\n |> distinct(column: \"source\")\n |> sort()",
"hide": 0,
"includeAll": true,
"label": "Router",
"multi": true,
"name": "router",
"options": [],
"query": "from(bucket: \"telemetry\")\n |> range(start: -1h)\n |> filter(fn: (r) => r._measurement == \"interface_counters\")\n |> keep(columns: [\"source\"])\n |> distinct(column: \"source\")\n |> sort()",
"refresh": 2,
"regex": "",
"type": "query"
},
{
"current": {},
"datasource": {"type": "influxdb","uid": "obmp_influxdb"},
"definition": "from(bucket: \"telemetry\")\n |> range(start: -1h)\n |> filter(fn: (r) => r._measurement == \"interface_counters\")\n |> filter(fn: (r) => r.source =~ /${router:regex}/)\n |> keep(columns: [\"name\"])\n |> distinct(column: \"name\")\n |> sort()",
"hide": 0,
"includeAll": true,
"label": "Interface",
"multi": true,
"name": "interface",
"options": [],
"query": "from(bucket: \"telemetry\")\n |> range(start: -1h)\n |> filter(fn: (r) => r._measurement == \"interface_counters\")\n |> filter(fn: (r) => r.source =~ /${router:regex}/)\n |> keep(columns: [\"name\"])\n |> distinct(column: \"name\")\n |> sort()",
"refresh": 2,
"regex": "",
"type": "query"
}
]
},
"panels": [
{
"datasource": {"type": "influxdb","uid": "obmp_influxdb"},
"description": "Interface error counters over time: input errors, output errors, and CRC errors. A rising trend indicates physical or configuration issues.",
"fieldConfig": {
"defaults": {
"color": {"mode": "palette-classic"},
"custom": {"axisBorderShow": false,"axisCenteredZero": false,"axisLabel": "","axisPlacement": "auto","barAlignment": 0,"drawStyle": "line","fillOpacity": 10,"gradientMode": "none","hideFrom": {"legend": false,"tooltip": false,"viz": false},"lineInterpolation": "linear","lineWidth": 1,"pointSize": 5,"scaleDistribution": {"type": "linear"},"showPoints": "never","spanNulls": false,"stacking": {"group": "A","mode": "none"},"thresholdsStyle": {"mode": "off"}},
"mappings": [],
"thresholds": {"mode": "absolute","steps": [{"color": "green","value": null},{"color": "yellow","value": 1},{"color": "red","value": 100}]},
"unit": "short"
}
},
"gridPos": {"h": 10,"w": 24,"x": 0,"y": 0},
"id": 1,
"options": {"legend": {"calcs": ["mean","max","last"],"displayMode": "table","placement": "bottom"},"tooltip": {"mode": "multi","sort": "desc"}},
"targets": [
{
"datasource": {"type": "influxdb","uid": "obmp_influxdb"},
"query": "from(bucket: \"telemetry\")\n |> range(start: v.timeRangeStart, stop: v.timeRangeStop)\n |> filter(fn: (r) => r._measurement == \"interface_counters\")\n |> filter(fn: (r) => r.source =~ /${router:regex}/)\n |> filter(fn: (r) => r.name =~ /${interface:regex}/)\n |> filter(fn: (r) => r._field == \"in-errors\" or r._field == \"out-errors\" or r._field == \"in-fcs-errors\")\n |> derivative(unit: 1s, nonNegative: true)",
"refId": "A"
}
],
"title": "Interface Errors",
"type": "timeseries"
},
{
"datasource": {"type": "influxdb","uid": "obmp_influxdb"},
"description": "Interface drop counters over time: input drops and output drops. Drops indicate congestion or queue overflow.",
"fieldConfig": {
"defaults": {
"color": {"mode": "palette-classic"},
"custom": {"axisBorderShow": false,"axisCenteredZero": false,"axisLabel": "","axisPlacement": "auto","barAlignment": 0,"drawStyle": "line","fillOpacity": 10,"gradientMode": "none","hideFrom": {"legend": false,"tooltip": false,"viz": false},"lineInterpolation": "linear","lineWidth": 1,"pointSize": 5,"scaleDistribution": {"type": "linear"},"showPoints": "never","spanNulls": false,"stacking": {"group": "A","mode": "none"},"thresholdsStyle": {"mode": "off"}},
"mappings": [],
"thresholds": {"mode": "absolute","steps": [{"color": "green","value": null},{"color": "yellow","value": 1},{"color": "red","value": 100}]},
"unit": "short"
}
},
"gridPos": {"h": 10,"w": 24,"x": 0,"y": 10},
"id": 2,
"options": {"legend": {"calcs": ["mean","max","last"],"displayMode": "table","placement": "bottom"},"tooltip": {"mode": "multi","sort": "desc"}},
"targets": [
{
"datasource": {"type": "influxdb","uid": "obmp_influxdb"},
"query": "from(bucket: \"telemetry\")\n |> range(start: v.timeRangeStart, stop: v.timeRangeStop)\n |> filter(fn: (r) => r._measurement == \"interface_counters\")\n |> filter(fn: (r) => r.source =~ /${router:regex}/)\n |> filter(fn: (r) => r.name =~ /${interface:regex}/)\n |> filter(fn: (r) => r._field == \"in-discards\" or r._field == \"out-discards\")\n |> derivative(unit: 1s, nonNegative: true)",
"refId": "A"
}
],
"title": "Interface Drops",
"type": "timeseries"
},
{
"datasource": {"type": "influxdb","uid": "obmp_influxdb"},
"description": "Summary table showing the latest error and drop counter values per interface. Useful for quickly identifying problematic interfaces.",
"fieldConfig": {
"defaults": {
"color": {"mode": "thresholds"},
"custom": {"align": "auto","displayMode": "auto","filterable": true,"inspect": true},
"mappings": [],
"thresholds": {"mode": "absolute","steps": [{"color": "green","value": null},{"color": "yellow","value": 1},{"color": "red","value": 100}]}
},
"overrides": [
{"matcher": {"id": "byName","options": "in-errors"},"properties": [{"id": "custom.displayMode","value": "color-background-solid"},{"id": "thresholds","value": {"mode": "absolute","steps": [{"color": "green","value": null},{"color": "yellow","value": 1},{"color": "red","value": 100}]}}]},
{"matcher": {"id": "byName","options": "out-errors"},"properties": [{"id": "custom.displayMode","value": "color-background-solid"},{"id": "thresholds","value": {"mode": "absolute","steps": [{"color": "green","value": null},{"color": "yellow","value": 1},{"color": "red","value": 100}]}}]},
{"matcher": {"id": "byName","options": "in-discards"},"properties": [{"id": "custom.displayMode","value": "color-background-solid"},{"id": "thresholds","value": {"mode": "absolute","steps": [{"color": "green","value": null},{"color": "yellow","value": 1},{"color": "red","value": 100}]}}]},
{"matcher": {"id": "byName","options": "out-discards"},"properties": [{"id": "custom.displayMode","value": "color-background-solid"},{"id": "thresholds","value": {"mode": "absolute","steps": [{"color": "green","value": null},{"color": "yellow","value": 1},{"color": "red","value": 100}]}}]}
]
},
"gridPos": {"h": 12,"w": 24,"x": 0,"y": 20},
"id": 3,
"options": {"footer": {"fields": "","reducer": ["sum"],"show": false},"showHeader": true,"sortBy": [{"desc": true,"displayName": "in-errors"}]},
"targets": [
{
"datasource": {"type": "influxdb","uid": "obmp_influxdb"},
"query": "from(bucket: \"telemetry\")\n |> range(start: v.timeRangeStart, stop: v.timeRangeStop)\n |> filter(fn: (r) => r._measurement == \"interface_counters\")\n |> filter(fn: (r) => r.source =~ /${router:regex}/)\n |> filter(fn: (r) => r.name =~ /${interface:regex}/)\n |> filter(fn: (r) => r._field == \"in-errors\" or r._field == \"out-errors\" or r._field == \"in-fcs-errors\" or r._field == \"in-discards\" or r._field == \"out-discards\")\n |> last()\n |> pivot(rowKey: [\"_time\"], columnKey: [\"_field\"], valueColumn: \"_value\")\n |> keep(columns: [\"source\", \"name\", \"in-errors\", \"out-errors\", \"in-fcs-errors\", \"in-discards\", \"out-discards\"])\n |> sort(columns: [\"in-errors\"], desc: true)",
"refId": "A"
}
],
"title": "Error Summary Table",
"type": "table"
}
],
"schemaVersion": 39,
"style": "dark",
"tags": ["obmp-telemetry"],
"time": {"from": "now-1h","to": "now"},
"timepicker": {},
"timezone": "browser",
"title": "Interface Errors & Drops",
"uid": "obmp-telem-02",
"version": 1
}

141
obmp-grafana/dashboards/Telemetry-3001/interface_utilization.json
View File

@ -1,141 +0,0 @@
{
"annotations": {"list": [{"builtIn": 1,"datasource": {"type": "datasource","uid": "grafana"},"enable": true,"hide": true,"iconColor": "rgba(0, 211, 255, 1)","name": "Annotations & Alerts","type": "dashboard"}]},
"description": "Interface utilization metrics collected via gNMI streaming telemetry from IOS-XR routers. Shows byte rates, packet rates, and top interfaces by traffic volume.",
"editable": true,
"fiscalYearStartMonth": 0,
"graphTooltip": 1,
"id": null,
"links": [],
"templating": {
"list": [
{
"current": {},
"datasource": {"type": "influxdb","uid": "obmp_influxdb"},
"definition": "from(bucket: \"telemetry\")\n |> range(start: -1h)\n |> filter(fn: (r) => r._measurement == \"interface_counters\")\n |> keep(columns: [\"source\"])\n |> distinct(column: \"source\")\n |> sort()",
"hide": 0,
"includeAll": true,
"label": "Router",
"multi": true,
"name": "router",
"options": [],
"query": "from(bucket: \"telemetry\")\n |> range(start: -1h)\n |> filter(fn: (r) => r._measurement == \"interface_counters\")\n |> keep(columns: [\"source\"])\n |> distinct(column: \"source\")\n |> sort()",
"refresh": 2,
"regex": "",
"type": "query"
},
{
"current": {},
"datasource": {"type": "influxdb","uid": "obmp_influxdb"},
"definition": "from(bucket: \"telemetry\")\n |> range(start: -1h)\n |> filter(fn: (r) => r._measurement == \"interface_counters\")\n |> filter(fn: (r) => r.source =~ /${router:regex}/)\n |> keep(columns: [\"name\"])\n |> distinct(column: \"name\")\n |> sort()",
"hide": 0,
"includeAll": true,
"label": "Interface",
"multi": true,
"name": "interface",
"options": [],
"query": "from(bucket: \"telemetry\")\n |> range(start: -1h)\n |> filter(fn: (r) => r._measurement == \"interface_counters\")\n |> filter(fn: (r) => r.source =~ /${router:regex}/)\n |> keep(columns: [\"name\"])\n |> distinct(column: \"name\")\n |> sort()",
"refresh": 2,
"regex": "",
"type": "query"
}
]
},
"panels": [
{
"datasource": {"type": "influxdb","uid": "obmp_influxdb"},
"description": "Rate of bytes received and sent per interface, computed as the derivative of cumulative counters. Unit: bytes per second.",
"fieldConfig": {
"defaults": {
"color": {"mode": "palette-classic"},
"custom": {"axisBorderShow": false,"axisCenteredZero": false,"axisLabel": "","axisPlacement": "auto","barAlignment": 0,"drawStyle": "line","fillOpacity": 10,"gradientMode": "none","hideFrom": {"legend": false,"tooltip": false,"viz": false},"lineInterpolation": "linear","lineWidth": 1,"pointSize": 5,"scaleDistribution": {"type": "linear"},"showPoints": "never","spanNulls": false,"stacking": {"group": "A","mode": "none"},"thresholdsStyle": {"mode": "off"}},
"mappings": [],
"thresholds": {"mode": "absolute","steps": [{"color": "green","value": null},{"color": "red","value": 80}]},
"unit": "Bps"
}
},
"gridPos": {"h": 10,"w": 24,"x": 0,"y": 0},
"id": 1,
"options": {"legend": {"calcs": ["mean","max"],"displayMode": "table","placement": "bottom"},"tooltip": {"mode": "multi","sort": "desc"}},
"targets": [
{
"datasource": {"type": "influxdb","uid": "obmp_influxdb"},
"query": "from(bucket: \"telemetry\")\n |> range(start: v.timeRangeStart, stop: v.timeRangeStop)\n |> filter(fn: (r) => r._measurement == \"interface_counters\")\n |> filter(fn: (r) => r.source =~ /${router:regex}/)\n |> filter(fn: (r) => r.name =~ /${interface:regex}/)\n |> filter(fn: (r) => r._field == \"in-octets\" or r._field == \"out-octets\")\n |> derivative(unit: 1s, nonNegative: true)\n |> map(fn: (r) => ({r with _value: if r._value < 0.0 then 0.0 else r._value}))",
"refId": "A"
}
],
"title": "Input/Output Bytes Rate",
"type": "timeseries"
},
{
"datasource": {"type": "influxdb","uid": "obmp_influxdb"},
"description": "Rate of packets received and sent per interface, computed as the derivative of cumulative counters. Unit: packets per second.",
"fieldConfig": {
"defaults": {
"color": {"mode": "palette-classic"},
"custom": {"axisBorderShow": false,"axisCenteredZero": false,"axisLabel": "","axisPlacement": "auto","barAlignment": 0,"drawStyle": "line","fillOpacity": 10,"gradientMode": "none","hideFrom": {"legend": false,"tooltip": false,"viz": false},"lineInterpolation": "linear","lineWidth": 1,"pointSize": 5,"scaleDistribution": {"type": "linear"},"showPoints": "never","spanNulls": false,"stacking": {"group": "A","mode": "none"},"thresholdsStyle": {"mode": "off"}},
"mappings": [],
"thresholds": {"mode": "absolute","steps": [{"color": "green","value": null},{"color": "red","value": 80}]},
"unit": "pps"
}
},
"gridPos": {"h": 10,"w": 24,"x": 0,"y": 10},
"id": 2,
"options": {"legend": {"calcs": ["mean","max"],"displayMode": "table","placement": "bottom"},"tooltip": {"mode": "multi","sort": "desc"}},
"targets": [
{
"datasource": {"type": "influxdb","uid": "obmp_influxdb"},
"query": "from(bucket: \"telemetry\")\n |> range(start: v.timeRangeStart, stop: v.timeRangeStop)\n |> filter(fn: (r) => r._measurement == \"interface_counters\")\n |> filter(fn: (r) => r.source =~ /${router:regex}/)\n |> filter(fn: (r) => r.name =~ /${interface:regex}/)\n |> filter(fn: (r) => r._field == \"in-pkts\" or r._field == \"out-pkts\")\n |> derivative(unit: 1s, nonNegative: true)\n |> map(fn: (r) => ({r with _value: if r._value < 0.0 then 0.0 else r._value}))",
"refId": "A"
}
],
"title": "Input/Output Packets Rate",
"type": "timeseries"
},
{
"datasource": {"type": "influxdb","uid": "obmp_influxdb"},
"description": "Top interfaces ranked by total bytes (received + sent) over the selected time range.",
"fieldConfig": {
"defaults": {
"color": {"mode": "palette-classic"},
"custom": {"axisBorderShow": false,"axisCenteredZero": false,"axisLabel": "","axisPlacement": "auto","fillOpacity": 80,"gradientMode": "none","hideFrom": {"legend": false,"tooltip": false,"viz": false},"lineWidth": 1,"scaleDistribution": {"type": "linear"},"thresholdsStyle": {"mode": "off"}},
"mappings": [],
"thresholds": {"mode": "absolute","steps": [{"color": "green","value": null}]},
"unit": "decbytes"
}
},
"gridPos": {"h": 10,"w": 24,"x": 0,"y": 20},
"id": 3,
"options": {"barRadius": 0,"barWidth": 0.6,"fullHighlight": false,"groupWidth": 0.7,"legend": {"calcs": [],"displayMode": "list","placement": "bottom"},"orientation": "horizontal","showValue": "auto","stacking": "none","tooltip": {"mode": "single","sort": "none"},"xTickLabelRotation": 0},
"targets": [
{
"datasource": {"type": "influxdb","uid": "obmp_influxdb"},
"query": "from(bucket: \"telemetry\")\n |> range(start: v.timeRangeStart, stop: v.timeRangeStop)\n |> filter(fn: (r) => r._measurement == \"interface_counters\")\n |> filter(fn: (r) => r.source =~ /${router:regex}/)\n |> filter(fn: (r) => r.name =~ /${interface:regex}/)\n |> filter(fn: (r) => r._field == \"in-octets\" or r._field == \"out-octets\")\n |> derivative(unit: 1s, nonNegative: true)\n |> group(columns: [\"source\", \"name\", \"_field\"])\n |> sum()\n |> group(columns: [\"source\", \"name\"])\n |> sum()\n |> group()\n |> sort(columns: [\"_value\"], desc: true)\n |> limit(n: 20)",
"refId": "A"
}
],
"title": "Top Interfaces by Traffic",
"type": "barchart"
},
{
"datasource": {"type": "datasource","uid": "grafana"},
"gridPos": {"h": 4,"w": 24,"x": 0,"y": 30},
"id": 4,
"options": {
"code": {"language": "plaintext","showLineNumbers": false,"showMiniMap": false},
"content": "## Interface Utilization Dashboard\n\nThis dashboard displays real-time interface utilization metrics collected via **gNMI streaming telemetry** from IOS-XR routers.\n\n- **Data source:** InfluxDB (Telegraf gNMI input plugin)\n- **YANG model:** OpenConfig (`openconfig-interfaces`)\n- **Subscription path:** `/interfaces/interface/state/counters`\n- **Sample interval:** 10 seconds\n\nUse the **Router** and **Interface** template variables at the top to filter the view.",
"mode": "markdown"
},
"title": "About",
"type": "text"
}
],
"schemaVersion": 39,
"style": "dark",
"tags": ["obmp-telemetry"],
"time": {"from": "now-1h","to": "now"},
"timepicker": {},
"timezone": "browser",
"title": "Interface Utilization",
"uid": "obmp-telem-01",
"version": 1
}

11
obmp-grafana/provisioning/dashboards/openbmp-dashboards.yml
View File

@ -134,14 +134,3 @@ providers:
options:
path: /var/lib/grafana/dashboards/Learning
foldersFromFilesStructure: false
- name: 'OpenBMP-Telemetry'
orgId: 1
folder: 'OBMP-Telemetry'
folderUid: '3001'
type: file
disableDeletion: false
updateIntervalSeconds: 30
allowUiUpdates: true
options:
path: /var/lib/grafana/dashboards/Telemetry-3001
foldersFromFilesStructure: false

16
obmp-grafana/provisioning/datasources/influxdb-ds.yml
View File

@ -1,16 +0,0 @@
apiVersion: 1
datasources:
- name: InfluxDB-Telemetry
uid: obmp_influxdb
type: influxdb
access: proxy
url: http://obmp-influxdb:8086
jsonData:
version: Flux
organization: openbmp
defaultBucket: telemetry
secureJsonData:
token: openbmp-telemetry-token
isDefault: false
editable: true

2
telegraf/Dockerfile
View File

@ -1,2 +0,0 @@
FROM telegraf:1.28-alpine
COPY telegraf.conf /etc/telegraf/telegraf.conf

63
telegraf/telegraf.conf
View File

@ -1,63 +0,0 @@
# Telegraf Configuration for gNMI Streaming Telemetry
# Collects interface counters and data rates from IOS-XR routers
[global_tags]
[agent]
interval = "10s"
round_interval = true
metric_batch_size = 1000
metric_buffer_limit = 10000
collection_jitter = "0s"
flush_interval = "10s"
flush_jitter = "0s"
precision = "0s"
###############################################################################
# INPUT PLUGINS #
###############################################################################
## CORE routers (directly reachable on port 57400 from host)
## R9K routers (10.100.0.1-7) are blocked by CML management network filtering
[[inputs.gnmi]]
addresses = [
"10.100.0.100:57400",
"10.100.0.200:57400"
]
username = "webui"
password = "cisco"
## No TLS (lab environment)
enable_tls = false
## Use json_ietf encoding (supported by IOS-XR 24.3.1)
encoding = "json_ietf"
## Redial in case of failures after
redial = "10s"
## OpenConfig interface counters (bytes, packets, errors, discards)
[[inputs.gnmi.subscription]]
name = "interface_counters"
origin = "openconfig-interfaces"
path = "/interfaces/interface/state/counters"
subscription_mode = "sample"
sample_interval = "10s"
## OpenConfig interface state (admin/oper status, description, type)
[[inputs.gnmi.subscription]]
name = "interface_state"
origin = "openconfig-interfaces"
path = "/interfaces/interface/state"
subscription_mode = "sample"
sample_interval = "30s"
###############################################################################
# OUTPUT PLUGINS #
###############################################################################
[[outputs.influxdb_v2]]
urls = ["http://localhost:8086"]
token = "${INFLUXDB_TOKEN}"
organization = "openbmp"
bucket = "telemetry"

12
traffic-gen-ui/Dockerfile
View File

@ -1,12 +0,0 @@
FROM node:20-alpine AS build
WORKDIR /app
COPY package.json ./
RUN npm install
COPY . .
RUN npm run build
FROM nginx:alpine
COPY --from=build /app/dist /usr/share/nginx/html
COPY nginx.conf /etc/nginx/conf.d/default.conf
EXPOSE 5002
CMD ["nginx", "-g", "daemon off;"]

12
traffic-gen-ui/index.html
View File

@ -1,12 +0,0 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>Traffic Generator</title>
</head>
<body>
<div id="app"></div>
<script type="module" src="/src/main.js"></script>
</body>
</html>

15
traffic-gen-ui/nginx.conf
View File

@ -1,15 +0,0 @@
server {
listen 5002;
root /usr/share/nginx/html;
index index.html;
location /api/ {
proxy_pass http://localhost:5051/;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
}
location / {
try_files $uri $uri/ /index.html;
}
}

17
traffic-gen-ui/package.json
View File

@ -1,17 +0,0 @@
{
"name": "traffic-gen-ui",
"version": "1.0.0",
"private": true,
"scripts": {
"dev": "vite",
"build": "vite build",
"preview": "vite preview"
},
"dependencies": {
"vue": "^3.3.0"
},
"devDependencies": {
"@vitejs/plugin-vue": "^4.2.0",
"vite": "^4.4.0"
}
}

313
traffic-gen-ui/src/App.vue
View File

@ -1,313 +0,0 @@
<template>
<div class="app-layout">
<!-- HEADER -->
<header class="app-header">
<div class="header-title">
<span class="logo-icon">&#9889;</span>
<h1>Traffic Generator</h1>
</div>
<StatusBar :health="health" :api-error="apiError" />
</header>
<!-- ERROR BANNER -->
<div v-if="apiError" class="error-banner">
<span class="error-icon">&#9888;</span>
API unreachable: {{ apiError }} &mdash; retrying every 5s
</div>
<!-- MAIN CONTENT -->
<div class="main-content">
<!-- LEFT COLUMN: Flow Builder -->
<aside class="left-col">
<FlowBuilder @created="fetchFlows" @updated="fetchFlows" />
</aside>
<!-- RIGHT COLUMN: Tabs -->
<main class="right-col">
<div class="tabs">
<button
v-for="tab in tabs"
:key="tab.id"
class="tab-btn"
:class="{ active: activeTab === tab.id }"
@click="activeTab = tab.id"
>
{{ tab.label }}
</button>
</div>
<div class="tab-content">
<FlowTable v-if="activeTab === 'flows'" :flows="flows" @refresh="fetchFlows" />
<TestBuilder v-else-if="activeTab === 'tests'" :flows="flows" @created="fetchTests" @refresh="fetchAll" />
<TestRunner v-else-if="activeTab === 'runner'" :tests="tests" @refresh="fetchTests" />
<ResultsPanel v-else-if="activeTab === 'results'" :tests="tests" />
<StatsMonitor v-else-if="activeTab === 'monitor'" :flows="flows" />
</div>
</main>
</div>
<!-- FOOTER -->
<footer class="app-footer">
<span>Refreshing every 5s (health) / 3s (flows)</span>
<span class="footer-sep">|</span>
<a href="http://localhost:3000" target="_blank" class="footer-link">Grafana: :3000</a>
<span class="footer-sep">|</span>
<a href="http://localhost:5001" target="_blank" class="footer-link">Route Injector: :5001</a>
</footer>
</div>
</template>
<script setup>
import { ref, onMounted, onUnmounted } from 'vue'
import { api } from './api.js'
import StatusBar from './components/StatusBar.vue'
import FlowBuilder from './components/FlowBuilder.vue'
import FlowTable from './components/FlowTable.vue'
import TestBuilder from './components/TestBuilder.vue'
import TestRunner from './components/TestRunner.vue'
import ResultsPanel from './components/ResultsPanel.vue'
import StatsMonitor from './components/StatsMonitor.vue'
const health = ref(null)
const flows = ref([])
const tests = ref([])
const apiError = ref(null)
const activeTab = ref('flows')
const tabs = [
{ id: 'flows', label: 'Flows' },
{ id: 'tests', label: 'Tests' },
{ id: 'runner', label: 'Runner' },
{ id: 'results', label: 'Results' },
{ id: 'monitor', label: 'Monitor' },
]
async function fetchHealth() {
try {
health.value = await api.health()
apiError.value = null
} catch (e) {
apiError.value = e.message
health.value = null
}
}
async function fetchFlows() {
try {
const data = await api.flows()
flows.value = data.flows || []
} catch (_) {}
}
async function fetchTests() {
try {
const data = await api.tests()
tests.value = data.tests || []
} catch (_) {}
}
async function fetchAll() {
await Promise.all([fetchFlows(), fetchTests()])
}
let healthTimer = null
let dataTimer = null
onMounted(() => {
fetchHealth()
fetchAll()
healthTimer = setInterval(fetchHealth, 5000)
dataTimer = setInterval(fetchAll, 3000)
})
onUnmounted(() => {
clearInterval(healthTimer)
clearInterval(dataTimer)
})
</script>
<style>
:root {
--bg: #0f1117;
--card-bg: #1a1f2e;
--border: #2d3748;
--accent: #4f9cf9;
--success: #48bb78;
--danger: #fc8181;
--warning: #f6ad55;
--text: #e2e8f0;
--muted: #718096;
--radius: 8px;
}
* { box-sizing: border-box; margin: 0; padding: 0; }
body {
background: var(--bg);
color: var(--text);
font-family: 'Segoe UI', system-ui, -apple-system, sans-serif;
font-size: 14px;
line-height: 1.5;
}
button {
cursor: pointer;
font-family: inherit;
font-size: 13px;
border: none;
border-radius: var(--radius);
transition: opacity 0.15s, background 0.15s;
}
button:disabled {
opacity: 0.5;
cursor: not-allowed;
}
input, select {
font-family: inherit;
font-size: 13px;
background: var(--bg);
color: var(--text);
border: 1px solid var(--border);
border-radius: var(--radius);
padding: 6px 10px;
outline: none;
}
input:focus {
border-color: var(--accent);
}
</style>
<style scoped>
.app-layout {
display: grid;
grid-template-rows: auto auto 1fr auto;
min-height: 100vh;
}
.app-header {
display: flex;
align-items: center;
justify-content: space-between;
padding: 12px 20px;
background: var(--card-bg);
border-bottom: 1px solid var(--border);
gap: 16px;
flex-wrap: wrap;
}
.header-title {
display: flex;
align-items: center;
gap: 10px;
}
.logo-icon {
font-size: 22px;
color: var(--warning);
}
.app-header h1 {
font-size: 18px;
font-weight: 600;
color: var(--text);
letter-spacing: 0.02em;
}
.error-banner {
background: rgba(252, 129, 129, 0.12);
border-bottom: 1px solid var(--danger);
color: var(--danger);
padding: 8px 20px;
font-size: 13px;
display: flex;
align-items: center;
gap: 8px;
}
.error-icon {
font-size: 16px;
}
.main-content {
display: grid;
grid-template-columns: 340px 1fr;
overflow: hidden;
height: calc(100vh - 110px);
}
.left-col {
border-right: 1px solid var(--border);
overflow-y: auto;
padding: 16px;
}
.right-col {
display: flex;
flex-direction: column;
overflow: hidden;
}
.tabs {
display: flex;
gap: 2px;
padding: 12px 16px 0;
background: var(--card-bg);
border-bottom: 1px solid var(--border);
}
.tab-btn {
background: transparent;
color: var(--muted);
padding: 8px 18px;
border-radius: var(--radius) var(--radius) 0 0;
border: 1px solid transparent;
border-bottom: none;
font-weight: 500;
}
.tab-btn:hover {
color: var(--text);
background: rgba(79, 156, 249, 0.08);
}
.tab-btn.active {
color: var(--accent);
background: var(--bg);
border-color: var(--border);
border-bottom: 1px solid var(--bg);
margin-bottom: -1px;
}
.tab-content {
flex: 1;
overflow-y: auto;
padding: 16px;
}
.app-footer {
padding: 8px 20px;
background: var(--card-bg);
border-top: 1px solid var(--border);
color: var(--muted);
font-size: 12px;
display: flex;
align-items: center;
gap: 10px;
}
.footer-sep {
color: var(--border);
}
.footer-link {
color: var(--accent);
text-decoration: none;
}
.footer-link:hover {
text-decoration: underline;
}
</style>

44
traffic-gen-ui/src/api.js
View File

@ -1,44 +0,0 @@
const BASE = '/api'
async function req(method, path, body) {
const opts = { method, headers: { 'Content-Type': 'application/json' } }
if (body) opts.body = JSON.stringify(body)
const r = await fetch(BASE + path, opts)
if (!r.ok) throw new Error(`${method} ${path} -> ${r.status}`)
return r.json()
}
export const api = {
health: () => req('GET', '/healthz'),
interfaces: () => req('GET', '/interfaces'),
mode: () => req('GET', '/mode'),
// Flows
flows: () => req('GET', '/flows'),
createFlow: (f) => req('POST', '/flows', f),
getFlow: (id) => req('GET', `/flows/${id}`),
updateFlow: (id, f) => req('PUT', `/flows/${id}`, f),
deleteFlow: (id) => req('DELETE', `/flows/${id}`),
startFlow: (id) => req('POST', `/flows/${id}/start`),
stopFlow: (id) => req('POST', `/flows/${id}/stop`),
flowStats: (id) => req('GET', `/flows/${id}/stats`),
// Tests
tests: () => req('GET', '/tests'),
createTest: (t) => req('POST', '/tests', t),
getTest: (id) => req('GET', `/tests/${id}`),
startTest: (id) => req('POST', `/tests/${id}/start`),
stopTest: (id) => req('POST', `/tests/${id}/stop`),
testResults: (id) => req('GET', `/tests/${id}/results`),
// Presets
presets: () => req('GET', '/presets'),
loadPreset: (name, overrides) => req('POST', `/presets/${name}`, overrides),
// Stats
statsHistory: () => req('GET', '/stats/history'),
// Responder
responderStats: () => req('GET', '/responder/stats'),
responderReset: () => req('POST', '/responder/reset'),
}

126
traffic-gen-ui/src/components/FlowBuilder.vue
View File

@ -1,126 +0,0 @@
<template>
<div class="flow-builder">
<h3>{{ editing ? 'Edit Flow' : 'Create Flow' }}</h3>
<form @submit.prevent="submit">
<div class="form-row">
<label>Name</label>
<input v-model="form.name" placeholder="My Flow" />
</div>
<div class="form-row">
<label>Destination IP *</label>
<input v-model="form.dst_ip" placeholder="10.100.0.100" required />
</div>
<div class="form-row">
<label>Source IP</label>
<input v-model="form.src_ip" placeholder="auto (from interface)" />
</div>
<div class="form-row">
<label>Dst MAC</label>
<input v-model="form.dst_mac" placeholder="auto" />
</div>
<div class="form-row">
<label>Protocol</label>
<select v-model="form.protocol">
<option value="udp">UDP</option>
<option value="tcp">TCP</option>
<option value="icmp">ICMP</option>
</select>
</div>
<div v-if="form.protocol !== 'icmp'" class="form-row-pair">
<div class="form-row">
<label>Src Port</label>
<input v-model.number="form.src_port" type="number" min="1" max="65535" />
</div>
<div class="form-row">
<label>Dst Port</label>
<input v-model.number="form.dst_port" type="number" min="1" max="65535" />
</div>
</div>
<div class="form-row-pair">
<div class="form-row">
<label>Frame Size (bytes)</label>
<input v-model.number="form.frame_size" type="number" min="64" max="9000" />
</div>
<div class="form-row">
<label>Rate (pps)</label>
<input v-model.number="form.rate_pps" type="number" min="1" max="100000" />
</div>
</div>
<div class="form-row-pair">
<div class="form-row">
<label>Duration (sec)</label>
<input v-model.number="form.duration" type="number" min="0" />
</div>
<div class="form-row">
<label>DSCP</label>
<input v-model.number="form.dscp" type="number" min="0" max="63" />
</div>
</div>
<div class="form-row">
<label>Responder URL (optional)</label>
<input v-model="form.responder_url" placeholder="http://host:5053" />
</div>
<div class="form-actions">
<button type="submit" class="btn btn-accent" :disabled="!form.dst_ip">
{{ editing ? 'Update' : 'Create Flow' }}
</button>
<button v-if="editing" type="button" class="btn btn-muted" @click="$emit('cancel')">Cancel</button>
</div>
</form>
</div>
</template>
<script setup>
import { reactive, watch } from 'vue'
import { api } from '../api.js'
const props = defineProps({ editFlow: Object })
const emit = defineEmits(['created', 'updated', 'cancel'])
const editing = !!props.editFlow
const defaults = {
name: '', dst_ip: '', src_ip: '', dst_mac: '',
protocol: 'udp', src_port: 50000, dst_port: 5001,
frame_size: 512, rate_pps: 1000, duration: 30,
dscp: 0, responder_url: '',
}
const form = reactive({ ...defaults, ...(props.editFlow || {}) })
async function submit() {
try {
const payload = { ...form }
if (!payload.src_ip) delete payload.src_ip
if (!payload.dst_mac) delete payload.dst_mac
if (!payload.responder_url) delete payload.responder_url
if (!payload.name) payload.name = `${payload.protocol.toUpperCase()} -> ${payload.dst_ip}`
if (editing) {
await api.updateFlow(props.editFlow.id, payload)
emit('updated')
} else {
await api.createFlow(payload)
Object.assign(form, defaults)
emit('created')
}
} catch (e) {
alert('Error: ' + e.message)
}
}
</script>
<style scoped>
.flow-builder { padding: 0; }
h3 { font-size: 15px; margin-bottom: 12px; color: var(--accent); }
.form-row { margin-bottom: 10px; }
.form-row label { display: block; font-size: 11px; color: var(--muted); margin-bottom: 3px; text-transform: uppercase; letter-spacing: 0.05em; }
.form-row input, .form-row select { width: 100%; }
.form-row-pair { display: grid; grid-template-columns: 1fr 1fr; gap: 8px; }
.form-actions { display: flex; gap: 8px; margin-top: 14px; }
.btn { padding: 8px 16px; font-weight: 600; font-size: 13px; }
.btn-accent { background: var(--accent); color: #fff; }
.btn-accent:hover { opacity: 0.9; }
.btn-accent:disabled { opacity: 0.4; }
.btn-muted { background: var(--border); color: var(--text); }
</style>

92
traffic-gen-ui/src/components/FlowTable.vue
View File

@ -1,92 +0,0 @@
<template>
<div class="flow-table">
<div v-if="!flows.length" class="empty">No flows created yet. Use the builder to create one.</div>
<table v-else>
<thead>
<tr>
<th>Name</th>
<th>Dst IP</th>
<th>Proto</th>
<th>Size</th>
<th>Rate</th>
<th>State</th>
<th>TX pps</th>
<th>Actions</th>
</tr>
</thead>
<tbody>
<tr v-for="f in flows" :key="f.id" :class="{ running: f.state === 'running' }">
<td>{{ f.name || '-' }}</td>
<td class="mono">{{ f.dst_ip }}</td>
<td>{{ f.protocol.toUpperCase() }}</td>
<td>{{ f.frame_size }}B</td>
<td>{{ f.rate_pps }} pps</td>
<td>
<span class="state-badge" :class="'state-' + f.state">{{ f.state }}</span>
</td>
<td class="mono">{{ stats[f.id]?.tx_pps || 0 }}</td>
<td class="actions">
<button v-if="f.state !== 'running'" class="btn-sm btn-go" @click="start(f.id)">Start</button>
<button v-else class="btn-sm btn-stop" @click="stop(f.id)">Stop</button>
<button class="btn-sm btn-del" @click="del(f.id)" :disabled="f.state === 'running'">Del</button>
</td>
</tr>
</tbody>
</table>
</div>
</template>
<script setup>
import { ref, onMounted, onUnmounted } from 'vue'
import { api } from '../api.js'
const props = defineProps({ flows: Array })
const emit = defineEmits(['refresh'])
const stats = ref({})
let statsTimer = null
async function fetchStats() {
for (const f of (props.flows || [])) {
if (f.state === 'running') {
try {
const s = await api.flowStats(f.id)
stats.value[f.id] = s
} catch (_) {}
}
}
}
onMounted(() => { statsTimer = setInterval(fetchStats, 1000) })
onUnmounted(() => { clearInterval(statsTimer) })
async function start(id) {
try { await api.startFlow(id); emit('refresh') } catch (e) { alert(e.message) }
}
async function stop(id) {
try { await api.stopFlow(id); emit('refresh') } catch (e) { alert(e.message) }
}
async function del(id) {
try { await api.deleteFlow(id); emit('refresh') } catch (e) { alert(e.message) }
}
</script>
<style scoped>
.flow-table { overflow-x: auto; }
.empty { color: var(--muted); padding: 20px; text-align: center; }
table { width: 100%; border-collapse: collapse; }
th { text-align: left; font-size: 11px; color: var(--muted); text-transform: uppercase; letter-spacing: 0.05em; padding: 6px 8px; border-bottom: 1px solid var(--border); }
td { padding: 8px; border-bottom: 1px solid rgba(45,55,72,0.5); font-size: 13px; }
tr.running { background: rgba(79,156,249,0.05); }
.mono { font-family: 'Cascadia Code', 'Fira Code', monospace; font-size: 12px; }
.state-badge { font-size: 11px; padding: 2px 8px; border-radius: 10px; font-weight: 600; }
.state-idle { background: rgba(113,128,150,0.2); color: var(--muted); }
.state-running { background: rgba(72,187,120,0.15); color: var(--success); }
.state-stopped { background: rgba(246,173,85,0.15); color: var(--warning); }
.actions { display: flex; gap: 4px; }
.btn-sm { padding: 3px 10px; font-size: 11px; font-weight: 600; border-radius: 6px; }
.btn-go { background: var(--success); color: #fff; }
.btn-stop { background: var(--warning); color: #000; }
.btn-del { background: rgba(252,129,129,0.15); color: var(--danger); }
.btn-del:disabled { opacity: 0.3; }
</style>

126
traffic-gen-ui/src/components/ResultsPanel.vue
View File

@ -1,126 +0,0 @@
<template>
<div class="results-panel">
<h3>Test Results</h3>
<div v-if="!completedTests.length" class="empty">No completed tests yet.</div>
<div v-for="t in completedTests" :key="t.id" class="result-card">
<div class="result-header">
<strong>{{ t.type }} Test</strong>
<span class="result-time">{{ t.completed_at }}</span>
<div class="export-btns">
<button class="btn-sm btn-export" @click="exportJSON(t)">Export JSON</button>
<button class="btn-sm btn-export" @click="exportCSV(t)">Export CSV</button>
</div>
</div>
<div v-if="t.results" class="result-table">
<table>
<thead>
<tr>
<th>Frame Size (B)</th>
<th v-for="col in resultColumns(t)" :key="col">{{ col }}</th>
</tr>
</thead>
<tbody>
<tr v-for="(val, size) in t.results" :key="size">
<td>{{ size }}</td>
<td v-for="col in resultColumns(t)" :key="col" class="mono">
{{ formatVal(val, col) }}
</td>
</tr>
</tbody>
</table>
</div>
<div v-if="t.results" class="result-chart">
<div class="bar-chart">
<div v-for="(val, size) in t.results" :key="size" class="bar-item">
<div class="bar-fill" :style="{ height: barHeight(t, val) + '%' }"></div>
<span class="bar-label">{{ size }}B</span>
</div>
</div>
</div>
</div>
</div>
</template>
<script setup>
import { computed } from 'vue'
const props = defineProps({ tests: Array })
const completedTests = computed(() =>
(props.tests || []).filter(t => t.state === 'complete' && t.results)
)
function resultColumns(t) {
if (t.type === 'throughput') return ['Max Rate (pps)', 'Throughput (Mbps)']
if (t.type === 'latency') return ['Min (ms)', 'Avg (ms)', 'Max (ms)', 'Jitter (ms)']
if (t.type === 'frame_loss') return ['Loss %']
if (t.type === 'back_to_back') return ['Max Burst']
return ['Value']
}
function formatVal(val, col) {
if (typeof val === 'object') {
if (col.includes('Rate')) return val.max_rate_pps ?? '-'
if (col.includes('Throughput')) return val.throughput_mbps ?? '-'
if (col.includes('Min')) return val.min_ms ?? '-'
if (col.includes('Avg')) return val.avg_ms ?? '-'
if (col.includes('Max') && col.includes('ms')) return val.max_ms ?? '-'
if (col.includes('Jitter')) return val.jitter_ms ?? '-'
if (col.includes('Loss')) return val.loss_pct ?? '-'
if (col.includes('Burst')) return val.max_burst ?? '-'
return JSON.stringify(val)
}
return val
}
function barHeight(t, val) {
const v = typeof val === 'object' ? (val.max_rate_pps || val.avg_ms || val.loss_pct || val.max_burst || 0) : val
return Math.min(100, Math.max(5, v / 100))
}
function exportJSON(t) {
const blob = new Blob([JSON.stringify(t, null, 2)], { type: 'application/json' })
downloadBlob(blob, `test_${t.type}_${t.id}.json`)
}
function exportCSV(t) {
if (!t.results) return
const cols = resultColumns(t)
let csv = 'Frame Size,' + cols.join(',') + '\n'
for (const [size, val] of Object.entries(t.results)) {
csv += size + ',' + cols.map(c => formatVal(val, c)).join(',') + '\n'
}
downloadBlob(new Blob([csv], { type: 'text/csv' }), `test_${t.type}_${t.id}.csv`)
}
function downloadBlob(blob, name) {
const url = URL.createObjectURL(blob)
const a = document.createElement('a')
a.href = url; a.download = name; a.click()
URL.revokeObjectURL(url)
}
</script>
<style scoped>
h3 { font-size: 15px; margin-bottom: 12px; color: var(--accent); }
.empty { color: var(--muted); padding: 20px; text-align: center; }
.result-card { background: var(--card-bg); border: 1px solid var(--border); border-radius: var(--radius); padding: 12px; margin-bottom: 12px; }
.result-header { display: flex; align-items: center; gap: 12px; margin-bottom: 10px; flex-wrap: wrap; }
.result-header strong { font-size: 14px; text-transform: capitalize; }
.result-time { font-size: 11px; color: var(--muted); }
.export-btns { margin-left: auto; display: flex; gap: 4px; }
.btn-sm { padding: 3px 10px; font-size: 11px; font-weight: 600; border-radius: 6px; }
.btn-export { background: rgba(79,156,249,0.12); color: var(--accent); }
.btn-export:hover { background: rgba(79,156,249,0.25); }
table { width: 100%; border-collapse: collapse; }
th { font-size: 11px; color: var(--muted); text-align: left; padding: 4px 8px; border-bottom: 1px solid var(--border); }
td { font-size: 13px; padding: 4px 8px; }
.mono { font-family: monospace; }
.bar-chart { display: flex; align-items: flex-end; gap: 8px; height: 80px; margin-top: 12px; padding: 0 8px; }
.bar-item { flex: 1; display: flex; flex-direction: column; align-items: center; height: 100%; }
.bar-fill { width: 100%; background: var(--accent); border-radius: 3px 3px 0 0; min-height: 4px; transition: height 0.3s; margin-top: auto; }
.bar-label { font-size: 10px; color: var(--muted); margin-top: 4px; }
</style>

129
traffic-gen-ui/src/components/StatsMonitor.vue
View File

@ -1,129 +0,0 @@
<template>
<div class="stats-monitor">
<h3>Live Statistics</h3>
<div class="flow-selector">
<label>Flow:</label>
<select v-model="selectedFlow">
<option value="">All Flows</option>
<option v-for="f in flows" :key="f.id" :value="f.id">{{ f.name || f.dst_ip }}</option>
</select>
</div>
<div class="stats-grid">
<div class="stat-card">
<div class="stat-value tx">{{ current.tx_pps || 0 }}</div>
<div class="stat-label">TX pps</div>
</div>
<div class="stat-card">
<div class="stat-value rx">{{ current.rx_pps || 0 }}</div>
<div class="stat-label">RX pps</div>
</div>
<div class="stat-card">
<div class="stat-value tx">{{ (current.tx_mbps || 0).toFixed(2) }}</div>
<div class="stat-label">TX Mbps</div>
</div>
<div class="stat-card">
<div class="stat-value rx">{{ (current.rx_mbps || 0).toFixed(2) }}</div>
<div class="stat-label">RX Mbps</div>
</div>
<div class="stat-card">
<div class="stat-value" :class="lossClass">{{ (current.loss_pct || 0).toFixed(1) }}%</div>
<div class="stat-label">Loss</div>
</div>
<div class="stat-card">
<div class="stat-value">{{ current.avg_latency_ms ? current.avg_latency_ms.toFixed(1) : '-' }}</div>
<div class="stat-label">Avg Latency (ms)</div>
</div>
</div>
<div class="totals">
<span>TX Packets: {{ current.tx_packets || 0 }}</span>
<span>RX Packets: {{ current.rx_packets || 0 }}</span>
<span>TX Bytes: {{ formatBytes(current.tx_bytes || 0) }}</span>
<span>RX Bytes: {{ formatBytes(current.rx_bytes || 0) }}</span>
</div>
<div class="history-chart">
<div class="chart-header">TX/RX Rate History (last 60s)</div>
<div class="sparkline">
<div v-for="(s, i) in history" :key="i" class="spark-bar">
<div class="spark-tx" :style="{ height: sparkHeight(s.tx_pps) + 'px' }"></div>
<div class="spark-rx" :style="{ height: sparkHeight(s.rx_pps) + 'px' }"></div>
</div>
</div>
</div>
</div>
</template>
<script setup>
import { ref, computed, onMounted, onUnmounted } from 'vue'
import { api } from '../api.js'
const props = defineProps({ flows: Array })
const selectedFlow = ref('')
const current = ref({})
const history = ref([])
const lossClass = computed(() => {
const l = current.value.loss_pct || 0
if (l > 5) return 'loss-high'
if (l > 0) return 'loss-med'
return 'loss-ok'
})
let timer = null
async function fetchStats() {
try {
if (selectedFlow.value) {
const s = await api.flowStats(selectedFlow.value)
current.value = s
} else {
const h = await api.statsHistory()
if (h.current) current.value = h.current
if (h.history) history.value = h.history.slice(-60)
}
} catch (_) {}
}
function sparkHeight(val) {
if (!val) return 0
const max = Math.max(...history.value.map(s => Math.max(s.tx_pps || 0, s.rx_pps || 0)), 1)
return Math.max(1, (val / max) * 40)
}
function formatBytes(b) {
if (b < 1024) return b + ' B'
if (b < 1048576) return (b / 1024).toFixed(1) + ' KB'
if (b < 1073741824) return (b / 1048576).toFixed(1) + ' MB'
return (b / 1073741824).toFixed(2) + ' GB'
}
onMounted(() => { fetchStats(); timer = setInterval(fetchStats, 1000) })
onUnmounted(() => { clearInterval(timer) })
</script>
<style scoped>
h3 { font-size: 15px; margin-bottom: 12px; color: var(--accent); }
.flow-selector { margin-bottom: 12px; display: flex; align-items: center; gap: 8px; }
.flow-selector label { font-size: 12px; color: var(--muted); }
.flow-selector select { flex: 1; }
.stats-grid { display: grid; grid-template-columns: repeat(3, 1fr); gap: 10px; margin-bottom: 12px; }
.stat-card { background: var(--card-bg); border: 1px solid var(--border); border-radius: var(--radius); padding: 10px; text-align: center; }
.stat-value { font-size: 22px; font-weight: 700; font-family: monospace; }
.stat-value.tx { color: var(--accent); }
.stat-value.rx { color: var(--success); }
.stat-label { font-size: 11px; color: var(--muted); margin-top: 2px; }
.loss-ok { color: var(--success); }
.loss-med { color: var(--warning); }
.loss-high { color: var(--danger); }
.totals { display: flex; gap: 16px; font-size: 12px; color: var(--muted); margin-bottom: 16px; flex-wrap: wrap; }
.history-chart { background: var(--card-bg); border: 1px solid var(--border); border-radius: var(--radius); padding: 12px; }
.chart-header { font-size: 12px; color: var(--muted); margin-bottom: 8px; }
.sparkline { display: flex; align-items: flex-end; gap: 1px; height: 50px; }
.spark-bar { flex: 1; display: flex; flex-direction: column; justify-content: flex-end; gap: 1px; }
.spark-tx { background: var(--accent); border-radius: 1px; min-width: 2px; }
.spark-rx { background: var(--success); border-radius: 1px; min-width: 2px; }
</style>

37
traffic-gen-ui/src/components/StatusBar.vue
View File

@ -1,37 +0,0 @@
<template>
<div class="status-bar">
<div class="status-badges">
<span class="badge" :class="connected ? 'badge-ok' : 'badge-err'">
{{ connected ? 'Connected' : 'Offline' }}
</span>
<span v-if="health" class="badge badge-info">
Mode: {{ health.mode || 'sender' }}
</span>
<span v-if="health" class="badge badge-info">
Flows: {{ health.active_flows || 0 }}
</span>
<span v-if="health" class="badge badge-info">
Tests: {{ health.active_tests || 0 }}
</span>
</div>
</div>
</template>
<script setup>
import { computed } from 'vue'
const props = defineProps({ health: Object, apiError: String })
const connected = computed(() => !props.apiError && props.health)
</script>
<style scoped>
.status-bar { display: flex; align-items: center; gap: 8px; }
.status-badges { display: flex; gap: 6px; flex-wrap: wrap; }
.badge {
font-size: 11px; padding: 3px 10px; border-radius: 12px;
font-weight: 600; letter-spacing: 0.03em;
}
.badge-ok { background: rgba(72,187,120,0.15); color: var(--success); }
.badge-err { background: rgba(252,129,129,0.15); color: var(--danger); animation: pulse 1.5s infinite; }
.badge-info { background: rgba(79,156,249,0.12); color: var(--accent); }
@keyframes pulse { 0%,100% { opacity: 1; } 50% { opacity: 0.5; } }
</style>

126
traffic-gen-ui/src/components/TestBuilder.vue
View File

@ -1,126 +0,0 @@
<template>
<div class="test-builder">
<h3>RFC 2544 Test</h3>
<div class="form-row">
<label>Test Type</label>
<select v-model="form.type">
<option value="throughput">Throughput (binary search for max rate)</option>
<option value="latency">Latency (measure RTT)</option>
<option value="frame_loss">Frame Loss (loss vs rate curve)</option>
<option value="back_to_back">Back-to-Back (max burst)</option>
</select>
</div>
<div class="form-row">
<label>Base Flow</label>
<select v-model="form.flow_id">
<option value="" disabled>Select a flow...</option>
<option v-for="f in flows" :key="f.id" :value="f.id">
{{ f.name || f.dst_ip }} ({{ f.protocol }})
</option>
</select>
</div>
<div class="form-row">
<label>Frame Sizes</label>
<div class="frame-sizes">
<label v-for="s in standardSizes" :key="s" class="checkbox-label">
<input type="checkbox" :value="s" v-model="form.frame_sizes" /> {{ s }}
</label>
</div>
</div>
<div class="form-row-pair">
<div class="form-row">
<label>Trial Duration (sec)</label>
<input v-model.number="form.trial_duration" type="number" min="5" max="300" />
</div>
<div class="form-row">
<label>Max Rate (pps)</label>
<input v-model.number="form.max_rate_pps" type="number" min="10" max="100000" />
</div>
</div>
<div class="form-row">
<label>Acceptable Loss %</label>
<input v-model.number="form.acceptable_loss_pct" type="number" min="0" max="100" step="0.1" />
</div>
<button class="btn btn-accent" @click="create" :disabled="!form.flow_id">
Create & Run Test
</button>
<div class="presets-section">
<h4>Quick Presets</h4>
<div class="preset-list">
<button v-for="(p, name) in presets" :key="name" class="btn-preset" @click="loadPreset(name)">
<strong>{{ name }}</strong>
<span>{{ p.description }}</span>
</button>
</div>
</div>
</div>
</template>
<script setup>
import { reactive, ref, onMounted } from 'vue'
import { api } from '../api.js'
const props = defineProps({ flows: Array })
const emit = defineEmits(['created', 'refresh'])
const standardSizes = [64, 128, 256, 512, 1024, 1280, 1518]
const presets = ref({})
const form = reactive({
type: 'throughput',
flow_id: '',
frame_sizes: [64, 512, 1518],
trial_duration: 30,
max_rate_pps: 10000,
acceptable_loss_pct: 0.0,
})
onMounted(async () => {
try { const r = await api.presets(); presets.value = r.presets || r } catch (_) {}
})
async function create() {
try {
const test = await api.createTest({ ...form })
await api.startTest(test.id)
emit('created')
} catch (e) { alert(e.message) }
}
async function loadPreset(name) {
try {
await api.loadPreset(name, {})
emit('refresh')
} catch (e) { alert(e.message) }
}
</script>
<style scoped>
h3 { font-size: 15px; margin-bottom: 12px; color: var(--accent); }
h4 { font-size: 13px; margin: 16px 0 8px; color: var(--muted); }
.form-row { margin-bottom: 10px; }
.form-row label { display: block; font-size: 11px; color: var(--muted); margin-bottom: 3px; text-transform: uppercase; letter-spacing: 0.05em; }
.form-row input, .form-row select { width: 100%; }
.form-row-pair { display: grid; grid-template-columns: 1fr 1fr; gap: 8px; }
.frame-sizes { display: flex; flex-wrap: wrap; gap: 8px; }
.checkbox-label { font-size: 12px; display: flex; align-items: center; gap: 4px; color: var(--text); cursor: pointer; }
.btn { padding: 8px 16px; font-weight: 600; font-size: 13px; width: 100%; margin-top: 8px; }
.btn-accent { background: var(--accent); color: #fff; }
.btn-accent:disabled { opacity: 0.4; }
.preset-list { display: flex; flex-direction: column; gap: 6px; }
.btn-preset {
display: flex; flex-direction: column; align-items: flex-start;
padding: 8px 12px; background: var(--card-bg); border: 1px solid var(--border);
border-radius: var(--radius); text-align: left;
}
.btn-preset:hover { border-color: var(--accent); }
.btn-preset strong { font-size: 12px; color: var(--accent); }
.btn-preset span { font-size: 11px; color: var(--muted); }
</style>

97
traffic-gen-ui/src/components/TestRunner.vue
View File

@ -1,97 +0,0 @@
<template>
<div class="test-runner">
<div v-if="!tests.length" class="empty">No tests created yet. Use the Test Builder tab.</div>
<div v-for="t in tests" :key="t.id" class="test-card" :class="'state-' + t.state">
<div class="test-header">
<div>
<strong>{{ t.type }}</strong>
<span class="test-state">{{ t.state }}</span>
</div>
<div class="test-actions">
<button v-if="t.state === 'idle'" class="btn-sm btn-go" @click="start(t.id)">Start</button>
<button v-if="t.state === 'running'" class="btn-sm btn-stop" @click="stop(t.id)">Stop</button>
</div>
</div>
<div v-if="t.state === 'running'" class="progress-section">
<div class="progress-label">Running {{ t.type }} test...</div>
<div class="progress-bar">
<div class="progress-fill" :style="{ width: progressPct(t) + '%' }"></div>
</div>
</div>
<div v-if="t.state === 'complete' && t.results" class="results-preview">
<table>
<thead>
<tr>
<th>Frame Size</th>
<th v-if="t.type === 'throughput'">Max Rate (pps)</th>
<th v-if="t.type === 'latency'">Avg Latency (ms)</th>
<th v-if="t.type === 'frame_loss'">Loss @ Max %</th>
<th v-if="t.type === 'back_to_back'">Max Burst</th>
</tr>
</thead>
<tbody>
<tr v-for="(val, size) in t.results" :key="size">
<td>{{ size }} B</td>
<td v-if="t.type === 'throughput'">{{ val.max_rate_pps || val }}</td>
<td v-if="t.type === 'latency'">{{ val.avg_ms || val }}</td>
<td v-if="t.type === 'frame_loss'">{{ val.loss_pct || val }}%</td>
<td v-if="t.type === 'back_to_back'">{{ val.max_burst || val }}</td>
</tr>
</tbody>
</table>
</div>
<div class="test-meta">
<span>Created: {{ t.created_at || '-' }}</span>
<span v-if="t.started_at">Started: {{ t.started_at }}</span>
<span v-if="t.completed_at">Completed: {{ t.completed_at }}</span>
</div>
</div>
</div>
</template>
<script setup>
import { api } from '../api.js'
const props = defineProps({ tests: Array })
const emit = defineEmits(['refresh'])
function progressPct(t) {
if (!t.results || !t.frame_sizes) return 20
const done = Object.keys(t.results).length
return Math.min(95, (done / t.frame_sizes.length) * 100)
}
async function start(id) {
try { await api.startTest(id); emit('refresh') } catch (e) { alert(e.message) }
}
async function stop(id) {
try { await api.stopTest(id); emit('refresh') } catch (e) { alert(e.message) }
}
</script>
<style scoped>
.empty { color: var(--muted); padding: 20px; text-align: center; }
.test-card { background: var(--card-bg); border: 1px solid var(--border); border-radius: var(--radius); padding: 12px; margin-bottom: 10px; }
.test-card.state-running { border-color: var(--accent); }
.test-card.state-complete { border-color: var(--success); }
.test-header { display: flex; justify-content: space-between; align-items: center; margin-bottom: 8px; }
.test-header strong { font-size: 14px; text-transform: capitalize; }
.test-state { font-size: 11px; padding: 2px 8px; border-radius: 10px; margin-left: 8px; font-weight: 600; background: rgba(113,128,150,0.2); color: var(--muted); }
.state-running .test-state { background: rgba(79,156,249,0.15); color: var(--accent); }
.state-complete .test-state { background: rgba(72,187,120,0.15); color: var(--success); }
.test-actions { display: flex; gap: 4px; }
.btn-sm { padding: 3px 10px; font-size: 11px; font-weight: 600; border-radius: 6px; }
.btn-go { background: var(--success); color: #fff; }
.btn-stop { background: var(--warning); color: #000; }
.progress-section { margin: 8px 0; }
.progress-label { font-size: 12px; color: var(--muted); margin-bottom: 4px; }
.progress-bar { height: 6px; background: var(--border); border-radius: 3px; overflow: hidden; }
.progress-fill { height: 100%; background: var(--accent); border-radius: 3px; transition: width 0.5s; }
.results-preview table { width: 100%; border-collapse: collapse; margin-top: 8px; }
.results-preview th { font-size: 11px; color: var(--muted); text-align: left; padding: 4px 8px; border-bottom: 1px solid var(--border); }
.results-preview td { font-size: 13px; padding: 4px 8px; font-family: monospace; }
.test-meta { display: flex; gap: 12px; margin-top: 8px; font-size: 11px; color: var(--muted); }
</style>

3
traffic-gen-ui/src/main.js
View File

@ -1,3 +0,0 @@
import { createApp } from 'vue'
import App from './App.vue'
createApp(App).mount('#app')

14
traffic-gen-ui/vite.config.js
View File

@ -1,14 +0,0 @@
import { defineConfig } from 'vite'
import vue from '@vitejs/plugin-vue'
export default defineConfig({
plugins: [vue()],
server: {
proxy: {
'/api': {
target: 'http://localhost:5051',
rewrite: path => path.replace(/^\/api/, '')
}
}
}
})

8
traffic-gen/Dockerfile
View File

@ -1,8 +0,0 @@
FROM python:3.11-slim
RUN apt-get update && apt-get install -y --no-install-recommends \
tcpreplay libpcap-dev procps && rm -rf /var/lib/apt/lists/*
RUN pip install --no-cache-dir flask scapy psutil
COPY . /traffic-gen/
WORKDIR /traffic-gen
EXPOSE 5051
CMD ["python3", "server.py"]

1
traffic-gen/engine/__init__.py
View File

@ -1 +0,0 @@
# traffic-gen engine package

120
traffic-gen/engine/packet_builder.py
View File

@ -1,120 +0,0 @@
"""
Packet Builder - constructs Scapy packets from flow configuration.
Each generated packet embeds:
- Magic bytes b'TGEN' (4 bytes)
- Sequence number (4 bytes, big-endian)
- Sender timestamp in nanoseconds (8 bytes, big-endian)
- Padding to reach requested frame_size
"""
import struct
import time
from scapy.all import (
Ether, IP, UDP, TCP, ICMP, Dot1Q, Raw, conf,
)
MAGIC = b'TGEN'
HEADER_LEN = 4 + 4 + 8 # magic + seq + timestamp_ns
def _build_payload(seq: int, frame_size: int, header_overhead: int) -> Raw:
"""Build payload with magic bytes, sequence number, timestamp placeholder,
and padding to reach the desired frame_size."""
timestamp_ns = time.time_ns()
header = MAGIC + struct.pack('!I', seq) + struct.pack('!Q', timestamp_ns)
# frame_size includes Ethernet header (14) + FCS (4) in standard accounting,
# but Scapy doesn't add FCS, so we target frame_size - 4 total bytes on wire.
# header_overhead accounts for Ether + IP + L4 headers already present.
pad_len = max(0, frame_size - 4 - header_overhead - HEADER_LEN)
return Raw(load=header + (b'\x00' * pad_len))
def stamp_payload(payload_bytes: bytes, seq: int) -> bytes:
"""Re-stamp an existing payload with a new sequence number and fresh timestamp."""
timestamp_ns = time.time_ns()
return (
MAGIC
+ struct.pack('!I', seq)
+ struct.pack('!Q', timestamp_ns)
+ payload_bytes[HEADER_LEN:]
)
def parse_payload(payload_bytes: bytes):
"""Extract (seq, timestamp_ns) from a TGEN payload, or None if invalid."""
if len(payload_bytes) < HEADER_LEN:
return None
if payload_bytes[:4] != MAGIC:
return None
seq = struct.unpack('!I', payload_bytes[4:8])[0]
timestamp_ns = struct.unpack('!Q', payload_bytes[8:16])[0]
return seq, timestamp_ns
def build_packet(flow_config: dict, seq: int = 0):
"""Build a Scapy packet from a flow configuration dict.
Required keys:
dst_ip, protocol
Optional keys:
src_mac, dst_mac, src_ip, src_port, dst_port, dscp, vlan_id, frame_size
"""
protocol = flow_config.get('protocol', 'udp').lower()
frame_size = flow_config.get('frame_size', 512)
# --- Layer 2 ---
src_mac = flow_config.get('src_mac', 'auto')
dst_mac = flow_config.get('dst_mac')
ether_kwargs = {}
if src_mac and src_mac != 'auto':
ether_kwargs['src'] = src_mac
if dst_mac:
ether_kwargs['dst'] = dst_mac
pkt = Ether(**ether_kwargs)
header_overhead = 14 # Ethernet
# --- VLAN ---
vlan_id = flow_config.get('vlan_id')
if vlan_id is not None:
pkt = pkt / Dot1Q(vlan=int(vlan_id))
header_overhead += 4
# --- Layer 3 ---
ip_kwargs = {'dst': flow_config['dst_ip']}
src_ip = flow_config.get('src_ip')
if src_ip:
ip_kwargs['src'] = src_ip
dscp = flow_config.get('dscp', 0)
if dscp:
ip_kwargs['tos'] = int(dscp) << 2
pkt = pkt / IP(**ip_kwargs)
header_overhead += 20 # IP (no options)
# --- Layer 4 ---
if protocol == 'udp':
src_port = flow_config.get('src_port', 12000)
dst_port = flow_config.get('dst_port', 5001)
pkt = pkt / UDP(sport=int(src_port), dport=int(dst_port))
header_overhead += 8
elif protocol == 'tcp':
src_port = flow_config.get('src_port', 12000)
dst_port = flow_config.get('dst_port', 80)
pkt = pkt / TCP(sport=int(src_port), dport=int(dst_port), flags='S')
header_overhead += 20
elif protocol == 'icmp':
pkt = pkt / ICMP()
header_overhead += 8
else:
raise ValueError(f'Unsupported protocol: {protocol}')
# --- Payload ---
pkt = pkt / _build_payload(seq, frame_size, header_overhead)
return pkt

185
traffic-gen/engine/responder.py
View File

@ -1,185 +0,0 @@
"""
Responder - listens for TGEN-tagged packets and collects receive statistics.
Two sub-modes:
- echo: swaps src/dst MAC and IP, sends packet back with receive timestamp
- log: records rx stats only, exposed via API
"""
import logging
import struct
import threading
import time
from scapy.all import (
AsyncSniffer, Ether, IP, Raw, send, conf,
)
from engine.packet_builder import MAGIC, HEADER_LEN, parse_payload
log = logging.getLogger(__name__)
conf.verb = 0
class Responder:
"""Listens for TGEN packets on an interface and collects stats."""
def __init__(self, mode: str = 'log'):
"""
Args:
mode: 'echo' to reflect packets back, 'log' to only record stats.
"""
self._mode = mode
self._lock = threading.Lock()
self._sniffer = None
self._running = False
# Stats
self._rx_packets = 0
self._rx_bytes = 0
self._latency_samples = [] # list of (latency_ms,)
self._seen_seqs = set()
self._last_seq = -1
self._out_of_order = 0
self._duplicates = 0
# ------------------------------------------------------------------
# Control
# ------------------------------------------------------------------
def start(self, interface: str = None):
"""Start sniffing for TGEN packets."""
if self._running:
log.warning('Responder already running')
return
bpf_filter = 'ip' # broad filter; we check magic in callback
kwargs = {
'prn': self._handle_packet,
'store': False,
'filter': bpf_filter,
}
if interface:
kwargs['iface'] = interface
self._sniffer = AsyncSniffer(**kwargs)
self._sniffer.start()
self._running = True
log.info('Responder started on interface=%s mode=%s', interface or 'all', self._mode)
def stop(self):
"""Stop sniffing."""
if self._sniffer and self._running:
self._sniffer.stop()
self._running = False
log.info('Responder stopped')
def is_running(self) -> bool:
return self._running
# ------------------------------------------------------------------
# Stats
# ------------------------------------------------------------------
def get_stats(self) -> dict:
with self._lock:
latency = {}
if self._latency_samples:
vals = self._latency_samples
latency = {
'min_ms': round(min(vals), 3),
'max_ms': round(max(vals), 3),
'avg_ms': round(sum(vals) / len(vals), 3),
'jitter_ms': round(
sum(abs(vals[i] - vals[i - 1]) for i in range(1, len(vals))) / max(1, len(vals) - 1),
3
) if len(vals) > 1 else 0.0,
'samples': len(vals),
}
return {
'rx_packets': self._rx_packets,
'rx_bytes': self._rx_bytes,
'out_of_order': self._out_of_order,
'duplicates': self._duplicates,
'latency': latency,
'running': self._running,
}
def reset_stats(self):
with self._lock:
self._rx_packets = 0
self._rx_bytes = 0
self._latency_samples = []
self._seen_seqs.clear()
self._last_seq = -1
self._out_of_order = 0
self._duplicates = 0
log.info('Responder stats reset')
# ------------------------------------------------------------------
# Packet handling
# ------------------------------------------------------------------
def _handle_packet(self, pkt):
"""Process a received packet, checking for TGEN magic bytes."""
if not pkt.haslayer(Raw):
return
payload = bytes(pkt[Raw].load)
parsed = parse_payload(payload)
if parsed is None:
return
seq, sender_ts_ns = parsed
rx_time_ns = time.time_ns()
pkt_len = len(bytes(pkt))
# Compute one-way latency (only meaningful if clocks are synced)
latency_ms = (rx_time_ns - sender_ts_ns) / 1_000_000
with self._lock:
self._rx_packets += 1
self._rx_bytes += pkt_len
# Duplicate detection
if seq in self._seen_seqs:
self._duplicates += 1
else:
self._seen_seqs.add(seq)
# Keep set bounded
if len(self._seen_seqs) > 100000:
# Remove oldest entries (approximate)
to_remove = sorted(self._seen_seqs)[:50000]
self._seen_seqs -= set(to_remove)
# Out-of-order detection
if seq < self._last_seq and seq not in self._seen_seqs:
self._out_of_order += 1
self._last_seq = seq
# Record latency (only if plausible: 0 < latency < 60s)
if 0 < latency_ms < 60000:
self._latency_samples.append(latency_ms)
if len(self._latency_samples) > 10000:
self._latency_samples = self._latency_samples[-5000:]
# Echo mode: swap and send back
if self._mode == 'echo' and pkt.haslayer(Ether) and pkt.haslayer(IP):
try:
echo_pkt = pkt.copy()
# Swap MACs
echo_pkt[Ether].src, echo_pkt[Ether].dst = pkt[Ether].dst, pkt[Ether].src
# Swap IPs
echo_pkt[IP].src, echo_pkt[IP].dst = pkt[IP].dst, pkt[IP].src
# Append receive timestamp to payload
rx_ts_bytes = struct.pack('!Q', rx_time_ns)
echo_pkt[Raw].load = payload + rx_ts_bytes
# Clear checksums so Scapy recalculates
del echo_pkt[IP].chksum
if echo_pkt.haslayer('UDP'):
del echo_pkt['UDP'].chksum
elif echo_pkt.haslayer('TCP'):
del echo_pkt['TCP'].chksum
send(echo_pkt[IP], verbose=0)
except Exception as e:
log.debug('Echo send error: %s', e)

336
traffic-gen/engine/rfc2544.py
View File

@ -1,336 +0,0 @@
"""
RFC 2544 test implementations:
- ThroughputTest: binary search for max zero-loss throughput
- LatencyTest: measure latency at a given rate
- FrameLossTest: measure loss at decreasing rates
- BackToBackTest: find max burst length with zero loss
"""
import logging
import threading
import time
from typing import Dict, List, Optional
from scapy.all import send, sr, conf, IP, ICMP
from engine.packet_builder import build_packet, parse_payload
log = logging.getLogger(__name__)
conf.verb = 0
class _BaseTest:
"""Base class for RFC 2544 tests."""
def __init__(self, test_id: str, flow_config: dict, frame_sizes: List[int],
trial_duration: float = 60, max_rate_pps: int = 10000,
acceptable_loss_pct: float = 0.0):
self.test_id = test_id
self.flow_config = dict(flow_config)
self.frame_sizes = frame_sizes
self.trial_duration = trial_duration
self.max_rate_pps = max_rate_pps
self.acceptable_loss_pct = acceptable_loss_pct
self.state = 'idle' # idle -> running -> complete/error
self.results = {}
self.error = None
self.started_at = None
self.completed_at = None
self._thread: Optional[threading.Thread] = None
self._stop_event = threading.Event()
self._lock = threading.Lock()
def start(self):
if self.state == 'running':
return
self._stop_event.clear()
self.state = 'running'
self.started_at = time.strftime('%Y-%m-%dT%H:%M:%SZ', time.gmtime())
self.results = {}
self.error = None
self._thread = threading.Thread(target=self._run_safe, daemon=True,
name=f'test-{self.test_id[:8]}')
self._thread.start()
def stop(self):
self._stop_event.set()
if self._thread and self._thread.is_alive():
self._thread.join(timeout=self.trial_duration + 5)
if self.state == 'running':
self.state = 'error'
self.error = 'Cancelled by user'
def _run_safe(self):
try:
self._run()
if self.state == 'running':
self.state = 'complete'
except Exception as e:
log.error('Test %s error: %s', self.test_id[:8], e)
self.state = 'error'
self.error = str(e)
finally:
self.completed_at = time.strftime('%Y-%m-%dT%H:%M:%SZ', time.gmtime())
def _run(self):
raise NotImplementedError
def _is_stopped(self) -> bool:
return self._stop_event.is_set()
def _send_trial(self, frame_size: int, rate_pps: int, duration: float):
"""Send packets at a given rate for a duration. Returns (tx_count, rx_count, latencies)."""
flow = dict(self.flow_config)
flow['frame_size'] = frame_size
interval = 1.0 / rate_pps if rate_pps > 0 else 1.0
tx_count = 0
rx_count = 0
latencies = []
protocol = flow.get('protocol', 'udp').lower()
start = time.time()
seq = 0
while (time.time() - start) < duration and not self._is_stopped():
pkt = build_packet(flow, seq=seq)
seq += 1
if protocol == 'icmp':
# Use sr() for ICMP to get responses
answered, _ = sr(pkt[IP], timeout=1, verbose=0)
tx_count += 1
for sent_pkt, recv_pkt in answered:
rx_count += 1
rtt_ms = (recv_pkt.time - sent_pkt.sent_time) * 1000
latencies.append(rtt_ms)
else:
send(pkt[IP], verbose=0)
tx_count += 1
# Rate limiting
elapsed = time.time() - start
expected_sent = elapsed * rate_pps
if tx_count > expected_sent:
sleep_time = (tx_count - expected_sent) / rate_pps
if sleep_time > 0:
self._stop_event.wait(min(sleep_time, 0.1))
# For non-ICMP, we can't easily measure rx without a responder.
# rx_count stays 0 for UDP/TCP unless a responder is configured.
return tx_count, rx_count, latencies
def get_info(self) -> dict:
return {
'test_id': self.test_id,
'type': self.__class__.__name__,
'state': self.state,
'results': self.results,
'error': self.error,
'started_at': self.started_at,
'completed_at': self.completed_at,
}
class ThroughputTest(_BaseTest):
"""Binary search for maximum throughput with acceptable loss."""
def _run(self):
for fs in self.frame_sizes:
if self._is_stopped():
break
low = 0
high = self.max_rate_pps
best_rate = 0
convergence_threshold = max(1, int(self.max_rate_pps * 0.01))
log.info('Throughput test: frame_size=%d, searching [%d, %d] pps', fs, low, high)
while (high - low) > convergence_threshold and not self._is_stopped():
mid = (low + high) // 2
if mid == 0:
break
tx, rx, _ = self._send_trial(fs, mid, self.trial_duration)
if tx == 0:
loss_pct = 100.0
else:
# For ICMP we have rx; for UDP assume zero loss if no responder
protocol = self.flow_config.get('protocol', 'udp').lower()
if protocol == 'icmp':
loss_pct = ((tx - rx) / tx) * 100
else:
# Without responder, assume success (user should use responder for accurate test)
loss_pct = 0.0
log.info(' frame=%d rate=%d tx=%d rx=%d loss=%.2f%%',
fs, mid, tx, rx, loss_pct)
if loss_pct <= self.acceptable_loss_pct:
best_rate = mid
low = mid + 1
else:
high = mid - 1
self.results[str(fs)] = {
'max_throughput_pps': best_rate,
'frame_size': fs,
}
log.info('Throughput result: frame_size=%d -> %d pps', fs, best_rate)
class LatencyTest(_BaseTest):
"""Measure latency at a specified rate."""
def _run(self):
rate = self.flow_config.get('rate_pps', 100)
for fs in self.frame_sizes:
if self._is_stopped():
break
log.info('Latency test: frame_size=%d at %d pps for %ds', fs, rate, self.trial_duration)
_, _, latencies = self._send_trial(fs, rate, self.trial_duration)
if latencies:
avg_ms = sum(latencies) / len(latencies)
min_ms = min(latencies)
max_ms = max(latencies)
jitter_ms = (
sum(abs(latencies[i] - latencies[i - 1]) for i in range(1, len(latencies)))
/ max(1, len(latencies) - 1)
) if len(latencies) > 1 else 0.0
self.results[str(fs)] = {
'frame_size': fs,
'min_ms': round(min_ms, 3),
'avg_ms': round(avg_ms, 3),
'max_ms': round(max_ms, 3),
'jitter_ms': round(jitter_ms, 3),
'samples': len(latencies),
}
else:
self.results[str(fs)] = {
'frame_size': fs,
'min_ms': None, 'avg_ms': None, 'max_ms': None,
'jitter_ms': None, 'samples': 0,
'note': 'No responses received (use ICMP or configure responder)',
}
log.info('Latency result: frame_size=%d -> %s', fs, self.results[str(fs)])
class FrameLossTest(_BaseTest):
"""Measure frame loss at decreasing rates (100%, 90%, 80%, ...)."""
def _run(self):
for fs in self.frame_sizes:
if self._is_stopped():
break
results_for_size = []
for pct in range(100, 0, -10):
if self._is_stopped():
break
rate = int(self.max_rate_pps * pct / 100)
if rate == 0:
continue
log.info('FrameLoss test: frame_size=%d rate=%d (%d%%)', fs, rate, pct)
tx, rx, _ = self._send_trial(fs, rate, self.trial_duration)
protocol = self.flow_config.get('protocol', 'udp').lower()
if tx > 0 and protocol == 'icmp':
loss_pct = ((tx - rx) / tx) * 100
else:
loss_pct = 0.0 # Cannot measure without responder for non-ICMP
results_for_size.append({
'rate_pct': pct,
'rate_pps': rate,
'tx_packets': tx,
'rx_packets': rx,
'loss_pct': round(loss_pct, 3),
})
self.results[str(fs)] = results_for_size
class BackToBackTest(_BaseTest):
"""Find maximum burst length with zero loss."""
def _run(self):
for fs in self.frame_sizes:
if self._is_stopped():
break
low = 1
high = self.max_rate_pps # Use max_rate_pps as max burst length
best_burst = 0
convergence = max(1, high // 100)
log.info('BackToBack test: frame_size=%d searching burst [%d, %d]', fs, low, high)
while (high - low) > convergence and not self._is_stopped():
mid = (low + high) // 2
if mid == 0:
break
flow = dict(self.flow_config)
flow['frame_size'] = fs
protocol = flow.get('protocol', 'udp').lower()
# Send burst of 'mid' packets as fast as possible
tx_count = 0
rx_count = 0
for seq in range(mid):
if self._is_stopped():
break
pkt = build_packet(flow, seq=seq)
if protocol == 'icmp':
answered, _ = sr(pkt[IP], timeout=0.5, verbose=0)
tx_count += 1
rx_count += len(answered)
else:
send(pkt[IP], verbose=0)
tx_count += 1
if protocol == 'icmp' and tx_count > 0:
loss_pct = ((tx_count - rx_count) / tx_count) * 100
else:
loss_pct = 0.0 # Can't measure without responder
log.info(' burst=%d tx=%d rx=%d loss=%.2f%%', mid, tx_count, rx_count, loss_pct)
if loss_pct <= self.acceptable_loss_pct:
best_burst = mid
low = mid + 1
else:
high = mid - 1
self.results[str(fs)] = {
'frame_size': fs,
'max_burst_frames': best_burst,
}
log.info('BackToBack result: frame_size=%d -> %d frames', fs, best_burst)
# Factory function
TEST_TYPES = {
'throughput': ThroughputTest,
'latency': LatencyTest,
'frame_loss': FrameLossTest,
'back_to_back': BackToBackTest,
}
def create_test(test_id: str, test_type: str, flow_config: dict, **kwargs):
"""Create an RFC 2544 test instance by type name."""
cls = TEST_TYPES.get(test_type)
if cls is None:
raise ValueError(f'Unknown test type: {test_type}. Available: {list(TEST_TYPES)}')
return cls(test_id=test_id, flow_config=flow_config, **kwargs)

247
traffic-gen/engine/sender.py
View File

@ -1,247 +0,0 @@
"""
FlowSender - manages traffic generation with background threads per flow.
"""
import logging
import shutil
import threading
import time
import urllib.request
import json
from scapy.all import send, sendpfast, sr, conf
from engine.packet_builder import build_packet, stamp_payload
log = logging.getLogger(__name__)
# Suppress Scapy verbosity globally
conf.verb = 0
HAS_TCPREPLAY = shutil.which('tcpreplay') is not None
class FlowSender:
"""Manages sending threads for multiple flows."""
def __init__(self):
self._lock = threading.Lock()
self._flows = {} # flow_id -> flow_config dict
self._threads = {} # flow_id -> Thread
self._stop_events = {} # flow_id -> Event
self._stats = {} # flow_id -> {tx_packets, tx_bytes, ...}
# ------------------------------------------------------------------
# Flow CRUD
# ------------------------------------------------------------------
def add_flow(self, flow_id: str, flow_config: dict):
with self._lock:
self._flows[flow_id] = flow_config
self._stats[flow_id] = {
'tx_packets': 0, 'tx_bytes': 0,
'rx_packets': 0, 'rx_bytes': 0,
'latency_samples': [],
}
def get_flow(self, flow_id: str):
with self._lock:
return self._flows.get(flow_id)
def get_all_flows(self):
with self._lock:
return dict(self._flows)
def update_flow(self, flow_id: str, updates: dict):
with self._lock:
if flow_id not in self._flows:
return False
self._flows[flow_id].update(updates)
return True
def remove_flow(self, flow_id: str):
self.stop(flow_id)
with self._lock:
self._flows.pop(flow_id, None)
self._stats.pop(flow_id, None)
# ------------------------------------------------------------------
# Start / Stop
# ------------------------------------------------------------------
def start(self, flow_id: str):
with self._lock:
if flow_id not in self._flows:
raise KeyError(f'Flow {flow_id} not found')
if flow_id in self._threads and self._threads[flow_id].is_alive():
return # already running
self._flows[flow_id]['state'] = 'running'
self._stats[flow_id] = {
'tx_packets': 0, 'tx_bytes': 0,
'rx_packets': 0, 'rx_bytes': 0,
'latency_samples': [],
}
stop_event = threading.Event()
self._stop_events[flow_id] = stop_event
t = threading.Thread(
target=self._send_loop,
args=(flow_id, stop_event),
daemon=True,
name=f'sender-{flow_id[:8]}',
)
self._threads[flow_id] = t
t.start()
def stop(self, flow_id: str):
with self._lock:
ev = self._stop_events.pop(flow_id, None)
if ev:
ev.set()
t = self._threads.pop(flow_id, None)
if flow_id in self._flows:
self._flows[flow_id]['state'] = 'stopped'
if t and t.is_alive():
t.join(timeout=5)
def is_running(self, flow_id: str) -> bool:
with self._lock:
t = self._threads.get(flow_id)
return t is not None and t.is_alive()
# ------------------------------------------------------------------
# Stats
# ------------------------------------------------------------------
def get_stats(self, flow_id: str) -> dict:
with self._lock:
s = self._stats.get(flow_id, {})
return dict(s)
def get_all_stats(self) -> dict:
with self._lock:
return {fid: dict(s) for fid, s in self._stats.items()}
# ------------------------------------------------------------------
# Internal send loop
# ------------------------------------------------------------------
def _send_loop(self, flow_id: str, stop_event: threading.Event):
with self._lock:
flow = dict(self._flows[flow_id])
rate_pps = flow.get('rate_pps', 1000)
duration = flow.get('duration', 30)
protocol = flow.get('protocol', 'udp').lower()
responder_url = flow.get('responder_url')
use_icmp_sr = (protocol == 'icmp' and not responder_url)
# Build template packet
pkt_template = build_packet(flow, seq=0)
pkt_bytes_len = len(bytes(pkt_template))
# Calculate sleep interval: send in batches for efficiency
batch_size = max(1, min(rate_pps // 10, 100))
interval = batch_size / rate_pps if rate_pps > 0 else 1.0
seq = 0
start_time = time.time()
last_responder_poll = 0
log.info('Flow %s: starting send loop at %d pps for %ds', flow_id[:8], rate_pps, duration)
try:
while not stop_event.is_set():
elapsed = time.time() - start_time
if duration and elapsed >= duration:
break
if use_icmp_sr:
# ICMP mode: use sr() to measure latency from responses
pkt = build_packet(flow, seq=seq)
answered, _ = sr(pkt[pkt.firstlayer().payload.__class__],
timeout=1, verbose=0)
with self._lock:
stats = self._stats.get(flow_id)
if stats:
stats['tx_packets'] += 1
stats['tx_bytes'] += pkt_bytes_len
for sent_pkt, recv_pkt in answered:
rtt_ms = (recv_pkt.time - sent_pkt.sent_time) * 1000
stats['rx_packets'] += 1
stats['rx_bytes'] += len(bytes(recv_pkt))
stats['latency_samples'].append(rtt_ms)
# Keep only last 1000 samples
if len(stats['latency_samples']) > 1000:
stats['latency_samples'] = stats['latency_samples'][-1000:]
seq += 1
# Rate limit for ICMP
sleep_time = (1.0 / rate_pps) - (time.time() - start_time - elapsed)
if sleep_time > 0:
stop_event.wait(sleep_time)
else:
# UDP/TCP mode: send batches
packets = []
for _ in range(batch_size):
pkt = build_packet(flow, seq=seq)
packets.append(pkt)
seq += 1
try:
if HAS_TCPREPLAY and rate_pps >= 1000:
sendpfast(packets, pps=rate_pps, loop=0)
else:
for p in packets:
send(p[p.firstlayer().payload.__class__], verbose=0)
except Exception as e:
# Fallback: basic send
log.debug('Send error (falling back): %s', e)
for p in packets:
try:
send(p[p.firstlayer().payload.__class__], verbose=0)
except Exception:
pass
with self._lock:
stats = self._stats.get(flow_id)
if stats:
stats['tx_packets'] += len(packets)
stats['tx_bytes'] += pkt_bytes_len * len(packets)
# Poll responder for rx stats periodically
if responder_url and (time.time() - last_responder_poll) >= 2.0:
self._poll_responder(flow_id, responder_url)
last_responder_poll = time.time()
# Rate limit
stop_event.wait(interval)
except Exception as e:
log.error('Flow %s: send loop error: %s', flow_id[:8], e)
finally:
with self._lock:
if flow_id in self._flows:
self._flows[flow_id]['state'] = 'stopped'
# Final responder poll
if responder_url:
self._poll_responder(flow_id, responder_url)
log.info('Flow %s: send loop finished. seq=%d', flow_id[:8], seq)
def _poll_responder(self, flow_id: str, responder_url: str):
"""Poll a responder's /responder/stats endpoint for rx metrics."""
try:
url = responder_url.rstrip('/') + '/responder/stats'
req = urllib.request.Request(url, method='GET')
req.add_header('Accept', 'application/json')
with urllib.request.urlopen(req, timeout=2) as resp:
data = json.loads(resp.read().decode())
with self._lock:
stats = self._stats.get(flow_id)
if stats:
stats['rx_packets'] = data.get('rx_packets', 0)
stats['rx_bytes'] = data.get('rx_bytes', 0)
lat = data.get('latency', {})
if lat.get('avg_ms') is not None:
stats['latency_samples'].append(lat['avg_ms'])
if len(stats['latency_samples']) > 1000:
stats['latency_samples'] = stats['latency_samples'][-1000:]
except Exception as e:
log.debug('Responder poll error for flow %s: %s', flow_id[:8], e)

119
traffic-gen/engine/stats.py
View File

@ -1,119 +0,0 @@
"""
StatsCollector - ring buffer of per-flow traffic statistics.
Stores the last 300 samples (5 minutes at 1-second intervals) per flow,
with derived rates and loss calculations.
"""
import threading
import time
from collections import defaultdict, deque
class StatsCollector:
"""Collects and stores per-flow traffic statistics in a ring buffer."""
def __init__(self, max_samples: int = 300):
self._lock = threading.Lock()
self._max_samples = max_samples
# flow_id -> deque of sample dicts
self._history = defaultdict(lambda: deque(maxlen=self._max_samples))
# flow_id -> previous counters for rate calculation
self._prev = {}
def record(self, flow_id: str, tx_packets: int, tx_bytes: int,
rx_packets: int = 0, rx_bytes: int = 0,
latency: dict = None):
"""Record a stats sample for a flow.
Args:
flow_id: Flow identifier.
tx_packets: Cumulative transmitted packets.
tx_bytes: Cumulative transmitted bytes.
rx_packets: Cumulative received packets.
rx_bytes: Cumulative received bytes.
latency: Optional dict with min_ms, avg_ms, max_ms, jitter_ms.
"""
now = time.time()
with self._lock:
prev = self._prev.get(flow_id)
if prev is not None:
dt = now - prev['time']
if dt > 0:
d_tx_pkts = tx_packets - prev['tx_packets']
d_tx_bytes = tx_bytes - prev['tx_bytes']
d_rx_pkts = rx_packets - prev['rx_packets']
d_rx_bytes = rx_bytes - prev['rx_bytes']
tx_pps = d_tx_pkts / dt
rx_pps = d_rx_pkts / dt
tx_mbps = (d_tx_bytes * 8) / (dt * 1_000_000)
rx_mbps = (d_rx_bytes * 8) / (dt * 1_000_000)
else:
tx_pps = rx_pps = tx_mbps = rx_mbps = 0.0
else:
tx_pps = rx_pps = tx_mbps = rx_mbps = 0.0
# Loss calculation
loss_pct = 0.0
if tx_packets > 0 and rx_packets > 0:
loss_pct = max(0.0, ((tx_packets - rx_packets) / tx_packets) * 100)
sample = {
'timestamp': now,
'tx_packets': tx_packets,
'tx_bytes': tx_bytes,
'rx_packets': rx_packets,
'rx_bytes': rx_bytes,
'tx_pps': round(tx_pps, 2),
'rx_pps': round(rx_pps, 2),
'tx_mbps': round(tx_mbps, 4),
'rx_mbps': round(rx_mbps, 4),
'loss_pct': round(loss_pct, 3),
}
if latency:
sample['latency'] = latency
self._history[flow_id].append(sample)
self._prev[flow_id] = {
'time': now,
'tx_packets': tx_packets,
'tx_bytes': tx_bytes,
'rx_packets': rx_packets,
'rx_bytes': rx_bytes,
}
def get_history(self, flow_id: str, count: int = 60) -> list:
"""Get the last N samples for a flow."""
with self._lock:
history = self._history.get(flow_id)
if not history:
return []
samples = list(history)
return samples[-count:]
def get_latest(self, flow_id: str) -> dict:
"""Get the most recent sample for a flow."""
with self._lock:
history = self._history.get(flow_id)
if not history:
return {}
return dict(history[-1])
def get_all_latest(self) -> dict:
"""Get the most recent sample for every flow."""
with self._lock:
result = {}
for flow_id, history in self._history.items():
if history:
result[flow_id] = dict(history[-1])
return result
def remove_flow(self, flow_id: str):
"""Remove all data for a flow."""
with self._lock:
self._history.pop(flow_id, None)
self._prev.pop(flow_id, None)

75
traffic-gen/presets/__init__.py
View File

@ -1,75 +0,0 @@
"""
Built-in test presets for the traffic generator.
"""
PRESETS = {
'quick_icmp': {
'description': 'Quick ICMP ping test to verify connectivity',
'flow': {
'protocol': 'icmp',
'frame_size': 64,
'rate_pps': 10,
'duration': 10,
},
},
'udp_flood_small': {
'description': 'UDP flood with 64-byte frames at 1000 pps',
'flow': {
'protocol': 'udp',
'dst_port': 5001,
'frame_size': 64,
'rate_pps': 1000,
'duration': 30,
},
},
'udp_flood_large': {
'description': 'UDP flood with 1518-byte frames at 500 pps',
'flow': {
'protocol': 'udp',
'dst_port': 5001,
'frame_size': 1518,
'rate_pps': 500,
'duration': 30,
},
},
'rfc2544_throughput': {
'description': 'RFC 2544 throughput test across standard frame sizes',
'flow': {
'protocol': 'udp',
'dst_port': 5001,
'frame_size': 64,
'rate_pps': 10000,
'duration': 60,
},
'test': {
'type': 'throughput',
'frame_sizes': [64, 128, 256, 512, 1024, 1280, 1518],
'trial_duration': 60,
'acceptable_loss_pct': 0.0,
},
},
'rfc2544_latency': {
'description': 'RFC 2544 latency test at moderate rate',
'flow': {
'protocol': 'icmp',
'frame_size': 64,
'rate_pps': 100,
'duration': 30,
},
'test': {
'type': 'latency',
'frame_sizes': [64, 512, 1518],
'trial_duration': 30,
},
},
'tcp_session': {
'description': 'TCP SYN flood at 100 pps (for testing ACL/firewall)',
'flow': {
'protocol': 'tcp',
'dst_port': 80,
'frame_size': 64,
'rate_pps': 100,
'duration': 30,
},
},
}

3
traffic-gen/requirements.txt
View File

@ -1,3 +0,0 @@
flask
scapy
psutil

656
traffic-gen/server.py
View File

@ -1,656 +0,0 @@
#!/usr/bin/env python3
"""
Traffic Generator API Server
Scapy-based traffic generator with a Flask API. Runs in two modes
controlled by TRAFFIC_GEN_MODE env var:
- sender (default): generates traffic, runs RFC 2544 tests
- responder: listens for test packets, echoes/timestamps, reports rx stats
API endpoints:
GET /healthz - health + active flows/tests count
GET /interfaces - list available network interfaces
GET /mode - current mode
Sender mode:
GET /flows - list all flows
POST /flows - create flow
GET /flows/<id> - get flow details + stats
PUT /flows/<id> - update flow (only if idle)
DELETE /flows/<id> - delete flow
POST /flows/<id>/start - start sending
POST /flows/<id>/stop - stop sending
GET /flows/<id>/stats - real-time stats
GET /tests - list all tests
POST /tests - create RFC 2544 test
GET /tests/<id> - test details + results
POST /tests/<id>/start - start test
POST /tests/<id>/stop - abort test
GET /tests/<id>/results - exportable results
GET /presets - list presets
POST /presets/<name> - create flow/test from preset
GET /stats/history - historical stats for all flows
Responder mode:
GET /responder/stats - receive statistics
POST /responder/reset - reset receive counters
"""
import sys
import os
import json
import threading
import time
import logging
import uuid
import psutil
from flask import Flask, request, jsonify
# --- logging to stderr so stdout stays clean ---
logging.basicConfig(
stream=sys.stderr,
level=logging.INFO,
format='%(asctime)s [TGEN] %(levelname)s %(message)s',
)
log = logging.getLogger(__name__)
app = Flask(__name__)
# ---------------------------------------------------------------------------
# Global state (set in main())
# ---------------------------------------------------------------------------
MODE = 'sender' # or 'responder'
# Sender-mode globals
_sender = None # FlowSender instance
_stats_collector = None # StatsCollector instance
_flows_meta = {} # flow_id -> metadata dict (includes config)
_flows_lock = threading.Lock()
_tests = {} # test_id -> RFC 2544 test instance
_tests_lock = threading.Lock()
# Responder-mode globals
_responder = None # Responder instance
# ---------------------------------------------------------------------------
# Helper
# ---------------------------------------------------------------------------
def _now_iso():
return time.strftime('%Y-%m-%dT%H:%M:%SZ', time.gmtime())
def _flow_response(flow_id: str) -> dict:
"""Build a serializable flow dict."""
with _flows_lock:
meta = _flows_meta.get(flow_id)
if meta is None:
return None
result = dict(meta)
if _sender:
result['is_running'] = _sender.is_running(flow_id)
result['stats'] = _sender.get_stats(flow_id)
return result
# ---------------------------------------------------------------------------
# Common endpoints
# ---------------------------------------------------------------------------
@app.route('/healthz')
def health():
with _flows_lock:
active_flows = sum(
1 for fid in _flows_meta
if _sender and _sender.is_running(fid)
) if MODE == 'sender' else 0
with _tests_lock:
active_tests = sum(
1 for t in _tests.values() if t.state == 'running'
) if MODE == 'sender' else 0
resp = {
'status': 'ok',
'mode': MODE,
'active_flows': active_flows,
'active_tests': active_tests,
}
if MODE == 'responder' and _responder:
resp['responder_running'] = _responder.is_running()
return jsonify(resp)
@app.route('/interfaces')
def list_interfaces():
ifaces = []
addrs = psutil.net_if_addrs()
stats = psutil.net_if_stats()
for name, addr_list in addrs.items():
info = {'name': name, 'ip': None, 'mac': None, 'is_up': False}
for addr in addr_list:
if addr.family.name == 'AF_INET':
info['ip'] = addr.address
elif addr.family.name == 'AF_PACKET':
info['mac'] = addr.address
if name in stats:
info['is_up'] = stats[name].isup
ifaces.append(info)
return jsonify({'interfaces': ifaces})
@app.route('/mode')
def get_mode():
return jsonify({'mode': MODE})
# ---------------------------------------------------------------------------
# Sender-mode: Flow endpoints
# ---------------------------------------------------------------------------
@app.route('/flows', methods=['GET'])
def list_flows():
if MODE != 'sender':
return jsonify({'error': 'Not in sender mode'}), 400
with _flows_lock:
flow_ids = list(_flows_meta.keys())
flows = [_flow_response(fid) for fid in flow_ids]
return jsonify({'count': len(flows), 'flows': flows})
@app.route('/flows', methods=['POST'])
def create_flow():
if MODE != 'sender':
return jsonify({'error': 'Not in sender mode'}), 400
data = request.get_json(force=True)
# Validate required fields
if 'dst_ip' not in data:
return jsonify({'error': 'dst_ip is required'}), 400
if 'protocol' not in data:
return jsonify({'error': 'protocol is required'}), 400
if data['protocol'].lower() not in ('udp', 'tcp', 'icmp'):
return jsonify({'error': 'protocol must be udp, tcp, or icmp'}), 400
flow_id = str(uuid.uuid4())
flow_config = {
'id': flow_id,
'name': data.get('name', f'flow-{flow_id[:8]}'),
'src_mac': data.get('src_mac', 'auto'),
'dst_mac': data.get('dst_mac'),
'src_ip': data.get('src_ip'),
'dst_ip': data['dst_ip'],
'protocol': data['protocol'].lower(),
'src_port': data.get('src_port'),
'dst_port': data.get('dst_port'),
'frame_size': int(data.get('frame_size', 512)),
'rate_pps': int(data.get('rate_pps', 1000)),
'duration': int(data.get('duration', 30)),
'dscp': int(data.get('dscp', 0)),
'vlan_id': data.get('vlan_id'),
'state': 'idle',
'responder_url': data.get('responder_url'),
'created_at': _now_iso(),
}
with _flows_lock:
_flows_meta[flow_id] = flow_config
_sender.add_flow(flow_id, dict(flow_config))
log.info('Created flow %s (%s -> %s, %s)',
flow_id[:8], flow_config.get('src_ip', 'auto'),
flow_config['dst_ip'], flow_config['protocol'])
return jsonify(_flow_response(flow_id)), 201
@app.route('/flows/<flow_id>', methods=['GET'])
def get_flow(flow_id):
if MODE != 'sender':
return jsonify({'error': 'Not in sender mode'}), 400
result = _flow_response(flow_id)
if result is None:
return jsonify({'error': 'Flow not found'}), 404
return jsonify(result)
@app.route('/flows/<flow_id>', methods=['PUT'])
def update_flow(flow_id):
if MODE != 'sender':
return jsonify({'error': 'Not in sender mode'}), 400
with _flows_lock:
meta = _flows_meta.get(flow_id)
if meta is None:
return jsonify({'error': 'Flow not found'}), 404
if meta.get('state') == 'running':
return jsonify({'error': 'Cannot update a running flow'}), 409
data = request.get_json(force=True)
updatable = [
'name', 'src_mac', 'dst_mac', 'src_ip', 'dst_ip', 'protocol',
'src_port', 'dst_port', 'frame_size', 'rate_pps', 'duration',
'dscp', 'vlan_id', 'responder_url',
]
with _flows_lock:
for key in updatable:
if key in data:
_flows_meta[flow_id][key] = data[key]
_sender.update_flow(flow_id, dict(_flows_meta[flow_id]))
return jsonify(_flow_response(flow_id))
@app.route('/flows/<flow_id>', methods=['DELETE'])
def delete_flow(flow_id):
if MODE != 'sender':
return jsonify({'error': 'Not in sender mode'}), 400
with _flows_lock:
if flow_id not in _flows_meta:
return jsonify({'error': 'Flow not found'}), 404
_sender.remove_flow(flow_id)
if _stats_collector:
_stats_collector.remove_flow(flow_id)
with _flows_lock:
_flows_meta.pop(flow_id, None)
log.info('Deleted flow %s', flow_id[:8])
return jsonify({'deleted': flow_id})
@app.route('/flows/<flow_id>/start', methods=['POST'])
def start_flow(flow_id):
if MODE != 'sender':
return jsonify({'error': 'Not in sender mode'}), 400
with _flows_lock:
if flow_id not in _flows_meta:
return jsonify({'error': 'Flow not found'}), 404
_flows_meta[flow_id]['state'] = 'running'
try:
_sender.start(flow_id)
except KeyError:
return jsonify({'error': 'Flow not found in sender'}), 404
log.info('Started flow %s', flow_id[:8])
return jsonify(_flow_response(flow_id))
@app.route('/flows/<flow_id>/stop', methods=['POST'])
def stop_flow(flow_id):
if MODE != 'sender':
return jsonify({'error': 'Not in sender mode'}), 400
with _flows_lock:
if flow_id not in _flows_meta:
return jsonify({'error': 'Flow not found'}), 404
_flows_meta[flow_id]['state'] = 'stopped'
_sender.stop(flow_id)
log.info('Stopped flow %s', flow_id[:8])
return jsonify(_flow_response(flow_id))
@app.route('/flows/<flow_id>/stats', methods=['GET'])
def flow_stats(flow_id):
if MODE != 'sender':
return jsonify({'error': 'Not in sender mode'}), 400
with _flows_lock:
if flow_id not in _flows_meta:
return jsonify({'error': 'Flow not found'}), 404
stats = _sender.get_stats(flow_id)
latest = _stats_collector.get_latest(flow_id) if _stats_collector else {}
return jsonify({'flow_id': flow_id, 'counters': stats, 'rates': latest})
# ---------------------------------------------------------------------------
# Sender-mode: Test endpoints
# ---------------------------------------------------------------------------
@app.route('/tests', methods=['GET'])
def list_tests():
if MODE != 'sender':
return jsonify({'error': 'Not in sender mode'}), 400
with _tests_lock:
tests = [t.get_info() for t in _tests.values()]
return jsonify({'count': len(tests), 'tests': tests})
@app.route('/tests', methods=['POST'])
def create_test():
if MODE != 'sender':
return jsonify({'error': 'Not in sender mode'}), 400
from engine.rfc2544 import create_test as _create_test
data = request.get_json(force=True)
flow_id = data.get('flow_id')
test_type = data.get('type')
if not flow_id or not test_type:
return jsonify({'error': 'flow_id and type are required'}), 400
with _flows_lock:
flow_meta = _flows_meta.get(flow_id)
if flow_meta is None:
return jsonify({'error': 'Flow not found'}), 404
test_id = str(uuid.uuid4())
kwargs = {
'frame_sizes': data.get('frame_sizes', [64, 512, 1518]),
'trial_duration': float(data.get('trial_duration', 60)),
'max_rate_pps': int(data.get('max_rate_pps', flow_meta.get('rate_pps', 10000))),
'acceptable_loss_pct': float(data.get('acceptable_loss_pct', 0.0)),
}
try:
test = _create_test(test_id, test_type, dict(flow_meta), **kwargs)
except ValueError as e:
return jsonify({'error': str(e)}), 400
with _tests_lock:
_tests[test_id] = test
log.info('Created test %s (type=%s, flow=%s)', test_id[:8], test_type, flow_id[:8])
return jsonify(test.get_info()), 201
@app.route('/tests/<test_id>', methods=['GET'])
def get_test(test_id):
if MODE != 'sender':
return jsonify({'error': 'Not in sender mode'}), 400
with _tests_lock:
test = _tests.get(test_id)
if test is None:
return jsonify({'error': 'Test not found'}), 404
return jsonify(test.get_info())
@app.route('/tests/<test_id>/start', methods=['POST'])
def start_test(test_id):
if MODE != 'sender':
return jsonify({'error': 'Not in sender mode'}), 400
with _tests_lock:
test = _tests.get(test_id)
if test is None:
return jsonify({'error': 'Test not found'}), 404
test.start()
log.info('Started test %s', test_id[:8])
return jsonify(test.get_info())
@app.route('/tests/<test_id>/stop', methods=['POST'])
def stop_test(test_id):
if MODE != 'sender':
return jsonify({'error': 'Not in sender mode'}), 400
with _tests_lock:
test = _tests.get(test_id)
if test is None:
return jsonify({'error': 'Test not found'}), 404
test.stop()
log.info('Stopped test %s', test_id[:8])
return jsonify(test.get_info())
@app.route('/tests/<test_id>/results', methods=['GET'])
def test_results(test_id):
if MODE != 'sender':
return jsonify({'error': 'Not in sender mode'}), 400
with _tests_lock:
test = _tests.get(test_id)
if test is None:
return jsonify({'error': 'Test not found'}), 404
return jsonify({
'test_id': test_id,
'type': test.__class__.__name__,
'state': test.state,
'results': test.results,
'started_at': test.started_at,
'completed_at': test.completed_at,
})
# ---------------------------------------------------------------------------
# Sender-mode: Presets
# ---------------------------------------------------------------------------
@app.route('/presets', methods=['GET'])
def list_presets():
if MODE != 'sender':
return jsonify({'error': 'Not in sender mode'}), 400
from presets import PRESETS
out = {}
for name, preset in PRESETS.items():
out[name] = {
'description': preset.get('description', ''),
'has_test': 'test' in preset,
}
return jsonify({'presets': out})
@app.route('/presets/<name>', methods=['POST'])
def load_preset(name):
if MODE != 'sender':
return jsonify({'error': 'Not in sender mode'}), 400
from presets import PRESETS
from engine.rfc2544 import create_test as _create_test
if name not in PRESETS:
return jsonify({'error': f'Unknown preset: {name}. Available: {list(PRESETS)}'}), 404
preset = PRESETS[name]
data = request.get_json(silent=True) or {}
# The preset flow needs a dst_ip; caller can override
flow_data = dict(preset['flow'])
flow_data['dst_ip'] = data.get('dst_ip', flow_data.get('dst_ip', '10.0.0.1'))
if 'src_ip' in data:
flow_data['src_ip'] = data['src_ip']
if 'responder_url' in data:
flow_data['responder_url'] = data['responder_url']
# Validate protocol present
if 'protocol' not in flow_data:
flow_data['protocol'] = 'udp'
flow_id = str(uuid.uuid4())
flow_config = {
'id': flow_id,
'name': data.get('name', f'{name}-{flow_id[:8]}'),
'src_mac': flow_data.get('src_mac', 'auto'),
'dst_mac': flow_data.get('dst_mac'),
'src_ip': flow_data.get('src_ip'),
'dst_ip': flow_data['dst_ip'],
'protocol': flow_data['protocol'],
'src_port': flow_data.get('src_port'),
'dst_port': flow_data.get('dst_port'),
'frame_size': int(flow_data.get('frame_size', 512)),
'rate_pps': int(flow_data.get('rate_pps', 1000)),
'duration': int(flow_data.get('duration', 30)),
'dscp': int(flow_data.get('dscp', 0)),
'vlan_id': flow_data.get('vlan_id'),
'state': 'idle',
'responder_url': flow_data.get('responder_url'),
'created_at': _now_iso(),
}
with _flows_lock:
_flows_meta[flow_id] = flow_config
_sender.add_flow(flow_id, dict(flow_config))
result = {'flow': _flow_response(flow_id)}
# Optionally create a test
if 'test' in preset:
test_cfg = preset['test']
test_id = str(uuid.uuid4())
kwargs = {
'frame_sizes': test_cfg.get('frame_sizes', [64, 512, 1518]),
'trial_duration': float(test_cfg.get('trial_duration', 60)),
'max_rate_pps': int(test_cfg.get('max_rate_pps', flow_config.get('rate_pps', 10000))),
'acceptable_loss_pct': float(test_cfg.get('acceptable_loss_pct', 0.0)),
}
test = _create_test(test_id, test_cfg['type'], dict(flow_config), **kwargs)
with _tests_lock:
_tests[test_id] = test
result['test'] = test.get_info()
log.info('Loaded preset %s -> flow %s', name, flow_id[:8])
return jsonify(result), 201
# ---------------------------------------------------------------------------
# Sender-mode: Stats history
# ---------------------------------------------------------------------------
@app.route('/stats/history', methods=['GET'])
def stats_history():
if MODE != 'sender':
return jsonify({'error': 'Not in sender mode'}), 400
count = int(request.args.get('count', 60))
with _flows_lock:
flow_ids = list(_flows_meta.keys())
history = {}
for fid in flow_ids:
history[fid] = _stats_collector.get_history(fid, count)
return jsonify({'history': history})
# ---------------------------------------------------------------------------
# Responder-mode endpoints
# ---------------------------------------------------------------------------
@app.route('/responder/stats', methods=['GET'])
def responder_stats():
if MODE != 'responder' or _responder is None:
return jsonify({'error': 'Not in responder mode'}), 400
return jsonify(_responder.get_stats())
@app.route('/responder/reset', methods=['POST'])
def responder_reset():
if MODE != 'responder' or _responder is None:
return jsonify({'error': 'Not in responder mode'}), 400
_responder.reset_stats()
return jsonify({'status': 'reset'})
# ---------------------------------------------------------------------------
# Stats collection loop (sender mode)
# ---------------------------------------------------------------------------
def _stats_loop(stop_event: threading.Event):
"""Runs every 1s, recording per-flow stats into the StatsCollector."""
while not stop_event.is_set():
try:
all_stats = _sender.get_all_stats()
for flow_id, s in all_stats.items():
latency = None
samples = s.get('latency_samples', [])
if samples:
latency = {
'min_ms': round(min(samples), 3),
'max_ms': round(max(samples), 3),
'avg_ms': round(sum(samples) / len(samples), 3),
}
_stats_collector.record(
flow_id,
tx_packets=s.get('tx_packets', 0),
tx_bytes=s.get('tx_bytes', 0),
rx_packets=s.get('rx_packets', 0),
rx_bytes=s.get('rx_bytes', 0),
latency=latency,
)
except Exception as e:
log.debug('Stats loop error: %s', e)
stop_event.wait(1.0)
# ---------------------------------------------------------------------------
# Main
# ---------------------------------------------------------------------------
def main():
global MODE, _sender, _stats_collector, _responder
MODE = os.environ.get('TRAFFIC_GEN_MODE', 'sender').lower()
api_port = int(os.environ.get('TRAFFIC_GEN_PORT', 5051))
listen_iface = os.environ.get('TRAFFIC_GEN_INTERFACE', None)
responder_sub_mode = os.environ.get('TRAFFIC_GEN_RESPONDER_MODE', 'log') # echo or log
log.info('Traffic Generator starting in %s mode', MODE)
if MODE == 'sender':
from engine.sender import FlowSender
from engine.stats import StatsCollector
_sender = FlowSender()
_stats_collector = StatsCollector()
# Start stats collection loop
stats_stop = threading.Event()
stats_thread = threading.Thread(
target=_stats_loop, args=(stats_stop,), daemon=True, name='stats-loop'
)
stats_thread.start()
# Start Flask in background thread
flask_thread = threading.Thread(
target=lambda: app.run(host='0.0.0.0', port=api_port, threaded=True),
daemon=True,
)
flask_thread.start()
log.info('Flask API listening on port %d', api_port)
# Main thread: keep alive
try:
while True:
time.sleep(1)
except KeyboardInterrupt:
log.info('Shutting down sender...')
stats_stop.set()
elif MODE == 'responder':
from engine.responder import Responder
_responder = Responder(mode=responder_sub_mode)
_responder.start(interface=listen_iface)
# Start Flask in background thread
flask_thread = threading.Thread(
target=lambda: app.run(host='0.0.0.0', port=api_port, threaded=True),
daemon=True,
)
flask_thread.start()
log.info('Flask API listening on port %d', api_port)
# Main thread: keep alive
try:
while True:
time.sleep(1)
except KeyboardInterrupt:
log.info('Shutting down responder...')
_responder.stop()
else:
log.error('Unknown mode: %s. Use "sender" or "responder".', MODE)
sys.exit(1)
if __name__ == '__main__':
main()