sam/obmp-docker
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
obmp-docker/DB_SCHEMA.md
sam f23e222bc0 Add Phase 3: TE/SR analytics, anomaly detection, DB schema reference 
- 4 new Grafana dashboards:
  - Database Schema Map (obmp-learn-07): interactive schema reference
    with live row counts, relationship diagrams, column details
  - TE & Segment Routing Analytics (obmp-learn-08): exposes BGP-LS TE/SR
    fields (bandwidth, admin groups, SRLG, SR SIDs, protection types)
  - Topology Change & Anomaly Detection (obmp-learn-09): link state
    change tracking, origin AS hijack detection, convergence timeline
  - Link Utilization & TE Thought Experiment (obmp-learn-10): capacity
    data from BGP-LS + streaming telemetry integration guide
- DB_SCHEMA.md: standalone database reference (33 tables, 11 views)
- 3 new ExaBGP scenarios: te_community_steering, origin_shift, path_diversity
- Updated DOCS.md with Phase 3 dashboards and scenarios

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 13:31:03 -07:00

388 lines
14 KiB
Markdown
Raw Blame History

# OpenBMP Database Schema Reference
PostgreSQL database `openbmp` with TimescaleDB extension for time-series data.
## Entity Relationship Diagram
```
collectors
└── routers (collector_hash_id)
└── bgp_peers (router_hash_id)
├── ip_rib (peer_hash_id) ──► base_attrs (base_attr_hash_id)
├── ip_rib_log (peer_hash_id)
├── l3vpn_rib (peer_hash_id) ──► base_attrs
├── ls_nodes (peer_hash_id)
├── ls_links (peer_hash_id) ──► ls_nodes (local/remote_node_hash_id)
├── ls_prefixes (peer_hash_id) ──► ls_nodes (local_node_hash_id)
├── peer_event_log (peer_hash_id)
├── stat_reports (peer_hash_id)
└── stats_* tables (peer_hash_id)
ip_rib.prefix ◄──► global_ip_rib.prefix (aggregated view)
├── rpki_origin_as ◄── rpki_validator
└── irr_origin_as ◄── info_route
base_attrs.origin_as ──► info_asn.asn (ASN enrichment)
routers.geo_ip_start ──► geo_ip.ip (geolocation)
```
---
## BMP Core Tables
### routers
BMP-monitored routers (one row per monitored device).
| Column | Type | Description |
|--------|------|-------------|
| hash_id | uuid | Primary key |
| name | varchar(200) | Router hostname |
| ip_address | inet | Router management IP |
| router_as | bigint | Router ASN |
| bgp_id | inet | BGP router-id |
| collector_hash_id | uuid | FK to collectors |
| state | opstate | up / down |
| timestamp | timestamp | Last update time |
| description | varchar(255) | Router description |
| init_data | text | BMP init message data |
| term_reason_code | int | BMP termination reason |
### collectors
BMP collector instances.
| Column | Type | Description |
|--------|------|-------------|
| hash_id | uuid | Primary key |
| admin_id | varchar(64) | Admin identifier |
| name | varchar(200) | Collector name |
| ip_address | varchar(40) | Collector IP |
| state | opstate | up / down |
| router_count | smallint | Number of monitored routers |
### bgp_peers
BGP sessions per router (one row per peer per router).
| Column | Type | Description |
|--------|------|-------------|
| hash_id | uuid | Primary key (composite with router_hash_id) |
| router_hash_id | uuid | FK to routers |
| peer_addr | inet | Peer IP address |
| peer_as | bigint | Peer ASN |
| peer_bgp_id | inet | Peer BGP router-id |
| name | varchar(200) | Peer name |
| state | opstate | up / down |
| isl3vpnpeer | boolean | L3VPN peer flag |
| isipv4 | boolean | IPv4 peer |
| isprepolicy | boolean | Pre-policy RIB |
| islocrib | boolean | Local RIB |
| local_ip | inet | Local IP |
| local_asn | bigint | Local ASN |
| local_hold_time | smallint | Local hold time |
| remote_hold_time | smallint | Remote hold time |
| sent_capabilities | varchar(4096) | BGP capabilities sent |
| recv_capabilities | varchar(4096) | BGP capabilities received |
| table_name | varchar(255) | VRF/table name |
### peer_event_log (TimescaleDB)
Historical BGP session state changes.
| Column | Type | Description |
|--------|------|-------------|
| id | bigint | Event sequence |
| peer_hash_id | uuid | FK to bgp_peers |
| state | opstate | up / down |
| timestamp | timestamp | Event time (partition key) |
| bmp_reason | smallint | BMP reason code |
| bgp_err_code | smallint | BGP error code |
| bgp_err_subcode | smallint | BGP error subcode |
| error_text | varchar(255) | Error description |
---
## BGP Path Attributes
### base_attrs
BGP path attributes shared across routes.
| Column | Type | Description |
|--------|------|-------------|
| hash_id | uuid | Primary key |
| peer_hash_id | uuid | FK to bgp_peers |
| origin | varchar(16) | IGP / EGP / Incomplete |
| as_path | bigint[] | AS path array |
| as_path_count | smallint | AS path length |
| origin_as | bigint | Origin ASN |
| next_hop | inet | BGP next-hop |
| med | bigint | Multi-Exit Discriminator |
| local_pref | bigint | Local preference |
| community_list | varchar(15)[] | Standard communities |
| ext_community_list | varchar(50)[] | Extended communities (RT, etc.) |
| large_community_list | varchar(40)[] | Large communities (RFC 8092) |
| cluster_list | varchar(40)[] | Route reflector cluster list |
| isatomicagg | boolean | Atomic aggregate flag |
| originator_id | inet | RR originator ID |
| aggregator | varchar(64) | Aggregator |
**Indexes**: GIN on as_path, community_list, ext_community_list, large_community_list
---
## IP RIB Tables
### ip_rib
Current IPv4/IPv6 unicast routing table.
| Column | Type | Description |
|--------|------|-------------|
| hash_id | uuid | Route hash |
| peer_hash_id | uuid | FK to bgp_peers (composite PK) |
| base_attr_hash_id | uuid | FK to base_attrs |
| prefix | inet | IP prefix |
| prefix_len | smallint | Prefix length |
| origin_as | bigint | Origin ASN |
| isipv4 | boolean | IPv4 flag |
| iswithdrawn | boolean | Withdrawn flag |
| labels | varchar(255) | MPLS labels |
| path_id | bigint | Add-Path ID |
| isprepolicy | boolean | Pre-policy flag |
| isadjribin | boolean | Adj-RIB-In flag |
| timestamp | timestamp | Last update |
| first_added_timestamp | timestamp | First seen |
### ip_rib_log (TimescaleDB)
Historical RIB changes — every advertisement and withdrawal.
| Column | Type | Description |
|--------|------|-------------|
| id | bigint | Change event ID |
| peer_hash_id | uuid | FK to bgp_peers |
| base_attr_hash_id | uuid | FK to base_attrs |
| prefix | inet | IP prefix |
| prefix_len | smallint | Prefix length |
| origin_as | bigint | Origin ASN |
| iswithdrawn | boolean | Withdrawal flag |
| timestamp | timestamp | Event time (partition key) |
### global_ip_rib
Aggregated prefix summary across all peers.
| Column | Type | Description |
|--------|------|-------------|
| prefix | inet | IP prefix (composite PK) |
| prefix_len | smallint | Prefix length |
| recv_origin_as | bigint | Received origin AS |
| rpki_origin_as | bigint | RPKI-validated origin AS |
| irr_origin_as | bigint | IRR-registered origin AS |
| irr_source | varchar(32) | IRR source (RADB, RIPE, etc.) |
| num_peers | int | Total advertising peers |
| iswithdrawn | boolean | Withdrawn flag |
---
## L3VPN Tables
### l3vpn_rib
L3VPN (RFC 4364) routes with Route Distinguisher.
| Column | Type | Description |
|--------|------|-------------|
| hash_id | uuid | Route hash |
| peer_hash_id | uuid | FK to bgp_peers |
| base_attr_hash_id | uuid | FK to base_attrs |
| rd | varchar(128) | Route Distinguisher |
| prefix | inet | VPN prefix |
| prefix_len | smallint | Prefix length |
| origin_as | bigint | Origin ASN |
| labels | varchar(255) | MPLS VPN labels |
| ext_community_list | varchar(50)[] | Route Targets |
| path_id | bigint | Add-Path ID |
| iswithdrawn | boolean | Withdrawn flag |
### l3vpn_rib_log (TimescaleDB)
Historical L3VPN route changes.
---
## Link-State Tables (BGP-LS / RFC 7752)
### ls_nodes
IS-IS / OSPF node information from BGP-LS.
| Column | Type | Description |
|--------|------|-------------|
| hash_id | uuid | Node hash |
| peer_hash_id | uuid | FK to bgp_peers (composite PK) |
| base_attr_hash_id | uuid | FK to base_attrs |
| asn | bigint | Node ASN |
| bgp_ls_id | bigint | BGP-LS Identifier |
| igp_router_id | varchar(46) | IGP Router ID |
| router_id | varchar(46) | BGP Router ID |
| protocol | ls_proto | IS-IS_L1, IS-IS_L2, OSPFv2, OSPFv3 |
| isis_area_id | varchar(46) | IS-IS area |
| ospf_area_id | varchar(16) | OSPF area |
| name | varchar(255) | Node hostname |
| flags | varchar(20) | Node flags |
| mt_ids | varchar(128) | Multi-Topology IDs |
| **sr_capabilities** | **varchar(255)** | **SR Global Block (SRGB) ranges** |
| iswithdrawn | boolean | Withdrawn flag |
### ls_links
IS-IS / OSPF links with full TE and SR attributes.
| Column | Type | Description |
|--------|------|-------------|
| hash_id | uuid | Link hash |
| peer_hash_id | uuid | FK to bgp_peers (composite PK) |
| local_node_hash_id | uuid | FK to ls_nodes (local end) |
| remote_node_hash_id | uuid | FK to ls_nodes (remote end) |
| local_router_id | varchar(46) | Local BGP Router ID |
| remote_router_id | varchar(46) | Remote BGP Router ID |
| local_igp_router_id | varchar(46) | Local IGP Router ID |
| remote_igp_router_id | varchar(46) | Remote IGP Router ID |
| interface_addr | inet | Local interface IP |
| neighbor_addr | inet | Remote interface IP |
| igp_metric | bigint | IGP metric |
| protocol | ls_proto | IGP protocol |
| mt_id | int | Multi-Topology ID |
| local_link_id | bigint | Local link identifier |
| remote_link_id | bigint | Remote link identifier |
| name | varchar(255) | Link name |
| **admin_group** | **bigint** | **TE admin group / link color bitmap** |
| **max_link_bw** | **bigint** | **Maximum link bandwidth (bytes/sec)** |
| **max_resv_bw** | **bigint** | **Maximum reservable bandwidth** |
| **unreserved_bw** | **varchar(128)** | **Unreserved BW per priority (8 values)** |
| **te_def_metric** | **bigint** | **TE default metric (for CSPF)** |
| **protection_type** | **varchar(60)** | **Link protection (FRR type)** |
| **mpls_proto_mask** | **ls_mpls_proto_mask** | **MPLS protocol support flags** |
| **srlg** | **varchar(128)** | **Shared Risk Link Group** |
| **peer_node_sid** | **varchar(128)** | **SR Peer Node SID (EPE, RFC 9086)** |
| **sr_adjacency_sids** | **varchar(255)** | **SR Adjacency SIDs** |
| iswithdrawn | boolean | Withdrawn flag |
**Bold** = TE/SR fields available via BGP-LS but not used by default dashboards.
### ls_prefixes
IS-IS / OSPF prefix information.
| Column | Type | Description |
|--------|------|-------------|
| hash_id | uuid | Prefix hash |
| peer_hash_id | uuid | FK to bgp_peers (composite PK) |
| local_node_hash_id | uuid | FK to ls_nodes |
| prefix | inet | Advertised prefix |
| prefix_len | smallint | Prefix length |
| protocol | ls_proto | IGP protocol |
| metric | bigint | Prefix metric |
| mt_id | int | Multi-Topology ID |
| ospf_route_type | ospf_route_type | Intra/Inter/Ext-1/Ext-2/NSSA |
| igp_flags | varchar(20) | IGP flags |
| route_tag | bigint | Route tag |
| **sr_prefix_sids** | **varchar(255)** | **SR Prefix SIDs (node SIDs)** |
| iswithdrawn | boolean | Withdrawn flag |
### ls_nodes_log, ls_links_log, ls_prefixes_log (TimescaleDB)
Historical link-state changes. Same columns as parent tables plus `id` (bigint) and timestamp as partition key.
---
## Statistics Tables (TimescaleDB)
| Table | Purpose | Key Columns |
|-------|---------|-------------|
| **stat_reports** | BMP stat messages per peer | prefixes_rejected, known_dup_prefixes, num_routes_adj_rib_in, num_routes_local_rib |
| **stats_chg_byprefix** | Per-prefix update/withdrawal counts | interval_time, prefix, updates, withdraws |
| **stats_chg_byasn** | Per-ASN update/withdrawal counts | interval_time, origin_as, updates, withdraws |
| **stats_chg_bypeer** | Per-peer update/withdrawal counts | interval_time, updates, withdraws |
| **stats_peer_rib** | Per-peer RIB size over time | interval_time, v4_prefixes, v6_prefixes |
| **stats_peer_update_counts** | Update rate statistics | interval_time, advertise_avg/min/max, withdraw_avg/min/max |
| **stats_ip_origins** | Per-ASN IP prefix counts | interval_time, asn, v4_prefixes, v6_prefixes, v4_with_rpki, v4_with_irr |
| **stats_l3vpn_chg_byprefix** | L3VPN per-prefix stats | interval_time, rd, prefix, updates, withdraws |
| **stats_l3vpn_chg_bypeer** | L3VPN per-peer stats | interval_time, updates, withdraws |
| **stats_l3vpn_chg_byrd** | L3VPN per-RD stats | interval_time, rd, updates, withdraws |
---
## Reference & Enrichment Tables
| Table | Purpose | Key Columns |
|-------|---------|-------------|
| **rpki_validator** | RPKI ROAs | prefix, prefix_len, prefix_len_max, origin_as |
| **info_asn** | ASN WHOIS/IRR data | asn, as_name, org_name, country, source |
| **info_route** | Route IRR data | prefix, origin_as, descr, source |
| **geo_ip** | IP geolocation (DB-IP) | ip, country, city, latitude, longitude, isp_name |
| **pdb_exchange_peers** | PeeringDB IXP peering | ix_name, peer_name, peer_asn, speed, peer_ipv4/ipv6 |
---
## Views
| View | Joins | Purpose |
|------|-------|---------|
| **v_peers** | bgp_peers + routers + info_asn | Complete peer info with router name and ASN details |
| **v_ip_routes** | ip_rib + bgp_peers + base_attrs + routers | Full route detail with path attributes |
| **v_ip_routes_geo** | v_ip_routes + geo_ip | Routes with geolocation |
| **v_ip_routes_history** | ip_rib_log + base_attrs + bgp_peers + routers | Historical route changes with attributes |
| **v_l3vpn_routes** | l3vpn_rib + bgp_peers + base_attrs + routers | L3VPN routes with path attributes |
| **v_l3vpn_routes_history** | l3vpn_rib_log + base_attrs + bgp_peers + routers | Historical L3VPN changes |
| **v_ls_nodes** | ls_nodes + base_attrs + bgp_peers + routers | Link-state nodes with peer/router info |
| **v_ls_links** | ls_links + ls_nodes(x2) + routers | Links with local/remote node names + all TE/SR fields |
| **v_ls_prefixes** | ls_prefixes + ls_nodes + routers | LS prefixes with originating node info |
---
## Custom Enum Types
| Type | Values |
|------|--------|
| **opstate** | up, down |
| **ls_proto** | IS-IS_L1, IS-IS_L2, OSPFv2, OSPFv3, Direct, Static |
| **ospf_route_type** | Intra, Inter, Ext-1, Ext-2, NSSA-1, NSSA-2 |
| **ls_mpls_proto_mask** | MPLS protocol bitmask |
| **user_role** | admin, oper |
---
## Key Query Patterns
### Get all active routes with full attributes
```sql
SELECT r.prefix, r.prefix_len, ba.origin_as, ba.as_path,
ba.med, ba.local_pref, ba.community_list, ba.next_hop
FROM ip_rib r
JOIN base_attrs ba ON ba.hash_id = r.base_attr_hash_id
WHERE r.iswithdrawn = false AND r.isipv4 = true
```
### Get link-state topology with TE attributes
```sql
SELECT local_router_name, remote_router_name,
igp_metric, te_def_metric, max_link_bw, admin_group, srlg,
sr_adjacency_sids
FROM v_ls_links
WHERE peer_hash_id = '<peer_hash>' AND iswithdrawn = false
```
### Time-series RIB changes
```sql
SELECT date_trunc('minute', timestamp) as time,
SUM(CASE WHEN iswithdrawn = false THEN 1 ELSE 0 END) as ads,
SUM(CASE WHEN iswithdrawn = true THEN 1 ELSE 0 END) as withdrawals
FROM ip_rib_log
WHERE timestamp > NOW() - INTERVAL '24 hours'
GROUP BY 1 ORDER BY 1
```
### RPKI validation status
```sql
SELECT CASE
WHEN rv.origin_as IS NOT NULL AND rv.origin_as = r.origin_as THEN 'Valid'
WHEN rv.origin_as IS NOT NULL THEN 'Invalid'
ELSE 'NotFound'
END as status,
COUNT(*)
FROM ip_rib r
LEFT JOIN rpki_validator rv ON rv.prefix = r.prefix AND rv.prefix_len = r.prefix_len
WHERE r.iswithdrawn = false
GROUP BY 1
```
Reference in New Issue View Git Blame Copy Permalink