obmp-docker/DB_SCHEMA.md
sam f23e222bc0 Add Phase 3: TE/SR analytics, anomaly detection, DB schema reference
- 4 new Grafana dashboards:
  - Database Schema Map (obmp-learn-07): interactive schema reference
    with live row counts, relationship diagrams, column details
  - TE & Segment Routing Analytics (obmp-learn-08): exposes BGP-LS TE/SR
    fields (bandwidth, admin groups, SRLG, SR SIDs, protection types)
  - Topology Change & Anomaly Detection (obmp-learn-09): link state
    change tracking, origin AS hijack detection, convergence timeline
  - Link Utilization & TE Thought Experiment (obmp-learn-10): capacity
    data from BGP-LS + streaming telemetry integration guide
- DB_SCHEMA.md: standalone database reference (33 tables, 11 views)
- 3 new ExaBGP scenarios: te_community_steering, origin_shift, path_diversity
- Updated DOCS.md with Phase 3 dashboards and scenarios

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 13:31:03 -07:00

14 KiB

OpenBMP Database Schema Reference

PostgreSQL database openbmp with TimescaleDB extension for time-series data.

Entity Relationship Diagram

collectors
  └── routers (collector_hash_id)
        └── bgp_peers (router_hash_id)
              ├── ip_rib (peer_hash_id) ──► base_attrs (base_attr_hash_id)
              ├── ip_rib_log (peer_hash_id)
              ├── l3vpn_rib (peer_hash_id) ──► base_attrs
              ├── ls_nodes (peer_hash_id)
              ├── ls_links (peer_hash_id) ──► ls_nodes (local/remote_node_hash_id)
              ├── ls_prefixes (peer_hash_id) ──► ls_nodes (local_node_hash_id)
              ├── peer_event_log (peer_hash_id)
              ├── stat_reports (peer_hash_id)
              └── stats_* tables (peer_hash_id)

ip_rib.prefix ◄──► global_ip_rib.prefix (aggregated view)
                     ├── rpki_origin_as ◄── rpki_validator
                     └── irr_origin_as ◄── info_route

base_attrs.origin_as ──► info_asn.asn (ASN enrichment)
routers.geo_ip_start ──► geo_ip.ip (geolocation)

BMP Core Tables

routers

BMP-monitored routers (one row per monitored device).

Column Type Description
hash_id uuid Primary key
name varchar(200) Router hostname
ip_address inet Router management IP
router_as bigint Router ASN
bgp_id inet BGP router-id
collector_hash_id uuid FK to collectors
state opstate up / down
timestamp timestamp Last update time
description varchar(255) Router description
init_data text BMP init message data
term_reason_code int BMP termination reason

collectors

BMP collector instances.

Column Type Description
hash_id uuid Primary key
admin_id varchar(64) Admin identifier
name varchar(200) Collector name
ip_address varchar(40) Collector IP
state opstate up / down
router_count smallint Number of monitored routers

bgp_peers

BGP sessions per router (one row per peer per router).

Column Type Description
hash_id uuid Primary key (composite with router_hash_id)
router_hash_id uuid FK to routers
peer_addr inet Peer IP address
peer_as bigint Peer ASN
peer_bgp_id inet Peer BGP router-id
name varchar(200) Peer name
state opstate up / down
isl3vpnpeer boolean L3VPN peer flag
isipv4 boolean IPv4 peer
isprepolicy boolean Pre-policy RIB
islocrib boolean Local RIB
local_ip inet Local IP
local_asn bigint Local ASN
local_hold_time smallint Local hold time
remote_hold_time smallint Remote hold time
sent_capabilities varchar(4096) BGP capabilities sent
recv_capabilities varchar(4096) BGP capabilities received
table_name varchar(255) VRF/table name

peer_event_log (TimescaleDB)

Historical BGP session state changes.

Column Type Description
id bigint Event sequence
peer_hash_id uuid FK to bgp_peers
state opstate up / down
timestamp timestamp Event time (partition key)
bmp_reason smallint BMP reason code
bgp_err_code smallint BGP error code
bgp_err_subcode smallint BGP error subcode
error_text varchar(255) Error description

BGP Path Attributes

base_attrs

BGP path attributes shared across routes.

Column Type Description
hash_id uuid Primary key
peer_hash_id uuid FK to bgp_peers
origin varchar(16) IGP / EGP / Incomplete
as_path bigint[] AS path array
as_path_count smallint AS path length
origin_as bigint Origin ASN
next_hop inet BGP next-hop
med bigint Multi-Exit Discriminator
local_pref bigint Local preference
community_list varchar(15)[] Standard communities
ext_community_list varchar(50)[] Extended communities (RT, etc.)
large_community_list varchar(40)[] Large communities (RFC 8092)
cluster_list varchar(40)[] Route reflector cluster list
isatomicagg boolean Atomic aggregate flag
originator_id inet RR originator ID
aggregator varchar(64) Aggregator

Indexes: GIN on as_path, community_list, ext_community_list, large_community_list


IP RIB Tables

ip_rib

Current IPv4/IPv6 unicast routing table.

Column Type Description
hash_id uuid Route hash
peer_hash_id uuid FK to bgp_peers (composite PK)
base_attr_hash_id uuid FK to base_attrs
prefix inet IP prefix
prefix_len smallint Prefix length
origin_as bigint Origin ASN
isipv4 boolean IPv4 flag
iswithdrawn boolean Withdrawn flag
labels varchar(255) MPLS labels
path_id bigint Add-Path ID
isprepolicy boolean Pre-policy flag
isadjribin boolean Adj-RIB-In flag
timestamp timestamp Last update
first_added_timestamp timestamp First seen

ip_rib_log (TimescaleDB)

Historical RIB changes — every advertisement and withdrawal.

Column Type Description
id bigint Change event ID
peer_hash_id uuid FK to bgp_peers
base_attr_hash_id uuid FK to base_attrs
prefix inet IP prefix
prefix_len smallint Prefix length
origin_as bigint Origin ASN
iswithdrawn boolean Withdrawal flag
timestamp timestamp Event time (partition key)

global_ip_rib

Aggregated prefix summary across all peers.

Column Type Description
prefix inet IP prefix (composite PK)
prefix_len smallint Prefix length
recv_origin_as bigint Received origin AS
rpki_origin_as bigint RPKI-validated origin AS
irr_origin_as bigint IRR-registered origin AS
irr_source varchar(32) IRR source (RADB, RIPE, etc.)
num_peers int Total advertising peers
iswithdrawn boolean Withdrawn flag

L3VPN Tables

l3vpn_rib

L3VPN (RFC 4364) routes with Route Distinguisher.

Column Type Description
hash_id uuid Route hash
peer_hash_id uuid FK to bgp_peers
base_attr_hash_id uuid FK to base_attrs
rd varchar(128) Route Distinguisher
prefix inet VPN prefix
prefix_len smallint Prefix length
origin_as bigint Origin ASN
labels varchar(255) MPLS VPN labels
ext_community_list varchar(50)[] Route Targets
path_id bigint Add-Path ID
iswithdrawn boolean Withdrawn flag

l3vpn_rib_log (TimescaleDB)

Historical L3VPN route changes.


ls_nodes

IS-IS / OSPF node information from BGP-LS.

Column Type Description
hash_id uuid Node hash
peer_hash_id uuid FK to bgp_peers (composite PK)
base_attr_hash_id uuid FK to base_attrs
asn bigint Node ASN
bgp_ls_id bigint BGP-LS Identifier
igp_router_id varchar(46) IGP Router ID
router_id varchar(46) BGP Router ID
protocol ls_proto IS-IS_L1, IS-IS_L2, OSPFv2, OSPFv3
isis_area_id varchar(46) IS-IS area
ospf_area_id varchar(16) OSPF area
name varchar(255) Node hostname
flags varchar(20) Node flags
mt_ids varchar(128) Multi-Topology IDs
sr_capabilities varchar(255) SR Global Block (SRGB) ranges
iswithdrawn boolean Withdrawn flag

IS-IS / OSPF links with full TE and SR attributes.

Column Type Description
hash_id uuid Link hash
peer_hash_id uuid FK to bgp_peers (composite PK)
local_node_hash_id uuid FK to ls_nodes (local end)
remote_node_hash_id uuid FK to ls_nodes (remote end)
local_router_id varchar(46) Local BGP Router ID
remote_router_id varchar(46) Remote BGP Router ID
local_igp_router_id varchar(46) Local IGP Router ID
remote_igp_router_id varchar(46) Remote IGP Router ID
interface_addr inet Local interface IP
neighbor_addr inet Remote interface IP
igp_metric bigint IGP metric
protocol ls_proto IGP protocol
mt_id int Multi-Topology ID
local_link_id bigint Local link identifier
remote_link_id bigint Remote link identifier
name varchar(255) Link name
admin_group bigint TE admin group / link color bitmap
max_link_bw bigint Maximum link bandwidth (bytes/sec)
max_resv_bw bigint Maximum reservable bandwidth
unreserved_bw varchar(128) Unreserved BW per priority (8 values)
te_def_metric bigint TE default metric (for CSPF)
protection_type varchar(60) Link protection (FRR type)
mpls_proto_mask ls_mpls_proto_mask MPLS protocol support flags
srlg varchar(128) Shared Risk Link Group
peer_node_sid varchar(128) SR Peer Node SID (EPE, RFC 9086)
sr_adjacency_sids varchar(255) SR Adjacency SIDs
iswithdrawn boolean Withdrawn flag

Bold = TE/SR fields available via BGP-LS but not used by default dashboards.

ls_prefixes

IS-IS / OSPF prefix information.

Column Type Description
hash_id uuid Prefix hash
peer_hash_id uuid FK to bgp_peers (composite PK)
local_node_hash_id uuid FK to ls_nodes
prefix inet Advertised prefix
prefix_len smallint Prefix length
protocol ls_proto IGP protocol
metric bigint Prefix metric
mt_id int Multi-Topology ID
ospf_route_type ospf_route_type Intra/Inter/Ext-1/Ext-2/NSSA
igp_flags varchar(20) IGP flags
route_tag bigint Route tag
sr_prefix_sids varchar(255) SR Prefix SIDs (node SIDs)
iswithdrawn boolean Withdrawn flag

Historical link-state changes. Same columns as parent tables plus id (bigint) and timestamp as partition key.


Statistics Tables (TimescaleDB)

Table Purpose Key Columns
stat_reports BMP stat messages per peer prefixes_rejected, known_dup_prefixes, num_routes_adj_rib_in, num_routes_local_rib
stats_chg_byprefix Per-prefix update/withdrawal counts interval_time, prefix, updates, withdraws
stats_chg_byasn Per-ASN update/withdrawal counts interval_time, origin_as, updates, withdraws
stats_chg_bypeer Per-peer update/withdrawal counts interval_time, updates, withdraws
stats_peer_rib Per-peer RIB size over time interval_time, v4_prefixes, v6_prefixes
stats_peer_update_counts Update rate statistics interval_time, advertise_avg/min/max, withdraw_avg/min/max
stats_ip_origins Per-ASN IP prefix counts interval_time, asn, v4_prefixes, v6_prefixes, v4_with_rpki, v4_with_irr
stats_l3vpn_chg_byprefix L3VPN per-prefix stats interval_time, rd, prefix, updates, withdraws
stats_l3vpn_chg_bypeer L3VPN per-peer stats interval_time, updates, withdraws
stats_l3vpn_chg_byrd L3VPN per-RD stats interval_time, rd, updates, withdraws

Reference & Enrichment Tables

Table Purpose Key Columns
rpki_validator RPKI ROAs prefix, prefix_len, prefix_len_max, origin_as
info_asn ASN WHOIS/IRR data asn, as_name, org_name, country, source
info_route Route IRR data prefix, origin_as, descr, source
geo_ip IP geolocation (DB-IP) ip, country, city, latitude, longitude, isp_name
pdb_exchange_peers PeeringDB IXP peering ix_name, peer_name, peer_asn, speed, peer_ipv4/ipv6

Views

View Joins Purpose
v_peers bgp_peers + routers + info_asn Complete peer info with router name and ASN details
v_ip_routes ip_rib + bgp_peers + base_attrs + routers Full route detail with path attributes
v_ip_routes_geo v_ip_routes + geo_ip Routes with geolocation
v_ip_routes_history ip_rib_log + base_attrs + bgp_peers + routers Historical route changes with attributes
v_l3vpn_routes l3vpn_rib + bgp_peers + base_attrs + routers L3VPN routes with path attributes
v_l3vpn_routes_history l3vpn_rib_log + base_attrs + bgp_peers + routers Historical L3VPN changes
v_ls_nodes ls_nodes + base_attrs + bgp_peers + routers Link-state nodes with peer/router info
v_ls_links ls_links + ls_nodes(x2) + routers Links with local/remote node names + all TE/SR fields
v_ls_prefixes ls_prefixes + ls_nodes + routers LS prefixes with originating node info

Custom Enum Types

Type Values
opstate up, down
ls_proto IS-IS_L1, IS-IS_L2, OSPFv2, OSPFv3, Direct, Static
ospf_route_type Intra, Inter, Ext-1, Ext-2, NSSA-1, NSSA-2
ls_mpls_proto_mask MPLS protocol bitmask
user_role admin, oper

Key Query Patterns

Get all active routes with full attributes

SELECT r.prefix, r.prefix_len, ba.origin_as, ba.as_path,
       ba.med, ba.local_pref, ba.community_list, ba.next_hop
FROM ip_rib r
JOIN base_attrs ba ON ba.hash_id = r.base_attr_hash_id
WHERE r.iswithdrawn = false AND r.isipv4 = true
SELECT local_router_name, remote_router_name,
       igp_metric, te_def_metric, max_link_bw, admin_group, srlg,
       sr_adjacency_sids
FROM v_ls_links
WHERE peer_hash_id = '<peer_hash>' AND iswithdrawn = false

Time-series RIB changes

SELECT date_trunc('minute', timestamp) as time,
       SUM(CASE WHEN iswithdrawn = false THEN 1 ELSE 0 END) as ads,
       SUM(CASE WHEN iswithdrawn = true THEN 1 ELSE 0 END) as withdrawals
FROM ip_rib_log
WHERE timestamp > NOW() - INTERVAL '24 hours'
GROUP BY 1 ORDER BY 1

RPKI validation status

SELECT CASE
         WHEN rv.origin_as IS NOT NULL AND rv.origin_as = r.origin_as THEN 'Valid'
         WHEN rv.origin_as IS NOT NULL THEN 'Invalid'
         ELSE 'NotFound'
       END as status,
       COUNT(*)
FROM ip_rib r
LEFT JOIN rpki_validator rv ON rv.prefix = r.prefix AND rv.prefix_len = r.prefix_len
WHERE r.iswithdrawn = false
GROUP BY 1