11 Commits

Author SHA1 Message Date
sam
a6180196e9 Fix MTU=0 validation and create_message_chunks import errors
- Send mtu=None instead of mtu=0 (NetBox requires MTU >= 1)
- Remove create_message_chunks usage (not in installed SDK version)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-28 16:58:18 -07:00
sam
37b92c166a Fix create_message_chunks import error across all collectors
The installed Diode SDK version does not export create_message_chunks.
Replace chunked ingestion with direct client.ingest() calls.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-28 16:49:25 -07:00
sam
427dbf456d Fix network collector float-to-int cast for interface speed and MTU
NAPALM can return speed/mtu as floats, but the Diode SDK expects integers.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-28 16:30:36 -07:00
sam
40d3814316 Fix UniFi collector Prefix site parameter for Diode SDK
Prefix uses scope_site instead of site in the Diode SDK ingester.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-28 16:26:58 -07:00
sam
5748bad765 Add PBS collector, multi-host PVE support, and collector fixes
- proxmox_collector: support numbered PVE_HOST_1/2/3 env vars with
  backward compat for legacy single PVE_HOST; fix MTU string-to-int cast
- pbs_collector: new collector for Proxmox Backup Server — discovers
  devices, interfaces, IPs, and datastores (as Services) via PBS API
- vmware_collector: fix mac_address → primary_mac_address for Diode SDK
- network_collector: add Netmiko SSH fallback for Brocade/NOS devices,
  add Brocade ICX interface type patterns
- unifi_collector: new collector for UniFi UDM-SE/switches/APs
- ENV_REFERENCE.md: document all collector env vars and setup steps
- .gitignore: exclude collectors/inventory.yaml (contains credentials)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-28 16:10:12 -07:00
sam
b4fcdfa277 Add network, CML, Zabbix, Observium, VMware, and Docker collectors
Six new collectors for ingesting infrastructure data into NetBox via
the Diode SDK pipeline:

- network_collector: Cisco/Brocade devices via NAPALM + pyATS/Genie
  with LLDP/CDP cable discovery, VLANs, VRFs, prefixes, device configs,
  inventory items, and BGP push to netbox-bgp plugin API
- cml_collector: Cisco Modeling Labs topology sync (nodes, links, configs)
- zabbix_collector: Brownfield import from Zabbix API with cross-ref
  custom fields
- observium_collector: Device/port/IP import from Observium REST API
- vmware_collector: vCenter/ESXi hosts, VMs, interfaces, disks, IPs
- docker_collector: Container discovery via Docker API (tested: 21
  containers found on local host)

Also adds inventory.yaml.example template for network device credentials.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-28 03:17:40 -07:00
sam
a5b37c0dd5 Add Proxmox VE collector for NetBox ingestion via Diode SDK
Single-file collector that discovers PVE host infrastructure (nodes,
LXC containers, QEMU VMs, interfaces, IPs, disks) and ingests it
into NetBox through the Diode pipeline. Supports DHCP IP discovery
via PVE runtime interfaces API and two-pass convergence for
primary_ip4 assignment.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-28 02:37:02 -07:00
sam
fbde598be3 Add ingestion test and fix OAuth2 scopes and bootstrap logic
- Add tests/test_ingestion.py for end-to-end Diode pipeline verification
- Fix OAuth2 client scopes: reconciler uses diode:reconcile, netbox-to-diode
  needs diode:read diode:write netbox:read netbox:write
- Rewrite bootstrap-clients.sh with upsert behavior (delete+recreate) so
  scope and secret changes are applied on restart
- Rewrite nginx.conf in setup.sh to match upstream auth_request architecture
- Update .claude/settings.json with expanded tool permissions

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-28 01:55:37 -07:00
sam
2f459e6f4a Fix Diode stack configuration to match upstream architecture
Major changes to align with the official netboxlabs/diode docker-compose:

docker-compose.yml:
- Upgrade Hydra from v2.2 to v25.4.0 (latest stable)
- Fix env var names: drop DIODE_ prefix on ingester/reconciler/auth
  (DIODE_REDIS_HOST -> REDIS_HOST, DIODE_GRPC_PORT -> removed, etc.)
- Remove AUTH_GRPC_TARGET from ingester — auth is handled by nginx
  via HTTP subrequests to diode-auth, not by the ingester directly
- Point DIODE_AUTH_TOKEN_URL to diode-auth:8080/token (not Hydra)
- Add Postgres connection vars to reconciler (MIGRATION_ENABLED, etc.)
- Mount nginx.conf as /etc/nginx/conf.d/default.conf (not nginx.conf)
- Use netboxlabs/diode-auth image for bootstrap (has hydra CLI + jq)
- Add Hydra JWT strategy config (STRATEGIES_ACCESS_TOKEN: jwt)
- Add orb-agent run command with explicit config path
- Expose Hydra ports 4444/4445 for external token requests
- Add Hydra DB env vars to Postgres for init script

nginx/nginx.conf:
- Rewrite to match official Diode nginx architecture
- Auth via auth_request subrequests to diode-auth:8080 (HTTP)
- gRPC pass to ingester/reconciler on :8081 after auth
- Add /diode/auth proxy for token endpoint access
- Add error handlers for 401/403

oauth2/client/bootstrap-clients.sh:
- Rewrite to use hydra CLI (create/get oauth2-client) instead of
  raw wget calls to Hydra admin API
- Use jq for JSON parsing instead of python3
- Idempotent: checks if client exists before creating

setup.sh:
- Fix OAuth2 scope: diode:ingester -> diode:ingest
- Rewrite orb-agent config to match current agent.yaml schema
  (config_manager, policies with cron schedules, scope-based targets)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-27 21:51:40 -07:00
sam
c5a0245dd2 Add project infrastructure and configuration files
Docker Compose stack, nginx config, OAuth2 client bootstrap,
Hydra DB init, setup script, and gitignore for secrets.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-27 20:46:59 -07:00
sam
955618a44a Initial commit 2026-02-27 20:28:57 -07:00