sam
5748bad765
- proxmox_collector: support numbered PVE_HOST_1/2/3 env vars with backward compat for legacy single PVE_HOST; fix MTU string-to-int cast - pbs_collector: new collector for Proxmox Backup Server — discovers devices, interfaces, IPs, and datastores (as Services) via PBS API - vmware_collector: fix mac_address → primary_mac_address for Diode SDK - network_collector: add Netmiko SSH fallback for Brocade/NOS devices, add Brocade ICX interface type patterns - unifi_collector: new collector for UniFi UDM-SE/switches/APs - ENV_REFERENCE.md: document all collector env vars and setup steps - .gitignore: exclude collectors/inventory.yaml (contains credentials) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
245 lines
7.9 KiB
Markdown
245 lines
7.9 KiB
Markdown
# Environment Variables Reference
|
|
|
|
All variables go in `.env` at the project root. Variables marked **[HAVE]**
|
|
are already configured. Variables marked **[NEED]** are what you need to gather.
|
|
|
|
---
|
|
|
|
## Diode Pipeline [HAVE]
|
|
Already configured and working.
|
|
|
|
```bash
|
|
INGESTER_CLIENT_ID=diode-ingester
|
|
INGESTER_CLIENT_SECRET=<already set>
|
|
NETBOX_API_URL=http://172.19.77.160:8000
|
|
NETBOX_API_TOKEN=nbt_<already set>
|
|
```
|
|
|
|
## Proxmox VE Collector [HAVE — partial]
|
|
Supports multiple standalone PVE hosts. prox940 already configured.
|
|
|
|
```bash
|
|
# Legacy single-host (still works)
|
|
PVE_HOST=192.168.1.190
|
|
PVE_USER=root@pam
|
|
PVE_TOKEN_NAME=diode
|
|
PVE_TOKEN_VALUE=<already set>
|
|
PVE_VERIFY_SSL=false
|
|
|
|
# Additional PVE hosts (numbered)
|
|
PVE_HOST_2=10.40.40.107 # proxmox2
|
|
PVE_USER_2=diode@pve
|
|
PVE_TOKEN_NAME_2=diode
|
|
PVE_TOKEN_VALUE_2= # NEED — create token on proxmox2
|
|
|
|
PVE_HOST_3=10.40.40.110 # proxmox3
|
|
PVE_USER_3=diode@pve
|
|
PVE_TOKEN_NAME_3=diode
|
|
PVE_TOKEN_VALUE_3= # NEED — create token on proxmox3
|
|
```
|
|
|
|
**Setup on each PVE host:**
|
|
```bash
|
|
pveum user add diode@pve --comment "Diode NetBox collector"
|
|
pveum aclmod / -user diode@pve -role PVEAuditor
|
|
pveum user token add diode@pve diode --privsep 0 --comment "NetBox Diode"
|
|
```
|
|
|
|
## Proxmox Backup Server Collector [NEED]
|
|
|
|
```bash
|
|
PBS_HOST_1=10.40.40.150 # PBS-01
|
|
PBS_USER_1=diode@pbs
|
|
PBS_TOKEN_NAME_1=diode
|
|
PBS_TOKEN_VALUE_1= # NEED — create token on PBS-01
|
|
|
|
PBS_HOST_2=192.168.1.241 # PBS-02
|
|
PBS_USER_2=diode@pbs
|
|
PBS_TOKEN_NAME_2=diode
|
|
PBS_TOKEN_VALUE_2= # NEED — create token on PBS-02
|
|
|
|
PBS_HOST_3=pbs.apodacalabs.com # PBS
|
|
PBS_USER_3=diode@pbs
|
|
PBS_TOKEN_NAME_3=diode
|
|
PBS_TOKEN_VALUE_3= # NEED — create token on PBS
|
|
```
|
|
|
|
**Setup on each PBS host:**
|
|
```bash
|
|
proxmox-backup-manager user create diode@pbs --comment "Diode NetBox collector"
|
|
proxmox-backup-manager acl update / Audit --auth-id diode@pbs
|
|
proxmox-backup-manager user generate-token diode@pbs diode
|
|
```
|
|
|
|
---
|
|
|
|
## Network Collector [NEED]
|
|
Credentials go in `collectors/inventory.yaml`, not `.env`.
|
|
Only these optional vars go in `.env`:
|
|
|
|
```bash
|
|
# Optional: skip pyATS even if installed (run with --no-pyats flag instead)
|
|
# No env vars strictly required — everything is in inventory.yaml
|
|
```
|
|
|
|
## CML Topology Collector [NEED]
|
|
|
|
```bash
|
|
CML_HOST= # CML controller IP or hostname (e.g., 10.40.40.50)
|
|
CML_USER=admin # CML admin username
|
|
CML_PASSWORD= # CML admin password
|
|
CML_LAB= # Optional: specific lab name/ID (blank = all labs)
|
|
CML_VERIFY_SSL=false # Set true if CML has valid TLS cert
|
|
CML_SITE=CML # NetBox site name for CML devices (default: CML)
|
|
```
|
|
|
|
**Setup on CML side:** Just need the controller address and admin creds.
|
|
The virl2_client library handles the REST API.
|
|
|
|
## Zabbix Collector [NEED]
|
|
|
|
```bash
|
|
ZABBIX_URL= # Full URL to API (e.g., http://10.40.40.20/api_jsonrpc.php)
|
|
ZABBIX_USER=Admin # Zabbix username
|
|
ZABBIX_PASSWORD= # Zabbix password
|
|
ZABBIX_API_TOKEN= # OR use an API token instead of user/pass (Zabbix 5.4+)
|
|
ZABBIX_SITE=main # NetBox site to assign devices to
|
|
ZABBIX_DEFAULT_ROLE=Server # Default role if group-based detection fails
|
|
```
|
|
|
|
**Setup on Zabbix side:** No setup needed — just need read access creds.
|
|
If using API token (Zabbix 5.4+): Administration → API tokens → Create.
|
|
|
|
## Observium Collector [NEED]
|
|
|
|
```bash
|
|
OBSERVIUM_URL= # API base URL (e.g., http://10.40.40.30/api/v0)
|
|
OBSERVIUM_USER=admin # Observium username
|
|
OBSERVIUM_PASSWORD= # Observium password
|
|
OBSERVIUM_SITE=main # NetBox site to assign devices to
|
|
OBSERVIUM_DEFAULT_ROLE=Network Device
|
|
```
|
|
|
|
**NOTE:** Observium REST API requires Professional or Enterprise edition.
|
|
Community Edition does not expose a REST API. If you're on Community,
|
|
skip this collector (Zabbix can cover similar ground).
|
|
|
|
## VMware Collector [NEED]
|
|
|
|
```bash
|
|
VCENTER_HOST= # vCenter or ESXi IP/hostname
|
|
VCENTER_USER=administrator@vsphere.local
|
|
VCENTER_PASSWORD= # vCenter/ESXi password
|
|
VCENTER_PORT=443 # API port (default: 443)
|
|
VCENTER_VERIFY_SSL=false # Set true if valid TLS cert
|
|
VCENTER_SITE=main # NetBox site to assign devices to
|
|
```
|
|
|
|
**Setup on vCenter side:** Just need a read-only account.
|
|
Minimum role: Read-only → Assign at vCenter root.
|
|
|
|
## Docker Collector [NEED — only if remote hosts]
|
|
Works immediately for local Docker (no env vars needed).
|
|
For remote Docker hosts:
|
|
|
|
```bash
|
|
DOCKER_HOSTS= # Comma-separated (e.g., tcp://10.0.0.5:2375,tcp://10.0.0.6:2375)
|
|
DOCKER_SITE=main # NetBox site
|
|
DOCKER_TLS_VERIFY=false # Set true if Docker TLS is configured
|
|
```
|
|
|
|
**Setup on remote Docker hosts:** Enable TCP API:
|
|
`dockerd -H unix:///var/run/docker.sock -H tcp://0.0.0.0:2375`
|
|
Or use TLS: https://docs.docker.com/engine/security/protect-access/
|
|
|
|
## UniFi Collector [NEED]
|
|
Discovers UDM-SE, switches, and APs from the local UniFi Controller API.
|
|
|
|
```bash
|
|
UNIFI_HOST=192.168.1.1 # UDM-SE / Controller IP (or hostname)
|
|
UNIFI_USER= # UniFi local admin username
|
|
UNIFI_PASSWORD= # UniFi local admin password
|
|
UNIFI_SITE=default # UniFi site name (usually "default")
|
|
UNIFI_VERIFY_SSL=false # UDM-SE uses self-signed cert by default
|
|
UNIFI_IS_UDM=true # true for UDM/UDM-SE/UDR, false for legacy controller
|
|
UNIFI_NETBOX_SITE=main # NetBox site to assign devices to
|
|
```
|
|
|
|
**What it discovers:**
|
|
- UDM-SE, switches, APs as Devices with model/serial/firmware
|
|
- Switch ports with speed, PoE, SFP detection
|
|
- WiFi radios with band/channel/power
|
|
- VLANs and subnets from network configurations
|
|
- WLANs (SSIDs) with auth type
|
|
- LLDP neighbors → Cables for topology mapping
|
|
|
|
**Setup on UDM-SE:** Just need a local admin account.
|
|
The API is built-in — no additional setup required.
|
|
|
|
## NAPALM Plugin (live device status in NetBox UI) [NEED]
|
|
These go in the NetBox Docker env, not the project `.env`.
|
|
Add to `/home/user/netbox-docker/env/netbox.env`:
|
|
|
|
```bash
|
|
NAPALM_USERNAME=admin # Same SSH creds as your network devices
|
|
NAPALM_PASSWORD= # SSH password
|
|
```
|
|
|
|
Then assign NAPALM drivers to Platforms in NetBox:
|
|
Devices → Platforms → edit each platform → set NAPALM driver
|
|
(e.g., Platform "Cisco IOS" → NAPALM driver: ios)
|
|
|
|
---
|
|
|
|
## Quick Checklist
|
|
|
|
| Collector | What to gather | Priority |
|
|
|-----------|---------------|----------|
|
|
| Network | SSH creds for routers/switches, fill in inventory.yaml | HIGH |
|
|
| CML | Controller IP + admin creds | HIGH |
|
|
| Zabbix | API URL + creds or API token | MEDIUM |
|
|
| Observium | API URL + creds (needs paid edition) | LOW |
|
|
| VMware | vCenter IP + read-only account | MEDIUM |
|
|
| Docker | Nothing (local works), or remote TCP URLs | LOW |
|
|
| Proxmox VE | Token per host (diode@pve + API token) | HIGH |
|
|
| PBS | Token per host (diode@pbs + API token) | HIGH |
|
|
| UniFi | UDM-SE IP + local admin creds | HIGH |
|
|
| NAPALM | SSH creds in netbox.env + assign drivers to platforms | MEDIUM |
|
|
|
|
## Testing Each Collector
|
|
|
|
All collectors support `--dry-run` for safe testing:
|
|
|
|
```bash
|
|
# Activate venv first
|
|
source .venv/bin/activate
|
|
|
|
# Network devices (highest value — discovers cables + topology)
|
|
python collectors/network_collector.py -i collectors/inventory.yaml --dry-run
|
|
|
|
# CML topology
|
|
python collectors/cml_collector.py --dry-run
|
|
|
|
# Zabbix import
|
|
python collectors/zabbix_collector.py --dry-run
|
|
|
|
# Observium import
|
|
python collectors/observium_collector.py --dry-run
|
|
|
|
# VMware
|
|
python collectors/vmware_collector.py --dry-run
|
|
|
|
# Docker (works immediately)
|
|
python collectors/docker_collector.py --dry-run
|
|
|
|
# UniFi (UDM-SE + APs)
|
|
python collectors/unifi_collector.py --dry-run
|
|
|
|
# Proxmox VE (multi-host — already tested)
|
|
python collectors/proxmox_collector.py --dry-run
|
|
|
|
# Proxmox Backup Server
|
|
python collectors/pbs_collector.py --dry-run
|
|
```
|
|
|
|
Add `--log-level DEBUG` to any command for verbose output. |