netbox-diode-project/collectors/ENV_REFERENCE.md

# Environment Variables Reference

All variables go in `.env` at the project root. Variables marked **[HAVE]**
are already configured. Variables marked **[NEED]** are what you need to gather.

---

## Diode Pipeline [HAVE]
Already configured and working.

```bash
INGESTER_CLIENT_ID=diode-ingester
INGESTER_CLIENT_SECRET=<already set>
NETBOX_API_URL=http://172.19.77.160:8000
NETBOX_API_TOKEN=nbt_<already set>
```

## Proxmox VE Collector [HAVE — partial]
Supports multiple standalone PVE hosts. prox940 already configured.

```bash
# Legacy single-host (still works)
PVE_HOST=192.168.1.190
PVE_USER=root@pam
PVE_TOKEN_NAME=diode
PVE_TOKEN_VALUE=<already set>
PVE_VERIFY_SSL=false

# Additional PVE hosts (numbered)
PVE_HOST_2=10.40.40.107           # proxmox2
PVE_USER_2=diode@pve
PVE_TOKEN_NAME_2=diode
PVE_TOKEN_VALUE_2=                # NEED — create token on proxmox2

PVE_HOST_3=10.40.40.110           # proxmox3
PVE_USER_3=diode@pve
PVE_TOKEN_NAME_3=diode
PVE_TOKEN_VALUE_3=                # NEED — create token on proxmox3
```

**Setup on each PVE host:**
```bash
pveum user add diode@pve --comment "Diode NetBox collector"
pveum aclmod / -user diode@pve -role PVEAuditor
pveum user token add diode@pve diode --privsep 0 --comment "NetBox Diode"
```

## Proxmox Backup Server Collector [NEED]

```bash
PBS_HOST_1=10.40.40.150           # PBS-01
PBS_USER_1=diode@pbs
PBS_TOKEN_NAME_1=diode
PBS_TOKEN_VALUE_1=                # NEED — create token on PBS-01

PBS_HOST_2=192.168.1.241          # PBS-02
PBS_USER_2=diode@pbs
PBS_TOKEN_NAME_2=diode
PBS_TOKEN_VALUE_2=                # NEED — create token on PBS-02

PBS_HOST_3=pbs.apodacalabs.com    # PBS
PBS_USER_3=diode@pbs
PBS_TOKEN_NAME_3=diode
PBS_TOKEN_VALUE_3=                # NEED — create token on PBS
```

**Setup on each PBS host:**
```bash
proxmox-backup-manager user create diode@pbs --comment "Diode NetBox collector"
proxmox-backup-manager acl update / Audit --auth-id diode@pbs
proxmox-backup-manager user generate-token diode@pbs diode
```

---

## Network Collector [NEED]
Credentials go in `collectors/inventory.yaml`, not `.env`.
Only these optional vars go in `.env`:

```bash
# Optional: skip pyATS even if installed (run with --no-pyats flag instead)
# No env vars strictly required — everything is in inventory.yaml
```

## CML Topology Collector [NEED]

```bash
CML_HOST=                    # CML controller IP or hostname (e.g., 10.40.40.50)
CML_USER=admin               # CML admin username
CML_PASSWORD=                # CML admin password
CML_LAB=                     # Optional: specific lab name/ID (blank = all labs)
CML_VERIFY_SSL=false         # Set true if CML has valid TLS cert
CML_SITE=CML                 # NetBox site name for CML devices (default: CML)
```

**Setup on CML side:** Just need the controller address and admin creds.
The virl2_client library handles the REST API.

## Zabbix Collector [NEED]

```bash
ZABBIX_URL=                  # Full URL to API (e.g., http://10.40.40.20/api_jsonrpc.php)
ZABBIX_USER=Admin            # Zabbix username
ZABBIX_PASSWORD=             # Zabbix password
ZABBIX_API_TOKEN=            # OR use an API token instead of user/pass (Zabbix 5.4+)
ZABBIX_SITE=main             # NetBox site to assign devices to
ZABBIX_DEFAULT_ROLE=Server   # Default role if group-based detection fails
```

**Setup on Zabbix side:** No setup needed — just need read access creds.
If using API token (Zabbix 5.4+): Administration → API tokens → Create.

## Observium Collector [NEED]

```bash
OBSERVIUM_URL=               # API base URL (e.g., http://10.40.40.30/api/v0)
OBSERVIUM_USER=admin         # Observium username
OBSERVIUM_PASSWORD=          # Observium password
OBSERVIUM_SITE=main          # NetBox site to assign devices to
OBSERVIUM_DEFAULT_ROLE=Network Device
```

**NOTE:** Observium REST API requires Professional or Enterprise edition.
Community Edition does not expose a REST API. If you're on Community,
skip this collector (Zabbix can cover similar ground).

## VMware Collector [NEED]

```bash
VCENTER_HOST=                # vCenter or ESXi IP/hostname
VCENTER_USER=administrator@vsphere.local
VCENTER_PASSWORD=            # vCenter/ESXi password
VCENTER_PORT=443             # API port (default: 443)
VCENTER_VERIFY_SSL=false     # Set true if valid TLS cert
VCENTER_SITE=main            # NetBox site to assign devices to
```

**Setup on vCenter side:** Just need a read-only account.
Minimum role: Read-only → Assign at vCenter root.

## Docker Collector [NEED — only if remote hosts]
Works immediately for local Docker (no env vars needed).
For remote Docker hosts:

```bash
DOCKER_HOSTS=                # Comma-separated (e.g., tcp://10.0.0.5:2375,tcp://10.0.0.6:2375)
DOCKER_SITE=main             # NetBox site
DOCKER_TLS_VERIFY=false      # Set true if Docker TLS is configured
```

**Setup on remote Docker hosts:** Enable TCP API:
`dockerd -H unix:///var/run/docker.sock -H tcp://0.0.0.0:2375`
Or use TLS: https://docs.docker.com/engine/security/protect-access/

## UniFi Collector [NEED]
Discovers UDM-SE, switches, and APs from the local UniFi Controller API.

```bash
UNIFI_HOST=192.168.1.1       # UDM-SE / Controller IP (or hostname)
UNIFI_USER=                  # UniFi local admin username
UNIFI_PASSWORD=              # UniFi local admin password
UNIFI_SITE=default           # UniFi site name (usually "default")
UNIFI_VERIFY_SSL=false       # UDM-SE uses self-signed cert by default
UNIFI_IS_UDM=true            # true for UDM/UDM-SE/UDR, false for legacy controller
UNIFI_NETBOX_SITE=main       # NetBox site to assign devices to
```

**What it discovers:**
- UDM-SE, switches, APs as Devices with model/serial/firmware
- Switch ports with speed, PoE, SFP detection
- WiFi radios with band/channel/power
- VLANs and subnets from network configurations
- WLANs (SSIDs) with auth type
- LLDP neighbors → Cables for topology mapping

**Setup on UDM-SE:** Just need a local admin account.
The API is built-in — no additional setup required.

## NAPALM Plugin (live device status in NetBox UI) [NEED]
These go in the NetBox Docker env, not the project `.env`.
Add to `/home/user/netbox-docker/env/netbox.env`:

```bash
NAPALM_USERNAME=admin        # Same SSH creds as your network devices
NAPALM_PASSWORD=             # SSH password
```

Then assign NAPALM drivers to Platforms in NetBox:
  Devices → Platforms → edit each platform → set NAPALM driver
  (e.g., Platform "Cisco IOS" → NAPALM driver: ios)

---

## Quick Checklist

| Collector | What to gather | Priority |
|-----------|---------------|----------|
| Network   | SSH creds for routers/switches, fill in inventory.yaml | HIGH |
| CML       | Controller IP + admin creds | HIGH |
| Zabbix    | API URL + creds or API token | MEDIUM |
| Observium | API URL + creds (needs paid edition) | LOW |
| VMware    | vCenter IP + read-only account | MEDIUM |
| Docker    | Nothing (local works), or remote TCP URLs | LOW |
| Proxmox VE | Token per host (diode@pve + API token) | HIGH |
| PBS       | Token per host (diode@pbs + API token) | HIGH |
| UniFi     | UDM-SE IP + local admin creds | HIGH |
| NAPALM    | SSH creds in netbox.env + assign drivers to platforms | MEDIUM |

## Testing Each Collector

All collectors support `--dry-run` for safe testing:

```bash
# Activate venv first
source .venv/bin/activate

# Network devices (highest value — discovers cables + topology)
python collectors/network_collector.py -i collectors/inventory.yaml --dry-run

# CML topology
python collectors/cml_collector.py --dry-run

# Zabbix import
python collectors/zabbix_collector.py --dry-run

# Observium import
python collectors/observium_collector.py --dry-run

# VMware
python collectors/vmware_collector.py --dry-run

# Docker (works immediately)
python collectors/docker_collector.py --dry-run

# UniFi (UDM-SE + APs)
python collectors/unifi_collector.py --dry-run

# Proxmox VE (multi-host — already tested)
python collectors/proxmox_collector.py --dry-run

# Proxmox Backup Server
python collectors/pbs_collector.py --dry-run
```

Add `--log-level DEBUG` to any command for verbose output.