Arvandor

Deprecated — This is the legacy (pre-HCI) infrastructure stack. It has been superseded by Arvandor HCI, which replaces per-service VMs with a two-tier cattle node model, Nomad workload scheduling, and multi-site quorum. This repo is preserved as a reference for the original architecture.

Production-grade infrastructure-as-code for running services on Proxmox with enterprise HA patterns.

Overview

Arvandor provides a complete infrastructure stack:

• Terraform - VM provisioning on Proxmox (Linux and Windows)
• Ansible - Configuration management (26 playbooks)
• Nebula - Encrypted overlay network (3-lighthouse mesh)
• Vault - Secrets management (3-node Raft cluster) + Internal PKI
• Consul - Service discovery (3-site cluster, Patroni DCS)
• Forgejo - Git forge (bare-metal binary)
• Stalwart - Mail server (SMTP, IMAP, JMAP, DKIM, SPF, DMARC)
• PostgreSQL - Database (3-node Patroni + Consul DCS, TLS)
• Valkey - Cache/queue (single instance, ACL per-service)
• Garage - S3-compatible storage (3-node cluster)
• Ory Stack - Identity platform (Kratos, Hydra, Keto, Oathkeeper)
• Monitoring - Prometheus + Grafana + Loki + Uptime Kuma

Architecture

┌─────────────────────────────────────────────────────────────────────────────┐
│                            Proxmox Host                                     │
├─────────────────────────────────────────────────────────────────────────────┤
│  ┌─────────────┐  ┌─────────────┐  ┌─────────────┐  ┌─────────────┐       │
│  │ Management  │  │  Platform   │  │    Data     │  │  Workloads  │       │
│  │ 1000-1999   │  │ 2000-2999   │  │ 3000-3999   │  │ 4000-4999   │       │
│  │             │  │             │  │             │  │             │       │
│  │ DNS, Caddy  │  │ Vault, Loom │  │ PostgreSQL  │  │ Your Apps   │       │
│  │             │  │ Mail, Ident │  │ Consul      │  │             │       │
│  │             │  │ Gateway     │  │ Valkey, S3  │  │             │       │
│  └──────┬──────┘  └──────┬──────┘  └──────┬──────┘  └──────┬──────┘       │
│         │                │                │                │              │
│         └────────────────┴────────────────┴────────────────┘              │
│                                  │                                         │
│                         Nebula Overlay (10.10.10.0/24)                    │
│                    3 Lighthouses: Proxmox + Home + Cloud                  │
└─────────────────────────────────────────────────────────────────────────────┘
         ┌──────────┐                              ┌──────────┐
         │ Home NUC │  Vault + Consul follower     │Cloud VPS │  Vault + Consul + DR
         │ .10      │  Nebula lighthouse           │ .110     │  Nebula lighthouse
         └──────────┘                              └──────────┘

Quick Start

1. Prerequisites

• Proxmox VE host
• Arch Linux VM template (VMID 9000)
• Terraform, Ansible installed locally
• Nebula binary for certificate generation

2. Configure

git clone <repo-url> arvandor
cd arvandor

# Configure Terraform
cp terraform/terraform.tfvars.example terraform/terraform.tfvars
vim terraform/terraform.tfvars

# Configure Ansible
cp ansible/inventory.ini.example ansible/inventory.ini
cp ansible/services.yml.example ansible/services.yml
cp ansible/group_vars/all.yml.example ansible/group_vars/all.yml
vim ansible/inventory.ini

# Set up Ansible Vault
echo "your-vault-password" > ~/.ansible-vault-pass
ansible-vault create ansible/vault/secrets.yml

# Generate Nebula CA
cd nebula
nebula-cert ca -name "Arvandor CA"

3. Provision VMs

cd terraform
cp terraform.tfvars.example terraform.tfvars  # Edit with your Proxmox API creds
terraform init
terraform plan
terraform apply

4. Onboard VMs

cd ansible

# Onboarding trilogy — uses bridge IP + SSH proxy through Proxmox host
ansible-playbook -i inventory.ini playbooks/bootstrap.yml --limit "new-vm" -e @vars/onboarding.yml
ansible-playbook -i inventory.ini playbooks/security.yml --limit "new-vm" -e @vars/onboarding.yml
ansible-playbook -i inventory.ini playbooks/nebula.yml --limit "new-vm" -e @vars/onboarding.yml

# After nebula.yml: VM is on the mesh — all subsequent playbooks use Nebula directly

5. Deploy Infrastructure

# Core services (deploy in order)
ansible-playbook -i inventory.ini playbooks/dns.yml
ansible-playbook -i inventory.ini playbooks/consul.yml
ansible-playbook -i inventory.ini playbooks/vault.yml
ansible-playbook -i inventory.ini playbooks/pki.yml                    # Internal PKI + TLS

# Data tier
ansible-playbook -i inventory.ini playbooks/postgres-ha.yml            # Patroni + Consul DCS
ansible-playbook -i inventory.ini playbooks/valkey.yml
ansible-playbook -i inventory.ini playbooks/garage.yml

# Platform services
ansible-playbook -i inventory.ini playbooks/caddy.yml
ansible-playbook -i inventory.ini playbooks/forgejo.yml
ansible-playbook -i inventory.ini playbooks/stalwart.yml               # Mail server
ansible-playbook -i inventory.ini playbooks/identity.yml               # Ory auth stack
ansible-playbook -i inventory.ini playbooks/gateway.yml                # Oathkeeper + Boundary

# Monitoring
ansible-playbook -i inventory.ini playbooks/observe.yml
ansible-playbook -i inventory.ini playbooks/exporters.yml
ansible-playbook -i inventory.ini playbooks/uptime-kuma.yml

# Per-service data provisioning (PostgreSQL + Valkey + S3 + Vault creds)
ansible-playbook -i inventory.ini playbooks/data-service.yml -e "service=myapp"

Directory Structure

arvandor/
├── terraform/           # VM provisioning (Linux + Windows)
│   ├── modules/vm/      # Reusable VM module (os_type: linux/windows)
│   ├── management.tf    # DNS, Caddy
│   ├── services.tf      # Vault, Forgejo, Mail, Identity, Gateway
│   └── data.tf          # PostgreSQL, Valkey, Garage, Consul
├── ansible/             # Configuration management
│   ├── playbooks/       # 26 playbooks
│   ├── templates/       # 75 Jinja2 templates
│   ├── group_vars/      # Global variables
│   ├── vars/            # Override variables (onboarding)
│   └── vault/           # Ansible Vault secrets
├── nebula/              # Overlay network (3-lighthouse mesh)
│   └── v2/              # Per-host certificates
├── network/             # Host networking (port-forward.sh, IP schema)
└── docs/                # Architecture documentation

Network Design

Two-Network Model

Network	CIDR	Purpose
Bridge (vmbr1)	192.168.100.0/24	Provisioning only
Nebula	10.10.10.0/24	All application traffic

VMs only accept traffic from the Proxmox host (for Ansible) and the Nebula overlay. Guest iptables enforce this.

Three-Lighthouse Mesh

Location	Nebula IP	Purpose
Proxmox host	10.10.10.1	Primary lighthouse + relay
Home NUC	10.10.10.10	Low-latency relay
Cloud VPS	10.10.10.110	Geographic redundancy + DR

Security Groups (Nebula)

Group	Purpose
admin	Full access (your devices)
infrastructure	Core services
platform	Identity/auth services
projects	Application workloads
games	Isolated game servers

Key Patterns

Internal PKI

Three-tier Vault PKI: Root CA (10y, EC P-384) → Intermediate CA (1y, EC P-384) → Leaf certs (72h, EC P-256). Distributed to all VMs. Vault Agent sidecars auto-renew leaf certs for PostgreSQL and Stalwart TLS.

Consul Service Discovery

All services register in Consul with health checks. DNS resolution via <service>.service.consul and <tag>.<service>.service.consul. Patroni uses Consul as DCS with health-aware leader election: master.postgres-cluster.service.consul always resolves to the current primary.

Vault Agent Sidecar

Applications needing dynamic Vault credentials use a Vault Agent sidecar. Architecture: vault-agent renders credentials to bind-mounted ./run/app/.env, app container reads via entrypoint.sh that sources the file.

Data Service Provisioning

data-service.yml provisions per-service: PostgreSQL database + roles, Valkey ACL user, S3 bucket, Vault AppRoles + policies. All driven by the service manifest (services.yml).

High Availability

Service	Topology	Notes
Vault	3-node Raft (Proxmox + Home + Cloud)	Auto-unseal, Transit KMS
PostgreSQL	3-node Patroni	Consul DCS, streaming replication, TLS
Consul	3-site cluster	Service discovery, Patroni DCS
Garage S3	3-node cluster	Replication factor 3
Nebula	3-lighthouse mesh	NAT traversal, geographic redundancy

Playbooks

Category	Playbooks
Onboarding	bootstrap.yml, security.yml, nebula.yml
DNS	dns.yml, dns-client.yml
Service Discovery	consul.yml, consul-client.yml, consul-register.yml
Secrets & PKI	vault.yml, pki.yml
Data	postgres-ha.yml, valkey.yml, garage.yml, data-service.yml
Platform	caddy.yml, forgejo.yml, stalwart.yml, identity.yml, gateway.yml
Monitoring	observe.yml, exporters.yml, uptime-kuma.yml
Deployment	deploy.yml, update.yml
Utility	docker.yml

Documentation

• Getting Started - Detailed setup guide
• Architecture - Design decisions
• Provisioning Guide - Adding new VMs

Related

• Arvandor HCI — Current-generation HCI platform (successor to this repo)