Self-Hosted AI with Ollama: Local LLMs for your homelab

Why self-hosted AI

Running AI models locally offers concrete advantages for a SysAdmin:

• Total privacy — your data never leaves the infrastructure
• Zero cost — no API key, no subscription
• Offline — works even without internet
• Customizable — you can fine-tune on your data
• Speed — with a dedicated GPU, very low latency

---

Recommended hardware

| Configuration | GPU RAM | Supported models | Notes | |---------------|---------|-------------------|------| | Entry (RX580 4GB) | 4 GB | 3B-7B quantized | Slow response | | Mid (RTX 3080 10GB) | 10 GB | 7B-13B | Good speed | | My setup | NVIDIA A10 24GB | 14B-34B | Great for production | | High-end (A100 80GB) | 80 GB | Full 70B | Enterprise |

---

Ollama installation

curl -fsSL https://ollama.com/install.sh | sh


ollama --version


systemctl enable ollama
systemctl start ollama

Configuration for remote access

[Service]
Environment="OLLAMA_HOST=0.0.0.0:11434"
Environment="OLLAMA_ORIGINS=*"

systemctl daemon-reload
systemctl restart ollama

---

Recommended models

ollama pull llama3.1:8b          # 5GB — veloce, buona qualità
ollama pull llama3.1:70b         # 40GB — qualità elevata
ollama pull mistral:7b           # 4GB — ottimo per italiano


ollama pull deepseek-r1:14b      # 9GB — ragionamento complesso
ollama pull deepseek-coder:6.7b  # 4GB — generazione codice
ollama pull llava:13b            # 8GB — analisi immagini (multimodale)
ollama pull nomic-embed-text     # 274MB — embedding per RAG


ollama list


ollama rm nome-modello

---

REST API

Ollama exposes an OpenAI-compatible API on port 11434:

curl http://localhost:11434/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "llama3.1:8b",
    "messages": [
      {"role": "system", "content": "Sei un assistente IT esperto."},
      {"role": "user", "content": "Come configuro il firewall su Ubuntu?"}
    ]
  }'


curl http://localhost:11434/v1/embeddings \
  -H "Content-Type: application/json" \
  -d '{"model": "nomic-embed-text", "input": "testo da embeddare"}'

---

Integration with n8n

Workflow: automatic log analysis

// Nodo "Code" in n8n - Analisi log con AI
const logs = $input.first().json.logs;

const response = await $http.post('http://192.168.1.20:11434/v1/chat/completions', {
  model: 'llama3.1:8b',
  messages: [
    {
      role: 'system',
      content: 'Sei un esperto di sicurezza informatica. Analizza i log e identifica anomalie, possibili attacchi o errori critici. Rispondi in italiano.'
    },
    {
      role: 'user',
      content: `Analizza questi log:\n\n${logs}`
    }
  ],
  max_tokens: 500
});

return [{ json: { 
  analisi: response.choices[0].message.content,
  timestamp: new Date().toISOString()
}}];

Workflow: PowerShell script generation

// Genera script PS da descrizione naturale
const task = $input.first().json.task;

const response = await $http.post('http://192.168.1.20:11434/v1/chat/complet