Secure Ollama and Open WebUI on a public server

AI and LLMsWritten by GLWS13 min readUbuntu 24.04 LTS / AlmaLinux 9

Commands run on your server unless a section says otherwise. Replace example usernames, addresses and paths with your own values. Keep a working session open when changing remote access.

Why AI services are a target

Ollama's API has no password. Anyone who can reach port 11434 can run your models, fill your disk and use your CPU, and some versions have had security bugs. Automated scanners look for exposed AI services and other self-hosted tools continuously, so a service left open is usually found within hours or days.

The rule is simple. Services listen on 127.0.0.1 only. The only things reachable from the internet are your SSH port and, if you publish something, a reverse proxy with HTTPS and a login.

On the VPS: list what is listening and on which addresses

sudo ss -lntp

Look for 0.0.0.0 and [::] in the Local Address column. Those are reachable from outside unless a firewall blocks them. 127.0.0.1 and [::1] are private to the server.

Bind each service to localhost

For a native Ollama install, make sure OLLAMA_HOST is not set to 0.0.0.0. The default is localhost. For containers, publish ports with the 127.0.0.1 prefix, as in 127.0.0.1:3000:8080. A mapping without the prefix opens the port to everyone.

Check the Ollama environment

systemctl show ollama -p Environment
sudo ss -lntp | grep 11434

Find containers that publish ports publicly

sudo docker ps --format "{{.Names}}\t{{.Ports}}" | grep -E "0\.0\.0\.0|\[::\]"

Do not trust the firewall with Docker

UFW and firewalld rules do not protect ports published by Docker, because Docker inserts its own rules ahead of them. A closed port in your firewall status can still be open. Fix the publishing, rather than the firewall: bind to 127.0.0.1, or put the container on a private network with no published port, as the Open WebUI guide does for Ollama.

Still enable the host firewall for everything that is not Docker, and allow only SSH, 80 and 443.

Review host firewall rules

sudo ufw status verbose 2>/dev/null
sudo firewall-cmd --list-all 2>/dev/null

Reach private services over an SSH tunnel

The safest way to use a private service is to tunnel to it. The command below forwards port 3000 on your computer to port 3000 on the server through your normal encrypted SSH login. Nothing extra is exposed, and it works with the key-only SSH settings from the hardening guide.

On your computer

ssh -N -L 3000:127.0.0.1:3000 deploy@YOUR_SERVER
# then open http://localhost:3000

Publish through a reverse proxy with a login

If other people need to reach the service, publish it through Nginx with HTTPS as in the reverse proxy guide, and require a login. Open WebUI has its own accounts. Services without any login, such as a raw Ollama API, need one added. Basic authentication in Nginx is a simple option for a small private group.

Create a password file with htpasswd, which is in apache2-utils on Ubuntu and httpd-tools on AlmaLinux, then reference it in the Nginx location block.

Ubuntu

sudo apt install apache2-utils
sudo htpasswd -c /etc/nginx/.htpasswd alice

AlmaLinux

sudo dnf install httpd-tools
sudo htpasswd -c /etc/nginx/.htpasswd alice

Add these two lines inside the location block of your Nginx file

auth_basic "Private";
auth_basic_user_file /etc/nginx/.htpasswd;

Test and reload

sudo nginx -t && sudo systemctl reload nginx

Basic authentication sends credentials with every request, so use it only over HTTPS. Do not expose a raw Ollama API on the internet, even behind basic authentication, unless you understand the risk.

Lock down accounts and secrets

Turn off open registration after you create your admin account, use long unique passwords, and keep API keys and secret keys in files readable only by their owner, not in compose files that you share or paste into chats. Keep the system, Docker images and AI software updated, because these projects release security fixes often.

Check secret file permissions

ls -l ~/openwebui/.env
chmod 600 ~/openwebui/.env

Ubuntu: update the system

sudo apt update && sudo apt upgrade

AlmaLinux: update the system

sudo dnf upgrade

Both systems: update container images

cd ~/openwebui && sudo docker compose pull && sudo docker compose up -d

Test from outside

Check from a different network, such as your phone on mobile data or another server, rather than from the VPS itself. These tests should all fail to connect. If one succeeds, the port is exposed.

From another computer (replace YOUR_SERVER_IP)

curl -m 5 http://YOUR_SERVER_IP:11434/api/tags
curl -m 5 http://YOUR_SERVER_IP:3000/
curl -m 5 http://YOUR_SERVER_IP:8080/

Watch resource use and logs

Unexpected CPU load, a full disk or unfamiliar models are signs of misuse. Check them from time to time, and install Fail2ban as described in the hardening guide. A small VPS that suddenly runs slowly is worth a look.

Both systems

ollama list 2>/dev/null
sudo docker stats --no-stream 2>/dev/null
df -h /
sudo journalctl -u ollama --since "1 day ago" --no-pager | tail -n 20

Official documentation

These upstream references provide more detail on the commands and workflows in this guide.

Secure Ollama, Open WebUI and other AI services on a public server