What to expect on a CPU-only server
Ollama runs open language models on your own server, so prompts and answers stay on your machine. Our plans have no GPU, so the model runs on shared vCPUs and system memory. Small models work, but generation is slower than hosted services, and speed varies with the model, the plan and how busy the host is.
That makes a CPU VPS a good fit for private experiments, light personal use, internal tools and automation that can wait a few seconds. It is a poor fit for many simultaneous users or large models. If you only need answers from a strong model, a hosted API is simpler and often cheaper.
free -h
nproc
df -h /
grep -o -m1 -w avx2 /proc/cpuinfo || echo "No AVX2 reported"Install Ollama
The official installer sets up the program, a dedicated service user and a systemd service. Download the script and read it before you run it, instead of piping it straight into a shell. If it asks for a missing tool such as zstd, install that with your package manager and run it again.
The current installer extracts a Zstandard archive, so install zstd as well as curl before running it.
sudo apt update
sudo apt install curl zstdsudo dnf install curl zstdcurl -fsSL https://ollama.com/install.sh -o ollama-install.sh
less ollama-install.sh
sh ollama-install.shollama --version
systemctl is-active ollama
sudo ss -lntp | grep 11434Choose a model that fits in memory
A model needs roughly its file size in memory, plus extra for the conversation context and your other services. As a rough guide, a 4-bit quantised model uses about 0.6 to 0.7 GB per billion parameters, so a 3-billion-parameter model needs around 2 to 3 GB in total. Leave room for the operating system and anything else running.
Start small: llama3.2:1b suits a 2 GB plan, and llama3.2:3b suits 4 GB or more. Browse the Ollama library for newer small models, and read each model's licence on its page. Licences differ, and some restrict commercial use or require attribution. The Llama 3.2 licence, for example, has its own conditions.
ollama pull llama3.2:3b
ollama listChat with the model and measure its speed
Run the model interactively. The verbose flag prints timing figures after each answer, including eval rate in tokens per second. That figure is the honest measure for your plan. Try a few prompts at different times of day, because shared vCPU speed is not constant.
ollama run llama3.2:3b --verbose
# type a question, then /bye to leaveollama pstotal duration: 18.836281278s
load duration: 8.436195373s
prompt eval count: 34 token(s)
prompt eval duration: 2.759876s
prompt eval rate: 12.32 tokens/s
eval count: 43 token(s)
eval duration: 7.634548s
eval rate: 5.63 tokens/sCall the local API
Applications talk to Ollama through a simple HTTP API on localhost. This request asks for one complete answer. Without stream set to false, the server returns many small JSON lines as it generates.
curl -s http://127.0.0.1:11434/api/generate -d '{"model":"llama3.2:3b","prompt":"Explain what a reverse proxy does in two sentences.","stream":false}'Control memory use and idle behaviour
By default Ollama keeps a model loaded for five minutes after the last request, which makes the next answer start quickly but holds the memory. Change it with a systemd override. A shorter keep-alive frees memory for other services. You can also limit how many models load at once and how many requests run in parallel, which matters on small plans.
sudo systemctl edit ollama[Service]
Environment="OLLAMA_KEEP_ALIVE=2m"
Environment="OLLAMA_MAX_LOADED_MODELS=1"
Environment="OLLAMA_NUM_PARALLEL=1"sudo systemctl restart ollama
systemctl show ollama -p EnvironmentUpdate and remove
Run the installer again to update Ollama. Remove a model you no longer use to recover disk space. To uninstall the whole service, stop it, delete its files and the service user as described in the Ollama documentation.
ollama rm llama3.2:3b
du -sh /usr/share/ollama/.ollama 2>/dev/nullOfficial documentation
These upstream references provide more detail on the commands and workflows in this guide.