Choose a VPS size for a local AI model

Memory decides what will run

The first limit is memory. A model has to fit in RAM, with room left for the operating system, the context for your conversation and everything else on the server. If it does not fit, the system swaps or the process is killed, and swapping a model is too slow to be useful.

Language models are measured in billions of parameters, written as 1B, 3B, 7B and so on. Most models you download are quantised, which means stored at lower precision to save memory. A common 4-bit quantisation needs roughly 0.6 to 0.7 GB for each billion parameters, plus extra for context. Treat these as estimates and check the size shown on the model's page.

A rough sizing guide

These are planning estimates for a 4-bit quantised model on a server with nothing else heavy running. They are not benchmarks.

1B models need about 1.5 GB in total, so a 2 GB plan is the practical minimum. 3B models need about 3 GB, so choose 4 GB or more. 7B to 8B models need about 5 to 6 GB, so choose 8 GB. 13B to 14B models need about 9 to 10 GB, so choose 16 GB. Models of 30B and above need 20 GB or more, which is beyond a VPS and into dedicated server territory.

On our plans, that means VPS 4 or Dev 4 (4 GB) for a 3B model, VPS 8 or Dev 8 (8 GB) for a 7B model, and Dev 16 (16 GB) for a 13B model with room for other services. Running an interface such as Open WebUI beside the model adds roughly 1 GB.

Speed depends on the CPU, not only memory

Memory lets a model run, while the CPU decides how fast. Generation is limited mainly by how quickly the processor can read the model from memory for each word it writes. More vCPUs help up to a point, and a larger model is slower in proportion to its size, so a 7B model is roughly twice as slow as a 3B model on the same plan.

Shared vCPUs give variable speed because other customers use the same hardware. Dedicated cores give steadier results. Never rely on a number from another provider's hardware or a blog post. Measure on the plan you intend to buy.

nproc
lscpu | grep -E "Model name|Thread|Core|Flags" | cut -c1-120
grep -o -m1 -w avx2 /proc/cpuinfo || echo "No AVX2 reported"

Disk space and bandwidth

Each model is a separate download, and people usually try several. Plan for 2 to 5 GB for each small model, and for the Docker images and application data if you use containers. On a 25 GB plan, a couple of small models fit but not much more. An 80 GB plan leaves comfortable room.

Downloads count against your monthly transfer, which on our plans is in terabytes, so model downloads are unlikely to matter. Once a model is downloaded, running it uses no bandwidth.

df -h /
sudo du -sh /usr/share/ollama/.ollama 2>/dev/null
sudo docker system df 2>/dev/null

Test small before you buy big

Start on a small plan and a small model, measure real speed, and move up only if it falls short. Ollama's verbose mode reports tokens per second, and a tokens-per-second figure of a few per second is readable but slow, while below one is painful. Judge by how the tool feels for your own task.

Because plans can be changed, there is little risk in starting small. Keep your configuration in files so that you can rebuild on a bigger server quickly. The backup guide covers what to keep.

ollama run llama3.2:3b --verbose "Write a short paragraph about servers."
# look for the "eval rate" line

When a hosted API is the better choice

A CPU server cannot match a large hosted model for quality or speed. If you need strong answers, many concurrent users, long documents or reliable low latency, call a hosted API from your VPS instead and keep the server small. A common design runs the application, database and queue on the VPS and sends only the difficult prompts to an API.

Weigh privacy, cost and quality for each task. Local models keep data on your server. Hosted models usually give better answers.