AI and LLMsWritten by GLWS14 min readUbuntu 24.04 LTS / AlmaLinux 9
Commands run on your server unless a section says otherwise. Replace example usernames, addresses and paths with your own values. Keep a working session open when changing remote access.

What a vector database does

A vector database stores embeddings: lists of numbers that represent the meaning of text, images or other data. To search, you embed the question and ask for the stored vectors nearest to it. This is the retrieval step in systems that answer questions about your own documents.

Two good options on a small VPS are Qdrant, a dedicated vector database, and pgvector, an extension that adds vector search to PostgreSQL. Choose pgvector if you already run PostgreSQL and have modest amounts of data. Choose Qdrant for a separate, purpose-built service.

Option 1: run Qdrant in Docker

Qdrant publishes an official image. Bind its HTTP port to localhost, store its data in a volume and set an API key, so any process that can reach the port still needs a secret. Install Docker first (see the Docker guide).

Both systems
mkdir -p ~/qdrant && cd ~/qdrant
(umask 077; printf "QDRANT_API_KEY=%s\n" "$(openssl rand -hex 24)" > .env)
cat > compose.yaml <<'EOF'
services:
  qdrant:
    image: qdrant/qdrant:latest
    ports:
      - "127.0.0.1:6333:6333"
    environment:
      - QDRANT__SERVICE__API_KEY=${QDRANT_API_KEY}
    volumes:
      - qdrant_storage:/qdrant/storage
    restart: unless-stopped

volumes:
  qdrant_storage:
EOF
sudo docker compose up -d
sudo docker compose ps

Try it with a tiny dataset

Create a collection that holds four-number vectors and add three points. Real embeddings are much longer, such as 768 numbers, but the commands are the same. Then search for the point nearest to a query vector. The result lists the stored ids with scores.

Both systems
cd ~/qdrant; set -a; . ./.env; set +a
curl --fail --show-error -s -X PUT http://127.0.0.1:6333/collections/demo -H "api-key: $QDRANT_API_KEY" -H "Content-Type: application/json" -d '{"vectors":{"size":4,"distance":"Cosine"}}'
curl --fail --show-error -s -X PUT http://127.0.0.1:6333/collections/demo/points?wait=true -H "api-key: $QDRANT_API_KEY" -H "Content-Type: application/json" -d '{"points":[{"id":1,"vector":[0.9,0.1,0.1,0.1],"payload":{"text":"cats"}},{"id":2,"vector":[0.1,0.9,0.1,0.1],"payload":{"text":"servers"}},{"id":3,"vector":[0.1,0.1,0.9,0.1],"payload":{"text":"tea"}}]}'
curl --fail --show-error -s -X POST http://127.0.0.1:6333/collections/demo/points/search -H "api-key: $QDRANT_API_KEY" -H "Content-Type: application/json" -d '{"vector":[0.15,0.85,0.1,0.1],"limit":2,"with_payload":true}'

Option 2: add pgvector to PostgreSQL

On Ubuntu 24.04 the extension is a package that matches the installed PostgreSQL version (16). Install it, then enable it in the database that needs it. On AlmaLinux the default repositories do not include pgvector, so use the PostgreSQL community repository as its documentation describes, or use the Qdrant option.

Ubuntu only
sudo apt install postgresql postgresql-16-pgvector
sudo -u postgres createdb vectors
sudo -u postgres psql -d vectors -c "CREATE EXTENSION vector;"
Ubuntu only: a small search example
sudo -u postgres psql -d vectors <<'EOF'
CREATE TABLE items (id serial PRIMARY KEY, text text, embedding vector(4));
INSERT INTO items (text, embedding) VALUES ('cats', '[0.9,0.1,0.1,0.1]'), ('servers', '[0.1,0.9,0.1,0.1]'), ('tea', '[0.1,0.1,0.9,0.1]');
SELECT text, embedding <=> '[0.15,0.85,0.1,0.1]' AS distance FROM items ORDER BY distance LIMIT 2;
EOF
The operator <=> measures cosine distance, and smaller means closer. Add an index (HNSW or IVFFlat) once you have thousands of rows, as the pgvector documentation explains.

Make real embeddings with Ollama

An embedding model turns text into the vectors you store. A small, popular one is nomic-embed-text, which makes 768-number vectors and runs fine on a CPU. Create a collection with that size when you store real data, and always embed questions with the same model as the documents.

Both systems
ollama pull nomic-embed-text
curl -s http://127.0.0.1:11434/api/embed -d '{"model":"nomic-embed-text","input":"A reverse proxy forwards requests."}' | head -c 200

Keep it private and backed up

Neither service needs a public port. Keep Qdrant on 127.0.0.1 and PostgreSQL on localhost, and reach them from your applications on the same server or through an SSH tunnel. Back up the Qdrant volume or use its snapshot feature, and back up PostgreSQL with a dump as in the database guide. Vectors can be rebuilt from the source documents, so keep those safe too.

Check what is listening
sudo ss -lntp | grep -E "6333|5432"
Qdrant: create a snapshot of a collection
cd ~/qdrant; set -a; . ./.env; set +a
curl --fail --show-error -s -X POST http://127.0.0.1:6333/collections/demo/snapshots -H "api-key: $QDRANT_API_KEY"
RECORDED LAB OUTPUT · AlmaLinux 9
{"result":true,"status":"ok","time":0.0072395}{"result":true,"status":"ok","time":0.164724631}Restored vector search top result: servers
Recorded search after deleting the collection and recovering an uploaded snapshot. The tiny lab dataset returned its expected top match.

Clean up the test data

Delete the demo collection or table when you are done, so it does not mix with real data.

Qdrant
cd ~/qdrant; set -a; . ./.env; set +a
curl --fail --show-error -s -X DELETE http://127.0.0.1:6333/collections/demo -H "api-key: $QDRANT_API_KEY"
pgvector
sudo -u postgres psql -d vectors -c "DROP TABLE items;"

Official documentation

These upstream references provide more detail on the commands and workflows in this guide.

KEEP LEARNING

Related GLWS guides

Browse all guides