How to Deploy Open WebUI and Ollama on a Server

Local language models are quietly moving from a hobbyist toy to a working tool for businesses and developers. The reasons are straightforward: conversations stay on your own infrastructure, costs are tied to a server rather than to token counts, and access to the chatbot doesn't depend on an external API's rate limits. Ollama and Open WebUI form one of the most convenient pairings for this kind of setup: one handles loading and running models, the other provides a clear interface on top of them.

Let's walk through how to set up this pair on a regular server: what resources you'll need, which steps to follow, and which mistakes are worth avoiding. This guide assumes you've configured servers before, but not necessarily worked with local models — we'll cover the basics along the way.

What Ollama and Open WebUI Actually Are

Ollama is a runtime for large language models: it downloads, runs, and serves them through a simple REST API. It works as a background service — it accepts a request, loads the needed model into memory, and returns a response. Ollama's model library includes dozens of open models, from compact 2–3 billion parameter versions to large 70-billion-plus ones, memory permitting.

Open WebUI is a web interface for talking to models, visually similar to familiar chatbots. It connects to Ollama over the network and adds multi-user access, conversation history, document upload for retrieval-based search, and system prompt configuration. The project was originally called Ollama WebUI, then expanded support to other backends with an OpenAI-compatible API. Put simply, Ollama is the engine that runs models, and Open WebUI is the storefront on top of it that people actually interact with.

In this setup, the server is just a regular VPS: it stores models on disk, allocates RAM and CPU for them, and serves the web interface to users through a browser.

Ubuntu VPS server Virtual private servers
with preinstalled Ubuntu.
Starting from €4Get Ubuntu VPS

How It Works: Architecture and Step-by-Step Deployment

The architecture is simple: Ollama runs as a separate service on port 11434, Open WebUI runs on its own port and talks to Ollama over the internal network, and the user connects through a regular HTTPS address in their browser. For isolation and easier management, both components are typically run in Docker containers, with a reverse proxy — nginx or Caddy — handling incoming traffic.

Step 1: Preparing the Server

Before installing anything, it's worth deciding which model you plan to run, since that determines how much memory you'll need. Compact 7–8 billion parameter models in quantized form typically need somewhere around 6–10 GB of RAM depending on quantization level and context length; 13-billion-parameter models start higher, and 30-billion-plus versions without a GPU aren't realistic on most servers. Give yourself headroom on disk too: models range from 2 GB to 40+ GB each, and disk space disappears fast if you experiment with several.

For most use cases — an internal chatbot, model testing, a team assistant — a server with 4–8 cores, 16 GB RAM, and an SSD of 60 GB or more is a reasonable baseline. Since actual memory use depends heavily on quantization and context length rather than a single fixed number, it's worth checking real usage with ollama ps after loading a model rather than relying on a table alone. This is where Serverspace VPS servers come in handy — plans scale RAM and disk easily if a model turns out to be more demanding than expected.

Step 2: Installing Docker and Docker Compose

Next comes standard environment setup. On a clean Ubuntu server, this takes a couple of commands:

sudo apt update && sudo apt upgrade -y
curl -fsSL https://get.docker.com | sh
sudo usermod -aG docker $USER
sudo apt install docker-compose-plugin -y

After adding your user to the docker group, log back in over SSH so the change takes effect without needing sudo.

Step 3: Launching Ollama and Open WebUI

Both services can be described in a single docker-compose.yml:

version: "3.8"
services:
  ollama:
    image: ollama/ollama
    container_name: ollama
    restart: unless-stopped
    volumes:
      - ollama_data:/root/.ollama
    ports:
      - "11434:11434"

  open-webui:
    image: ghcr.io/open-webui/open-webui:main
    container_name: open-webui
    restart: unless-stopped
    depends_on:
      - ollama
    environment:
      - OLLAMA_BASE_URL=http://ollama:11434
    volumes:
      - webui_data:/app/backend/data
    ports:
      - "3000:8080"

volumes:
  ollama_data:
  webui_data:

This file defines two services. Ollama stores models in a separate volume so they survive container rebuilds, and Open WebUI gets Ollama's address through an environment variable and listens on port 3000 externally. Everything starts with a single command:

docker compose up -d

After about thirty seconds you can check that both containers are running:

docker ps

Step 4: Downloading a Model

Models are pulled with a command inside the Ollama container. For example, for Llama 3 8B:

docker exec -it ollama ollama pull llama3

The process takes anywhere from a few minutes to half an hour, depending on model size and network speed. To see which models are already downloaded:

docker exec -it ollama ollama list

Which Model to Choose

It's worth noting that model choice directly affects response speed and server requirements. Below are a few popular options from Ollama's library, along with approximate memory ballparks — actual usage depends on quantization level and context length, so treat these as a starting point rather than a hard number.

These numbers assume CPU-only inference with a moderate context window. Longer conversations and higher-precision quantization push actual usage up — always confirm with ollama ps after loading rather than trusting a table alone.

Step 5: Setting Up a Domain and HTTPS

Exposing the interface over plain HTTP is a bad idea — passwords and conversations would travel unencrypted. Put nginx in front of the service:

server {
    listen 80;
    server_name chat.example.com;

    location / {
        proxy_pass http://127.0.0.1:3000;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
    }
}

Then issue a certificate with a single certbot command:

sudo certbot --nginx -d chat.example.com

After that, Open WebUI becomes available over https, and certbot will renew the certificate automatically.

Step 6: Firewall and First Run

Ollama's port shouldn't face the internet directly — it only needs to be reachable by Open WebUI inside the container network. Externally, open only 22, 80, and 443:

sudo ufw allow 22
sudo ufw allow 80
sudo ufw allow 443
sudo ufw enable

From here you can visit https://chat.example.com, create the first user — who automatically gets admin rights — and pick a model from the ones you've downloaded.

Advantages and Disadvantages

Is it worth deploying a model on your own server when a cloud API is already available? If data control and predictable costs matter to you, yes — here's why.

• Full control over data: conversations and uploaded documents never leave your infrastructure.
• Predictable costs: you pay for the server, not per request token.
• Works without access to external APIs — suitable for isolated networks.
• Freedom to choose the model, with the option to keep several available at once.
• Flexible control over the interface, prompts, and user permissions.

There's a flip side too, and it's worth factoring in early.

• Open models still lag behind top commercial APIs on complex tasks.
• You need a server with headroom on memory and CPU — larger models struggle on weak configurations.
• Keeping Ollama and Open WebUI versions compatible is something you have to track yourself.
• Security and backups are entirely your responsibility.

Limitations and Risks

On CPU without a GPU, models above 13 billion parameters respond noticeably slower — a long answer can take tens of seconds. If speed matters, it's worth looking at GPU-equipped configurations or sticking to compact models.

An open Ollama port without authentication is a common source of trouble: if the API ends up reachable from outside, anyone can load the server and burn through its resources. A reverse proxy combined with firewall-level access restrictions closes this gap.

Licensing is worth checking too: some models only permit research use, others restrict commercial use beyond a certain audience size. Before production use, it's worth checking a model's card in the Ollama library separately.

Finally, self-hosting means self-support: if the service goes down, your own team has to bring it back — unlike cloud APIs with a guaranteed SLA.

Practical Use Cases

An Internal Team Assistant

A company deploys Open WebUI on its own server and connects employees through a shared domain. Everyone gets a separate account, conversation history stays separate per user, and sensitive material — code, financial data, contract drafts — never leaves the infrastructure.

A Sandbox for Choosing a Model

Before embedding an LLM into a product, developers spin up several models on one server and compare answer quality on real examples. Ollama lets you keep 3–4 models around at once and switch between them without reinstalling anything.

A Knowledge Base with Document Search

Open WebUI can index internal policies, technical documentation, or a support ticket archive, then search across them on every query. This handles first-line support without sending documents to an external service.

Learning and Experimentation

Students and junior developers use this pairing for practice: prompt tuning, comparing model architectures, exploring how the API behaves — the whole stack can be set up on a budget server in one evening, without bumping into the free-tier limits of external services.

A Backup Channel for External API Outages

If a business process depends on a cloud API and occasionally runs into rate limits or downtime, a local server running Ollama and Open WebUI works as a fallback channel: simple requests can be routed there without stopping the whole process.

Common Mistakes

Conclusion

The Ollama and Open WebUI pairing handles quick local chatbot deployment without relying on third-party APIs. The upside is data control and predictable costs; the downside is having to track server resources and component updates yourself.

Getting started only takes Docker, one docker-compose.yml, and a server with enough memory headroom for your chosen model — from there, the setup scales: add models, hook up search over your own documents, connect multiple users. As load grows, server resources are easy to scale up without migrating to new infrastructure — for example, through the Serverspace VPS control panel, where memory and disk change without reinstalling the system. For more hands-on server deep-dives, check out the Serverspace blog.

Cloud Servers from €4 / moIntel Xeon Gold 6254 3.1 GHz CPU, SLA 99,9%, 100 Mbps channeltry

Frequently Asked Questions (FAQ)