How to Build a ChatGPT-Like AI Assistant on a VPS

Ask ten people why they want their own AI assistant instead of a ChatGPT subscription, and you'll get ten different answers. Some are worried about sending internal documents to a third-party API. Others have hit a rate limit at the worst possible moment. A few just want to stop paying a recurring fee for something that, on paper, they could run themselves. All three reasons point to the same solution: a self-hosted assistant running on a virtual private server you control.

This is more achievable than it sounds. You don't need a rack of GPUs or a machine learning degree. A properly sized VPS, an open-source inference engine, and a web interface are enough to get a genuinely useful chat assistant running in an afternoon. What follows is a practical walkthrough: what the pieces are, how they fit together, what it costs in resources, and where people typically get it wrong. It's written for anyone comfortable with a terminal — you don't need prior experience with machine learning specifically, just a willingness to follow commands and read error messages when something doesn't start on the first try.

What You're Actually Building

Before touching a terminal, it helps to know what each piece does. A large language model (LLM) is the "brain" of the assistant — a file containing billions of numeric parameters that were trained to predict text. Inference is simply the process of running that model to generate a response to your prompt. Quantization compresses those parameters into smaller data types, trading a small amount of precision for a large reduction in memory use — it's the difference between a model needing 140 GB of RAM and needing 40 GB.

An inference engine (Ollama, vLLM, LocalAI, and similar tools) loads the model and exposes it through an API, usually one compatible with the same format OpenAI uses. A chat interface like Open WebUI sits on top of that API and gives you the familiar browser-based chat window, complete with conversation history and multiple user accounts. The VPS is just the rented computer everything runs on — and this is the one variable you have full control over.

Put together, the stack looks like this: VPS → Docker → inference engine → model → chat interface → your browser. Every layer is replaceable, which is part of the appeal.

Step 1: Choose and Size the VPS

This is where most projects succeed or fail before a single line of code gets written. Model weights have to fit in memory, and there's no way around that requirement — if a model doesn't fit, it either refuses to load or crawls at unusable speed. The table below reflects real-world figures rather than theoretical minimums, since the operating system, Docker, and the web interface all take a share of RAM on top of the model itself.

Treat these numbers as a starting range rather than a hard rule — actual memory use shifts with context window length, the number of concurrent users, and which quantization level you pick, so it's worth checking your chosen model's documentation before committing to a plan. A CPU-only VPS handles the 3B–13B range reasonably well for a single user; anything larger becomes noticeably slow without a GPU-backed instance. If you're not sure where to start, a mid-tier configuration — something in the 8–16 vCPU, 32 GB RAM range — covers most single-team use cases without over-provisioning. Serverspace's VPS server line-up lets you configure vCPU, RAM, and SSD independently through a calculator, which is convenient here: you can start modestly and scale RAM up later if a bigger model turns out to be worth it, without rebuilding the server from scratch.

Step 2: Prepare the Server

Once the server is running, update the system and install Docker — nearly every piece of this stack ships as a container, which keeps dependencies isolated and makes cleanup trivial if something goes wrong.

Log out and back in for the group change to apply, then confirm Docker is working with docker --version. It's a small step, but skipping it is a common reason people end up running everything as root out of frustration later on.

Step 3: Install an Inference Engine

This is the component that actually loads and runs the model. Several mature options exist, and the right pick depends mostly on your hardware and how many people will use the assistant at once.

For a first build, Ollama is the pragmatic choice — installation is a single command, and it handles model downloads and quantization selection automatically.

If that last command gives you a working conversation in the terminal, the hard part is already done. Everything from here is about making it accessible and pleasant to use.

Step 4: Add a Chat Interface

A terminal prompt isn't what anyone pictures when they say "ChatGPT-like." Open WebUI closes that gap: it's a self-hosted, browser-based interface that talks to Ollama (or vLLM, or LocalAI) and adds conversation history, multiple accounts, and document upload for retrieval-augmented answers.

Point a browser at http://your-server-ip:3000, create an admin account, and the model you pulled earlier should already appear in the model selector. At this point you have something that looks and feels like ChatGPT, running entirely on infrastructure you rent and control.

Step 5: Put a Domain and HTTPS in Front of It

Exposing port 3000 directly is fine for testing, but not for anything you'll actually rely on. A reverse proxy such as Nginx or Caddy, paired with a free certificate from Let's Encrypt, gets you a proper domain name and encrypted traffic with very little configuration:

Caddy handles certificate issuance and renewal automatically once that file is in place and the domain points at your server's IP address. It's a five-minute step that saves a great deal of trouble later, particularly once you add authentication and want cookies to behave correctly over a real domain.

Step 6: Harden Access Before You Rely on It

A working assistant behind a domain name is still an open invitation if nothing is guarding it. Before pointing coworkers or customers at the URL, a few small changes go a long way. Start by disabling public sign-ups in Open WebUI's admin settings, so the first person to stumble on the address can't create an account and start burning your server's CPU cycles. Next, restrict the raw inference API — port 11434 for Ollama, 8000 for a typical vLLM deployment — to localhost only, so it's reachable through the reverse proxy but never directly from the internet:

Adding fail2ban to watch the reverse proxy's access logs catches repeated login attempts and blocks the offending IP automatically, which matters more than it might seem — a public chat login page attracts automated probing within days of going live, not months. None of this takes more than twenty minutes, and doing it before launch is considerably less painful than doing it after an incident.

Advantages and Trade-offs

The case for self-hosting is straightforward on paper: your prompts and documents never leave your server, costs stay flat regardless of usage volume once the VPS is paid for, and you can swap models freely as better open-weight releases appear. There's also no dependency on a provider's uptime, pricing changes, or usage policy updates.

None of that comes free, though. Open-weight models at sizes that fit comfortably on a mid-range VPS still trail the largest hosted models on complex reasoning and long-context tasks — the gap has narrowed considerably, but it hasn't closed. You're also taking on the maintenance that a hosted provider would otherwise handle: security patches, backups, and monitoring all become your responsibility. And unlike a metered API, a VPS costs the same at 2 a.m. with zero users as it does at peak load, so the economics only favor self-hosting once usage is consistent enough to justify a dedicated server.

Limitations and Risks

A few risks deserve more attention than they usually get. First, an inference API left open on a public IP without authentication is a real and common mistake — anyone who finds the port can run inference on your server for free, and you pay the bill. Second, CPU-only inference is slower than most people expect going in; a 13B model on a modest CPU might produce only a handful of tokens per second, which feels sluggish compared to a commercial chatbot. Third, self-hosted doesn't automatically mean compliant — if you're handling regulated data, you still need to think through retention, logging, and access control just as carefully as you would with any other system holding sensitive information. Finally, models and tooling in this space move quickly; a setup that works well today may need occasional updates to stay compatible with newer model formats. None of these are reasons to avoid self-hosting — they're simply the maintenance cost that replaces the monthly subscription fee, and it's worth budgeting a small amount of ongoing attention rather than treating the server as something you set up once and never revisit.

Practical Scenarios

A few situations where this setup tends to earn its cost quickly:

1. Internal knowledge assistant. Point Open WebUI's document upload at your company wiki or policy documents, and staff get answers without those documents ever touching an external API.
2. Customer-facing chat widget. The OpenAI-compatible API that Ollama and vLLM expose can sit behind a simple website widget, giving visitors a chat experience without a per-message fee.
3. Personal coding assistant. Tools like Continue or Cody can point at your VPS instead of a cloud API, which is useful for anyone working with proprietary codebases.
4. Document analysis and RAG. Feeding contracts, reports, or research papers into a retrieval pipeline lets the assistant answer questions grounded in your own material rather than general training data.
5. Small team shared assistant. Open WebUI's multi-user support means a five-person team can share one server and one bill instead of five separate subscriptions.

Common Mistakes to Avoid

• Renting a VPS sized for the model's file size alone, forgetting that the OS, Docker, and the interface all need headroom too.
• Expecting a 70B model on a CPU-only server to feel like a commercial chatbot — it won't, and a smaller model with GPU acceleration is usually the better trade.
• Leaving the Ollama or vLLM API bound to 0.0.0.0 with no firewall rule or authentication in front of it.
• Skipping HTTPS because "it's just for internal use," then later exposing the same URL to a wider audience without revisiting that decision.
• Not backing up the Docker volumes holding chat history and configuration — a server rebuild without a backup means starting over from zero.

Conclusion

Building a private, ChatGPT-like assistant is no longer an exotic project reserved for teams with a machine learning background. A correctly sized VPS, Docker, an inference engine like Ollama, and a chat interface like Open WebUI are enough to get a working assistant online in under an hour, with a reverse proxy and HTTPS added shortly after. The trade-offs are real — you give up some raw model quality and take on some operational responsibility — but for anyone who values keeping data in-house or wants predictable monthly costs, those trade-offs are usually worth making. Start with a modest configuration and a mid-sized model, and scale the server up if the assistant proves its worth.

Frequently Asked Questions (FAQ)