How to Set Up Headscale on a VPS

Suppose you have several devices—a home computer, a laptop, a smartphone, and perhaps a server in a data center. You want them to communicate directly, securely, without intermediaries, regardless of each device's IP address or the router it sits behind. This is the problem solved by modern mesh solutions built on the WireGuard protocol.

One of the most convenient such solutions is Tailscale—a service that lets you assemble devices into a private network in just five minutes. However, Tailscale has a catch: its coordination server (control plane) is hosted by the company, and the free tier limits the number of devices. For many, this is acceptable, but if you want full control over your network, avoid reliance on a third‑party service, and plan to connect tens or hundreds of devices—you need Headscale.

Headscale is an open‑source implementation of Tailscale's coordination server that you can run on your own server. In essence, you get all the benefits of Tailscale (simple clients, automatic NAT traversal, WireGuard encryption) while the control centre remains under your control. The best way to deploy it is to use a VPS (Virtual Private Server) with a public IP address.

In this article, we will thoroughly explain what Headscale is, how it works, why you should run it on a VPS, and walk you through the entire installation and configuration process step by step. You will learn about available deployment options, typical pitfalls, and how to avoid them. The article is aimed at a broad audience—even if you are not a professional system administrator, following these instructions will get you up and running.

What Is Headscale and Why Do You Need It

To understand Headscale, you first need to grasp how Tailscale works. Tailscale builds mesh networks on top of the WireGuard protocol. WireGuard is a modern, high‑speed VPN protocol, but in its “raw” form it requires manual configuration of each device: generating keys, writing them into configuration files, specifying peer IP addresses. Once you have more than two or three devices, this process becomes a headache.

Tailscale solves this with a coordination server (control plane). This server handles:

• device registration;
• exchange of public keys;
• connection coordination;
• access policy distribution (ACL);
• NAT traversal using DERP relays.

Data transfer itself occurs directly between devices over WireGuard—not a single byte of your traffic passes through the coordination server. The coordination server is only needed for devices to “meet” each other.

Headscale is an open‑source, self‑hosted implementation of exactly that coordination server. You install Headscale on your server (e.g., a VPS), and all your devices connect to it instead of to Tailscale Inc.'s servers. At the same time, you use the same official Tailscale clients on all devices—they are fully compatible.

In other words, Headscale allows you to replace Tailscale's cloud coordination server with your own.

Key features of Headscale:

• Full compatibility with official Tailscale clients (Windows, macOS, Linux, Android, iOS).
• User and device management via command‑line interface (CLI) or a web interface (Headscale-UI).
• Support for ACLs (Access Control Lists) for fine‑grained permissions between devices.
• Built‑in MagicDNS—devices are accessible by name, not IP.
• Ability to use your own DERP relays for NAT traversal.
• Support for PostgreSQL and SQLite as the database.
• Unlimited users and devices.

How Headscale Works: Architecture and Principles

To better understand what we are configuring, let's break down the Headscale architecture. It consists of three main layers.

Layer 1: Coordination server (Headscale)

This is the server application you run on a VPS. It provides the API with which Tailscale clients interact. It stores:

• information about registered devices;
• public WireGuard keys;
• assigned IP addresses from a pool (typically from the 100.64.0.0/10 range);
• access policies (ACL);
• user and namespace data.

Headscale can use SQLite (for small deployments) or PostgreSQL (for production environments). Configuration is defined in a config.yaml file.

Layer 2: Clients (Tailscale)

On each device you want to join the network, you install the official Tailscale client. When started, the client:

1. Receives the address of your Headscale server (you specify it during connection).
2. Generates a WireGuard key pair.
3. Sends its public key to Headscale for registration.
4. Gets from Headscale the list of other devices in the network and their public keys.
5. Establishes direct WireGuard connections with those devices.

Important: the Tailscale client does not need to be modified—it “thinks” it is communicating with the official Tailscale server because Headscale implements the same protocol.

Layer 3: Data transfer (WireGuard P2P)

After devices have “met” via Headscale, they set up direct encrypted connections with each other using WireGuard. Traffic goes directly, bypassing your VPS. If a direct connection is impossible (e.g., both devices are behind strict NAT), DERP relays come into play—special intermediary servers that relay traffic. You can use Tailscale's public DERP servers or deploy your own.

Thus, Headscale is only a “dispatcher” that orchestrates connections but does not participate in data transfer. This ensures high performance and low load on your VPS.

Why Headscale Needs a VPS

Headscale is a server application that requires constant Internet access with a public IP address. Tailscale clients must be able to reach it at any time to update network information, obtain keys, or authenticate.

Running Headscale on a home computer or a Raspberry Pi behind a home router is not a good idea because:

• Home IP addresses can change (dynamic IP).
• The router may lose power or reboot.
• The ISP may block incoming connections on certain ports.
• Home Internet connections may be unstable.

VPS (Virtual Private Server) is the ideal place for Headscale because:

• It has a static public IP address.
• It runs 24/7 with guaranteed availability.
• You have full control over the operating system and network settings.
• The cost of a basic VPS is low—for Headscale, 1 vCPU and 512 MB – 1 GB of RAM are sufficient.

To deploy Headscale, you can rent a VPS server from a provider, for example, Serverspace. This is a reliable and easy‑to‑manage infrastructure that allows you to launch the coordination server in minutes. Serverspace's basic plans are perfect for Headscale: minimal resources are required, while stability and availability are ensured at the data‑centre level.

Cloud Servers from €4 / moIntel Xeon Gold 6254 3.1 GHz CPU, SLA 99,9%, 100 Mbps channeltry

Step‑by‑Step Guide to Installing Headscale on a VPS

Now let's get to the main part—practical installation. We will cover two primary ways to deploy Headscale on a VPS: using an automated script (the quickest method) and using Docker Compose (more flexible and convenient for management). Choose the one that suits you best.

Before you start, make sure you have:

• A VPS running Ubuntu 22.04 or 24.04 (recommended).
• A domain or subdomain with an A record pointing to your VPS IP address (e.g., headscale.your-domain.com). This is required for HTTPS.
• SSH access to the VPS with root or sudo privileges.

Option 1: Installation Using an Automated Script

The simplest way to deploy Headscale is to use a community‑provided script. It automatically downloads the binary, sets up a systemd service, opens firewall ports, and creates the first user.

Run the following commands on your VPS:

wget -O headscale.sh https://get.vpnsetup.net/hs
sudo bash headscale.sh --auto --serverurl https://headscale.your-domain.com

Replace https://headscale.your-domain.com with your actual domain. If you omit --serverurl, the script will use the public IP and HTTP, which is not recommended for production use.

The script will perform all necessary actions: install Headscale, set it up as a system service, generate a configuration file, and start the server. After completion, you will receive the first user's name (default admin) and a command to generate a pre‑authorization key.

This method is good for a quick start, but it is less flexible than the Docker option. If you want more control over the configuration, use the second method.

Option 2: Installation Using Docker Compose

Docker Compose allows you to describe the entire infrastructure in one file and manage it with simple commands. This approach is especially convenient if you also plan to deploy a web interface (Headscale-UI) or a reverse proxy for automatic HTTPS.

Step 1: Install Docker and Docker Compose on the VPS

sudo apt update
sudo apt install -y ca-certificates curl
sudo install -m 0755 -d /etc/apt/keyrings
sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc
sudo chmod a+r /etc/apt/keyrings/docker.asc
echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu $(. /etc/os-release && echo "$VERSION_CODENAME") stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt update
sudo apt install -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin

These commands add the official Docker repository and install the required packages.

Step 2: Create the directory structure

mkdir -p /opt/headscale/config
cd /opt/headscale

Step 3: Create the Headscale configuration file

Create /opt/headscale/config/config.yaml with the following minimal content (adapt to your domain):

server_url: https://headscale.your-domain.com
listen_addr: 0.0.0.0:8080
private_key_path: /var/lib/headscale/private.key
noise:
  private_key_path: /var/lib/headscale/noise_private.key
ip_prefixes:
  - 100.64.0.0/10
  - fd7a:115c:a1e0::/48
derp:
  server:
    enabled: false
  urls:
    - https://controlplane.tailscale.com/derpmap/default
database:
  type: sqlite3
  sqlite:
    path: /var/lib/headscale/db.sqlite

This is a basic configuration. Headscale will use SQLite as the database and Tailscale's public DERP servers for NAT traversal.

Step 4: Create the docker-compose.yml file

Create /opt/headscale/docker-compose.yml:

version: '3'
services:
  headscale:
    image: headscale/headscale:latest
    container_name: headscale
    restart: unless-stopped
    volumes:
      - ./config:/etc/headscale
      - ./data:/var/lib/headscale
    ports:
      - "8080:8080"
    command: headscale serve

This file describes a single service—Headscale. It mounts the local config and data folders into the container, where the configuration and database are stored respectively.

Step 5: Start Headscale

docker compose up -d

The container will start in the background. Check that it is running:

docker compose ps

Step 6: Set up HTTPS using a reverse proxy

For secure operation, Tailscale clients require HTTPS. The simplest way is to use Caddy or Nginx as a reverse proxy. Caddy can automatically obtain SSL certificates from Let's Encrypt.

Create a Caddyfile in a separate container or install Caddy on the host system. Example minimal Caddy configuration:

headscale.your-domain.com {
    reverse_proxy localhost:8080
}

Start Caddy, and it will automatically issue a certificate for your domain.

After this, Headscale will be available at https://headscale.your-domain.com.

Creating the First User and Connecting a Device

After installing Headscale (by any method), you need to create a user (in Headscale terminology, a namespace) and generate a key for device connection.

Create a user:

docker compose exec headscale headscale users create myuser

Replace myuser with any name.

Generate a pre‑authorization key (pre‑auth key):

docker compose exec headscale headscale preauthkeys create --user myuser --expiration 24h

You will get a key like abcdef123456.... Copy it.

Connecting different types of devices:

For computers (Linux, macOS, Windows with WSL or command line):

Install the official Tailscale client and run:

tailscale up --login-server https://headscale.your-domain.com --auth-key THE_KEY_YOU_GOT

The device will register with your Headscale and appear in the node list.

For smartphones (Android / iOS):

Mobile platforms have no command line, so connection is done through the graphical interface:

• Android: install the Tailscale app from Google Play or F‑Droid. Open the menu (three dots) → “Use an alternate server” → enter your Headscale URL. Then, in the same menu, select “Use an auth key” and paste the generated key.
• iOS: install the Tailscale app from the App Store. Make sure you are not logged in. Open system Settings → Tailscale → in the “ALTERNATE COORDINATION SERVER URL” field, enter your Headscale URL. Close and reopen the app, then tap “Sign in” — authentication will be performed via your server.

After successful registration, the device will appear in the node list, just like computers.

To list connected devices, run:

docker compose exec headscale headscale nodes list

Pros, Cons, and Limitations of Headscale

Like any solution, Headscale has its strengths and weaknesses. It is important to understand them before deciding to deploy.

It is worth noting that while a single Headscale instance serves one network (tailnet), you can use namespaces for logical separation of devices and emulate isolation between user groups. This allows you to organise, for example, separate spaces for development, testing, and production within a single server.

Practical Use Cases for Headscale

Headscale fits a wide variety of tasks. Here are several typical scenarios where it proves especially useful.

1. Secure access to a home lab (homelab)

You have a home server with Docker, a Raspberry Pi, or an old computer running various services (e.g., Plex, Home Assistant, Grafana). You want to access them from anywhere but do not want to open ports to the Internet. Headscale lets you bring all devices into a private network, and you can connect to home services via their internal IP addresses from anywhere, as if you were at home.

2. Secure connection of multiple VPS and cloud servers

If you have several VPS instances in different data centres or from different providers, Headscale allows you to unify them into one network. This simplifies administration: you can copy files between servers, set up monitoring, use shared databases—all over encrypted channels, without the need to expose SSH to the Internet.

3. Corporate private network for a small team

For a small company or startup, Headscale can be an excellent alternative to corporate VPN solutions. You can deploy the coordination server, grant each employee access to the private network, and configure ACLs so that, for example, developers have access to development servers while managers only access the CRM system. All data remains under your control, and the number of users is unlimited.

4. Remote access to IoT devices

If you have many IoT devices (cameras, sensors, controllers) spread across different locations, Headscale allows you to combine them into one network and manage them remotely. The devices may be behind NAT, but thanks to built‑in NAT traversal, they will be accessible from anywhere.

5. Private network for development and testing

Developers often use isolated environments for application testing. Headscale lets you create a private network containing only test servers and developers' workstations. This isolates the test environment from production networks and the Internet, enhancing security.

Common Mistakes When Installing and Configuring Headscale (and How to Avoid Them)

Even when following instructions, problems can arise. Here are the most frequent errors and how to fix them.

Error: client cannot connect to Headscale (“context deadline exceeded”)

Cause: the Tailscale client cannot reach your Headscale server. Often this is due to missing HTTPS or an incorrect URL.

Solution: ensure you are using https:// in the server address. Check that your domain resolves correctly to the VPS IP. Open port 443 (or the port your reverse proxy uses) in the firewall. Verify that Headscale is actually listening on the required port and that the reverse proxy (Caddy/Nginx) correctly forwards traffic.

Error: “failed to load config: open config.yaml: no such file or directory”

Cause: Headscale cannot find the configuration file. This often happens with DEB package installations or manual runs.

Solution: ensure the configuration file is in the correct location. For DEB installation it is /etc/headscale/config.yaml. For Docker, make sure you mounted the volume with the configuration correctly. Check file permissions.

Error: “noise handshake failed: decrypting machine key”

Cause: an error when updating Headscale to a new version—clients cannot establish a connection due to protocol or key changes.

Solution: before updating Headscale, read the release notes. In some cases, devices need to be re‑registered. To avoid this issue, always back up the database before updating and test on non‑critical devices.

Error: port already in use (“address already in use”)

Cause: another process is listening on the same port as Headscale (default 8080).

Solution: change the port in the Headscale configuration (parameter listen_addr) or stop the conflicting process. If you use a reverse proxy, Headscale may listen only on localhost (127.0.0.1:8080) to avoid conflicts with other services.

Error: device connects but does not appear in the node list

Cause: the pre‑authorization key may have expired, or the device did not complete registration.

Solution: generate a new key with a sufficient expiration period (e.g., --expiration 720h for 30 days) and retry the connection. Make sure you are using the correct --login-server URL.

Comparison of Headscale Deployment Methods

To help you choose the right installation method, here is a comparison table.

Frequently Asked Questions (FAQ)

Conclusion: Should You Deploy Headscale and What to Do Next

Headscale is a powerful and flexible tool for building your own private mesh network. It is ideal for those who value control over their data, do not want limits on the number of devices, and are willing to invest a little time in initial configuration.

Who should choose Headscale:

• Self‑hosting enthusiasts and home lab owners.
• Small companies needing a corporate VPN without a subscription fee.
• Developers testing distributed systems.
• Anyone wanting to connect their VPS, cloud servers, and home devices into a single secure network.

Who might prefer to stay with Tailscale:

• Those who do not want to administer a server and prefer an all‑in‑one solution.
• Users with a small number of devices (up to 100) who are satisfied with the free tier.
• Those who need ready‑made integrations with SSO and enterprise features out of the box.

If you decide to try Headscale, start by renting a VPS from a reliable provider, such as Serverspace. Choose a basic plan with Ubuntu 22.04, set up your domain, and follow our step‑by‑step guide. The entire process should take no more than 30–40 minutes, and the result—a fully controlled and secure private network—will repay the time invested.

After successful deployment, you can expand the infrastructure: install the Headscale-UI web interface for convenient management, configure ACLs to restrict access between users, and connect your own DERP relays to optimise NAT traversal. Further customisation depends on your needs and technical expertise.

Start building your private mesh network today—with Headscale and a VPS from Serverspace, it is easier than you might think.