Install Ollama Using Docker Compose

This article is for those looking for a detailed and straightforward guide on installing Ollama using Docker Compose.

Ollama is a streamlined, modular framework designed for developing and operating language models locally.

TIP

Architecture Context

Choose self-hosted Ollama when you need local LLM inference with full data privacy — no prompts or responses leave your infrastructure. The OpenAI API or AWS Bedrock provide managed alternatives with larger model selection and zero GPU maintenance. Self-hosting is justified when data sensitivity prohibits external API calls or when predictable inference costs outweigh the capital expense of GPU hardware.

💾 You can find the repository used in this guide on GitHub.

NOTE

We’ll use Traefik as our reverse proxy. It’ll handle obtaining cryptographic certificates from Let’s Encrypt for your domain names and route requests to the corresponding services based on those domains.

CAUTION

To obtain cryptographic certificates, you will need A-type records in the external DNS zone, which point to the IP address of your server where Traefik is installed. If you have created these records recently, you should wait before starting the installation of the services. Full replication of these records between DNS servers can take from a few minutes to 48 hours or even longer in rare cases.

IMPORTANT

Docker Engine and Docker Compose must be installed on the server.

For a step-by-step guide on installing Docker Engine on Ubuntu Server, see Install Docker Engine and Docker Compose on Ubuntu Server

IMPORTANT

OpenSSH must be installed on the server, and port 22 must be open in order to be able to connect to the server using the SSH protocol.

To install OpenSSH on the server you can use the command:

1
sudo apt install openssh-server

NOTE

To connect to the server from a Windows system, you can use tools like PuTTY or MobaXterm.

NOTE

This guide walks you through connecting to a server with the iTerm2 terminal emulator on macOS.

CAUTION

You will need to open the following TCP ports for access to the services:

• TCP port 80 - to obtain a free cryptographic certificate through the Let’s Encrypt certification center.
• TCP port 443 - to access the Ollama web interface.

We connect to the server on which Ollama is planned to be installed.

Now it is necessary to create networks for your services.

We create a network for Traefik using the command:

1
docker network create traefik-network

We create a network for Ollama using the command:

1
docker network create ollama-network

Next, you need to clone the repository that contains the configuration files, which include all the necessary conditions for Ollama to work.

You can clone the repository using the command:

1
git clone https://github.com/heyvaldemar/ollama-traefik-letsencrypt-docker-compose.git

Navigate to the directory with the repository using the command:

1
cd ollama-traefik-letsencrypt-docker-compose

Next, you need to change the variables in the .env file according to your requirements.

IMPORTANT

The .env file should be in the same directory as ollama-traefik-letsencrypt-docker-compose.yml.

Now let’s start Ollama with the command:

1
docker compose -f ollama-traefik-letsencrypt-docker-compose.yml -p ollama up -d

To access the Ollama management panel, go to https://ollama.heyvaldemar.net from your workstation, where ollama.heyvaldemar.net is the domain name of my service. Accordingly, you need to specify your domain name that points to the IP address of your server with the installed Traefik service, which will redirect the request to Ollama.

NOTE

You need to specify the domain name of the service, previously defined in the .env file.

Click on the “Sign up” button.

The next step is to provide: your full name, an email address and a password to create a Ollama administrator account.

Click on the “Create Account” button.

Welcome to the Ollama control panel.

Please wait for the models listed in your .env file to download; the duration will depend on your internet speed. Once downloaded, you can select any model from the left corner of the interface to start using it.

To access the Traefik control panel, go to https://traefik.ollama.heyvaldemar.net from your workstation, where traefik.ollama.heyvaldemar.net is the domain name of my service. Accordingly, you need to specify your domain name that points to the IP address of your server with the installed Traefik.

NOTE

You need to specify the domain name of the service, previously defined in the .env file.

Enter the username and password previously set in the .env file, and click the “OK” button.

Welcome to the Traefik control panel.