Docker Desktop's Performance Odyssey Over a Year of Innovations

Utilizing the hyperfine Tool#

To benchmark Docker commands, we’ll use the hyperfine tool. Here’s how you can execute it:

1
hyperfine -i -p 'docker stop ubuntu || :' 'docker run -it --rm ubuntu echo'

Breaking Down the Command:

• -i allows hyperfine to execute the benchmark in an interactive mode, providing real-time results.

• -p specifies the warmup and benchmark commands for hyperfine.

• The warmup command, docker stop ubuntu || :, instructs Docker to halt the ubuntu container if it’s active. If not, it simply proceeds without any action.

• The benchmark command, docker run -it --rm ubuntu echo, prompts Docker to initiate the ubuntu container interactively and subsequently display the current date and time.

Results for Docker Desktop v4.22#

First Attempt:

1
hyperfine -i -p 'docker stop ubuntu || :' 'docker run -it --rm ubuntu echo'

1
Benchmark 1: docker run -it --rm ubuntu echo
2
Time (mean ± σ): 44.3 ms ± 3.3 ms [User: 29.5 ms, System: 12.7 ms]
3
Range (min … max): 39.7 ms … 49.6 ms 10 runs
4
Warning: Ignoring non-zero exit code.

Second Attempt:

1
hyperfine -i -p 'docker stop ubuntu || :' 'docker run -it --rm ubuntu echo'

1
Benchmark 1: docker run -it --rm ubuntu echo
2
Time (mean ± σ): 46.3 ms ± 2.3 ms [User: 29.4 ms, System: 12.4 ms]
3
Range (min … max): 43.1 ms … 52.5 ms 28 runs
4
Warning: Ignoring non-zero exit code.

Third Attempt:

1
hyperfine -i -p 'docker stop ubuntu || :' 'docker run -it --rm ubuntu echo'

1
Benchmark 1: docker run -it --rm ubuntu echo
2
Time (mean ± σ): 46.5 ms ± 5.3 ms [User: 29.5 ms, System: 12.1 ms]
3
Range (min … max): 39.0 ms … 63.5 ms 28 runs
4
Warning: Ignoring non-zero exit code.

From the First Attempt, we discern an average execution time of 44.3 milliseconds, accompanied by a deviation of 3.3 milliseconds. This suggests that the majority of runs were completed between 41.0 milliseconds and 47.6 milliseconds. The runs varied from a brisk 39.7 milliseconds to a peak of 49.6 milliseconds.

In the Second Attempt, the average execution time was 46.3 milliseconds, with a deviation of 2.3 milliseconds. This indicates that most runs were executed between 44.0 milliseconds and 48.6 milliseconds. The fastest run was recorded at 43.1 milliseconds, while the longest took 52.5 milliseconds.

For the Third Attempt, the average execution time stood at 46.5 milliseconds, with a more pronounced deviation of 5.3 milliseconds. This means that the bulk of runs were executed between 41.2 milliseconds and 51.8 milliseconds. The runs ranged from a swift 39.0 milliseconds to a high of 63.5 milliseconds.

Results for Docker Desktop v4.11#

First Attempt:

1
hyperfine -i -p 'docker stop ubuntu || :' 'docker run -it --rm ubuntu echo'

1
Benchmark 1: docker run -it --rm ubuntu echo
2
Time (mean ± σ): 59.7 ms ± 3.2 ms [User: 36.4 ms, System: 15.6 ms]
3
Range (min … max): 55.0 ms … 68.3 ms 13 runs
4
Warning: Ignoring non-zero exit code.

Second Attempt:

1
hyperfine -i -p 'docker stop ubuntu || :' 'docker run -it --rm ubuntu echo'

1
Benchmark 1: docker run -it --rm ubuntu echo
2
Time (mean ± σ): 56.7 ms ± 2.7 ms [User: 36.3 ms, System: 15.3 ms]
3
Range (min … max): 51.7 ms … 62.5 ms 13 runs
4
Warning: Ignoring non-zero exit code.

Third Attempt:

1
hyperfine -i -p 'docker stop ubuntu || :' 'docker run -it --rm ubuntu echo'

1
Benchmark 1: docker run -it --rm ubuntu echo
2
Time (mean ± σ): 59.8 ms ± 4.5 ms [User: 36.4 ms, System: 15.3 ms]
3
Range (min … max): 51.2 ms … 69.5 ms 21 runs
4
Warning: Ignoring non-zero exit code.

From the First Attempt, we note an average execution time of 59.7 milliseconds, with a deviation of 3.2 milliseconds. This suggests that most runs were completed between 56.5 milliseconds and 62.9 milliseconds. The quickest run was at 55.0 milliseconds, while the longest took 68.3 milliseconds.

For the Second Attempt, the average execution time was 56.7 milliseconds, with a deviation of 2.7 milliseconds. This indicates that the majority of runs fell between 54.0 milliseconds and 59.4 milliseconds. The runs ranged from a swift 51.7 milliseconds to a peak of 62.5 milliseconds.

In the Third Attempt, the average execution time stood at 59.8 milliseconds, with a deviation of 4.5 milliseconds. This means that most runs were executed between 55.3 milliseconds and 64.3 milliseconds. The fastest run was recorded at 51.2 milliseconds, while the slowest reached 69.5 milliseconds.

Final Results (Docker Desktop 4.11 vs 4.22)#

Types of File Sharing Mechanism#

Enable VirtioFS#

In Docker Desktop settings, navigate to the “General” tab. Locate the section titled “Choose file sharing implementation for your containers” and select “VirtioFS”. Then, click on the “Apply & Restart” button.

Steps to Set Up and Test VirtioFS#

Create a directory using the command:

1
mkdir data

Populate the Directory with a Large Data File (1GB of random data) using the command:

1
dd if=/dev/zero of=data/data.img bs=1M count=1000

Next, let’s build the Docker Image using the command:

First, create a Dockerfile with the following content:

1
FROM ubuntu:latest
2

3
VOLUME /data
4

5
CMD ["bash"]

Then, build the Docker image using the command:

1
docker build -t virtiofs-demo:latest .

Run the Docker Container using the command:

1
docker run -it --rm -v "$(pwd)/data:/data" virtiofs-demo

List the Running Containers using the command:

1
docker ps

Expected output:

1
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
2
e47eb4731c80 virtiofs-demo "bash" 5 minutes ago Up 5 minutes gifted_lichterman

Next, measure File Copy Time into the Container

1
time docker cp gifted_lichterman:/data/data.img data/data.img

Expected result:

1
Successfully copied 1.05GB to /Users/ajeetsraina/july/virt/data/data.img
2
docker cp gifted_lichterman:/data/data.img data/data.img 0.61s user 1.84s system 36% cpu 6.750 total

Performance Comparison#

Comparing Docker Desktop 4.22 vs 4.11 (without VirtioFS enabled)

Comparing Docker Desktop 4.22 vs 4.11 (with VirtioFS enabled)

Redis Build Process#

1
# !/bin/bash
2

3
rm -rf /tmp/redis
4
docker run --rm -v /tmp:/tmp alpine sh -c \
5
 'cd /tmp ; apk add git ; git clone https://github.com/redis/redis --depth 1'
6

7
docker run --rm -v /tmp/redis:/tmp/redis ubuntu bash -c \
8
 'cd /tmp/redis ; apt update && apt install -y make gcc ; make distclean && time make'

Redis Build Performance#

Postgres Build Process#

Before diving into the Postgres build process, it’s essential to understand the setup. The following script showcases the steps involved in setting up a Postgres network, initiating a server, and launching a pgbench client.

1
# !/bin/bash
2

3
IMG=postgres:alpine
4
NET=postgresnet
5
TESTDIR=/tmp/postgrestest/data
6
PGNAME=postgrestest
7
PGPASS=postgrespass
8

9
docker network rm -q $NET &>/dev/null
10

11
echo "Setting up postgres network..."
12
docker network create $NET
13

14
rm -rf $TESTDIR
15
mkdir -p $TESTDIR
16

17
echo "Initiating postgres server..."
18
docker run --name=${PGNAME} -e POSTGRES_PASSWORD=postgrespass --network ${NET} -p 5432:5432 -v ${TESTDIR}:/var/lib/postgresql/data -d ${IMG}
19

20
echo "Launching postgres pgbench client..."
21
docker run --network ${NET} alpine sh -c \
22
 'apk add postgresql ; export PGPASSWORD=postgrespass; \
23
 pgbench --host=postgrestest -U postgres -i -s 10 postgres ; \
24
 pgbench --host=postgrestest -U postgres -c 10 -t 10 postgres'
25

26
docker stop ${PGNAME} &>/dev/null
27
docker rm ${PGNAME} &>/dev/null
28
docker network rm -q $NET &>/dev/null

Postgres Build Performance#

Docker Desktop version 4.22 significantly outperforms version 4.11.1 in terms of transactions per second (tps). The newer version achieves 2600 tps, which is more than double the 1000 tps of the older version.

The choice of file-sharing mechanism in Docker Desktop can have a substantial impact on build times. For tasks like building Redis images, VirtioFS offers the best performance.

Docker Desktop’s newer version (4.22) showcases a marked improvement in handling Postgres transactions, indicating optimizations or enhancements in this version.

While VirtioFS is the fastest for Redis builds, it’s essential to consider the specific needs and compatibility requirements of individual projects when choosing a file-sharing mechanism.

The significant jump in Postgres transaction performance from version 4.11.1 to 4.22 suggests that Docker Desktop’s newer versions come with substantial improvements, making upgrades worthwhile.

Enabling gVisor#

To ensure Docker Desktop uses gVisor by default, run the following command:

1
cat ~/Library/Group\ Containers/group.com.docker/settings.json | grep -i network

If you wish to revert to VPNKit, add “networkType”:“vpnkit” to your settings.json file.

Benefits of gVisor over VPNKit:

• Performance: gVisor’s speed outpaces VPNKit, enhancing container performance.
• Security: As a sandboxed runtime, gVisor offers better protection against vulnerabilities.
• Resource Efficiency: gVisor consumes fewer resources than VPNKit, optimizing your Mac’s performance.

Networking Benchmarks#

Container to Container

Benchmarking Tool: iperf3/netperf

1
# !/bin/bash
2

3
IMG=dockerpinata/iperf3:2.1
4
NET=iperf3net
5
SERVER=iperf3server
6
CLIENT=iperf3client
7

8
docker rm -f $SERVER &>/dev/null
9
docker network rm -q $NET &>/dev/null
10

11
echo "Creating iperf3 network..."
12
docker network create $NET
13

14
echo "Starting iperf3 server..."
15
docker run --rm -d --name=${SERVER} --network=${NET} ${IMG} /usr/bin/iperf3 -s -1
16

17
echo "Starting iperf3 client..."
18
docker run --rm --name=${CLIENT} --network=${NET} ${IMG} /usr/bin/iperf3 -c ${SERVER}
19

20
docker rm -f $SERVER &>/dev/null
21
docker network rm -q $NET &>/dev/null

Container to Host

Benchmarking Tool: iperf3/netperf

1
# !/bin/bash
2

3
IMG=dockerpinata/iperf3:2.1
4

5
# This can be used to remove previous hanging iperf3 servers
6

7
# pkill iperf3
8

9
echo "Starting iperf3 server..."
10
iperf3 -s -1 &
11
IPERF3_SERVER_PID=$!
12

13
echo "Starting iperf3 client..."
14
docker run --rm ${IMG} /usr/bin/iperf3 -c host.docker.internal
15

16
kill $IPERF3_SERVER_PID &>/dev/null

Host to Container

Benchmarking Tool: iperf3/netperf

1
# !/bin/bash
2

3
IMG=dockerpinata/iperf3:2.1
4
SERVER=iperf3server
5
docker rm -f $SERVER &>/dev/null
6
echo "Starting iperf3 server..."
7
docker run --rm -d -p 5201:5201 --name=${SERVER} ${IMG} /usr/bin/iperf3 -s -1
8
echo "Starting iperf3 client..."
9
iperf3 -c localhost
10
docker rm -f $SERVER &>/dev/null