Container Runtime Comparison

Owned by Jeremy Bowman (Deactivated)
Last updated: Feb 22, 2024 by Diana Huang
5 min read

Since Docker started charging for Docker Desktop licenses (under somewhat confusing terms and circumstances) in 2022, there has been an explosion of interest in alternative runtimes for Linux containers on macOS and Windows. Here are some notes on the ones we’ve tried with devstack on macOS so far (and ones that won’t work with devstack as it is right now).

Important notes:

  • If you run Linux as your native operating system, you don’t need any of these. Just install Docker directly from your distribution.

  • If you run Windows, some of these may work but devstack itself probably won’t. A few attempts at Windows support have been made, but there are no reports of it working properly yet.

Docker Desktop

Pricing

Free for “individuals, non-commercial open source developers, students and educators, and small businesses of fewer than than 250 employees AND less than $10 million in revenue.” Otherwise, $5-$24 per user per month depending on organization size (2U falls in the $24 bucket). See https://www.docker.com/pricing/ for details.

Installation

  1. Talk to your IT department if you work at a company that has one, to determine if you need a license and your company has paid for one. If you need a license and they don’t want to pay for one, pick a different runtime.

  2. Download the appropriate version for your operating system from https://www.docker.com/products/docker-desktop/

  3. Run the downloaded installer; give it admin privileges if it asks for them

  4. Go to Settings → Advanced and make sure that the CLI tools are installed via System and the default Docker socket is enabled (these are sometimes missed during installation if the installer somehow fails to get admin privileges).

Performance

  • make dev.provision - about 46 minutes on a 2021 14-inch MacBook Pro

Other Notes

  • RAM and CPU cores must be pre-allocated and are unavailable to the rest of the system whenever Docker Desktop is running, even if there are no running containers.

  • Disk space for Docker images and volumes must also be pre-allocated, and is unavailable to the rest of the system as soon as it’s assigned.

  • Default choice for most developers until a recent licensing change made it more expensive (and worse, complicated to manage).

OrbStack

Pricing

Free during the beta period. After that, tentatively free for personal/education use or $8 per month per user for commercial use. See https://docs.orbstack.dev/faq#free for details.

Installation

  1. Verify on https://docs.orbstack.dev/faq that OrbStack is still in the free beta period. If the beta period has ended and you work for a business, check with your IT department if they are willing to pay for a license for you. If not, pick a different runtime.

  2. Download the appropriate version for your operating system from https://orbstack.dev/download

  3. Open the downloaded installer and drag the app to the Applications folder.

  4. Run the application.

  5. Select “Docker” (instead of “Linux”) from the first dialog to ask you a question.

  6. Optionally click “Migrate” to preserve database volumes, etc. from an existing Docker Desktop installation.

Performance

  • make pull - about 3 minutes on a 300 Megabit Verizon Fios connection, 2021 14-inch MacBook Pro

  • make dev.provision - about 17 minutes on a 2021 14-inch MacBook Pro

Other Notes

  • RAM and CPU cores are only utilized when needed, up to the limits configured in settings

  • Disk space is automatically allocated as needed for Docker image storage, up to the smaller of 8 TB or available disk space

  • Supports browsing the filesystem of any installed Docker image or volume in Finder or the terminal. It looks like this allows using IDEs with code installed in volumes for even faster performance than bind mounts.

  • Also functions as a Linux quasi-VM manager, like Windows Subsystem for Linux.

  • Automatically assigns private IP addresses and domain names (such as lms.devstack.orb.local) to each running container. Port numbers aren’t needed when using domains to access web services in containers. Also hosts a page at orb.local providing links to all running containers.

Colima

Pricing

Free and open source (MIT license).

Installation

  1. If you haven’t already installed Homebrew, do so via https://brew.sh/ .

  2. Run brew install colima

  3. Run brew install docker

  4. Run colima start --edit and customize some of the settings (they will be saved to a config file as the defaults for future colima start runs):

    1. Set disk to 96

    2. Set memory to 8

    3. Set vmType to vz

    4. Set rosetta to true - only if you are on an Apple Silicon Mac

    5. Set mountType to virtiofs

Performance

  • make pull - about 4 minutes on a 300 Megabit Verizon Fios connection, 2021 14-inch MacBook Pro

    • Mine (Diana) was 9 minutes on a 2019/2020ish MacBook Pro

  • make dev.provision - about 41 minutes on a 2021 14-inch MacBook Pro

    • Over an hour for me

Other Notes

  • It has no GUI, just the CLI

  • This is practically the only option from this list that can be cleanly installed on macOS CI runners

Rancher Desktop

Pricing

Free and open source (Apache 2.0).

Installation

  1. Download the appropriate version for your operating system from https://rancherdesktop.io/

  2. Open the downloaded installer and drag the app to the Applications folder.

  3. Run the Rancher Desktop application and in the setup dialog choose the defaults as of this writing:

    1. Enable the latest stable Kubernetes

    2. The dockerd (moby) Container Engine

    3. Automatic path configuration

  4. Under Settings → Virtual Machine → Hardware, increase Memory to 8GB

  5. Under Settings → Virtual Machine → Emulation, select VZ and Enable Rosetta Support

  6. Under Settings → Virtual Machine → Volumes, select virtiofs

  7. Click the button to Apply the settings changes

Performance

  • make pull - about 4 minutes on a 300 Megabit Verizon Fios connection, 2021 14-inch MacBook Pro

  • make dev.provision - Consistently stalls and/or crashes on the LMS run of npm to install dependencies, despite otherwise seeming to work and perform pretty well with the settings above. The same problem occurs with QEMU emulation and reverse-sshfs for volumes. It’s probably worth making a minimal test case to reproduce this and submit it as a bug report; could serve as a good benchmark for filesystem performance.

    • Stalls indefinitely on webpack in credentials on a 2023 MacBook pro

---> pavelib.prereqs.install_node_prereqs rm -rf common/static/common/js/vendor/ common/static/common/css/vendor/ rmdir common/static/common/css || true make[1]: *** [impl-dev.provision] Error 255 make: *** [dev.provision] Error 2

Podman Desktop

Pricing

Free and open source (Apache 2.0).

Installation

  1. Download the appropriate version for your operating system from https://podman-desktop.io/

  2. Open the downloaded installer and drag the app to the Applications folder.

  3. Run the Podman Desktop application

Performance

Mac: 3h 15minutes on a 2019 Macbook Pro. So not a direct comparison with the above numbers, but pretty suggestive that you should avoid this if at all possible. Linux performance is reportedly much better.

Other Notes

  • The entire experience (UI, docs, etc.) seem intended to pressure you to switch from Docker to Podman. But Podman is primarily intended to run on Fedora/Red Hat, and is apparently tricky to get working on Ubuntu (our current distribution of choice).

  • While it will show information about containers in the currently active Docker engine context, it doesn’t actually provide one. So to use docker, docker-compose, etc., you also need one of the tools above.

  • There is good support for migrating Docker images and docker-compose configurations to Podman, but this feels like a non-starter if we’re using Docker for Ubuntu-based production and Linux dev environments.

Minikube

There doesn’t seem to be a clean way to use docker-compose directly with Minikube, although tools like https://kompose.io/conversion/ could feasibly get significant chunks of devstack functionality working in it. It may be a good choice for Tutor deployments via Kubernetes, though.