Rapid Ansible Iteration with Docker

Owned by Edward Zarecor (Do Not Use) (Deactivated)
Last updated: Nov 10, 2015

Docker is a useful tool for rapidly iterating on Ansible roles, but there are some important caveats to keep in mind.

A big one is that Docker won't run your Sys V init scripts.  In the edx environment this means that rsyslogd and supervisor won't be started in the normal way.

This guide is a "close to the metal" tour.  You may prefer to use packages and configure using the idiom of your OS or distribution.  If you do, feel free to contribute the details back to this page.

In order to build your ansible roles on Docker you will need to develop against the hack2015/dockers branch, which may be too much of a burden for features that need to land on master sooner-than-later.  All that being said, I still find it the shortest loop and a great way to iterate rapidly.

Step-by-step guide

  1. Checkout the edx configuration repository locally for hacking
  2. Create a local virtual environment on your host machine and install the configuration repository requirements that include edX fork of Ansible.

  3. Create a new role using our template

    cd edx/configuration/playbooks
ansible-playbook -i 'localhost,' create_role.yml --connection=local --extra-vars "role_name=my_shiny_new_role"
  4. As you are developing your role, keep our, freshly minted, role lifecycle convention in mind: Role Life-cycle Tags

  5. Create a play for running your role and per-requisities in configuration/playbooks/edx-east

    - name: Deploy my shiny new service
  hosts: all
  sudo: True
  gather_facts: True
  vars:
    serial_count: 1
  serial: "{{ serial_count }}"
  roles:
    - docker
    - supervisor
    - my_shiny_new_role
  6. Install the latest binary build of Docker, >= 1.9.0, so you can make use of productionalized Docker networking.  You can find the details here http://docs.docker.com/engine/installation/binaries/

  7. Start the Docker daemon

    sudo /home/edward/bin/docker daemon --bip  172.17.42.1/16 --dns 8.8.8.8

  8. Start up the precise-common container mounting your checked out code so it overlays the checkout in /edx/app/edx-ansible

    docker run -ti -v /home/me/Documents/git/edx/configuration:/edx/app/edx_ansible/edx_ansible edxops/precise-common /bin/bash

    Unpacking this command, it is saying

    • Run the Docker container edxops/precise-common:latest which comes with our edx-ansible application installed

    • Run with a pseudo TTY, -t
    • Run in interactive mode, -i
    • Overlay host directory /home/me/Documents/git/edx/configuration on top of the guest's directory /edx/app/edx_ansible/edx_ansible
    • Run the command /bin/bash dropping you into an interactive shell

  9. Activate the edx_ansible virtual environment

    . /edx/app/edx_ansible/venvs/edx_ansible/bin/activate

  10. Change to the edx_ansible/playbooks directory

    cd /edx/app/edx_ansible/edx_ansible/playbooks/

  11. Run your play

    sudo ansible-playbook -vvv my_shiny_new_play.yml -c local -t 'install:base,install:configuration'
  12. Lather, rinse, repeat.  Changes made on the host will immediately be available on the guest.
  13. We are actively working on formalizing and stabilizing Docker support, but for now you'll need to take care regarding the following:

    1. Most of our applications rely on rsyslogd which won't be started as normal in Docker.  You can hack around this by  running

      $ rsyslogd

      Ultimately we need to figure out the process manager story in Docker.  While we will push toward single services per container, I don't think we'll get there entirely.  We're likely to want rsyslogd, splunk-forwarder, dogstatsd or some analog each running along side our applications.

    2. Running supervisor is a little bit tricky, but doing so is not insurmountable.  It is assumed that your play is installing it as it does not come on the base box.  To run it, you'll need to connect to your running container and run supervisor

      $ docker ps
CONTAINER ID        IMAGE                   COMMAND             CREATED             STATUS              PORTS               NAMES
2b3ccb75c5f9        edxops/precise-common   "/bin/bash"         46 minutes ago      Up 46 minutes                           sleepy_perlman

$ docker exec -ti 2b3ccb75c5f9 /bin/bash
$ sudo /edx/app/supervisor/venvs/supervisor/bin/supervisord -n -c /edx/app/supervisor/supervisord.conf

      With supervisor running you can connect to the console from the first window with:

      sudo /edx/app/supervisor/venvs/supervisor/bin/supervisorctl -c /edx/app/supervisor/supervisord.conf