Docker for Python apps ====================== ---- Docker for Python apps ====================== .footer: `RoPython-Cluj 16 May 2016 `_ — generated with `Darkslide `_ .. class:: center **Ionel Cristian Mărieș** * Background: * Ported a big app from deb/rpm packaging to docker * Don't know that much about orchestration tools. Don't ask me about Mesos or Kubernetes 😂 * Contents: * Concepts * Building images * Running apps inside a container ---- Concepts ======== * Containers: * Lightweight isolation (various limits to processes in container). * Security is bad: policy is to disallow things, instead of only allowing certain things. * Less fuss managing dependencies. No more dependency conflicts. Like virtualenvs, but for everything. * Faster than virtualization. Only 1 kernel. ---- Concepts ======== Docker: * High level management tool for containers that hides the gnarly Linux APIs. * Relatively new (since 2013). * Lots of competition. But Docker is the most popular. More resources, better docs, more tooling. | Image: A snapshot (readonly). Container: An execution environment. Engine: It's just the docker daemon - a service that manages containers. ---- Overview ======== Running images: * ``docker run --rm -it ubuntu:xenial bash`` * ``--privileged`` for extra capabilities (eg: for using ``strace`` inside of container). * ``-v $PWD:/foobar`` for mounting cwd inside as ``/foobar``. * ``-e FOO=bar`` for running with that extra env var. For building images there are two ways: * ``docker run; docker commit $(docker ps -a -q | head -n 1)`` * ``docker build .`` Presenter notes --------------- .. sourcecode:: bash docker run --rm -it ubuntu:xenial bash # ^ don't keep stuff around # ^ short for --interactive and --tty # ^ image name # ^ optional image tag, like a version # ^ optional command to run, image # usually has a default ---- Building images with Dockerfiles ================================ ``docker build path`` * Docker makes an archive of ``path`` and sends it to the daemon (the "`context`"). * Runs instructions from ``path/Dockerfile``, example: .. sourcecode:: docker FROM ubuntu:xenial RUN apt-get update \ && apt-get install -y --no-install-recommends \ ca-certificates curl \ strace gdb lsof locate net-tools htop \ python2.7-dbg python2.7 \ && rm -rf /var/lib/apt/lists/* RUN curl -fSL 'https://bootstrap.pypa.io/get-pip.py' | python2.7 - ---- Building images with Dockerfiles ================================ * Alternate path to Dockerfile: ``docker build -f path/Dockerfile path`` The ``Dockerfile`` must be inside of ``path``. * No mounts! Caching for ``apt`` or ``pip`` becomes tricky. Solutions: * Use multiple Dockerfiles/build commands. * Use ``docker run/commit`` * Bunch of commands: ``RUN``, ``ADD`` (don't use), ``COPY``, ``EXPOSE``, ``VOLUME``, ``ENV``, ``ARG``, ``WORKDIR``, ``USER``, ``ENTRYPOINT``, ``CMD`` etc. ---- Tricky entrypoints ================== In a ``Dockerfile``: * ``ENTRYPOINT`` is rarely needed, avoid it. * If ``CMD=["/entrypoint.sh"]``: * Use ``exec``: .. sourcecode:: bash #!/bin/sh exec uwsgi ... * If changing user then use `gosu `_ or `pysu `_ instead of ``su``: .. sourcecode:: bash #!/bin/sh pysu username uwsgi ... .. list-table:: :header-rows: 1 :class: small - - - No ``ENTRYPOINT`` - .. sourcecode:: docker ENTRYPOINT exec_entry p1_entry - .. sourcecode:: docker ENTRYPOINT ["exec_entry", "p1_entry"] - - No ``CMD`` - *error, not allowed* - .. sourcecode:: shell /bin/sh -c exec_entry p1_entry - .. sourcecode:: shell exec_entry p1_entry - - .. sourcecode:: docker CMD ["exec_cmd", "p1_cmd"] - .. sourcecode:: shell exec_cmd p1_cmd - .. sourcecode:: shell /bin/sh -c exec_entry p1_entry exec_cmd p1_cmd - .. sourcecode:: shell exec_entry p1_entry exec_cmd p1_cmd - - .. sourcecode:: docker CMD ["p1_cmd", "p2_cmd"] - .. sourcecode:: shell p1_cmd p2_cmd - .. sourcecode:: shell /bin/sh -c exec_entry p1_entry p1_cmd p2_cmd - .. sourcecode:: shell exec_entry p1_entry p1_cmd p2_cmd - - .. sourcecode:: docker CMD exec_cmd p1_cmd - .. sourcecode:: shell /bin/sh -c exec_cmd p1_cmd - .. sourcecode:: shell /bin/sh -c exec_entry p1_entry /bin/sh -c exec_cmd p1_cmd - .. sourcecode:: shell exec_entry p1_entry /bin/sh -c exec_cmd p1_cmd ---- Running containers ================== * Runs as ``root`` by default. Mounts become inconvenient, if not dangerous:: ionel@newbox:~$ docker run --rm -it -v $PWD:/stuff ubuntu root@5b28926fb3a8:/# touch /stuff/foobar root@5b28926fb3a8:/# exit ionel@newbox:~$ ls -al foobar -rw-r--r-- 1 root root 0 May 16 10:26 foobar * Two ways to deal with it: * User namespaces (need os and docker daemon conf). * Just run as different user. ---- Running as different user ========================= .. sourcecode:: docker ARG USER ARG UID ARG GID RUN echo "Creating user: $USER ($UID:$GID)" \ && groupadd --system --gid=$GID $USER \ && useradd --system --create-home --gid=$GID --uid=$UID $USER WORKDIR /home/$USER USER $USER * UID should match something on the host ---- Mounts and volumes ================== * You can have this in your Dockerfile: .. sourcecode:: docker VOLUME /foo COPY stuff /foo/bar Files from image make the initial data for the volume. * You can mount volumes from a different container .. sourcecode:: bash docker run --volumes-from other-container my-image ---- Dependencies ============ * You can make containers depend on each other. Example: ``web`` container depends on ``postgres`` .. sourcecode:: bash docker run --name=pg postgres:9.5 docker run --link=pg web Then we can connect to ``pg`` inside the ``web`` container (Docker provides a DNS server) * A nicer way, using ``docker-compose``: .. sourcecode:: yaml version: '3' services: web: build: 'docker/web' ports: - '8080:80' links: - 'pg' pg: image: 'postgres:9.5' ---- Dependencies ============ * Services aren't ready right away. Solutions: * Let Docker restart the ``web`` container till it works 😒 * Use healthchecks (``docker-compose`` with v3 config has them, other orchestration tools have them too). * Wait for services to be ready. Several tools: `dockerize `_, `wait-for-it `_, `holdup `_. Example entrypoint script: .. sourcecode:: bash #!/bin/sh set -eux exec holdup tcp://pg:5432 -- uwsgi ... ---- Configuration ============= Ways to allow changing configuration without rebuilding images: * Environment variables. Have to change app to take various settings from ``os.environ`` (aka the `12-factor `_ way). * Volumes: .. sourcecode:: docker RUN mkdir /etc/app # default data for the volume COPY settings.conf /etc/app/ VOLUME /etc/app ---- Slides ====== .qr: https\://blog.ionelmc.ro/presentations/docker/