Docker for Python apps
======================
----
Docker for Python apps
======================
.footer: `RoPython-Cluj 16 May 2016 `_ — generated with `Darkslide `_
.. class:: center
**Ionel Cristian Mărieș**
* Background:
* Ported a big app from deb/rpm packaging to docker
* Don't know that much about orchestration tools. Don't ask me about Mesos or Kubernetes 😂
* Contents:
* Concepts
* Building images
* Running apps inside a container
----
Concepts
========
* Containers:
* Lightweight isolation (various limits to processes in container).
* Security is bad: policy is to disallow things, instead of only allowing certain things.
* Less fuss managing dependencies. No more dependency conflicts. Like virtualenvs, but for everything.
* Faster than virtualization. Only 1 kernel.
----
Concepts
========
Docker:
* High level management tool for containers that hides the gnarly Linux APIs.
* Relatively new (since 2013).
* Lots of competition. But Docker is the most popular. More resources, better docs, more tooling.
|
Image: A snapshot (readonly).
Container: An execution environment.
Engine: It's just the docker daemon - a service that manages containers.
----
Overview
========
Running images:
* ``docker run --rm -it ubuntu:xenial bash``
* ``--privileged`` for extra capabilities (eg: for using ``strace`` inside of container).
* ``-v $PWD:/foobar`` for mounting cwd inside as ``/foobar``.
* ``-e FOO=bar`` for running with that extra env var.
For building images there are two ways:
* ``docker run; docker commit $(docker ps -a -q | head -n 1)``
* ``docker build .``
Presenter notes
---------------
.. sourcecode:: bash
docker run --rm -it ubuntu:xenial bash
# ^ don't keep stuff around
# ^ short for --interactive and --tty
# ^ image name
# ^ optional image tag, like a version
# ^ optional command to run, image
# usually has a default
----
Building images with Dockerfiles
================================
``docker build path``
* Docker makes an archive of ``path`` and sends it to the daemon (the "`context`").
* Runs instructions from ``path/Dockerfile``, example:
.. sourcecode:: docker
FROM ubuntu:xenial
RUN apt-get update \
&& apt-get install -y --no-install-recommends \
ca-certificates curl \
strace gdb lsof locate net-tools htop \
python2.7-dbg python2.7 \
&& rm -rf /var/lib/apt/lists/*
RUN curl -fSL 'https://bootstrap.pypa.io/get-pip.py' | python2.7 -
----
Building images with Dockerfiles
================================
* Alternate path to Dockerfile:
``docker build -f path/Dockerfile path``
The ``Dockerfile`` must be inside of ``path``.
* No mounts! Caching for ``apt`` or ``pip`` becomes tricky. Solutions:
* Use multiple Dockerfiles/build commands.
* Use ``docker run/commit``
* Bunch of commands:
``RUN``,
``ADD`` (don't use),
``COPY``,
``EXPOSE``,
``VOLUME``,
``ENV``, ``ARG``,
``WORKDIR``, ``USER``,
``ENTRYPOINT``, ``CMD`` etc.
----
Tricky entrypoints
==================
In a ``Dockerfile``:
* ``ENTRYPOINT`` is rarely needed, avoid it.
* If ``CMD=["/entrypoint.sh"]``:
* Use ``exec``:
.. sourcecode:: bash
#!/bin/sh
exec uwsgi ...
* If changing user then use `gosu `_ or
`pysu `_ instead of ``su``:
.. sourcecode:: bash
#!/bin/sh
pysu username uwsgi ...
.. list-table::
:header-rows: 1
:class: small
- -
- No ``ENTRYPOINT``
- .. sourcecode:: docker
ENTRYPOINT exec_entry p1_entry
- .. sourcecode:: docker
ENTRYPOINT ["exec_entry", "p1_entry"]
- - No ``CMD``
- *error, not allowed*
- .. sourcecode:: shell
/bin/sh -c exec_entry p1_entry
- .. sourcecode:: shell
exec_entry p1_entry
- - .. sourcecode:: docker
CMD ["exec_cmd", "p1_cmd"]
- .. sourcecode:: shell
exec_cmd p1_cmd
- .. sourcecode:: shell
/bin/sh -c exec_entry p1_entry exec_cmd p1_cmd
- .. sourcecode:: shell
exec_entry p1_entry exec_cmd p1_cmd
- - .. sourcecode:: docker
CMD ["p1_cmd", "p2_cmd"]
- .. sourcecode:: shell
p1_cmd p2_cmd
- .. sourcecode:: shell
/bin/sh -c exec_entry p1_entry p1_cmd p2_cmd
- .. sourcecode:: shell
exec_entry p1_entry p1_cmd p2_cmd
- - .. sourcecode:: docker
CMD exec_cmd p1_cmd
- .. sourcecode:: shell
/bin/sh -c exec_cmd p1_cmd
- .. sourcecode:: shell
/bin/sh -c exec_entry p1_entry /bin/sh -c exec_cmd p1_cmd
- .. sourcecode:: shell
exec_entry p1_entry /bin/sh -c exec_cmd p1_cmd
----
Running containers
==================
* Runs as ``root`` by default. Mounts become inconvenient, if not dangerous::
ionel@newbox:~$ docker run --rm -it -v $PWD:/stuff ubuntu
root@5b28926fb3a8:/# touch /stuff/foobar
root@5b28926fb3a8:/# exit
ionel@newbox:~$ ls -al foobar
-rw-r--r-- 1 root root 0 May 16 10:26 foobar
* Two ways to deal with it:
* User namespaces (need os and docker daemon conf).
* Just run as different user.
----
Running as different user
=========================
.. sourcecode:: docker
ARG USER
ARG UID
ARG GID
RUN echo "Creating user: $USER ($UID:$GID)" \
&& groupadd --system --gid=$GID $USER \
&& useradd --system --create-home --gid=$GID --uid=$UID $USER
WORKDIR /home/$USER
USER $USER
* UID should match something on the host
----
Mounts and volumes
==================
* You can have this in your Dockerfile:
.. sourcecode:: docker
VOLUME /foo
COPY stuff /foo/bar
Files from image make the initial data for the volume.
* You can mount volumes from a different container
.. sourcecode:: bash
docker run --volumes-from other-container my-image
----
Dependencies
============
* You can make containers depend on each other. Example: ``web`` container depends on ``postgres``
.. sourcecode:: bash
docker run --name=pg postgres:9.5
docker run --link=pg web
Then we can connect to ``pg`` inside the ``web`` container (Docker provides a DNS server)
* A nicer way, using ``docker-compose``:
.. sourcecode:: yaml
version: '3'
services:
web:
build: 'docker/web'
ports:
- '8080:80'
links:
- 'pg'
pg:
image: 'postgres:9.5'
----
Dependencies
============
* Services aren't ready right away. Solutions:
* Let Docker restart the ``web`` container till it works 😒
* Use healthchecks (``docker-compose`` with v3 config has them, other orchestration tools have them too).
* Wait for services to be ready. Several tools:
`dockerize `_,
`wait-for-it `_,
`holdup `_. Example entrypoint script:
.. sourcecode:: bash
#!/bin/sh
set -eux
exec holdup tcp://pg:5432 -- uwsgi ...
----
Configuration
=============
Ways to allow changing configuration without rebuilding images:
* Environment variables. Have to change app to take various settings from ``os.environ``
(aka the `12-factor `_ way).
* Volumes:
.. sourcecode:: docker
RUN mkdir /etc/app
# default data for the volume
COPY settings.conf /etc/app/
VOLUME /etc/app
----
Slides
======
.qr: https\://blog.ionelmc.ro/presentations/docker/