Improve Docker ecosystem

Instead of using Dockerfile for everything, we've introduced
a docker-compose.yml file and kept the Dockerfile to producing
a pure base image for the services defined.

docker-compose services:

- `mariadb` - Setup mariadb
- `sharness` - Run sharness suites
- `pytest-mysql` - Run pytest suites with MariaDB
- `pytest-sqlite` - Run pytest suites with SQLite
- `test` - Run all tests and produce a collective coverage report
    - This target mounts a cache volume and copies any successful
      coverage report back to `./cache/.coverage`. Users can run
      `./util/fix-coverage ./cache/.coverage` to rewrite source
      code paths and move coverage into place to view reports
      on your local system.

== Get Started ==

Build `aurweb:latest`.

    $ docker build -t aurweb:latest .

Run all tests via `docker-compose`.

    $ docker-compose up test

You can also purely run `pytest` in SQLite or MariaDB modes.

    $ docker-compose up pytest-sqlite
    $ docker-compose up pytest-mysql

Or `sharness` alone, which only uses SQLite internally.

    $ docker-compose up sharness

After running tests, coverage reports are stored in `./cache/.coverage`.
This database was most likely created in a different path, and so it
needs to be sanitized with `./util/fix-coverage`.

    $ ./util/fix-coverage cache/.coverage
    Copied coverage db to /path/to/aurweb/.coverage.
    $ coverage report
    ...
    $ coverage html
    $ coverage xml
    ...

Defined components:

**Entrypoints**

- mariadb-entrypoint.sh - setup mariadb and run its daemon
- test-mysql-entrypoint.sh - setup mysql configurations
- test-sqlite-entrypoint.sh - setup sqlite configurations
- tests-entrypoint.sh - setup mysql and sqlite configurations

**Scripts**

- run-mariadb.sh - setup databases
- run-pytests.sh - run pytest suites
- run-sharness.sh - run sharness suites
- run-tests.sh - run both pytests and sharness

**Health**

- mariadb.sh - A healthcheck script for the mariadb service
- pytest.sh - A healthcheck script for the pytest-* services
- sharness.sh - A healthcheck script for the sharness service

This Docker configuration is setup for tests, but should be
extendable for web and git servers.

**Changes to Makefile**

- Remove `.coverage` in the `clean` target
- Add a `coverage` target which prints a report and outputs xml

Signed-off-by: Kevin Morris <kevr@0cost.org>

docker/health/mariadb.sh

@@ -0,0 +1,2 @@
+#!/bin/bash
+exec mysqladmin ping --silent

docker/mariadb-entrypoint.sh

@@ -0,0 +1,6 @@
+#!/bin/bash
+set -eou pipefail
+
+mariadb-install-db --user=mysql --basedir=/usr --datadir=/var/lib/mysql
+
+exec "$@"

docker/scripts/run-mariadb.sh

@@ -0,0 +1,19 @@
+#!/bin/bash
+
+mysqld_safe --datadir=/var/lib/mysql --skip-networking &
+until mysqladmin ping --silent; do
+    sleep 1s
+done
+
+# Create test database.
+mysql -u root -e "CREATE USER 'aur'@'%' IDENTIFIED BY 'aur'" \
+    2>/dev/null || /bin/true
+mysql -u root -e "DROP DATABASE aurweb_test" 2>/dev/null || /bin/true
+mysql -u root -e "CREATE DATABASE aurweb_test"
+mysql -u root -e "GRANT ALL PRIVILEGES ON aurweb_test.* TO 'aur'@'%'"
+mysql -u root -e "FLUSH PRIVILEGES"
+
+# Shutdown mariadb.
+mysqladmin -uroot shutdown
+
+exec "$@"

docker/scripts/run-pytests.sh

@@ -0,0 +1,41 @@
+#!/bin/bash
+set -eou pipefail
+
+COVERAGE=1
+PARAMS=()
+
+while [ $# -ne 0 ]; do
+    key="$1"
+    case "$key" in
+        --no-coverage)
+            COVERAGE=0
+            shift
+            ;;
+        -*)
+            echo "usage: $0 [--no-coverage] targets ..."
+            exit 1
+            ;;
+        *)
+            PARAMS+=("$key")
+            shift
+            ;;
+    esac
+done
+
+# Initialize the new database; ignore errors.
+python -m aurweb.initdb 2>/dev/null || /bin/true
+
+# Run pytest with optional targets in front of it.
+make -C test "${PARAMS[@]}" pytest
+
+# By default, report coverage and move it into cache.
+if [ $COVERAGE -eq 1 ]; then
+    make -C test coverage
+
+    # /cache is mounted as a volume. Copy coverage into it.
+    # Users can then sanitize the coverage locally in their
+    # aurweb root directory: ./util/fix-coverage ./cache/.coverage
+    rm -f /cache/.coverage
+    cp -v .coverage /cache/.coverage
+    chmod 666 /cache/.coverage
+fi

docker/scripts/run-sharness.sh

@@ -0,0 +1,7 @@
+#!/bin/bash
+set -eou pipefail
+
+# Initialize the new database; ignore errors.
+python -m aurweb.initdb 2>/dev/null || /bin/true
+
+make -C test sh

docker/scripts/run-tests.sh

@@ -0,0 +1,34 @@
+#!/bin/bash
+set -eou pipefail
+dir=$(dirname $0)
+
+# Clean up coverage and stuff.
+make -C test clean
+
+# Run sharness tests.
+bash $dir/run-sharness.sh
+
+# Run Python tests with MariaDB database.
+# Pass --silence to avoid reporting coverage. We will do that below.
+bash $dir/run-pytests.sh --no-coverage
+
+# Export SQLite aurweb configuration.
+export AUR_CONFIG=conf/config.sqlite
+
+# Run Python tests.
+bash $dir/run-pytests.sh --no-coverage
+
+make -C test coverage
+
+# /cache is mounted as a volume. Copy coverage into it.
+# Users can then sanitize the coverage locally in their
+# aurweb root directory: ./util/fix-coverage ./cache/.coverage
+rm -f /cache/.coverage
+cp -v .coverage /cache/.coverage
+chmod 666 /cache/.coverage
+
+# Run flake8 and isort checks.
+for dir in aurweb test migrations; do
+    flake8 --count $dir
+    isort --check-only $dir
+done

docker/test-mysql-entrypoint.sh

@@ -0,0 +1,23 @@
+#!/bin/bash
+set -eou pipefail
+
+DB_NAME="aurweb_test"
+DB_HOST="mariadb"
+DB_USER="aur"
+DB_PASS="aur"
+
+# Setup a config for our mysql db.
+cp -vf conf/config.dev conf/config
+sed -i "s;YOUR_AUR_ROOT;$(pwd);g" conf/config
+sed -ri "s/^(name) = .+/\1 = ${DB_NAME}/" conf/config
+sed -ri "s/^(host) = .+/\1 = ${DB_HOST}/" conf/config
+sed -ri "s/^(user) = .+/\1 = ${DB_USER}/" conf/config
+sed -ri "s/^;?(password) = .+/\1 = ${DB_PASS}/" conf/config
+
+# The port can be excluded from use if properly using
+# volumes to share the mysql socket from the mariadb service.
+# Example port sed:
+# sed -i "s/^;?(port = .+)$/\1/" conf/config
+
+# Continue onto the main command.
+exec "$@"

docker/test-sqlite-entrypoint.sh

@@ -0,0 +1,16 @@
+#!/bin/bash
+set -eou pipefail
+
+DB_BACKEND="sqlite"
+DB_NAME="aurweb.sqlite3"
+
+# Create an SQLite config from the default dev config.
+cp -vf conf/config.dev conf/config.sqlite
+cp -vf conf/config.defaults conf/config.sqlite.defaults
+
+# Modify it for SQLite.
+sed -i "s;YOUR_AUR_ROOT;$(pwd);g" conf/config.sqlite
+sed -ri "s/^(backend) = .+/\1 = ${DB_BACKEND}/" conf/config.sqlite
+sed -ri "s/^(name) = .+/\1 = ${DB_NAME}/" conf/config.sqlite
+
+exec "$@"

docker/tests-entrypoint.sh

@@ -0,0 +1,8 @@
+#!/bin/bash
+set -eou pipefail
+dir="$(dirname $0)"
+
+bash $dir/test-mysql-entrypoint.sh
+bash $dir/test-sqlite-entrypoint.sh
+
+exec "$@"