aecb649473
First off: This commit changes the default development database backend to mysql. sqlite, however, is still completely supported with the caveat that a user must now modify config.dev to use the sqlite backend. While looking into this, it was discovered that our SQLAlchemy backend for mysql (mysql-connector) completely broke model attributes when we switched to utf8mb4_bin (binary) -- it does not correct the correct conversion to and from binary utf8mb4. The new, replacement dependency mysqlclient does. mysqlclient is also recommended in SQLAlchemy documentation as the "best" one available. The mysqlclient backend uses a different exception flow then sqlite, and so tests expecting IntegrityError has to be modified to expect OperationalError from sqlalchemy.exc. So, for each model that we define, check keys that can't be NULL and raise sqlalchemy.exc.IntegrityError if we have to. This way we keep our exceptions uniform. Signed-off-by: Kevin Morris <kevr@0cost.org> |
||
|---|---|---|
| .. | ||
| scripts | ||
| __init__.py | ||
| Makefile | ||
| README.md | ||
| setup.sh | ||
| sharness.sh | ||
| t1100-git-auth.t | ||
| t1200-git-serve.t | ||
| t1300-git-update.t | ||
| t2100-mkpkglists.t | ||
| t2200-tuvotereminder.t | ||
| t2300-pkgmaint.t | ||
| t2400-aurblup.t | ||
| t2500-notify.t | ||
| t2600-rendercomment.t | ||
| t2700-usermaint.t | ||
| test_accepted_term.py | ||
| test_account_type.py | ||
| test_accounts_routes.py | ||
| test_api_rate_limit.py | ||
| test_asgi.py | ||
| test_auth.py | ||
| test_auth_routes.py | ||
| test_ban.py | ||
| test_captcha.py | ||
| test_config.py | ||
| test_db.py | ||
| test_dependency_type.py | ||
| test_exceptions.py | ||
| test_group.py | ||
| test_initdb.py | ||
| test_l10n.py | ||
| test_license.py | ||
| test_package.py | ||
| test_package_base.py | ||
| test_package_dependency.py | ||
| test_package_group.py | ||
| test_package_keyword.py | ||
| test_package_license.py | ||
| test_package_relation.py | ||
| test_popupdate.py | ||
| test_relation_type.py | ||
| test_routes.py | ||
| test_session.py | ||
| test_ssh_pub_key.py | ||
| test_term.py | ||
| test_time.py | ||
| test_user.py | ||
Running tests
To run all tests, you may run make check under test/ (alternative targets:
make pytest, make sh).
For more control, you may use the prove or pytest command, which receives a
directory or a list of files to run, and produces a report.
Each test script is standalone, so you may run them individually. Some tests may receive command-line options to help debugging. See for example sharness's documentation for shell test scripts: https://github.com/chriscool/sharness/blob/master/README.git
Dependencies
For all the test to run, the following Arch packages should be installed:
- pyalpm
- python-alembic
- python-bleach
- python-markdown
- python-pygit2
- python-sqlalchemy
- python-srcinfo
- python-coverage
- python-pytest
- python-pytest-cov
- python-pytest-asyncio
- postfix
- openssh
Running tests
Recommended method of running tests: make check.
First, setup the test configuration:
$ sed -r 's;YOUR_AUR_ROOT;$(pwd);g' conf/config.dev > conf/config
You'll need to make sure that emails can be sent out by aurweb.scripts.notify. If you don't have anything setup, just install postfix and start it before running tests.
With those installed, one can run Python tests manually with any AUR config
specified by AUR_CONFIG:
$ AUR_CONFIG=conf/config coverage run --append /usr/bin/pytest test/
After tests are run (particularly, with coverage run included), one can
produce coverage reports.
# Print out a CLI coverage report.
$ coverage report
# Produce an HTML-based coverage report.
$ coverage html
When running make -C test, all tests ran will produce coverage data via
coverage run --append. After running make -C test, one can continue with
coverage reporting steps above. Running tests through make will test and
cover code ran by both aurweb scripts and our pytest collection.
Writing tests
Shell test scripts must follow the Test Anything Protocol specification: http://testanything.org/tap-specification.html
Python tests must be compatible with pytest and included in pytest test/
execution after setting up a configuration.
Tests must support being run from any directory. They may use $0 to determine their location. Python scripts should expect aurweb to be installed and importable without toying with os.path or PYTHONPATH.
Tests written in shell should use sharness. In general, new tests should be consistent with existing tests unless they have a good reason not to.
Debugging tests
By default, make -C test is quiet and does not print out verbose information
about tests being run. If a test is failing, one can look into verbose details
of sharness tests by executing them with the --verbose flag. Example:
./t1100_git_auth.t --verbose. This is particularly useful when tests happen
to fail in a remote continuous integration environment, where the reader does
not have complete access to the runner.