We try to find packages when a user enters a URL like /somepkg
or accidentally opens /somepkg.git in the browser.
However, it currently also does this for URL's like /pkgbase/doesnotexist
and falsely interprets "pkgbase" part as a package or pkgbase name.
This in combination with a pkgbase that is named "pkgbase" generates
some misleading 404 message for URL's like /pkgbase/doesnotexist.
That being said, we should probably add pkgbase to the blacklist check
as well (we do this for pkgname already) and add things like
"pkgbase" to the blacklist -> Will be picked up in another commit.
Signed-off-by: moson <moson@archlinux.org>
The current query is a bit mixed up. The intention was to return the
number of unaccepted records. Now it does also count all records
that were accepted by some other user though.
Let's check the total number of terms vs. the number of accepted
records (by our user) instead.
Signed-off-by: moson <moson@archlinux.org>
we copy static files used by PHP and Python versions into /static
preparation work for the removal of the PHP version
Signed-off-by: moson-mo <mo-son@mailbox.org>
middleware must be added before startup:
fixes: "RuntimeError: Cannot add middleware after an application has started"
https://fastapi.tiangolo.com/release-notes/#0910
Signed-off-by: moson-mo <mo-son@mailbox.org>
swagger-ui uses inline javascript to bootstrap itself, so we need to
allow unsafe inline because we can't give swagger-ui a nonce to embed.
Signed-off-by: Kevin Morris <kevr@0cost.org>
As repeats of these traceback notifications were annoying some of
the devops staff, and it took coordination to share tracebacks with
developers, this commit removes that responsibility off of devops
by reporting tracebacks to Gitlab repositories in the form of issues.
- removed ServerErrorNotification
- removed notifications.postmaster configuration option
- added notifications.gitlab-instance option
- added notifications.error-project option
- added notifications.error-token option
- added aurweb.exceptions.handle_form_exceptions, a POST route decorator
Issues are filed confidentially. This change will need updates
in infrastructure's ansible configuration before this can be
applied to aur.archlinux.org.
Signed-off-by: Kevin Morris <kevr@0cost.org>
This commit does quite a bit:
- Catches unhandled exceptions raised in the route handler and
produces a 500 Internal Server Error Arch-themed response.
- Each unhandled exception causes a notification to be sent to new
`notifications.postmaster` email with a "Traceback ID."
- Traceback ID is logged to the server along with the traceback which
caused the 500: `docker-compose logs fastapi | grep '<traceback_id>'`
- If `options.traceback` is set to `1`, traceback is displayed in
the new 500.html template.
Signed-off-by: Kevin Morris <kevr@0cost.org>
This gets raised when a client closes a connection before receiving
a valid response; this is not controllable from our side.
Signed-off-by: Kevin Morris <kevr@0cost.org>
Now, when we want to add, remove routes, our base routes should
be defined in aurweb.routers.__init__.APP_ROUTES.
Signed-off-by: Kevin Morris <kevr@0cost.org>
Move package request routes and related routes to their
respective routers. In addition, move some utility used
for requests over from `aurweb.packages`.
Introduced routers:
- `aurweb.routers.requests`
Introduced package:
- `aurweb.requests`
Introduced module:
- `aurweb.requests.util`
Changes:
- Moved `aurweb.packages.validate` to `aurweb.pkgbase.validate`
- Moved requests listing & request closure routes to
`aurweb.routers.requests`
- Moved pkgbase request creation route to `aurweb.routers.pkgbase`
- Moved `get_pkgreq_by_id` from `aurweb.packages.util` to
`aurweb.requests.util` and fixed its return type hint.
Signed-off-by: Kevin Morris <kevr@0cost.org>
Introduces new router:
- `aurweb.routers.pkgbase`
Introduces new package:
- `aurweb.pkgbase`
Introduces new modules:
- `aurweb.pkgbase.actions`
- `aurweb.pkgbase.util`
Changes:
- `pkgbase_{action}_instance` functions are now located in
`aurweb.pkgbase.actions`.
- `pkgbase`-wise routes have been moved to
`aurweb.routers.pkgbase`.
- `make_single_context` was moved to
`aurweb.pkgbase.util.make_context`.
Signed-off-by: Kevin Morris <kevr@0cost.org>
Without the increment, we've seen tests failed due to recursion
errors caused by starlette's base middleware. Just make it safe
in case nobody supplies TEST_RECURSION_LIMIT.
Signed-off-by: Kevin Morris <kevr@0cost.org>
This commit provides custom metrics, so we can group requests
into their route paths and not by the arguments given, e.g.
/pkgbase/some-package -> /pkgbase/{name}. We also count RPC
requests as `http_api_requests_total`, split by the RPC
query "type" argument.
- `http_api_requests_total`
- Labels: ["type", "status"]
- `http_requests_total`
- Number of HTTP requests in total.
- Labels: ["method", "path", "status"]
Signed-off-by: Kevin Morris <kevr@0cost.org>
The default recursion limit used by Docker's archlinux:base-devel
Python package becomes problematic in some cases when running tests
against our FastAPI application using starlette.testclient.TestClient
(aliased to fastapi.testclient.TestClient). starlette ends up with
test failures because it exceeds the recursion limit, but this only
happens when using the `TestClient`. When the ASGI servers are run,
this is not an issue and so in that case, the recursion limit has
not been touched.
This change uses a `TEST_RECURSION_LIMIT` environment variable to
modify the recursion limit of the FastAPI application. This variable
is, by default, only supplied when running pytests in Docker, but
can be force-supplied by the user.
TEST_RECURSION_LIMIT=10000 has been added to `.env` and `.gitlab-ci.yml`.
Signed-off-by: Kevin Morris <kevr@0cost.org>
There are slight differences in that, with `python-feedgen`,
an empty description field completely omits the description,
but includes the description when there is one.
Signed-off-by: Kevin Morris <kevr@0cost.org>
A new middleware which redirects requests going to '/route?id=some_id'
to '/route/some_id'. In the FastAPI application, we'll prefer using
restful layouts where possible where resource-based ids are
parameters of the request uri: '/route/{resource_id}'.
Signed-off-by: Kevin Morris <kevr@0cost.org>
This commit implements the '/tu' Trusted User index page.
In addition to this functionality, this commit introduces
the following jinja2 filters:
- dt: util.timestamp_to_datetime
- as_timezone: util.as_timezone
- dedupe_qs: util.dedupe_qs
- urlencode: urllib.parse.quote_plus
There's also a new decorator that can be used to enforce
permissions: `account_type_required`. If a user does not
meet account type requirements, they are redirected to '/'.
```
@auth_required(True)
@account_type_required({"Trusted User"})
async def some_route(request: fastapi.Request):
return Response("You are a Trusted User!")
```
Routes added:
- `GET /tu`: aurweb.routers.trusted_user.trusted_user
Signed-off-by: Kevin Morris <kevr@0cost.org>
This clones the end goal behavior of PHP, but it does not
concern itself with the revision form array at all.
Since this page on PHP renders out the entire list of
terms that a user needs to accept, we can treat a
POST request with the "accept" checkbox enabled as a
request to accept all unaccepted (or outdated revision)
terms.
This commit also adds in a new http middleware used to
redirect authenticated users to '/tos' if they have not
yet accepted all terms.
Signed-off-by: Kevin Morris <kevr@0cost.org>
This resolves logging issues with alembic on aurweb.initdb
in addition to adding more logging utilities for aurweb
and tests in general.
Developers should fetch a logger for their specific module
via `logging.getLogger(__name__)`.
Signed-off-by: Kevin Morris <kevr@0cost.org>
This commit introduces a middleware function which adds
the following security headers to each response:
- Content-Security-Policy
- This includes a new `nonce`, which is tied to a user
via authentication middleware. Both an anonymous user
and an authenticated user recieve their own random nonces.
- X-Content-Type-Options
- Referrer-Policy
- X-Frame-Options
They are then tested for existence in test/test_routes.py.
Note: The overcomplicated-looking asyncio behavior in the
middleware function is used to avoid a warning about the old
coroutine awaits being deprecated. See
https://docs.python.org/3/library/asyncio-task.html#asyncio.wait
for more detail.
Signed-off-by: Kevin Morris <kevr@0cost.org>
AUR renders its own 404 Not Found page when a bad route
is encountered. Introducing the previous verification
caused an error in this case when setting a language
while viewing the Not Found page. So, instead of checking
through routes, just make sure that the next parameter
starts with a '/' character, which removes the possibility
of any cross attacks.
+ Removed aurweb.asgi.routes; no longer needed.
Signed-off-by: Kevin Morris <kevr@0cost.org>
Introduced `get|post` `/passreset` routes. These routes mimic the
behavior of the existing PHP implementation, with the exception of
HTTP status code returns.
Routes added:
GET /passreset
POST /passreset
Routers added:
aurweb.routers.accounts
* On an unknown user or mismatched resetkey (where resetkey must ==
user.resetkey), return HTTP status NOT_FOUND (404).
* On another error in the request, return HTTP status BAD_REQUEST (400).
Both `get|post` routes requires that the current user is **not**
authenticated, hence `@auth_required(False, redirect="/")`.
+ Added auth_required decorator to aurweb.auth.
+ Added some more utility to aurweb.models.user.User.
+ Added `partials/error.html` template.
+ Added `passreset.html` template.
+ Added aurweb.db.ConnectionExecutor functor for paramstyle logic.
Decoupling the executor logic from the database connection logic
is needed for us to easily use the same logic with a fastapi
database session, when we need to use aurweb.scripts modules.
At this point, notification configuration is now required to complete
tests involved with notifications properly, like passreset.
`conf/config.dev` has been modified to include [notifications] sendmail,
sender and reply-to overrides. Dockerfile and .gitlab-ci.yml have been
updated to setup /etc/hosts and start postfix before running tests.
* setup.cfg: ignore E741, C901 in aurweb.routers.accounts
These two warnings (shown in the commit) are not dangerous and a bi-product
of maintaining compatibility with our current code flow.
Signed-off-by: Kevin Morris <kevr@0cost.org>
+ Added route: GET `/login` via `aurweb.routers.auth.login_get`
+ Added route: POST `/login` via `aurweb.routers.auth.login_post`
+ Added route: GET `/logout` via `aurweb.routers.auth.logout`
+ Added route: POST `/logout` via `aurweb.routers.auth.logout_post`
* Modify archdev-navbar.html template to toggle displays on auth state
+ Added login.html template
Signed-off-by: Kevin Morris <kevr@0cost.org>
+ Added aurweb.auth.AnonymousUser
* An instance of this model is returned as the request user
when the request is not authenticated
+ Added aurweb.auth.BasicAuthBackend
+ Add starlette's AuthenticationMiddleware to app middleware,
which uses our BasicAuthBackend facility
+ Added User.is_authenticated()
+ Added User.authenticate(password)
+ Added User.login(request, password)
+ Added User.logout(request)
+ Added repr(User(...)) representation
+ Added aurweb.auth.auth_required decorator.
This change uses the same AURSID logic in the PHP implementation.
Additionally, introduce a few helpers for authentication,
one of which being `User.update_password(password, rounds = 12)`
where `rounds` is a configurable number of salt rounds.
Signed-off-by: Kevin Morris <kevr@0cost.org>
+ Added Session class and global session object to aurweb.db,
these are sessions created by sqlalchemy ORM's sessionmaker
and will allow us to use declarative/imperative models.
Signed-off-by: Kevin Morris <kevr@0cost.org>
+ Mounted static files (at web/html) to /static.
+ Added AURWEB_VERSION to aurweb.config (this is used around HTML
to refer back to aurweb's release on git.archlinux.org), so we
need it easily accessible in the Python codebase.
+ Implemented basic Jinja2 partials to put together whole aurweb
pages. This may be missing some things currently and is a WIP
until this set is ready to be merged.
+ Added config [options] aurwebdir = YOUR_AUR_ROOT; this configuration
option should specify the root directory of the aurweb project.
It is used by various parts of the FastAPI codebase to target
project directories.
Added routes via aurweb.routers.html:
* POST /language: Set your session language.
* GET /favicon.ico: Redirect to /static/images/favicon.ico.
* Some browsers always look for $ROOT/favicon.ico to get an icon
for the page being loaded, regardless of a specified "shortcut
icon" given in a <link> directive.
* GET /: Home page; WIP.
* Updated aurweb.routers.html.language passes query parameters to
its next redirection.
When calling aurweb.templates.render_template, the context passed should
be formed via the aurweb.templates.make_context. See
aurweb.routers.html.index for an example of this.
Signed-off-by: Kevin Morris <kevr@0cost.org>
Developers can go to /sso/login to get redirected to the SSO. On
successful login, the ID token is displayed.
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
aurweb.spawn used to launch only PHP’s built-in server. Now it spawns a
dummy FastAPI application too. Since both stacks spawn their own HTTP
server, aurweb.spawn also spawns nginx as a reverse proxy to mount them
under the same base URL, defined by aur_location in the configuration.
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
