Some fixes for the /rss endpoints
* Load all data in one go:
Previously data was lazy loaded thus it made several sub-queries per
package (> 200 queries for composing the rss data for a single request).
Now we are performing a single SQL query.
(request time improvement: 550ms -> 130ms)
This also fixes aurweb-errors#510 and alike
* Remove some "dead code":
The fields "source, author, link" were never included in the rss output
(wrong or insufficient data passed to the different entry.xyz functions)
Nobody seems to be missing them anyways, so let's remove em.
* Remove "Last-Modified" header:
Obsolete since nginx can/will only handle "If-Modified-Since" requests
in it's current configuration. All requests are passed to fastapi anyways.
Signed-off-by: moson <moson@archlinux.org>
In some translations we might be missing replacement placeholders (%).
This turns out to be problematic when calling the format function.
Wrap the jinja2 format function and just return the string unformatted
when % is missing.
Fixes: #341
Signed-off-by: moson <moson@archlinux.org>
Users might accidentally past their ssh key multiple times
when they try to register or edit their account.
Convert our of list of keys to a set, removing any double keys.
Signed-off-by: moson <moson@archlinux.org>
Package requests are sent to the ML as well as users (CC).
For those who chose to hide their mail address,
we should add them to the BCC list instead.
Signed-off-by: moson-mo <mo-son@mailbox.org>
Fix order of recipients for "FlagNotification" test.
Apply sorting to the recipients query.
(only relevant for tests, but who knows when they change things again)
MariaDB 11 includes some changes related to the
query optimizer. Turns out that this might have effects
on how records are ordered for certain queries.
(in case no ORDER BY clause was specified)
https://mariadb.com/kb/en/mariadb-11-0-0-release-notes/
Signed-off-by: moson <moson@archlinux.org>
The queries being done on the package search page are quite costly.
(Especially the default one ordered by "Popularity" when navigating to /packages)
Let's add the search results to the Redis cache:
Every result of a search query is being pushed to Redis until we hit our maximum of 50k.
An entry expires after 3 minutes before it's evicted from the cache.
Lifetime an Max values are configurable.
Signed-off-by: moson-mo <mo-son@mailbox.org>
We currently sorting on two columns in different tables which is quite
expensive in terms of performance:
MariaDB is first merging the data into some temporary table to apply the
sorting and record limiting.
We can tweak a couple of these queries by changing the "order by" clause
such that they refer to columns within the same table (PackageBases).
So instead performing the second sorting on "Packages.Name", we do
this on "PackageBases.Name" instead.
This should still be "good enough" to produce properly sorted results.
Signed-off-by: moson-mo <mo-son@mailbox.org>
We're currently creating a "PackageBases" when the "restore" command is executed.
This is problematic for pkgbases that never existed before.
In those cases it will create the record but fail in the update.py script.
Thus it leaves an orphan "PackageBases" record in the DB
(which does not have any related "Packages" record(s))
Navigating to such a packages /pkgbase/... URL will result in a crash
since it is not foreseen to have "orphan" pkgbase records.
We can safely remove the early creation of that record because
it'll be taken care of in the update.py script that is being called
We'll also fix some tests. Before it was executing a dummy script
instead of "update.py" which might be a bit misleading
since it did not check the real outcome of our "restore" action.
Signed-off-by: moson-mo <mo-son@mailbox.org>
This allows migration of git history for packages dropped from a repo to AUR
in case they contain PGP key material
Signed-off-by: moson-mo <mo-son@mailbox.org>
Use "contains" filtering instead of an exact match
when a package name filter is given.
This makes it easier to find requests for a "group" of packages.
Signed-off-by: moson-mo <mo-son@mailbox.org>
this is similar to the message that gitlab produces:
$ ssh -T aur.archlinux.org
Welcome to AUR, gromit! Interactive shell is disabled.
Try `ssh ssh://aur@aur.archlinux.org help` for a list of commands.
$ ssh -T gitlab.archlinux.org
Welcome to GitLab, @gromit!
Signed-off-by: Christian Heusel <christian@heusel.eu>
This is done in order to relax the constraints so that dropping packages
from the official repos can be done with preserving their history.
Its sufficient to also have this present in the latest commit of a push.
Signed-off-by: Christian Heusel <christian@heusel.eu>
This should match more closely the expectation of a user.
A session cookie should vanish on browser close
and you thus they need to authenticate again.
There is no need to bump the expiration of AURSID either,
so we can remove that part.
Signed-off-by: moson-mo <mo-son@mailbox.org>
If it's a session cookie it poses issues for users
whose browsers wipe session cookies after close.
They'd be logged out early even if they chose
the "remember me" option when they log in.
Signed-off-by: moson-mo <mo-son@mailbox.org>
We add a new config option for cookies with a 400 day lifetime.
AURLANG should survive longer for unauthenticated users.
Today they have to set this again after each browser restart.
(for users whose browsers wipe session cookies on close)
authenticated users don't need this cookie
since the setting is saved to the DB
Signed-off-by: moson-mo <mo-son@mailbox.org>
We don't need to set these cookies when logging in.
These settings are saved to the DB anyways.
(and they are picked up from there as well for any web requests,
when no cookies are given)
Signed-off-by: moson-mo <mo-son@mailbox.org>
We don't need to set these cookies when an account is edited.
These settings are saved to the DB anyways.
(and they are picked up from there as well for any web requests,
when no cookies are given)
Setting these cookies can even be counter-productive:
Imagine a TU/Dev editing another users account.
They would overwrite their own cookies with the other users TZ/LANG settings.
Signed-off-by: moson-mo <mo-son@mailbox.org>
Whenever the AURREMEMBER cookie was defined, regardless of its value,
"remember_me" is always set to True
The get method of a dict returns a string,
converting a value of str "False" into a bool -> True
We have to check AURREMEMBERs value instead.
Signed-off-by: moson-mo <mo-son@mailbox.org>
- Add package name textbox for filtering requests (with auto-suggest)
- Make "x pending requests" a link for TU/Dev on the package details page
Signed-off-by: moson-mo <mo-son@mailbox.org>
Currently, the "Repo" column in "OfficialProviders" is not updated
when a package is moved from one repository to another.
Note that we only save a package/provides combination once,
hence if a package is available in core and testing at the same time,
it would only put just one record into the OfficialProviders table.
We iterate through the repos one by one and the last value
is kept for mapping a (package/provides) combination to a repo.
Due to that, the repos listed in the "sync-db" config setting
should be ordered such that the "testing" repos are listed first.
Signed-off-by: moson-mo <mo-son@mailbox.org>
we copy static files used by PHP and Python versions into /static
preparation work for the removal of the PHP version
Signed-off-by: moson-mo <mo-son@mailbox.org>
TUs and Devs can delete and merge packages directly.
Currently the comments they enter, don't end up in the ML notification.
Include the comment in the notifications for direct deletion / merge
Signed-off-by: moson-mo <mo-son@mailbox.org>
middleware must be added before startup:
fixes: "RuntimeError: Cannot add middleware after an application has started"
https://fastapi.tiangolo.com/release-notes/#0910
Signed-off-by: moson-mo <mo-son@mailbox.org>
Data for packages that do not have a "Last Packager"
(e.g. because the user account was deleted)
should still be available from the /rpc and metadata archives.
Signed-off-by: moson-mo <mo-son@mailbox.org>
With the removal of the "setup-repo" command this script becomes obsolete,
because it is not possible to reserve a repo anymore.
Hence we don't need cleanup.
We've also seen issues in case the last packager's user account is removed,
leading to the deletion of a Package.
Let's deactivate this for now.
Issue report: #425
Signed-off-by: moson-mo <mo-son@mailbox.org>
Fixes errors that might occur when loading the package details page.
Problem:
We are querying a list of "Required by" packages.
This list is loaded with all details for a "PackageDependency" record.
Now we also have a reference to some attributes from the
related package (PackageDependency.Package.xxx)
This will effectively trigger the ORM to run another query (lazyload),
to fetch the missing Package data (for each PackageDependency record).
At that point it might have happened that a referenced package
got deleted / updated so that we can't retrieve this data anymore and
our dep.Package object is "None"
Fix:
We can force our query to include Package data right away.
Thus we can avoid running a separate query (per "required by"...)
As a side-effect we get better performance.
Signed-off-by: moson-mo <mo-son@mailbox.org>
When we edit a comment we can enable notifications (if not yet enabled).
We should also do this when the comment text is not changed.
Signed-off-by: moson-mo <mo-son@mailbox.org>
Provides a convenient way to check for responses on the
mailing list prior to Accepting/Rejecting requests.
We compute the Message-ID hash that can be used to
link back to the article in the mailing list archive.
Signed-off-by: moson-mo <mo-son@mailbox.org>
Improves performance for queries with large result sets.
The "group by" clause can be removed for all search types but the keywords.
Signed-off-by: moson-mo <mo-son@mailbox.org>
