Commit graph

18 commits

Author SHA1 Message Date
Kevin Morris
30e72d2db5 feat: archive git repository (experimental)
See doc/git-archive.md for general Git archive specifications
See doc/repos/metadata-repo.md for info and direction related to the new Git metadata archive
2022-09-24 16:51:25 +00:00
Joakim Saario
9c6c13b78a
style: Run pre-commit 2022-08-22 22:40:45 +02:00
Jelle van der Waa
a509e40474 fix(python): use standard dict/list type annotation
Since Python 3.9 list/dict can be used as type hint.
2022-08-02 12:06:58 +00:00
Awal Garg
b119db251b fixup: feat(archives): add .sha256 and construct archives in tmpdir 2022-02-09 07:03:12 +05:30
Kevin Morris
40a0e866e7
feat(archives): add {archive}.sha256 and construct archives in tmpdir
This change brings some new additions to our archives:
- SHA-256 .sha256 hexdigests
- We construct our archives in a tmpdir now and move them to the
archive destination when all are completed. This removes some
corrupted downloading when archiving is in-process.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2022-02-08 13:28:39 -08:00
Kevin Morris
33cddb36ff
fix: restore URL field in mkpkglists meta archives
Signed-off-by: Kevin Morris <kevr@0cost.org>
2022-02-07 01:20:42 -08:00
Kevin Morris
211ca5e49c
housekeep: define filters in their own modules
This patch cleans up aurweb.templates and removes direct
module-level initialization of the environment.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2022-01-18 03:06:17 -08:00
Kevin Morris
29c2d0de6b
change(mkpkglists): converted to use aurweb.db ORM
- Improved speed dramatically
- Removed mkpkglists sharness

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-11-28 19:55:08 -08:00
Kevin Morris
8788f99005
fix(mkpkglists): restore isort order
Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-11-10 13:54:18 -08:00
Kevin Morris
6e344ce9da
fix(mkpkglists): default keys to result[1]
Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-11-10 13:41:53 -08:00
Kevin Morris
4f7aeafa8d
feat(docker): host gzip archive downloads
- added config option [mkpkglists] archivedir
    - created by mkpkglists

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-11-10 07:39:23 -08:00
Kevin Morris
abbecf5194
change(mkpkglists): remove header comments
These comments change every time mkpkglists is run; which
would invalidate the ETag headers disbursed by the gzip
host. This commit removes those changing headers.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-11-10 07:39:17 -08:00
Kevin Morris
0155f4ea84
fix(mkpkglists): remove caching
We really need caching for this; however, our current caching
method will cause the script to bypass changes to columns
if they have nothing to do with IDs.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-11-09 02:07:13 -08:00
Kevin Morris
f3f662c696
fix(mkpkglists): improve package meta archive
The SQL logic in this file for package metadata now exactly
reflects RPC's search logic, without searching for specific
packages.

Two command line arguments are available:

    --extended | Include License, Keywords, Groups, relations
                 and dependencies.

When --extended is passed, the script will create a
packages-meta-ext-v1.json.gz, configured via packagesmetaextfile.

Archive JSON is in the following format: line-separated package objects
enclosed in a list:

    [
    {...},
    {...},
    {...}
    ]

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-11-09 02:06:50 -08:00
Kristian Klausen
f606140050
feat(PHP): Add packages dump file with more metadata 2021-11-09 02:04:58 -08:00
Lukas Fleischer
cb307bf01a Do not hardcode path to the Python interpreter
Use `/usr/bin/env python3` instead of `/usr/bin/python3` in the shebang
of Python scripts. This adds support for non-standard Python interpreter
paths such as the paths used in virtualenv environments.

Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2017-08-25 07:01:12 +02:00
Lukas Fleischer
d9883ee642 mkpkglists: Generate a list of user names
In addition to the packages list and the package base list, also create
a list of registered user names.

Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2017-08-01 07:08:29 +02:00
Lukas Fleischer
37188603b5 Make maintenance scripts installable
Add wrappers for the maintenance scripts to the setuptools
configuration.

Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2016-10-17 15:23:13 +02:00
Renamed from scripts/mkpkglists.py (Browse further)