mirror of
https://github.com/vinta/awesome-python.git
synced 2026-10-02 08:23:10 +08:00
refactor: rename download-fetch scripts to source-suffixed scheme
fetch_pypi_downloads.py becomes fetch_pypi_downloads_via_clickpy.py and fetch_pepy_downloads.py becomes fetch_pypi_downloads_via_pepy.py, ahead of splitting the BigQuery path into its own file. Updates the usage strings, the cross-file import, and the audit-the-list skill's references to match. Pure rename, no behavior change. Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
@@ -16,7 +16,7 @@ Resolve the scope from the arguments. Named sections mean exactly those, whether
|
||||
|
||||
Fetch live evidence for every entry in scope before judging anything (CLAUDE.md verification rule):
|
||||
|
||||
- **Downloads/month**: `cd website && UV_PYTHON=3.13 uv run python fetch_pypi_downloads.py --names-file <file>` — default source is ClickPy (free, keyless, one batched query covers the full README). `--bigquery` switches to the canonical cross-check on the maintainer's own GCP account; there, run `--dry-run` first and read the docstring's cost constraints before batching. Spot-checks: `uv run python fetch_pepy_downloads.py <name> ...` (needs `PEPY_TECH_API_KEY` in repo-root `.env`, throttled to 5 requests/minute) or `https://pypistats.org/api/packages/{name}/recent` paced 8s or slower. pypistats excludes mirror/CI traffic; ClickPy, BigQuery, and pepy include it — never mix sources within one comparison.
|
||||
- **Downloads/month**: `cd website && UV_PYTHON=3.13 uv run python fetch_pypi_downloads_via_clickpy.py --names-file <file>` — default source is ClickPy (free, keyless, one batched query covers the full README). `--bigquery` switches to the canonical cross-check on the maintainer's own GCP account; there, run `--dry-run` first and read the docstring's cost constraints before batching. Spot-checks: `uv run python fetch_pypi_downloads_via_pepy.py <name> ...` (needs `PEPY_TECH_API_KEY` in repo-root `.env`, throttled to 5 requests/minute) or `https://pypistats.org/api/packages/{name}/recent` paced 8s or slower. pypistats excludes mirror/CI traffic; ClickPy, BigQuery, and pepy include it — never mix sources within one comparison.
|
||||
- **Repo state**: archived flag, last push, created date, stars, description — `gh api repos/{owner}/{repo}`, GitLab API for GitLab-hosted projects.
|
||||
- **PyPI metadata** (`https://pypi.org/pypi/{name}/json`) wherever a name might not be the canonical package — ownership collisions and wrong display names surface here.
|
||||
|
||||
|
||||
@@ -30,7 +30,7 @@ running.
|
||||
Results are cached to data/pypi_downloads.tsv (merged with any existing
|
||||
rows); names with no PyPI rows are written as NOT_FOUND.
|
||||
|
||||
Usage: python fetch_pypi_downloads.py [--bigquery] [--dry-run] [--names-file FILE]
|
||||
Usage: python fetch_pypi_downloads_via_clickpy.py [--bigquery] [--dry-run] [--names-file FILE]
|
||||
"""
|
||||
|
||||
import re
|
||||
|
||||
@@ -4,7 +4,7 @@
|
||||
Cross-check for a handful of packages during audits — not for full-README
|
||||
sweeps: the free API key is throttled to 5 requests/minute (10 burst), so
|
||||
this script sleeps 12s between requests and 530 names would take ~2 hours
|
||||
(use fetch_pypi_downloads.py for bulk). Reads PEPY_TECH_API_KEY from the
|
||||
(use fetch_pypi_downloads_via_clickpy.py for bulk). Reads PEPY_TECH_API_KEY from the
|
||||
environment, falling back to the repo-root .env. The v2 endpoint returns
|
||||
~90 days of per-day per-version counts; this script sums the most recent
|
||||
30 days present in the response across all versions. pepy counts include
|
||||
@@ -13,7 +13,7 @@ ClickPy/BigQuery figures; pypistats.org excludes mirrors, so never mix
|
||||
the two in one comparison. Results print to stdout as TSV and are not
|
||||
cached — data/pypi_downloads.tsv stays single-source.
|
||||
|
||||
Usage: python fetch_pepy_downloads.py NAME [NAME ...]
|
||||
Usage: python fetch_pypi_downloads_via_pepy.py NAME [NAME ...]
|
||||
"""
|
||||
|
||||
import os
|
||||
@@ -22,7 +22,7 @@ import time
|
||||
from pathlib import Path
|
||||
|
||||
import httpx
|
||||
from fetch_pypi_downloads import normalize
|
||||
from fetch_pypi_downloads_via_clickpy import normalize
|
||||
|
||||
ENV_FILE = Path(__file__).parent.parent / ".env"
|
||||
PEPY_URL = "https://api.pepy.tech/api/v2/projects/{name}"
|
||||
@@ -49,7 +49,7 @@ def last_30_day_total(downloads_per_day: dict[str, dict[str, int]]) -> int:
|
||||
|
||||
def main() -> None:
|
||||
if len(sys.argv) < 2:
|
||||
print("Usage: python fetch_pepy_downloads.py NAME [NAME ...]", file=sys.stderr)
|
||||
print("Usage: python fetch_pypi_downloads_via_pepy.py NAME [NAME ...]", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
names = [normalize(name) for name in sys.argv[1:]]
|
||||
with httpx.Client(headers={"X-API-Key": load_api_key()}, timeout=30) as client:
|
||||
|
||||
Reference in New Issue
Block a user