Commit Graph
8 Commits
Author SHA1 Message Date
Vinta ChenandClaude b841db68ba refactor: make pypi_name_overrides.json entries self-documenting
Every entry is now {"package": str|null, "reason": str|null} instead of
a bare string/null. Reasons are required for null packages, explaining
why the name must never be queried (squatted name, stdlib module,
monorepo umbrella, GitHub-only project, and so on). Reasons are
optional for remaps and kept only on the six non-obvious ones: pytorch
(squatter), jinja (jinja is Jinja1), strawberry (unrelated bookmarking
service), django-rules (abandoned fork), django-rest-framework (dead
alias), and devpi (deprecated metapackage); plain publishes-as-X
remaps get a null reason.

load_overrides() in the clickpy fetcher now extracts the package field
from each entry; resolve() and the pepy/bigquery cross-check scripts
are unchanged since they consume load_overrides()'s output.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-16 19:08:55 +08:00
Vinta ChenandClaude fcfc65df00 data: record all remaining NOT_FOUND names as explicit null overrides
Every queried name now resolves 447/447. Adds 23 explicit null
overrides so squatters can never silently attach a PyPI number to
these names later: stdlib-named entries (concurrent-futures, difflib,
mimetypes, sqlite3, tkinter, tomllib, zoneinfo), interpreters
(micropython, pypy), monorepo umbrellas (azure-sdk-for-python,
google-cloud-python), self-hosted or distro-installed projects (odoo,
cloud-init, warehouse), GitHub-only projects (thealgorithms,
geodjango, django-db-models, django-ai-plugins, graphify,
sentry-skills, social-engineer-toolkit, trailofbits-skills), and
httpx-url (a class within httpx, not a package).

Caveat: graphify and django-ai-plugins are young projects that may
legitimately publish to PyPI later — flip their null to a remap
during a future audit if they do.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-16 18:57:13 +08:00
Vinta ChenandClaude f3ff733fd7 fix: add three more PyPI name remaps for NOT_FOUND triage
autobahn-python publishes as autobahn (7.1M/mo), pangu-py as pangu, and
strawberry-django as strawberry-graphql-django (1.5M/mo). httpx.URL is
left unmapped deliberately since it's a class within the httpx package,
not a package of its own.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-16 18:54:58 +08:00
Vinta ChenandClaude b440e64cd0 fix: resolve PyPI download counts through curated package overrides
A pypi.org identity sweep of all 438 cached rows (project_urls/home_page
vs entry GitHub URL) found download counts were looked up by README
display name, so entries whose name differs from the canonical package
silently measured squatters or dead predecessors: pytorch measured a
squatter (169,737/mo vs torch's 94M), jinja measured Jinja1 (3,168 vs
jinja2's 736M), django-rest-framework a dead alias package (real:
djangorestframework), django-rules an abandoned fork (real: rules),
strawberry an unrelated bookmarking service (real: strawberry-graphql),
devpi a deprecated metapackage (mapped to devpi-server).

New curated website/data/pypi_name_overrides.json maps normalized
README name to the real package, or null for projects not
pip-installable whose name is squatted or a relic (cpython, pyenv,
renpy, python-patterns, winpython); also maps mem0 to mem0ai, fasthtml
to python-fasthtml, and playwright-python to playwright.

All three fetch scripts resolve names through it; the clickpy TSV
cache gains a package column recording what each row actually
measured. .gitignore switches website/data/ to website/data/* with a
negation so the curated overrides file is tracked while caches stay
ignored.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-16 17:51:11 +08:00
Vinta ChenandClaude 0d9dfb2713 ci: consolidate star fetch into deploy workflow using Actions cache
Replace the separate fetch-github-stars.yml workflow (which committed
star data back to git) with an inline fetch step in deploy-website.yml.
Star data is now stored in Actions cache between runs, eliminating the
workflow_run trigger chain and the need to track github_stars.json in
the repository.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-03-18 22:44:29 +08:00
Vinta ChenandClaude 280f250ce0 feat: migrate README parser to markdown-it-py and refresh website
Switch readme_parser.py from regex-based parsing to markdown-it-py for
more robust and maintainable Markdown AST traversal. Update build pipeline,
templates, styles, and JS to support the new parser output. Refresh GitHub
stars data and update tests to match new parser behavior.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-03-18 20:33:36 +08:00
Vinta ChenandClaude 7eb9b11a67 data: remove zipline entry from github_stars.json
Follows the removal of zipline from README.md (see bd73b1f).

Co-Authored-By: Claude <noreply@anthropic.com>
2026-03-18 13:57:36 +08:00
Vinta ChenandClaude 177183d9bd add custom website build system
Replaces MkDocs with a bespoke Python site generator using Jinja2 templates
and Markdown. Adds uv for dependency management, GitHub Actions workflow for
deployment, and Makefile targets for local development (fetch_stars, build,
preview, deploy).

Co-Authored-By: Claude <noreply@anthropic.com>
2026-03-18 13:48:49 +08:00