Mirrors_Framework/awesome-python
1
0
Fork 0
mirror of https://github.com/vinta/awesome-python.git synced 2026-10-02 08:23:10 +08:00
Code Issues Packages Projects Releases Wiki Activity

audit: sweep Distributed Computing to the shortlist cap

Browse Source 
Removed luigi (1,546,263 downloads/month, repo active with a push 2026-07) — the drop is editorial: Spotify itself moved to Flyte, 2026 orchestration comparisons no longer name luigi, and the Workflow Orchestration slots in Job Schedulers go to airflow, prefect, dagster.

Flattened the sole Batch Processing subcategory — the label added nothing, the section is a single use case.

Reordered by tier and downloads: obvious choices ray 62.6M, pyspark 48.3M, dask 30.0M; challengers joblib 216.9M raw (scikit-learn-dependency-inflated, judgment call — raw count still sorts it first in tier) and mpi4py 1.0M (HPC standard, specific audience not small).

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
Vinta Chen
2026-08-16 12:15:51 +08:00
co-authored by Claude
parent 77e7b62d93
commit 4fc32225a9
1 changed files with 5 additions and 7 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+5 -7
View File
@@ -713,13 +713,11 @@ _Software and libraries for DevOps._
_Frameworks and libraries for Distributed Computing._
- Batch Processing
- [dask](https://github.com/dask/dask) - A flexible parallel computing library for analytic computing.
- [luigi](https://github.com/spotify/luigi) - A module that helps you build complex pipelines of batch jobs.
- [mpi4py](https://github.com/mpi4py/mpi4py) - Python bindings for MPI.
- [pyspark](https://github.com/apache/spark) - [Apache Spark](https://spark.apache.org/) Python API.
- [joblib](https://github.com/joblib/joblib) - A set of tools to provide lightweight pipelining in Python.
- [ray](https://github.com/ray-project/ray/) - A system for parallel and distributed Python that unifies the machine learning ecosystem.
- [ray](https://github.com/ray-project/ray/) - A system for parallel and distributed Python that unifies the machine learning ecosystem.
- [pyspark](https://github.com/apache/spark) - [Apache Spark](https://spark.apache.org/) Python API.
- [dask](https://github.com/dask/dask) - A flexible parallel computing library for analytic computing.
- [joblib](https://github.com/joblib/joblib) - A set of tools to provide lightweight pipelining in Python.
- [mpi4py](https://github.com/mpi4py/mpi4py) - Python bindings for MPI.
### Task Queues