Merge pull request #1366 from EpicWink/more-indexes-2

Add more package index projects
pypa · Feb 1, 2024 · 144ebf1 · 144ebf1
2 parents c8f51ab + 35c7f84
commit 144ebf1
Show file tree

Hide file tree

Showing 3 changed files with 305 additions and 33 deletions.
diff --git a/source/guides/hosting-your-own-index.rst b/source/guides/hosting-your-own-index.rst
@@ -52,10 +52,86 @@ directory with autoindex enabled. For an example using the built in Web server
 in `Twisted`_, you would simply run ``twistd -n web --path .`` and then
 instruct users to add the URL to their installer's configuration.
 
+
+Existing projects
+=================
+
+.. list-table::
+   :header-rows: 1
+
+   * - Project
+     - Package upload
+     - PyPI fall-through [2]_
+     - Additional notes
+
+   * - :ref:`devpi`
+     - ✔
+     - ✔
+     - multiple indexes with inheritance, with syncing, replication, fail-over;
+       mirroring
+
+   * - :ref:`simpleindex`
+     -
+     - ✔
+     -
+
+   * - :ref:`pypiserver`
+     - ✔
+     -
+     -
+
+   * - :ref:`pypiprivate`
+     -
+     -
+     -
+
+   * - :ref:`pypicloud`
+     -
+     -
+     - unmaintained; also cached proxying; authentication, authorisation
+
+   * - :ref:`pywharf`
+     -
+     -
+     - unmaintained; serve files in GitHub
+
+   * - :ref:`pulppython`
+     - ✔
+     -
+     - also mirroring, proxying; plugin for Pulp
+
+   * - :ref:`pip2pi`
+     -
+     -
+     - also mirroring; manual synchronisation
+
+   * - :ref:`dumb-pypi`
+     -
+     -
+     - not a server, but a static file site generator
+
+   * - :ref:`httpserver`
+     -
+     -
+     - standard-library
+
+   * - `Apache <https://httpd.apache.org/>`_
+     -
+     - ✔
+     - using
+       `mod_rewrite
+       <https://httpd.apache.org/docs/current/mod/mod_rewrite.html>`_
+       and
+       `mod_cache_disk
+       <https://httpd.apache.org/docs/current/mod/mod_cache_disk.html>`_,
+       you can cache requests to package indexes through an Apache server
+
 ----
 
 .. [1] For complete documentation of the simple repository protocol, see
        :ref:`simple repository API <simple-repository-api>`.
 
+.. [2] Can be configured to fall back to PyPI (or another package index)
+       if a requested package is missing.
 
 .. _Twisted: https://twistedmatrix.com/
diff --git a/source/guides/index-mirrors-and-caches.rst b/source/guides/index-mirrors-and-caches.rst
@@ -5,19 +5,24 @@ Package index mirrors and caches
 ================================
 
 :Page Status: Incomplete
-:Last Reviewed: 2014-12-24
+:Last Reviewed: 2023-11-08
 
-
-Mirroring or caching of PyPI can be used to speed up local package installation,
+Mirroring or caching of PyPI (and other
+:term:`package indexes <Package Index>`) can be used to speed up local
+package installation,
 allow offline work, handle corporate firewalls or just plain Internet flakiness.
 
-Three options are available in this area:
+There are multiple classes of options in this area:
+
+1. local/hosted caching of package indexes.
+
+2. local/hosted mirroring of a package index. A mirror is a (whole or
+   partial) copy of a package index, which can be used in place of the
+   original index.
 
-1. pip provides local caching options,
-2. devpi provides higher-level caching option, potentially shared amongst
-   many users or machines, and
-3. bandersnatch provides a local complete mirror of all PyPI :term:`packages
-   <Distribution Package>`.
+3. private package index with fall-through to public package indexes (for
+   example, to mitigate dependency confusion attacks), also known as a
+   proxy.
 
 
 Caching with pip
@@ -38,26 +43,75 @@ cached copies of :term:`packages <Distribution Package>`:
       python3 -m pip install --no-index --find-links=/tmp/wheelhouse SomeProject
 
 
-Caching with devpi
-------------------
-
-devpi is a caching proxy server which you run on your laptop, or some other
-machine you know will always be available to you. See the `devpi
-documentation for getting started`__.
-
-__ https://devpi.net/docs/devpi/devpi/latest/+d/quickstart-pypimirror.html
-
-
-Complete mirror with bandersnatch
-----------------------------------
-
-bandersnatch will set up a complete local mirror of all PyPI :term:`packages
-<Distribution Package>` (externally-hosted packages are not mirrored). See
-the `bandersnatch documentation for getting that going`__.
-
-__ https://github.com/pypa/bandersnatch/
-
-A benefit of devpi is that it will create a mirror which includes
-:term:`packages <Distribution Package>` that are external to PyPI, unlike
-bandersnatch which will only cache :term:`packages <Distribution Package>`
-hosted on PyPI.
+Existing projects
+-----------------
+
+.. list-table::
+   :header-rows: 1
+
+   * - Project
+     - Cache
+     - Mirror
+     - Proxy
+     - Additional notes
+
+   * - :ref:`devpi`
+     - ✔
+     - ✔
+     -
+     - multiple indexes with inheritance; syncing, replication, fail-over;
+       package upload
+
+   * - :ref:`bandersnatch`
+     - ✔
+     - ✔
+     -
+     -
+
+   * - :ref:`simpleindex`
+     -
+     -
+     - ✔
+     - custom plugin enables caching; re-routing to other package indexes
+
+   * - :ref:`pypicloud`
+     - ✔
+     -
+     - ✔
+     - unmaintained; authentication, authorisation
+
+   * - :ref:`pulppython`
+     -
+     - ✔
+     - ✔
+     - plugin for Pulp; multiple proxied indexes; package upload
+
+   * - :ref:`proxpi`
+     - ✔
+     -
+     - ✔
+     - multiple proxied indexes
+
+   * - :ref:`nginx_pypi_cache`
+     - ✔
+     -
+     - ✔
+     - multiple proxied indexes
+
+   * - :ref:`flaskpypiproxy`
+     - ✔
+     -
+     - ✔
+     - unmaintained
+
+   * - `Apache <https://httpd.apache.org/>`_
+     - ✔
+     -
+     - ✔
+     - using
+       `mod_rewrite
+       <https://httpd.apache.org/docs/current/mod/mod_rewrite.html>`_
+       and
+       `mod_cache_disk
+       <https://httpd.apache.org/docs/current/mod/mod_cache_disk.html>`_,
+       you can cache requests to package indexes through an Apache server