Add support to configure HTTPAdapter (#459)

docs/source/behavior&configuration.rst

@@ -3,13 +3,17 @@ Behavior & Configuration
 
 Host, Username and Password
 ***************************
-* The authentication credentials can be provided when instantiating :class:`~qbittorrentapi.client.Client`:
+* The authentication credentials can be provided when instantiating
+  :class:`~qbittorrentapi.client.Client`:
 
 .. code:: python
 
     qbt_client = Client(host="localhost:8080", username='...', password='...')
 
-* The credentials can also be specified after :class:`~qbittorrentapi.client.Client` is created but calling :meth:`~qbittorrentapi.auth.AuthAPIMixIn.auth_log_in` is not strictly necessary to authenticate the client; this will happen automatically for any API request.
+* The credentials can also be specified after :class:`~qbittorrentapi.client.Client`
+  is created but calling :meth:`~qbittorrentapi.auth.AuthAPIMixIn.auth_log_in` is not
+  strictly necessary to authenticate the client; this will happen automatically for any
+  API request.
 
 .. code:: python
 
@@ -23,10 +27,16 @@ Host, Username and Password
 
 qBittorrent Session Management
 ******************************
-* Any time a connection is established with qBittorrent, it instantiates a session to manage authentication for all subsequent API requests.
-* This client will transparently manage sessions by ensuring the client is always logged in in-line with any API request including requesting a new session upon expiration of an existing session.
-* However, each new :class:`~qbittorrentapi.client.Client` instantiation will create a new session in qBittorrent.
-* Therefore, if many :class:`~qbittorrentapi.client.Client` instances will be created be sure to call :class:`~qbittorrentapi.auth.AuthAPIMixIn.auth_log_out` for each instance or use a context manager.
+* Any time a connection is established with qBittorrent, it instantiates a session to
+  manage authentication for all subsequent API requests.
+* This client will transparently manage sessions by ensuring the client is always logged
+  in in-line with any API request including requesting a new session upon expiration of
+  an existing session.
+* However, each new :class:`~qbittorrentapi.client.Client` instantiation will create a
+  new session in qBittorrent.
+* Therefore, if many :class:`~qbittorrentapi.client.Client` instances will be created be
+  sure to call :class:`~qbittorrentapi.auth.AuthAPIMixIn.auth_log_out` for each instance
+  or use a context manager.
 * Otherwise, qBittorrent may experience abnormally high memory usage.
 
 .. code:: python
@@ -35,24 +45,36 @@ qBittorrent Session Management
         if qbt_client.torrents_add(urls="...") != "Ok.":
             raise Exception("Failed to add torrent.")
 
-Untrusted WebUI Certificate
-***************************
-* qBittorrent allows you to configure HTTPS with an untrusted certificate; this commonly includes self-signed certificates.
-* When using such a certificate, instantiate Client with ``VERIFY_WEBUI_CERTIFICATE=False`` or set environment variable ``QBITTORRENTAPI_DO_NOT_VERIFY_WEBUI_CERTIFICATE`` to a non-null value.
+Untrusted Web API Certificate
+*****************************
+* qBittorrent allows you to configure HTTPS with an untrusted certificate; this commonly
+  includes self-signed certificates.
+* When using such a certificate, instantiate Client with
+  ``VERIFY_WEBUI_CERTIFICATE=False`` or set environment variable
+  ``QBITTORRENTAPI_DO_NOT_VERIFY_WEBUI_CERTIFICATE`` to a non-null value.
 * Failure to do this for will cause connections to qBittorrent to fail.
-* As a word of caution, doing this actually does turn off certificate verification. Therefore, for instance, potential man-in-the-middle attacks will not be detected and reported (since the error is suppressed). However, the connection will remain encrypted.
+* As a word of caution, doing this actually does turn off certificate verification.
+  Therefore, for instance, potential man-in-the-middle attacks will not be detected and
+  reported (since the error is suppressed). However, the connection will remain encrypted.
 
 .. code:: python
 
     qbt_client = Client(..., VERIFY_WEBUI_CERTIFICATE=False}
 
 Requests Configuration
 **********************
-* The `Requests <https://requests.readthedocs.io/en/latest/>`_ package is used to issue HTTP requests to qBittorrent to facilitate this API.
-* Much of `Requests` configuration for making HTTP requests can be controlled with parameters passed along with the request payload.
-* For instance, HTTP Basic Authorization credentials can be provided via ``auth``, timeouts via ``timeout``, or Cookies via ``cookies``. See `Requests documentation <https://requests.readthedocs.io/en/latest/api/#requests.request>`_ for full details.
-* These parameters are exposed here in two ways; the examples below tell ``Requests`` to use a connect timeout of 3.1 seconds and a read timeout of 30 seconds.
-* When you instantiate :class:`~qbittorrentapi.client.Client`, you can specify the parameters to use in all HTTP requests to qBittorrent:
+* The `Requests <https://requests.readthedocs.io/en/latest/>`_ package is used to issue
+  HTTP requests to qBittorrent to facilitate this API.
+* Much of ``Requests`` configuration for making HTTP requests can be controlled with
+  parameters passed along with the request payload.
+* For instance, HTTP Basic Authorization credentials can be provided via ``auth``,
+  timeouts via ``timeout``, or Cookies via ``cookies``. See
+  `Requests documentation <https://requests.readthedocs.io/en/latest/api/#requests.request>`_
+  for full details.
+* These parameters are exposed here in two ways; the examples below tell ``Requests`` to
+  use a connect timeout of 3.1 seconds and a read timeout of 30 seconds.
+* When you instantiate :class:`~qbittorrentapi.client.Client`, you can specify the
+  parameters to use in all HTTP requests to qBittorrent:
 
 .. code:: python
 
@@ -64,11 +86,22 @@ Requests Configuration
 
     qbt_client.torrents_info(..., requests_args={'timeout': (3.1, 30)})
 
+* Additionally, configuration for the :class:`~requests.adapters.HTTPAdapter` for the
+  :class:`~requests.Session` can be specified via the ``HTTPADAPTER_ARGS`` parameter for
+  :class:`~qbittorrentapi.client.Client`:
+
+.. code:: python
+
+    qbt_client = Client(..., HTTPADAPTER_ARGS={"pool_connections": 100, "pool_maxsize": 100}
+
 Additional HTTP Headers
 ***********************
-* For consistency, HTTP Headers can be specified using the method above; for backwards compatibility, the methods below are supported as well.
-* Either way, these additional headers will be incorporated (using clobbering) into the rest of the headers to be sent.
-* To send a custom HTTP header in all requests made from an instantiated client, declare them during instantiation:
+* For consistency, HTTP Headers can be specified using the method above; for backwards
+  compatibility, the methods below are supported as well.
+* Either way, these additional headers will be incorporated (using clobbering) into the
+  rest of the headers to be sent.
+* To send a custom HTTP header in all requests made from an instantiated client, declare
+  them during instantiation:
 
 .. code:: python
 
@@ -82,33 +115,42 @@ Additional HTTP Headers
 
 Unimplemented API Endpoints
 ***************************
-* Since the qBittorrent Web API has evolved over time, some endpoints may not be available from the qBittorrent host.
-* By default, if a request is made to endpoint that doesn't exist for the version of the qBittorrent host (e.g., the Search endpoints were introduced in Web API v2.1.1), there's a debug logger output and None is returned.
-* To raise ``NotImplementedError`` instead, instantiate Client with:
+* Since the qBittorrent Web API has evolved over time, some endpoints may not be
+  available from the qBittorrent host.
+* By default, if a request is made to endpoint that doesn't exist for the version of the
+  qBittorrent host (e.g., the Search endpoints were introduced in Web API v2.1.1),
+  there's a debug logger output and None is returned.
+* To raise :any:`NotImplementedError` instead, instantiate Client with:
 
 .. code:: python
 
     qbt_client = Client(..., RAISE_NOTIMPLEMENTEDERROR_FOR_UNIMPLEMENTED_API_ENDPOINTS=True)
 
 qBittorrent Version Checking
 ****************************
-* It is also possible to either raise an Exception for qBittorrent hosts that are not "fully" supported or manually check for support.
-* The most likely situation for this to occur is if the qBittorrent team publishes a new release but its changes have not been incorporated in to this client yet.
-* Instantiate Client like below to raise ``UnsupportedQbittorrentVersion`` exception for versions not fully supported:
+* It is also possible to either raise an Exception for qBittorrent hosts that are not
+  "fully" supported or manually check for support.
+* The most likely situation for this to occur is if the qBittorrent team publishes a new
+  release but its changes have not been incorporated in to this client yet.
+* Instantiate Client like below to raise
+  :class:`~qbittorrentapi.exceptions.UnsupportedQbittorrentVersion` exception for versions
+  not fully supported:
 
 .. code:: python
 
     qbt_client = Client(..., RAISE_ERROR_FOR_UNSUPPORTED_QBITTORRENT_VERSIONS=True)
 
-* Additionally, the :doc:`qbittorrentapi.Version <apidoc/version>` class can be used for manual introspection of the versions.
+* Additionally, :class:`~qbittorrentapi._version_support.Version` can be used for manual introspection of
+  the versions.
 
 .. code:: python
 
     Version.is_app_version_supported(qbt_client.app.version)
 
 Disable Logging Debug Output
 ****************************
-* Instantiate Client with ``DISABLE_LOGGING_DEBUG_OUTPUT=True`` or manually disable logging for the relevant packages:
+* Instantiate Client with ``DISABLE_LOGGING_DEBUG_OUTPUT=True`` or manually disable
+  logging for the relevant packages:
 
 .. code:: python
 

src/qbittorrentapi/client.py

@@ -66,7 +66,7 @@ class Client(
         >>> client = Client(host='localhost:8080', username='admin', password='adminadmin')
         >>> torrents = client.torrents_info()
 
-    :param host: hostname for qBittorrent Web API, ``[http[s]://]localhost[:8080][/path]``
+    :param host: hostname for qBittorrent Web API, ``[http[s]://]hostname[:port][/path]``
     :param port: port number for qBittorrent Web API (ignored if host contains a port)
     :param username: username for qBittorrent Web API
     :param password: password for qBittorrent Web API
@@ -79,27 +79,30 @@ class Client(
         back. Alternatively, set this to True only for an individual method call.
         For instance, when requesting the files for a torrent:
         ``client.torrents_files(torrent_hash='...', SIMPLE_RESPONSES=True)``
-    :param VERIFY_WEBUI_CERTIFICATE: Set to False to skip verify certificate for
+    :param VERIFY_WEBUI_CERTIFICATE: Set to ``False`` to skip verify certificate for
         HTTPS connections; for instance, if the connection is using a self-signed
-        certificate. Not setting this to False for self-signed certs will cause a
+        certificate. Not setting this to ``False`` for self-signed certs will cause a
         :class:`~qbittorrentapi.exceptions.APIConnectionError` exception to be raised.
     :param EXTRA_HEADERS: Dictionary of HTTP Headers to include in all requests
         made to qBittorrent.
-    :param REQUESTS_ARGS: Dictionary of configuration for Requests package:
-        `<https://requests.readthedocs.io/en/latest/api/#requests.request>`_
+    :param REQUESTS_ARGS: Dictionary of configuration for each HTTP request made by
+        :any:`requests.request`.
+    :param HTTPADAPTER_ARGS: Dictionary of configuration for
+        :class:`~requests.adapters.HTTPAdapter`.
     :param FORCE_SCHEME_FROM_HOST: If a scheme (i.e. ``http`` or ``https``) is
         specified in host, it will be used regardless of whether qBittorrent is
         configured for HTTP or HTTPS communication. Normally, this client will
         attempt to determine which scheme qBittorrent is actually listening on...
         but this can cause problems in rare cases. Defaults ``False``.
-    :param RAISE_NOTIMPLEMENTEDERROR_FOR_UNIMPLEMENTED_API_ENDPOINTS: Some Endpoints
-        may not be implemented in older versions of qBittorrent. Setting this to True
-        will raise a :class:`NotImplementedError` instead of just returning``None``.
-    :param RAISE_ERROR_FOR_UNSUPPORTED_QBITTORRENT_VERSIONS: raise the
-        UnsupportedQbittorrentVersion exception if the connected version of
-        qBittorrent is not fully supported by this client. Defaults ``False``.
+    :param RAISE_NOTIMPLEMENTEDERROR_FOR_UNIMPLEMENTED_API_ENDPOINTS: Some endpoints
+        may not be implemented in older versions of qBittorrent. Setting this to ``True``
+        will raise a :class:`NotImplementedError` instead of just returning ``None``.
+    :param RAISE_ERROR_FOR_UNSUPPORTED_QBITTORRENT_VERSIONS: Raise
+        :class:`~qbittorrentapi.exceptions.UnsupportedQbittorrentVersion` if the
+        connected version of qBittorrent is not fully supported by this client.
+        Defaults ``False``.
     :param DISABLE_LOGGING_DEBUG_OUTPUT: Turn off debug output from logging for
-        this package as well as Requests & urllib3.
+        this package as well as ``requests`` & ``urllib3``.
     """  # noqa: E501
 
     def __init__(
@@ -108,8 +111,10 @@ def __init__(
         port: str | int | None = None,
         username: str | None = None,
         password: str | None = None,
+        *,
         EXTRA_HEADERS: Mapping[str, str] | None = None,
         REQUESTS_ARGS: Mapping[str, Any] | None = None,
+        HTTPADAPTER_ARGS: Mapping[str, Any] | None = None,
         VERIFY_WEBUI_CERTIFICATE: bool = True,
         FORCE_SCHEME_FROM_HOST: bool = False,
         RAISE_NOTIMPLEMENTEDERROR_FOR_UNIMPLEMENTED_API_ENDPOINTS: bool = False,
@@ -125,6 +130,7 @@ def __init__(
             password=password,
             EXTRA_HEADERS=EXTRA_HEADERS,
             REQUESTS_ARGS=REQUESTS_ARGS,
+            HTTPADAPTER_ARGS=HTTPADAPTER_ARGS,
             VERIFY_WEBUI_CERTIFICATE=VERIFY_WEBUI_CERTIFICATE,
             FORCE_SCHEME_FROM_HOST=FORCE_SCHEME_FROM_HOST,
             RAISE_NOTIMPLEMENTEDERROR_FOR_UNIMPLEMENTED_API_ENDPOINTS=RAISE_NOTIMPLEMENTEDERROR_FOR_UNIMPLEMENTED_API_ENDPOINTS,  # noqa: E501

src/qbittorrentapi/request.py

@@ -240,6 +240,7 @@ def __init__(
         password: str | None = None,
         EXTRA_HEADERS: Mapping[str, str] | None = None,
         REQUESTS_ARGS: Mapping[str, Any] | None = None,
+        HTTPADAPTER_ARGS: Mapping[str, Any] | None = None,
         VERIFY_WEBUI_CERTIFICATE: bool = True,
         FORCE_SCHEME_FROM_HOST: bool = False,
         RAISE_NOTIMPLEMENTEDERROR_FOR_UNIMPLEMENTED_API_ENDPOINTS: bool = False,
@@ -256,6 +257,7 @@ def __init__(
         self._initialize_settings(
             EXTRA_HEADERS=EXTRA_HEADERS,
             REQUESTS_ARGS=REQUESTS_ARGS,
+            HTTPADAPTER_ARGS=HTTPADAPTER_ARGS,
             VERIFY_WEBUI_CERTIFICATE=VERIFY_WEBUI_CERTIFICATE,
             FORCE_SCHEME_FROM_HOST=FORCE_SCHEME_FROM_HOST,
             RAISE_NOTIMPLEMENTEDERROR_FOR_UNIMPLEMENTED_API_ENDPOINTS=(
@@ -325,6 +327,7 @@ def _initialize_settings(
         self,
         EXTRA_HEADERS: Mapping[str, str] | None = None,
         REQUESTS_ARGS: Mapping[str, Any] | None = None,
+        HTTPADAPTER_ARGS: Mapping[str, Any] | None = None,
         VERIFY_WEBUI_CERTIFICATE: bool = True,
         FORCE_SCHEME_FROM_HOST: bool = False,
         RAISE_NOTIMPLEMENTEDERROR_FOR_UNIMPLEMENTED_API_ENDPOINTS: bool = False,
@@ -338,6 +341,9 @@ def _initialize_settings(
         # Configuration parameters
         self._EXTRA_HEADERS = dict(EXTRA_HEADERS) if EXTRA_HEADERS is not None else {}
         self._REQUESTS_ARGS = dict(REQUESTS_ARGS) if REQUESTS_ARGS is not None else {}
+        self._HTTPADAPTER_ARGS = (
+            dict(HTTPADAPTER_ARGS) if HTTPADAPTER_ARGS is not None else {}
+        )
         self._VERIFY_WEBUI_CERTIFICATE = bool(VERIFY_WEBUI_CERTIFICATE)
         self._VERBOSE_RESPONSE_LOGGING = bool(VERBOSE_RESPONSE_LOGGING)
         self._SIMPLE_RESPONSES = bool(SIMPLE_RESPONSES)
@@ -909,14 +915,20 @@ def _session(self) -> QbittorrentSession:
         # at any rate, the retries count in request_manager should always be
         # at least 2 to accommodate significant settings changes in qBittorrent
         # such as enabling HTTPs in Web UI settings.
-        adapter = HTTPAdapter(
-            max_retries=Retry(
+        default_adapter_config = {
+            "max_retries": Retry(
                 total=1,
                 read=1,
                 connect=1,
                 status_forcelist={500, 502, 504},
                 raise_on_status=False,
             )
+        }
+        adapter = HTTPAdapter(
+            **{
+                **default_adapter_config,
+                **self._HTTPADAPTER_ARGS,
+            }
         )
         self._http_session.mount("http://", adapter)
         self._http_session.mount("https://", adapter)

tests/test_request.py

@@ -10,6 +10,7 @@
 from qbittorrentapi.request import Request
 from qbittorrentapi.torrents import TorrentDictionary, TorrentInfoList
 from requests import Response
+from requests.adapters import DEFAULT_POOLBLOCK, DEFAULT_POOLSIZE
 
 from tests.conftest import IS_QBT_DEV
 from tests.utils import mkpath
@@ -708,3 +709,33 @@ def test_not_implemented_error(monkeypatch, client):
     monkeypatch.setattr(client, "app_web_api_version", MagicMock(return_value="10.0.0"))
     with pytest.raises(NotImplementedError, match=r"This endpoint was removed"):
         client.search_categories()
+
+
+def test_http_adapter_defaults():
+    client = Client(
+        RAISE_NOTIMPLEMENTEDERROR_FOR_UNIMPLEMENTED_API_ENDPOINTS=True,
+        VERIFY_WEBUI_CERTIFICATE=False,
+    )
+    assert client._session.adapters["http://"] is client._session.adapters["https://"]
+    assert client._session.adapters["http://"].max_retries.total == 1
+    assert client._session.adapters["http://"]._pool_connections == DEFAULT_POOLSIZE
+    assert client._session.adapters["http://"]._pool_maxsize == DEFAULT_POOLSIZE
+    assert client._session.adapters["http://"]._pool_block is DEFAULT_POOLBLOCK
+
+
+def test_http_adapter_overrides():
+    client = Client(
+        RAISE_NOTIMPLEMENTEDERROR_FOR_UNIMPLEMENTED_API_ENDPOINTS=True,
+        VERIFY_WEBUI_CERTIFICATE=False,
+        HTTPADAPTER_ARGS=dict(
+            pool_connections=100,
+            pool_maxsize=50,
+            max_retries=10,
+            pool_block=True,
+        ),
+    )
+    assert client._session.adapters["http://"] is client._session.adapters["https://"]
+    assert client._session.adapters["http://"].max_retries.total == 10
+    assert client._session.adapters["http://"]._pool_connections == 100
+    assert client._session.adapters["http://"]._pool_maxsize == 50
+    assert client._session.adapters["http://"]._pool_block is True