🔖 Release 3.5.4 (#98)

3.5.4 (2024-03-17)
------------------

**Added**
- Support to verify the peer certificate fingerprint using `verify=...`
by passing a string using the following format:

`verify="sha256_748c76348778cb4a536e7ec12bc9aa559c12770bd1419c7ffe516006e1dea0ec"`.
Doing so disable the certificate
  usual verification and only checks for its fingerprint match.

**Fixed**
- Multiplexed request in async did not support awaitable in hooks.
- Setting `verify=...` and `cert=...` then change it for the same host
did not apply to the underlying (existing) connection pool.

**Misc**
- Overall performance improvements in both async and sync requests.
- Update pre-commit dependencies (ruff, pyupgrade, and mypy).
- Fixed SessionRedirect in tests that produced an incomplete Response
instance that wasn't suitable for tests.

.github/workflows/publish.yml

@@ -83,4 +83,4 @@ jobs:
         gh release upload ${{ github.ref_name }} dist/* --repo ${{ github.repository }}
 
     - name: "Publish dists to PyPI"
-      uses: "pypa/gh-action-pypi-publish@f8c70e705ffc13c3b4d1221169b84f12a75d6ca8" # v1.8.8
+      uses: "pypa/gh-action-pypi-publish@81e9d935c883d0b210363ab89cf05f3894778450" # v1.8.14

.pre-commit-config.yaml

@@ -9,21 +9,21 @@ repos:
   - id: end-of-file-fixer
   - id: trailing-whitespace
 - repo: https://github.com/asottile/pyupgrade
-  rev: v3.10.1
+  rev: v3.15.1
   hooks:
     - id: pyupgrade
       args: [--py37-plus]
 - repo: https://github.com/astral-sh/ruff-pre-commit
   # Ruff version.
-  rev: v0.1.7
+  rev: v0.3.2
   hooks:
     # Run the linter.
     - id: ruff
       args: [ --fix ]
     # Run the formatter.
     - id: ruff-format
 - repo: https://github.com/pre-commit/mirrors-mypy
-  rev: v1.5.1
+  rev: v1.9.0
   hooks:
   -   id: mypy
       args: [--check-untyped-defs]

HISTORY.md

@@ -1,6 +1,23 @@
 Release History
 ===============
 
+3.5.4 (2024-03-17)
+------------------
+
+**Added**
+- Support to verify the peer certificate fingerprint using `verify=...` by passing a string using the following format:
+  `verify="sha256_748c76348778cb4a536e7ec12bc9aa559c12770bd1419c7ffe516006e1dea0ec"`. Doing so disable the certificate
+  usual verification and only checks for its fingerprint match.
+
+**Fixed**
+- Multiplexed request in async did not support awaitable in hooks.
+- Setting `verify=...` and `cert=...` then change it for the same host did not apply to the underlying (existing) connection pool.
+
+**Misc**
+- Overall performance improvements in both async and sync requests.
+- Update pre-commit dependencies (ruff, pyupgrade, and mypy).
+- Fixed SessionRedirect in tests that produced an incomplete Response instance that wasn't suitable for tests.
+
 3.5.3 (2024-03-06)
 ------------------
 

README.md

@@ -40,28 +40,33 @@ Niquests, is the “**Safest**, **Fastest[^10]**, **Easiest**, and **Most advanc
 </details>
 
 <details>
-  <summary>📈 <b>Look at the performance comparison</b> against <i>requests, httpx and aiohttp</i>!</summary>
+  <summary>📈 <b>Look at the performance comparison</b> against <i>them</i>!</summary>
 
 _Scenario:_ Fetch a thousand requests using 10 tasks or threads, each with a hundred requests using a single pool of connection.
 
 **High-Level APIs**
 
-| Client   | Average Delay to Complete | Notes                        |
-|----------|---------------------------|------------------------------|
-| requests | 987 ms                    | ThreadPoolExecutor. HTTP/1.1 |
-| httpx    | 735 ms                    | Asyncio. HTTP/2              |
-| niquests | 600 ms                    | Asyncio. HTTP/2              |
+| Client   | Average Delay to Complete                | Notes                        |
+|----------|------------------------------------------|------------------------------|
+| requests | <span style="color:red">987 ms</span>    | ThreadPoolExecutor. HTTP/1.1 |
+| httpx    | <span style="color:orange">735 ms</span> | Asyncio. HTTP/2              |
+| niquests | <span style="color:green">470 ms</span>  | Asyncio. HTTP/2              |
 
 **Simplified APIs**
 
-| Client        | Average Delay to Complete | Notes                        |
-|---------------|---------------------------|------------------------------|
-| requests core | 643 ms                    | ThreadPoolExecutor. HTTP/1.1 |
-| httpx core    | 550 ms                    | Asyncio. HTTP/2              |
-| aiohttp       | 220 ms                    | Asyncio. HTTP/1.1            |
-| niquests core | 210 ms                    | Asyncio. HTTP/2              |
+| Client        | Average Delay to Complete                | Notes                        |
+|---------------|------------------------------------------|------------------------------|
+| requests core | <span style="color:red">643 ms</span>    | ThreadPoolExecutor. HTTP/1.1 |
+| httpx core    | <span style="color:orange">550 ms</span> | Asyncio. HTTP/2              |
+| aiohttp       | <span style="color:green">220 ms</span>  | Asyncio. HTTP/1.1            |
+| niquests core | <span style="color:green">210 ms</span>  | Asyncio. HTTP/2              |
 
-Want to learn more about the tests? scripts? reasoning? Take a deeper look at https://github.com/Ousret/niquests-stats
+Did you give up on HTTP/2 due to performance concerns? Think again! Multiplexing and response lazyness open up a wide range
+of possibilities! Want to learn more about the tests? scripts? reasoning?
+
+Take a deeper look at https://github.com/Ousret/niquests-stats
+
+⚠️ Do the responsible thing with this library and do not attempt DoS remote servers using its abilities.
 </details>
 
 ```python
@@ -169,7 +174,7 @@ purchasing and maintaining their software, with professional grade assurances
 from the experts who know it best, while seamlessly integrating with existing
 tools.
 
-You may also be interested in unlocking specific advantages by looking at our [GitHub sponsor tiers](https://github.com/sponsors/Ousret).
+You may also be interested in unlocking specific advantages _(like access to a private issue tracker)_ by looking at our [GitHub sponsor tiers](https://github.com/sponsors/Ousret).
 
 ---
 
@@ -179,7 +184,7 @@ Niquests is a highly improved HTTP client that is based (forked) on Requests. Th
 [^2]: requests has no support for asynchronous request.
 [^3]: while the HTTP/2 connection object can handle concurrent requests, you cannot leverage its true potential.
 [^4]: loading client certificate without file can't be done.
-[^5]: httpx officially claim to be thread safe but recent tests demonstrate otherwise as of february 2024.
+[^5]: httpx officially claim to be thread safe but recent tests demonstrate otherwise as of february 2024. https://github.com/jawah/niquests/issues/83#issuecomment-1956065258 https://github.com/encode/httpx/issues/3072 https://github.com/encode/httpx/issues/3002
 [^6]: they do not expose anything to control network aspects such as IPv4/IPv6 toggles, and timings (e.g. DNS response time, established delay, TLS handshake delay, etc...) and such.
 [^7]: while advertised as possible, they refuse to make it the default due to performance issues. as of february 2024 an extra is required to enable it manually.
 [^8]: they don't support HTTP/3 at all.

docs/community/faq.rst

@@ -84,8 +84,8 @@ May actually import either urllib3 or urllib3.future.
 But fear not, if your script was compatible with urllib3, it will most certainly work
 out-of-the-box with urllib3.future.
 
-This behavior was chosen to ensure the highest level of compatibility with your migration,
-ensuring the minimum friction during the migration between Requests and Niquests.
+This behavior was chosen to ensure the highest level of compatibility for your migration,
+ensuring the minimum friction during the migration between Requests to Niquests.
 
 Cohabitation
 ~~~~~~~~~~~~
@@ -96,3 +96,104 @@ Niquests will use the secondary entrypoint for urllib3.future internally.
 It does not change anything for you. You may still pass ``urllib3.Retry`` and
 ``urllib3.Timeout`` regardless of the cohabitation, Niquests will do
 the translation internally.
+
+What are my headers are lowercased?
+-----------------------------------
+
+This may come as a surprise for some of you. Until Requests-era, header keys could arrive
+as they were originally sent (case-sensitive). This is possible thanks to HTTP/1.1 protocol.
+Nonetheless, RFCs specifies that header keys are *case-insensible*, that's why both Requests
+and Niquests ships with ``CaseInsensitiveDict`` class.
+
+So why did we alter it then?
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The answer is quite simple, we support HTTP/2, and HTTP/3 over QUIC! The newer protocols enforce
+header case-insensitivity and we can only forward them as-is (lowercased).
+
+Can we revert this behavior? Any fallback?
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Yes... kind of!
+Niquests ships with a nice alternative to ``CaseInsensitiveDict`` that is ``kiss_headers.Headers``.
+You may access it through the ``oheaders`` property of your usual Response, Request and PreparedRequest.
+
+Am I obligated to install qh3 and dependents?
+---------------------------------------------
+
+No. But by default, it could be picked for installation. You may remove it safely at the cost
+of loosing HTTP/3 over QUIC.
+
+A shortcut would be::
+
+    $ pip uninstall qh3 cryptography cffi pycparser
+
+.. warning:: Your site-packages is shared, thus it is possible that other libraries depends on some of the listed programs. Do it with care!
+
+What are "pem lib" errors?
+--------------------------
+
+Ever encountered something along::
+
+    $ SSLError: [SSL] PEM lib (_ssl.c:2532)
+
+Yes? Usually it means that you tried to load a certificate (CA or client cert) that is malformed.
+
+What does malformed means?
+~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Could be just a missing newline character *RC*, or wrong format like passing a DER file instead of a PEM
+encoded certificate.
+
+If none of those seems related to your situation, feel free to open an issue at https://github.com/jawah/niquests/issues
+
+Why HTTP/2 and HTTP/3 seems slower than HTTP/1.1?
+-------------------------------------------------
+
+Because you are not leveraging its potential properly. Most of the time, developers tends to
+make a request and immediately consume the response afterward. Let's call that making OneToOne requests.
+HTTP/2, and HTTP/3 both requires more computational power for a single request than HTTP/1.1 (in OneToOne context).
+The true reason for them to exist, is not the OneToOne scenario.
+
+So, how to remedy that?
+
+You have multiple choices:
+
+1. Using multiplexing in a synchronous context or asynchronous
+2. Starting threads
+3. Using async with concurrent tasks
+
+This example will quickly demonstrate, how to utilize and leverage your HTTP/2 connection with ease::
+
+    from time import time
+    from niquests import Session
+
+    #: You can adjust it as you want and verify the multiplexed advantage!
+    REQUEST_COUNT = 10
+    REQUEST_URL = "https://httpbin.org/delay/1"
+
+    def make_requests(url: str, count: int, use_multiplexed: bool):
+      before = time()
+
+      responses = []
+
+      with Session(multiplexed=use_multiplexed) as s:
+        for _ in range(count):
+          responses.append(s.get(url))
+          print(f"request {_+1}...OK")
+        print([r.status_code for r in responses])
+
+      print(
+          f"{time() - before} seconds elapsed ({'multiplexed' if use_multiplexed else 'standard'})"
+      )
+
+    #: Let's start with the same good old request one request at a time.
+    print("> Without multiplexing:")
+    make_requests(REQUEST_URL, REQUEST_COUNT, False)
+    #: Now we'll take advantage of a multiplexed connection.
+    print("> With multiplexing:")
+    make_requests(REQUEST_URL, REQUEST_COUNT, True)
+
+.. note:: This piece of code showcase how to emit concurrent requests in a synchronous context without threads and async.
+
+We would gladly discuss potential implementations if needed, just open a new issue at https://github.com/jawah/niquests/issues

docs/community/recommended.rst

@@ -36,3 +36,26 @@ to provide authentication. It also provides a lot of tweaks that handle ways
 that specific OAuth providers differ from the standard specifications.
 
 .. _requests-oauthlib: https://requests-oauthlib.readthedocs.io/en/latest/
+
+.. warning:: There's a catch when trying to use Niquests with `requests-oauthlib`_. You will need a quick patch prior to using it.
+
+Please patch your program as follow::
+
+    import niquests
+    from oauthlib.oauth2 import BackendApplicationClient
+    import requests_oauthlib
+
+    requests_oauthlib.OAuth2Session.__bases__ = (niquests.Session,)
+
+    client_id = "xxxxxxxxxxxxxxxxxxxxxxxxxxx"
+    client_secret = "xxxxxxxxxxxxxxxxxxxxxxxxxxx"
+    token_url = 'https://api.github.com/token'
+
+    if __name__ == "__main__":
+        client = BackendApplicationClient(client_id=client_id)
+        sample = requests_oauthlib.OAuth2Session(client=client)
+
+        token = sample.fetch_token(token_url, client_secret=client_secret)
+
+The key element to be considered is ``requests_oauthlib.OAuth2Session.__bases__ = (niquests.Session,)``.
+You may apply it to `requests_oauthlib.OAuth1Session` too.

docs/user/advanced.rst

@@ -1371,3 +1371,49 @@ It will be passed down the the lower stack. No effort required.
 .. note:: You can set **0** instead of 4444 to select a random port.
 
 .. note:: You can set **0.0.0.0** to select the network adapter automatically instead, if you wish to set the port only.
+
+Inspect network timings
+-----------------------
+
+You are probably used to calling ``response.elapsed`` to get a rough estimate on how long did the
+request took to complete.
+
+It is likely that you may be interested in knowing:
+
+- How long did the TCP/UDP established connection took?
+- How long did the DNS resolution cost me?
+
+... and so on.
+
+Here is a simple example::
+
+    import niquests
+
+    session = niquests.Session()
+
+    response = session.get("https://pie.dev/get")
+
+    print(response.conn_info.resolution_latency)  # output the DNS resolution latency
+    print(response.conn_info.tls_handshake_latency)  # the TLS handshake completion
+
+Here, ``conn_info`` is a ``urllib3.ConnectionInfo`` instance. The complete list of
+attributes is listed on the Hook bottom section.
+
+.. note:: Each response and request are linked to a unique ConnectionInfo.
+
+Verify Certificate Fingerprint
+------------------------------
+
+.. note:: Available since Niquests 3.5.4
+
+An alternative to the certificate verification can be asserting its fingerprint. We (absolutely) do
+not recommend using it unless you are left with no other alternative.
+
+Here is a simple example::
+
+    import niquests
+
+    session = niquests.Session()
+    session.get("https://pie.dev/get", verify="sha256_8fff956b66667ffe5801c8432b12c367254727782d91bc695b7a53d0b512d721")
+
+.. warning:: Supported fingerprinting algorithms are sha256, and sha1. The prefix is mandatory.

src/niquests/__version__.py

@@ -9,9 +9,9 @@
 __url__: str = "https://niquests.readthedocs.io"
 
 __version__: str
-__version__ = "3.5.3"
+__version__ = "3.5.4"
 
-__build__: int = 0x030503
+__build__: int = 0x030504
 __author__: str = "Kenneth Reitz"
 __author_email__: str = "me@kennethreitz.org"
 __license__: str = "Apache-2.0"