Protecting microdot from client errors due to self-signed certificate.

[ned-pcs]

I've got the latest Microdot source, and the latest Micropython, and can't get TLS working in my application.
I'm using an ESP32-WROOM-1-N16R8 (16MiB flash, 8MiB octal SPIRAM), and have plenty of memory.
My application is running Microdot as an asyncio task, and I've got the DER-format certificate and key files loaded in the same way as the Microdot tls/hello.py example:

sslctx = ssl.SSLContext(ssl.PROTOCOL_TLS_SERVER)
sslctx.load_cert_chain("/storage/ec_cert.der", "/storage/ec_key.der")

async def task():
    await server.start_server(host="0.0.0.0", port=443, debug=True, ssl=sslctx)

I get this exception when trying to do a GET request:

--> Traceback (most recent call last):
  File "microdot/microdot.py", line 1289, in handle_request
  File "microdot/microdot.py", line 398, in create
  File "microdot/microdot.py", line 512, in _safe_readline
  File "asyncio/stream.py", line 1, in readline
OSError: (-30592, 'MBEDTLS_ERR_SSL_FATAL_ALERT_MESSAGE')
Traceback (most recent call last):
  File "asyncio/core.py", line 1, in run_until_complete
  File "microdot/microdot.py", line 1183, in serve
  File "microdot/microdot.py", line 1295, in handle_request
  File "microdot/microdot.py", line 642, in write
  File "asyncio/stream.py", line 1, in stream_awrite
  File "asyncio/stream.py", line 1, in write
OSError: -30592
device disconnected

I've generated both an RSA key (as shown in the Microdot examples/tls/README.md):

openssl req -x509 -newkey rsa:4096 -nodes -out cert.pem -keyout key.pem -days 365
openssl x509 -in cert.pem -out cert.der -outform DER
openssl rsa -in key.pem -out key.der -outform DER

And an elliptic key:

openssl ecparam -name prime256v1 -genkey -noout -out ec_key.der -outform DER
openssl req -new -x509 -key ec_key.der -out ec_cert.der -outform DER -days 365 -nodes 

Both keys resulted in the same response.

I've tried accessing this with a web browser, and with curl -insecure with the same results.
What do I need to do to get this to work?

[miguelgrinberg]

@Carglglz would you know what the cause of this error could be?

[ned-pcs]

I should note also that I have @Carglglz fixed version of extmod/asyncio/stream.py, as it avoids the ECONNRESET problems I'd been having.

And I get the same exception with the Microdot examples/tls/hello.py example.

[ned-pcs]

On a Teensy 4.1, I get a slightly different exception:

Starting async server on 0.0.0.0:4443...
Traceback (most recent call last):
  File "/sdcard/lib/microdot/microdot.py", line 1289, in handle_request
  File "/sdcard/lib/microdot/microdot.py", line 398, in create
  File "/sdcard/lib/microdot/microdot.py", line 512, in _safe_readline
  File "asyncio/stream.py", line 1, in readline
OSError: (-30464, 'MBEDTLS_ERR_SSL_UNEXPECTED_MESSAGE')
Traceback (most recent call last):
  File "asyncio/core.py", line 1, in run_until_complete
  File "/sdcard/lib/microdot/microdot.py", line 1183, in serve
  File "/sdcard/lib/microdot/microdot.py", line 1295, in handle_request
  File "/sdcard/lib/microdot/microdot.py", line 642, in write
  File "asyncio/stream.py", line 1, in stream_awrite
  File "asyncio/stream.py", line 1, in write
OSError: -30464

[ned-pcs]

That was when accessing the endpoint with a web browser.

I am able to get the whole thing to work (on the Teensy) with openssl s_client -connect 192.168.2.136:443, then typing GET / HTTP/1.0.

[ned-pcs]

Here's openssl output:

% openssl s_client -no_tls1_3 -connect 10.100.2.167:443
Connecting to 10.100.2.167
CONNECTED(00000003)
Can't use SSL_get_servername
depth=0 CN=MQTT Server
verify error:num=20:unable to get local issuer certificate
verify return:1
depth=0 CN=MQTT Server
verify error:num=21:unable to verify the first certificate
verify return:1
depth=0 CN=MQTT Server
verify return:1
---
Certificate chain
 0 s:CN=MQTT Server
   i:CN=MQTT CA
   a:PKEY: id-ecPublicKey, 256 (bit); sigalg: ecdsa-with-SHA256
   v:NotBefore: Feb 12 16:59:46 2024 GMT; NotAfter: Feb 11 16:59:46 2025 GMT
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIBszCCAVigAwIBAgIUOSXoesaJTy8C26+daEINSyhXF/AwCgYIKoZIzj0EAwIw
EjEQMA4GA1UEAwwHTVFUVCBDQTAeFw0yNDAyMTIxNjU5NDZaFw0yNTAyMTExNjU5
NDZaMBYxFDASBgNVBAMMC01RVFQgU2VydmVyMFkwEwYHKoZIzj0CAQYIKoZIzj0D
AQcDQgAE84j9KrfSdOHk+6j/SB1CR2vY2Esu7x7OUT2jkCHxJTmp7v2vh7gOw2u1
ZwamkCqd/DnhlNnRXJ5vpEVZ16CgC6OBhzCBhDBCBgNVHREEOzA5ggpXaWxsby01
RjU0ghBXaWxsby01RjU0LmxvY2FsggsxOTIuMTY4LjQuMYIMMTAuMTAwLjIuMTY3
MB0GA1UdDgQWBBS1U4wqBXkPukO7SzZKr5PAcalUuzAfBgNVHSMEGDAWgBToO09i
U349KJ+DPKf+f8HmfZys9TAKBggqhkjOPQQDAgNJADBGAiEAkET9Hkmm/PsbacXX
YNUJymWVaHt7EqM7Sc6ZjJ49mDMCIQCPkRLPJQ17odUkW4d1CWx4c4Qtl+0TES/2
/oV6S+e4zg==
-----END CERTIFICATE-----
subject=CN=MQTT Server
issuer=CN=MQTT CA
---
No client certificate CA names sent
Peer signing digest: SHA256
Peer signature type: ECDSA
Server Temp Key: ECDH, prime256v1, 256 bits
---
SSL handshake has read 762 bytes and written 320 bytes
Verification error: unable to verify the first certificate
---
New, TLSv1.2, Cipher is ECDHE-ECDSA-AES256-GCM-SHA384
Server public key is 256 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : ECDHE-ECDSA-AES256-GCM-SHA384
    Session-ID: E4EAFE980D25EDC3BD649C6AC8CCAE23D2D783BFD1D92A261243C2F50957A518
    Session-ID-ctx: 
    Master-Key: 4CD85BAFA9F304AD4FBEB4EE3D47B522150C0F1B455272A40C35D29AF52A67C668E71A64EAE5A973A82593CDDD13114F
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    Start Time: 1707782932
    Timeout   : 7200 (sec)
    Verify return code: 21 (unable to verify the first certificate)
    Extended master secret: yes

[ned-pcs]

And I can get it to work with openssl s_client on the ESP32 as well. Apparently the exceptions come from clients (the web browser and curl) that didn't like the self-signed certificate.

So the question becomes: how can I protect against microdot crashing in these situations?

[miguelgrinberg]

Thanks. I'm not sure the correct way to handle these is to silence them. For all we know there might be many more situations that generate different errors, and I'm also not convinced these errors are exclusive to client errors, they appear to be very generic. As long as Microdot does not crash I think this is the way I'm going to leave it for now.

[ned-pcs]

The problem is that Microdot does crash. The exception leaks out to the asyncio exception handler.
I have an asyncio global exception handler set up and the second traceback gets caught there.

    def handle_exception(loop, context):
        exc = context["exception"]
        sys.print_exception(exc)
        with open(_EXCEPTION_FILE, "a") as f:
            print(f"at time {time.ticks_ms()} ms: ", file=f)
            sys.print_exception(exc, f)

        time.sleep(2)
        sys.exit()

    loop = asyncio.get_event_loop()
    loop.set_exception_handler(handle_exception

And I see this in my exception file:

at time 307136 ms: 
Traceback (most recent call last):
  File "asyncio/core.py", line 1, in run_until_complete
  File "microdot/microdot.py", line 1183, in serve
  File "microdot/microdot.py", line 1295, in handle_request
  File "microdot/microdot.py", line 642, in write
  File "asyncio/stream.py", line 1, in stream_awrite
  File "asyncio/stream.py", line 1, in write
OSError: -30592

[Carglglz]

I have an asyncio global exception handler set up and the second traceback gets caught there.

mmm that's interesting, don't use one and see if microdot continues running, what I see is this (unix port):

Traceback (most recent call last):
...
OSError: (-30464, 'MBEDTLS_ERR_SSL_UNEXPECTED_MESSAGE')
Task exception wasn't retrieved
future: <Task> coro= <generator object 'serve' at 7f8bd5826be0>
Traceback (most recent call last):
...
OSError: -30464

Then I can do requests normally e.g.

$ ab -n 1 -v 2 https://0.0.0.0:1234/check

GET /check 200

[EDIT]
OK I see it now: you're exiting here, so just delete sys.exit()

time.sleep(2)
sys.exit()

[ned-pcs]

OK, I got rid of the sys.exit() and it keeps running. I can even access it from a web browser (if I tell the browser to ignore certificate errors). Thanks for looking at this!

[miguelgrinberg]

You are actually right, there is a write call in microdot.py line 1295 that does not catch exceptions. I'll fix it, so that you don't have to install your own exception hander.