[Bug Report] DeviceClient.SendEventAsync takes over one minute after update from 1.42.0 to 1.42.2

[fabianmeyer]

Context

• OS, version, SKU and CPU architecture used: Ubuntu 23.10 64-bit
• Application's .NET Target Framework : net8
• Device: Laptop
• SDK version used: Microsoft.Azure.Devices.Client 1.42.2

Description of the issue

After updating from Microsoft.Azure.Devices.Client 1.42.0 to 1.42.2, calls to SendEventAsync take over one minute in comparison to ~700ms before.

Code sample exhibiting the issue

var deviceClient = DeviceClient.CreateFromConnectionString($"HostName={cloudGatewayHostname};DeviceId={deviceId};SharedAccessKey={sharedAccessKey}", TransportType.Mqtt);
var sw = Stopwatch.StartNew();
_logger.LogInformation("Sending event as device {DeviceId} to IoT Hub", deviceId);
await deviceClient.SendEventAsync(message, cancellationToken);
_logger.LogInformation("Sent event as device {DeviceId} to IoT Hub ({Duration}ms)", deviceId, sw.Elapsed.TotalMilliseconds);

Console log of the issue

1.42.0
Sent event as device 00000001-4505-009d-5741-ec8495e2bc03 to IoT Hub (755.3321ms)
1.42.2
Sent event as device 00000001-4505-009d-5741-ec8495e2bc03 to IoT Hub (71925.7335ms)

[TheDon79]

It seems to be, that I am running into the same/a very similar problem. When using Version 1.42.2 deviceClient.SendEventAsync runs into a Timeout and throws an IotHubCommunicationException with Transient network error occurred, please retry. Returning to Version 1.4.0 resolves this issue.

[timtay-microsoft]

Do you see this same behavior on .NET 7 as well? And do you see it when using AMQPS as well?

[fabianmeyer]

Besides NET8, I only have the NET6 SDK installed, for which the issue does not occur (same duration for 1.42.0 and 1.42.2).
The issue also does not occur when using AMQP with 1.42.2 on NET8.

[TheDon79]

• deviceClient.SendEventAsync works in .NET7 with 1.42.2
• it works in .NET8 with AMQP but not with MQTT (or MQTT_TCP_ONLY)
• it works in .NET8 with 1.42.0

It seems that the client stops to communicate with the IoT Hub after the Server Hello Done.

[timtay-microsoft]

We suspect this is related to a .NET 8 bug in DotNetty (the MQTT library we use). The fix for this bug in DotNetty has been merged and we are working on putting out a new DotNetty release. Once we have released DotNetty, we'll put out a release of this SDK with the new DotNetty version which should resolve this issue.

[jenscski]

Looks like DotNetty is out with new version, I have added direct refereces to DotNetty and it looks like it's working now

<PackageReference Include="DotNetty.Codecs.Mqtt" Version="0.7.6" />
<PackageReference Include="DotNetty.Handlers" Version="0.7.6" />

[timtay-microsoft]

This behavior should be fixed in the new release:
https://github.com/Azure/azure-iot-sdk-csharp/releases/tag/2024-03-28