Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FileUploadCompletionNotification not working with Iothub GWV2 #1771

Open
Robsmon opened this issue Dec 11, 2023 · 9 comments
Open

FileUploadCompletionNotification not working with Iothub GWV2 #1771

Robsmon opened this issue Dec 11, 2023 · 9 comments

Comments

@Robsmon
Copy link

Robsmon commented Dec 11, 2023

Context

  • OS and version used: Windows 10
  • Java runtime used: 17.0.4
  • SDK version used: 2.4.0

Description of the issue

Can not send FileUploadCompletionNotification. IotHubClientException with BadFormat is thrown.
As i used the FileUploadSample as a basis this should be reproducable relativly easily i hope.

Code sample exhibiting the issue

Slightly adapted FileUploadSample from the samples.

public class FileUploadSample
{
    private static final List<String> fileNameList = new ArrayList<>();

    /**
     * Upload file or directories to blobs using IoT Hub.
     *
     * @param args 
     * args[0] = IoT Hub connection string
     * args[1] = File or directory to upload
     */
    public static void main(String[] args)
            throws IOException, URISyntaxException
    {
        String connString = "XXX";

        System.out.println("Starting...");
        System.out.println("Beginning setup.");

        // File upload will always use HTTPS, DeviceClient will use this protocol only
        //   for the other services like Device Telemetry, Device Method and Device Twin.
        IotHubClientProtocol protocol = IotHubClientProtocol.MQTT;

        System.out.println("Successfully read input parameters.");
        System.out.format("Using communication protocol %s.\n",
                protocol.name());

        DeviceClient client = new DeviceClient(connString, protocol);

        System.out.println("Successfully created an IoT Hub client.");
        
        try
        {
            FileUploadSasUriResponse sasUriResponse = client.getFileUploadSasUri(new FileUploadSasUriRequest("JustSomeFile"));

            FileUploadCompletionNotification completionNotification = new FileUploadCompletionNotification(sasUriResponse.getCorrelationId(), false);
            client.completeFileUpload(completionNotification);
        }
        catch (IotHubClientException e) 
        {
            System.out.println("IotHubClientException: " + e.getStatusCode());
        }
        catch (Exception e)
        {
            System.out.println("On exception, shutting down \n" + " Cause: " + e.getCause() + " \nERROR: " +  e.getMessage());
            System.out.println("Shutting down...");
            client.close();
        }
        client.close();
        
    }

}

Console log of the issue

Starting...
Beginning setup.
Successfully read input parameters.
Using communication protocol MQTT.
2023-12-11 19:54:10,169 INFO (main) [com.microsoft.azure.sdk.iot.device.transport.ExponentialBackoffWithJitter] - NOTE: A new instance of ExponentialBackoffWithJitter has been created with the following properties. Retry Count: 2147483647, Min Backoff Interval: 100, Max Backoff Interval: 10000, Max Time Between Retries: 100, Fast Retry Enabled: true
2023-12-11 19:54:10,233 DEBUG (main) [com.microsoft.azure.sdk.iot.device.ClientConfiguration] - Device configured to use software based SAS authentication provider
2023-12-11 19:54:10,249 DEBUG (main) [com.microsoft.azure.sdk.iot.device.DeviceClient] - Initialized a DeviceClient instance using SDK version 2.4.0
Successfully created an IoT Hub client.
IotHubClientException: BAD_FORMAT
2023-12-11 19:54:10,593 INFO (main) [com.microsoft.azure.sdk.iot.device.DeviceClient] - Closing device client...
2023-12-11 19:54:10,594 INFO (main) [com.microsoft.azure.sdk.iot.device.DeviceClient] - Device client closed successfully
Consider setting the DEBUG environment variable to '*'. This will produce a much more verbose output that will help debugging
Don't forget to remove any connection string information!

@Robsmon Robsmon added the bug label Dec 11, 2023
@Robsmon Robsmon changed the title [Bug Report] FileUploadCompletionNotification not working Dec 11, 2023
@Robsmon
Copy link
Author

Robsmon commented Dec 11, 2023

I tried the same with a different iothub instance which is working as expected.
I could see the iothub i used for the test has the GWV2 feature enabled. This may be the cause for the Problem but i can not confirm it as it is not possible to change this feature.

@timtay-microsoft
Copy link
Member

Is there an actual file being uploaded when you repro this? Sometimes, the BAD_FORMAT error just indicates that IoT hub doesn't have a record of that SAS URI yet. It may take a second or two between requesting one and it being completable and normally that time is filled by uploading a file.

@timtay-microsoft
Copy link
Member

Adding a Thread.sleep(2000) between receiving the SAS URI and completing it could be another way to check if this is just a timing issue.

@Robsmon
Copy link
Author

Robsmon commented Dec 12, 2023

I think BAD_FORMAT is a bad error message in this case.
This is just a minimal sample i found the Problem in a more complex Szenario with a file upload.
I have run the sample with a sleep time of 20000 ms and it comes to the same result.

@Robsmon Robsmon changed the title FileUploadCompletionNotification not working FileUploadCompletionNotification not working with Iothub GWV2 Dec 12, 2023
@timtay-microsoft
Copy link
Member

I think BAD_FORMAT is a bad error message in this case.

Yep, this is something we have brought up to the service team before, but we can't change this behavior now, unfortunately.

How reliably do you see this behavior repro? 100% of the time?

@Robsmon
Copy link
Author

Robsmon commented Dec 12, 2023

I can see this behavior every time until i hit the limit of too much active downloads.

@Robsmon
Copy link
Author

Robsmon commented Dec 12, 2023

I think it would be nice to confirm the issue with another sdk or with a simple http client but I did not have the time yet.

@timtay-microsoft
Copy link
Member

This is likely a question of service behavior, so we'll try to investigate this when we have the time

@mischak
Copy link

mischak commented Feb 13, 2024

We ran into exactly the same issue with a hub on GWV2. The issue seems to be that the FileUploadCompletionNotification(String, Boolean) constructor does not fill in statusCode and statusDescription correctly. When they remain null, the GWV2 refuses our request with validation error (a BAD_FORMAT error message). If we set those fields explicitly using a different constructor, releasing the URI works.

I think, this additional validation is something that needs to be changed in GWV2, as it breaks existing clients that cannot be easily updated to workaround this new validation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants