[BUG] Protox.DecodingError when decoding valid message

[nTraum]

First of all thanks for providing this excellent library. 🥇

Describe the bug
This is my first time using protobuf, so I am not sure if this is a bug or an issue on my side.

When attempting to decode a valid (I think 😬) protobuf message, it errors unexpectedly with Protox.DecodingError.

A minimal reproduction script is attached below, which includes the proto schema and example payload.

To verify the schema definition and payload, I used https://www.protobufpal.com/, which yields correct results. The repro script outputs the payload as Base64 to easily copy & paste into the online service.

To Reproduce

Run the following repro script:

Mix.install([
  {:protox, "~> 1.7"}
])

File.mkdir_p("_build/dev/")

defmodule Quantified do
  use Protox,
    schema: """
    syntax="proto3";

    message FireFlyPayload {              //0x01
    //uint32 full_spectrum = 1;       //needs to be split to visible and ir light, and then calculate lux and/or par
    uint32 visible_spectrum = 1;
    uint32 ir_spectrum = 2;
    TslMetadata tsl = 3;
    sint32 temperature = 4;         //needs division by 100 for output in degrees C. value in tenths of millidegrees celcius
    uint32 air_pressure = 5;        //needs division by 100 for value in hPa
    uint32 relative_humidity = 6;   // Returns humidity in %RH as unsigned 32 bit integer in Q22.10 format (22 integer and 10 fractional bits).
                                  // Output value of “47445” represents 47445/1024 = 46.333 %RH
    }

    message GpsData {                     //0x02
    GpsLocation gps_location = 1;
    GpsMetadata gps_metadata = 2;
    bool fix = 3;
    }

    message GpsLocation {               //0x03
    int32 latitude = 1;             //given as int, needs to be divided by 10000 to get actual float value
    int32 longitude = 2;            //given as int, needs to be divided by 10000 to get actual float value
    }

    message GpsMetadata {                 //0x04
    uint32 nsats = 1;
    float hdop = 2;
    float timetofix = 3;
    }

    message TslMetadata {                 //0x05
    bool tsl_agl = 3;
    enum tsl_time {
    Aperture_3ms = 0;     //2.73ms
    Aperture_27ms = 1;    //27.3ms
    Aperture_101ms = 2;   //101ms
    Aperture_175ms = 3;   //175ms Default value
    Aperture_699ms = 4;   //699ms
    }
    enum tsl_gain {
    Gain_1X = 0;          //no gain (*1)
    Gain_8X = 1;          //Default value (*8)
    Gain_16X = 2;         // (*16)
    Gain_120X = 3;        // (*120)
    }
    tsl_time TSL_time = 4;
    tsl_gain TSL_gain = 5;
    }

    //Status from node
    message Status {                      //0x07
    bool GPS_available = 1;
    bool GPS_enabled = 2;
    uint32 GPS_rate = 3;
    uint32 update_rate = 4;
    uint32 firmwareVersion = 5;     // 4 bytes = 8 hex characters from hash
    float battery_voltage = 6;
    bool gpsmeta_enabled = 7;
    uint32 lora_port = 8;
    uint32 externalSensorRate = 18;
    }

    message externalSensor {            //0x10
    uint32 ID_serial = 1;                  //ID of external sensor = 8bit and serial number = 16bit
    uint32 size = 2;                //not used
    bytes data = 3;
    }
    """
end

message = <<8, 222, 205, 2, 16, 214, 159, 1, 26, 2, 32, 3, 32, 232, 43, 48, 199, 254, 3>>
base64 = Base.encode64(message)

dbg(message)
dbg(base64)

FireFlyPayload.decode!(message)

Expected behavior

Returns a FireFlyPayload struct with payload identical to:

{
  "visible_spectrum": 42718,
  "ir_spectrum": 20438,
  "tsl": {
    "tsl_agl": false,
    "TSL_time": "Aperture_175ms",
    "TSL_gain": "Gain_1X"
  },
  "temperature": 2804,
  "air_pressure": 0,
  "relative_humidity": 65351
}

Environment

elixir -v
Erlang/OTP 27 [erts-15.0.1] [source] [64-bit] [smp:16:16] [ds:16:16:10] [async-threads:1] [jit:ns]
Elixir 1.17.2 (compiled with Erlang/OTP 27)

protoc --version
libprotoc 3.19.6

I can provide further message payloads if needed. Thanks in advance! 🙇

[ahamez]

Hello, thank you for the report! Can you give me the full protobuf definition (I need TSL_time and TSL_gain)?

[nTraum]

Unfortunately I don't have a more complete protobuf definition, as this has been provided by a 3rd party. I reckon the enum definition in TslMetadata is not enough?

[ahamez]

I've found the issue: protox has a problem with enums in snake_case 🤔.
When fixing the case in the protobuf definition, it works:

    syntax="proto3";

    message FireFlyPayload {              //0x01
      //uint32 full_spectrum = 1;       //needs to be split to visible and ir light, and then calculate lux and/or par
      uint32 visible_spectrum = 1;
      uint32 ir_spectrum = 2;
      TslMetadata tsl = 3;
      sint32 temperature = 4;         //needs division by 100 for output in degrees C. value in tenths of millidegrees celcius
      uint32 air_pressure = 5;        //needs division by 100 for value in hPa
      uint32 relative_humidity = 6;   // Returns humidity in %RH as unsigned 32 bit integer in Q22.10 format (22 integer and 10 fractional bits).
                                    // Output value of “47445” represents 47445/1024 = 46.333 %RH
    }

    message GpsData {                     //0x02
      GpsLocation gps_location = 1;
      GpsMetadata gps_metadata = 2;
      bool fix = 3;
    }

    message GpsLocation {               //0x03
      int32 latitude = 1;             //given as int, needs to be divided by 10000 to get actual float value
      int32 longitude = 2;            //given as int, needs to be divided by 10000 to get actual float value
    }

    message GpsMetadata {                 //0x04
      uint32 nsats = 1;
      float hdop = 2;
      float timetofix = 3;
    }

    message TslMetadata {                 //0x05
      bool tsl_agl = 3;
      
      enum TslTime {
        Aperture_3ms = 0;     //2.73ms
        Aperture_27ms = 1;    //27.3ms
        Aperture_101ms = 2;   //101ms
        Aperture_175ms = 3;   //175ms Default value
        Aperture_699ms = 4;   //699ms
      }
      
      enum TslGain {
        Gain_1X = 0;          //no gain (*1)
        Gain_8X = 1;          //Default value (*8)
        Gain_16X = 2;         // (*16)
        Gain_120X = 3;        // (*120)
      }
      
      TslTime TSL_time = 4;
      TslGain TSL_gain = 5;
    }

    //Status from node
    message Status {                      //0x07
      bool GPS_available = 1;
      bool GPS_enabled = 2;
      uint32 GPS_rate = 3;
      uint32 update_rate = 4;
      uint32 firmwareVersion = 5;     // 4 bytes = 8 hex characters from hash
      float battery_voltage = 6;
      bool gpsmeta_enabled = 7;
      uint32 lora_port = 8;
      uint32 externalSensorRate = 18;
    }

    message externalSensor {            //0x10
      uint32 ID_serial = 1;                  //ID of external sensor = 8bit and serial number = 16bit
      uint32 size = 2;                //not used
      bytes data = 3;
    }

message = <<8, 222, 205, 2, 16, 214, 159, 1, 26, 2, 32, 3, 32, 232, 43, 48, 199, 254, 3>>
FireFlyPayload.decode!(message)

%FireFlyPayload{
  visible_spectrum: 42718,
  ir_spectrum: 20438,
  tsl: %TslMetadata{tsl_agl: false, TSL_time: :Aperture_175ms, TSL_gain: :Gain_1X, __uf__: []},
  temperature: 2804,
  air_pressure: 0,
  relative_humidity: 65351,
  __uf__: []
}

Even though enums should be in PascalCase as stated by the protobuf style guide, protox should be able to handle this case.

Until I have the time to come up with a fix, I suggest you change the case in the meantime 😅.