Different binary data representation when sending data through MQTT and HTTP

[mapedraza]

When sending data using HTTP or MQTT, the data stored in the IoT Agent differs.

MQTT

        "temperature": {
            "type": "Text",
            "value": "4d5f5f09426f",
        }

HTTP

        "temperature": {
            "type": "Text",
            "value": {
                "type": "Buffer",
                "data": [
                    65,
                    66,
                    67
                ]
            }
        },

How to reproduce it

When provisioning a device like this:

{
    "devices": [
        {
            "apikey": "4jggokgpepnvsb2uv4s40d59ov",
            "device_id": "devbin5",
            "entity_name": "devbin005",
            "entity_type": "dev",
            "protocol": "PDI-IoTA-UltraLight",
            "transport": "MQTT",
            "expressionLanguage": "jexl",
            "commands": [
                {
                    "name": "ring",
                    "type": "command"
                }
            ],
            "attributes": [
                {
                    "object_id": "t",
                    "name": "temperature",
                    "type": "Text"
                },
                {
                    "name": "temperature_mod",
                    "type": "Text",
                    "expression": "t?('0x'+(t|slice(2,6)))|parseint|tostring:'NO DATA'"
                }
            ]
        }
    ]
}

If sending data through MQTT like this:

/json/4jggokgpepnvsb2uv4s40d59ov/devbin5/attrs/t
0x4D,0x5F,0x5F,0x09,0x42,0x6f

The result on the entity attribute is correct:

    {
        "id": "devbin005",
        "type": "dev",
        "TimeInstant": {
            "type": "DateTime",
            "value": "2022-09-14T16:25:41.428Z",
            "metadata": {}
        },
        "ring_info": {
            "type": "commandResult",
            "value": " ",
            "metadata": {}
        },
        "ring_status": {
            "type": "commandStatus",
            "value": "UNKNOWN",
            "metadata": {}
        },
        "temperature": {
            "type": "Text",
            "value": "4d5f5f09426f",
            "metadata": {
                "TimeInstant": {
                    "type": "DateTime",
                    "value": "2022-09-14T16:25:41.428Z"
                }
            }
        },
        "temperature_mod": {
            "type": "Text",
            "value": "24415",
            "metadata": {
                "TimeInstant": {
                    "type": "DateTime",
                    "value": "2022-09-14T16:25:41.428Z"
                }
            }
        },
        "ring": {
            "type": "command",
            "value": "",
            "metadata": {}
        }
    }

But, when sending data through HTTP:

curl --location --request POST 'http://localhost:7896/iot/d/attrs/t?k=4jggokgpepnvsb2uv4s40d59ov&i=devbin5&getCmd=1' \
--header 'Content-Type: application/octet-stream' \
--data-raw 'ABC'

The result obtained is the following

    {
        "id": "devbin005",
        "type": "dev",
        "TimeInstant": {
            "type": "DateTime",
            "value": "2022-09-14T16:28:11.995Z",
            "metadata": {}
        },
        "ring_info": {
            "type": "commandResult",
            "value": " ",
            "metadata": {}
        },
        "ring_status": {
            "type": "commandStatus",
            "value": "UNKNOWN",
            "metadata": {}
        },
        "temperature": {
            "type": "Text",
            "value": {
                "type": "Buffer",
                "data": [
                    65,
                    66,
                    67
                ]
            },
            "metadata": {
                "TimeInstant": {
                    "type": "DateTime",
                    "value": "2022-09-14T16:28:11.995Z"
                }
            }
        },
        "temperature_mod": {
            "type": "Text",
            "value": "12",
            "metadata": {
                "TimeInstant": {
                    "type": "DateTime",
                    "value": "2022-09-14T16:28:11.995Z"
                }
            }
        },
        "ring": {
            "type": "command",
            "value": "",
            "metadata": {}
        }
    }

[Madhu1029]

Hi @mapedraza ,

I have tried to reproduce the issue in following manner:
Case 1: Send measure using MQTT protocol, some different data is stored.
Send measure to IOTA using MQTT:
mosquitto_pub -t /json/4jggokgpepnvsb2uv4s40d59ov/motion002/attrs/a -m 0xD,0xE,0xF -h localhost -p 1883

Get entity data from Orion:
{"id":"urn:ngsi-ld:Motion:002","type":"Motion","TimeInstant":{"type":"DateTime","value":"2022-12-05T10:17:28.342Z","metadata":{}},"county":{"type":"Text","value":"3078442c3078452c307846","metadata":{"TimeInstant":{"type":"DateTime","value":"2022-12-05T10:17:28.342Z"}}}}

Case 2: Send measure using HTTP protocol, ASCII code of letters is stored.
Send measure to IOTA using HTTP:
curl -X POST 'http://localhost:7896/iot/json/attrs/a?k=4jggokgpepnvsb2uv4s40d59ov&i=motion002&getCmd=1' --header 'Content-Type: application/octet-stream' --data-raw 'ABC'

Get entity data from Orion:
{"id":"urn:ngsi-ld:Motion:002","type":"Motion","TimeInstant":{"type":"DateTime","value":"2022-12-05T11:30:20.394Z","metadata":{}},"county":{"type":"Text","value":{"type":"Buffer","data":[65,66,67]},"metadata":{"TimeInstant":{"type":"DateTime","value":"2022-12-05T11:30:20.394Z"}}}}

I have understood that data is stored differently for MQTT and HTTP.
Please confirm my understanding for reproduction steps.

[mapedraza]

Hi @MadhuNEC your understanding is right and the steps reproduce well this issue. After some internal discussion, the expected behaviour for sending HTTP data should be the same that when sending the data through MQTT. This means a hexstring for both of them (I.E: {"type":"Text","value":"3078442c3078452c307846"})

[MayankSingh-NEC]

Hi @mapedraza and @fgalan.

A gentle reminder. Please review the PR raised for this issue.
Thank you.

[fgalan]

PR #760 (PR #736 + some extras)

CC: @Keshav-NEC

Should this issue be closed? Or is there anything pending?

[mapedraza]

@fgalan yes, this issue is solved with PR you mention. Thus, I close it.