FileToAzureIoTHub

Rewrite of filemsgtoazureiothub. The new FileToAzureIoTHub moves implementation from Python to C# and .NET5. This will deprecate filemsgtoazureiothub when finnished. The project is not generic and will only work with filetypes from two vendors: Østfold Energi and Storm Elektro. Thus public interest should only be as an example of how to move data from a file to Azure IoT Hub when the file is created.

Prerequisites

.NET 5 SDK and PowerShell 7 must be installed on the server. To Install .Net and Powershell on Linux or Windows, please follow Microsofts instructions (google it). You can use the latest versions. For Ubuntu: # sudo apt get dotnet-sdk-5.0

Git must be installed to download the needed files.

Installation

Clone the repository: git clone ...
cd FileToAzureIoTHub

Linux Specific
Build the project: dotnet build --configuration Release --runtime linux-x64
Copy binary files to /opt/FileToAzureIoTHub:
cp -R ./bin/Release/net5.0/linux-x64/* /opt/FileToAzureIoTHub
Copy FileToAzureIoTHub.service to /etc/systemd/system/:
sudo cp FileToAzureIoTHub.service /etc/systemd/system
Copy the configuration file to /etc and edit it:
sudo cp ConfigExample.json /etc/FileToAzureIoTHub.json
Enable and start the servie
sudo systemctl enable FileToAzureIoTHub
sudo systemctl start FileToAzureIoTHub
Check if everything is running:
sudo systemctl status FileToAzureIoTHub
sudo journalctl -r -u FileToAzureIoTHub
Check inotify max_user_watches, this value must be in the region 65536:
vim /etc/sysctl.conf.d/20-inotify.conf
fs.inotify.max_user_watches=65536
# echo /proc/sys/fs/inotify/max_user_watches to check
Boot, or echo 65536 | sudo tee /proc/sys/fs/inotify/max_user_watches

Configuration File

⚠️ Be VERY careful with the config file exposure after you add theAzure IoT Hub connection string. This string should not be exposed to unauthorized parties. You can prevent this by only editing the file in the configuration directory.

The configuration file FileToAzureIoTHub.json is placed differently based on the OS the file is installed on:
Linux: /etc/FileToAzureIoTHub.json
Windows: $env:ProgramData/Sarpsborgkommune/FileToAzureIoTHub/FileToAzureIoTHub.json

The file format is:

{
    "receiver": [
        {
            "id": "",
            "filePath": "",
            "filePattern": ""
        },
        {
            ...
        },
        ...
    ],
    "sender": [
        {
            "id": "",
            "connectionString": ""
        },
        {
            ...
        },
        ...
    ]
}

Currently only two receivers are implemented:
ostfoldeneergi, smartelektro

Currently only one sender is implemented:
energymanager

Sender

Receiver

Ostfold Energi

This json format does not adhere to standards. The format uses Norwegian special characters, and this should be avoided. The dataformat for the messages is:

{
    "måledata":[
        {
            "id": int,
            "k-nummer": "<string>",
            "dato": "<string>DD.MM.YYY",
            "periode": "<string>HH:MM",
            "forbruk": int,
            "målerstand": int
        },
        ...
    ]
}

EnergyManager

EnergyManager must receive messages in the following json format.

{
    "<måler-id>": [
        {"ts": <datetime>, "v": <double>},
        { ... },
        ...
    ],
    "<måler-id<": [
        {"ts": <datetime>, "v": <double>},
        { ... },
        ...
    ],
    ...
}

Multisensordata can also be formatted as:

{
    "<måler-id>": [
        {"ts": <datetime>, "v": { "<key1>": <double>, "<key2>": <double>, ...}}
    ]
}

Smart Elektro

The data is received as a CSV file with the following format.

"<DateTime>","<IDENT>",<METER READING>
"DD-MM-YYYY HH:MM:SS","<IDENT String>",<double>

The Ident String has the following format: Cluster<#>.ID_MEASUREMENTTYPE

ID example: BY3570_310_001_OE001

MEASUREMENTTYPE examples: Heat_energy_Y, kwh_T