Beta Progress

[PatrickRitchie]

Observations

After looking at implementing DataSet and Table observations and how to store them in the buffer, I revised the way observations are handled. Now all observations will implement the IObservation interface and inherit the Observation base class.

Each observation can now contain multiple values (this is the way an observation is described in the MTConnect standard) and a single Sequence and Timestamp. Previously I had designed it so that multiple observations could have the same sequence and then they were combined when requested from the buffer. This approach caused problems when I went to implement the way DataSet and Table entries are stored.

After re-reading the sections of the standard on how these observations are handled, it made since to rename the StreamingBuffer to ObservationBuffer since the term "Observation" seems to be the universal term for streaming data.

DataSet observations are now working correctly as of commit 949035a. I still have to implement the entries for Table observations.

Validation

I updated the DataItem.IsValid() method to now perform generic validation and call a separate OnValidation() method that derived classes can override. This prevents the same code from needing to be copied to each DataItem. These methods now return a DataItemValidationResult struct so that a descriptive message can be passed to the agent application to display and/or log.

Generic validation still needs to be finished so that it can check that, for example, the Observation matches the Representation, a Sample has a numeric value (will need to handle *_3D types and others that have a special format).

The difference betwen IsValid() and Process() methods will be that IsValid() validates the Observation that is added to the Agent and Process() removes/alters the object that gets returned/serialized in a Response Document. This is to allow compatibility with different MTConnect schema versions.

I added a similar Asset.IsValid() method to IAsset but it still needs to be implemented for each Asset Type.

SHDR DataSet conversion

I updated the way that DataSet entries are read from an SHDR string so that single quotes, double quotes, and curly braces are now supported as well as determining whether an entry is set to be removed. I used a fairly long Regex pattern so I'm thinking that performance may be an issue and might need to be rewritten before the first Release.

Compositions

Apparently I skipped over the part of the description of Compositions that don't allow DataEntities. I will need to update the Composition class as well as some of the processing. Also the Models for Compositions will need to be updated though I think I can still just add the DataItem to the parent component and set the CompositionId attribute while maintaining the same structure that the classes are currently in.

Adapter Examples

I'm still putting together some example adapter applications. The goal is to have an example similar to the MTConnect Adapter SDK and an example using Models.

MTConnect Http Server

I noticed the default address for the Http server to listen on was 'localhost' which I had set for testing. When changing it to '+' to accept any address, I was getting an Access Denied error which requires a windows shell command to allow that Url to be listened on. I think I will handle this in an installer for the MTConnect Agent application as I already have done that with other installers and it seemed to work smoothly.

To Do

• Table Entry Observations
• Path parameter in Streams requests. I'm still debating on how to handle this in order to support XPath (as defined in the standard) and most likely JSONPath which seems comparable. I think I would need to read the Devices document from the buffer first then use XPath on an XmlDocument to determine a list of DataItemIds to pass to the ObservationBuffer.
• At parameter in Streams requests. I think I have everything I need already, just need to test with DataSet and Table Entries.
• Go through Assets to make sure all properties serialize to JSON properly. Mainly thinking about CuttingTool as the elements can be nested several levels deep.
• Update Asset metrics when Assets are added
• Need to automatically add an 'Agent' device? I may need to get more information on what is required of the standard and what is left up to the implementation.

[PatrickRitchie]

Duration

In commit 7259b0e I added the ability to read/write a Duration in the SHDR line and also to output that in the MTConnectStreams response document. I used a Regex pattern to parse the Timestamp and Duration for lines that contain both. The Duration is now stored for an Observation using the 'Duration' ValueType.

Reset Trigger

I started adding properties to implement the ResetTrigger and ResetTriggered attributes. I'm a little confused on if the Agent is always/sometimes/never responsible for resetting a value. Some make sense such as Day, Week, Year but others like Shift would need to have the shift defined (which I don't believe MTConnect supports as of now). According to the SHDR protocol, you can pass a trigger to reset but I will have to implement that into Shdr classes once I get some more information about how all of this is supposed to work.

Discrete

I still need to implement the Discrete attribute for DataItems and how that works with DataSet and Table. It looks like for Value DataItems, it will just be a boolean to determine whether to filter out duplicates but looks like there is some other notes on how it works with DataSets and Tables so I will have to get some more information about that.

Initial Value

I have some implementation of the InitialValue attribute done but it looks like some more work will need to be done to implement that with the Reset Trigger.

Data Conversion

Data conversion when an observation is added still needs to be implemented. I have conversion classes for each Sample but after looking at it again, I think I will need to rethink my initial approach. I initially had a ToMetric() and ToInch() method in the SampleValue class but since multiple NativeUnits resolve to the same Units (ex. Inch and Foot are both Lengths that resolve to Millimeter), I think I will need to make one method with a parameter of the Units to convert from or something similar to that.

Conditions

It looks like I may need to review how Conditions are handled in order to make sure a Normal observation resets any other Faults or Warnings as I believe is the expected behavior. Also I need to make sure that multiple fault states can be present which I believe is also how they are supposed to work.

To Do

• Reset Trigger implementation
• Conditions
• Data conversion on Observation add
• Table Entry Observations
• Path parameter in Streams requests. I'm still debating on how to handle this in order to support XPath (as defined in the standard) and most likely JSONPath which seems comparable. I think I would need to read the Devices document from the buffer first then use XPath on an XmlDocument to determine a list of DataItemIds to pass to the ObservationBuffer.
• At parameter in Streams requests. I think I have everything I need already, just need to test with DataSet and Table Entries.
• Go through Assets to make sure all properties serialize to JSON properly. Mainly thinking about CuttingTool as the elements can be nested several levels deep.
• Update Asset metrics when Assets are added
• Need to automatically add an 'Agent' device? I may need to get more information on what is required of the standard and what is left up to the implementation.
• Agent Commands from SHDR

[PatrickRitchie]

Conditions

In commit 1ae89b4 I completely redid the way Conditions are handled in order to be compliant with MTConnect's ability to handle multiple Condition FaultStates at any point in time.

Adapter

The ShdrCondition class no longer inherits the Observation base class and now has a list of FaultState objects as a property. The ShdrFaultState implements the Observation base class and represents each FaultState that is active for the Condition. This allows multiple observations for the same condition to be sent to the agent.

var condition = new ShdrCondition("system_cond");
condition.AddWarning("Bad Request", "400");
condition.AddFault("Not Found", "404");
condition.AddFault("Method Not Found", "405");
_adapter.AddCondition(condition);

Agent

The agent handles multiple Conditions by storing them in a new local variable list _currentConditions. This basically just allows multiple observations to be stored per key in the list.

ObservationBuffer

The buffer now stores multiple current Condition observations and will manage updates according to the standard. The buffer has a new local variable list (also called) _currentConditions which allows multiple observations to be stored per key.

SHDR

2022-02-06T06:48:50.9090000Z|system_cond|WARNING|400|||Bad Request
2022-02-06T06:48:50.9090000Z|system_cond|FAULT|404|||Not Found
2022-02-06T06:48:50.9090000Z|system_cond|FAULT|405|||Method Not Found

MTConnectStreams Current Request

<Warning dataItemId="system_cond" timestamp="2022-02-06T06:48:50.9090000Z" sequence="121" type="SYSTEM" nativeCode="400">Bad Request</Warning>
<Fault dataItemId="system_cond" timestamp="2022-02-06T06:48:50.9090000Z" sequence="122" type="SYSTEM" nativeCode="404">Not Found</Fault>
<Fault dataItemId="system_cond" timestamp="2022-02-06T06:48:50.9090000Z" sequence="123" type="SYSTEM" nativeCode="405">Method Not Found</Fault>

TO DO

• Reset Trigger implementation
• Data conversion on Observation add
• Table Entry Observations
• Path parameter in Streams requests. I'm still debating on how to handle this in order to support XPath (as defined in the standard) and most likely JSONPath which seems comparable. I think I would need to read the Devices document from the buffer first then use XPath on an XmlDocument to determine a list of DataItemIds to pass to the ObservationBuffer.
• At parameter in Streams requests. I think I have everything I need already, just need to test with DataSet and Table Entries.
• Go through Assets to make sure all properties serialize to JSON properly. Mainly thinking about CuttingTool as the elements can be nested several levels deep.
• Update Asset metrics when Assets are added
• Need to automatically add an 'Agent' device? I may need to get more information on what is required of the standard and what is left up to the implementation.
• Agent Commands from SHDR

[PatrickRitchie]

ResetTriggered

In commit d86f894 I updated the SHDR classes, MTConnectAgent, and MTConnectObservationBuffer classes to implement the ResetTriggered function in the MTConnectStreams response document.

I also added a new Streams.ResetTriggered enum since it differs from the Devices.ResetTrigger enum (I initially read them as being the same but they have slightly different values and descriptions). This meant I needed to update some of the code to reflect this so it gets output correctly in the StreamsDocument.

Adapter

var dataSet = new ShdrDataSet("testDataSet");
dataSet.ResetTriggered = ResetTriggered.MANUAL;
dataSet.Entries = new List<DataSetEntry>
{
    new DataSetEntry("T1", 9.1234),
    new DataSetEntry("T2", 7.0000)
};
_adapter.AddDataSet(dataSet);

SHDR

2022-02-06T21:16:27.7160000Z|testDataSet|:MANUAL T1=9.1234 T2=7

MTConnectStreams

<ToolOffsetDataSet dataItemId="testDataSet" timestamp="2022-02-06T21:16:27.7160000Z" sequence="126" resetTriggered="MANUAL" count="2">
  <Entry key="T1">9.1234</Entry>
  <Entry key="T2">7</Entry>
</ToolOffsetDataSet>

TO DO

• Data conversion on Observation add
• Table Entry Observations
• Path parameter in Streams requests. I'm still debating on how to handle this in order to support XPath (as defined in the standard) and most likely JSONPath which seems comparable. I think I would need to read the Devices document from the buffer first then use XPath on an XmlDocument to determine a list of DataItemIds to pass to the ObservationBuffer.
• At parameter in Streams requests. I think I have everything I need already, just need to test with DataSet and Table Entries.
• Go through Assets to make sure all properties serialize to JSON properly. Mainly thinking about CuttingTool as the elements can be nested several levels deep.
• Update Asset metrics when Assets are added
• Need to automatically add an 'Agent' device? I may need to get more information on what is required of the standard and what is left up to the implementation.
• Agent Commands from SHDR

[PatrickRitchie]

Unit Conversion

In commit 2a55839 I added the functionality to the MTConnectAgent class to perform Unit Conversions on added Sample Observations.

My initial approach was to set the conversion value in each sample class that inherited SampleValue as overridable properties for the Units and the NativeUnits that matched. But this ended up in a lot of duplicated code when the Devices.DataItem class should really be the only place to contain the default Units. This also caused an obvious issue when I realized that multiple different NativeUnits convert to the same Units listed in the MTConnect standard (ex. INCH and FOOT both convert to MILLIMETER).

The new approach was to just add a universal Unit conversion static method in the Streams.DataItem class that takes the Units (to convert to) and the NativeUnits (to convert from) and use a switch statement to do all of the math needed for unit conversion. I left the math as verbose as possible to make it clear to read the code.

This new method is then called in the MTConnectAgent class in the ConvertObservationValue() method when a new Observation is added with a Category of SAMPLE. This method also handles *_3D types by parsing the CDATA of the Sample and converting each value and then reconstructing the CDATA string to pass to the Buffer.

Each individual SampleValue class now inherits from the SampleValue class which has Value and NativeValue properties which handle Unit Conversion (mainly used when using Models). The SampleValue generic was needed in order to handle the *_3D types as the CDATA value is not a double but rather a space-delimited string. I still need to test this a little more but the idea is that you can access the original native CDATA by using the NativeValue property and the converted CDATA by using the Value property. This should make writing data to an Agent (or Adapter) easier as well as when reading from a client.

Native Scale

The NativeScale attribute for DataItems is also handled in the new MTConnectAgent.ConvertObservationValue() method.

TO DO

• Table Entry Observations
• Path parameter in Streams requests. I'm still debating on how to handle this in order to support XPath (as defined in the standard) and most likely JSONPath which seems comparable. I think I would need to read the Devices document from the buffer first then use XPath on an XmlDocument to determine a list of DataItemIds to pass to the ObservationBuffer.
• At parameter in Streams requests. I think I have everything I need already, just need to test with DataSet and Table Entries.
• Go through Assets to make sure all properties serialize to JSON properly. Mainly thinking about CuttingTool as the elements can be nested several levels deep.
• Update Asset metrics when Assets are added
• Need to automatically add an 'Agent' device? I may need to get more information on what is required of the standard and what is left up to the implementation.
• Agent Commands from SHDR

[PatrickRitchie]

Type Description Properties

In commit 7a481be I added a TypeDescription property to each Device, Component, Composition, and DataItem class (and derived classes). This property references a constant string in each class that contains the description of the entity from the MTConnect Standard documents. I already had the descriptions as documentation for the source code (for Intellisense) but this will allow a client application to read the description in order to give more context to each entity.

public class ExecutionDataItem : DataItem
{
    public const DataItemCategory CategoryId = DataItemCategory.EVENT;
    public const string TypeId = "EXECUTION";
    public const string NameId = "execution";
    public new const string DescriptionText = "The execution status of a component.";

    public override string TypeDescription => DescriptionText;

For SubType descriptions, I added an overridable GetSubTypeDescription() method in the DataItem base class that returns the description for the SubType of the DataItem.

public override string GetSubTypeDescription() => GetSubTypeDescription(SubType);

public static string GetSubTypeDescription(string subType)
{
    var s = subType.ConvertEnum<SubTypes>();
    switch (s)
    {
        case SubTypes.DRY_RUN: return "When DRY_RUN is ON, the equipment performs all of its normal functions, except no part or product is produced.If the equipment has a spindle, spindle operation is suspended.";
        case SubTypes.SINGLE_BLOCK: return "Program execution is paused after each BLOCK of code is executed when SINGLE_BLOCK is ON";
        case SubTypes.MACHINE_AXIS_LOCK: return "When MACHINE_AXIS_LOCK is ON, program execution continues normally, but no equipment motion occurs";
        case SubTypes.OPTIONAL_STOP: return "The program execution is stopped after a specific program block is executed when OPTIONAL_STOP is ON.";
        case SubTypes.TOOL_CHANGE_STOP: return "Program execution is paused when a command is executed requesting a cutting tool to be changed.";
    }

    return null;
}

This should also support extended classes that inherit those base classes so custom types should be supported.

Usage

foreach (var dataItem in device.DataItems)
{
    Console.WriteLine($"DataItemId = {dataItem.Id}");
    Console.WriteLine($"Type = {dataItem.Type} : {dataItem.TypeDescription}");
    Console.WriteLine($"SubType = {dataItem.SubType} : {dataItem.GetSubTypeDescription()}");
    Console.WriteLine();
}

DataItemId = avail
Type = AVAILABILITY : Represents the Agent's ability to communicate with the data source.
SubType = :

DataItemId = functionalmode
Type = FUNCTIONAL_MODE : The current intended production status of the device or component.
SubType = :

DataItemId = emloaded
Type = EQUIPMENT_MODE : An indication that a piece of equipment, or a sub-part of a piece of equipment, is performing specific types of activities.
SubType = LOADED : Measurement of the time that the sub-parts of a piece of equipment are under load.

DataItemId = emworking
Type = EQUIPMENT_MODE : An indication that a piece of equipment, or a sub-part of a piece of equipment, is performing specific types of activities.
SubType = WORKING : Measurement of the time that a piece of equipment is performing any activity the equipment is active and performing a function under load or not.

[PatrickRitchie]

Observations

Major changes to the way Observations are handled and structured in commit 0e50b3b. Observations are now handled by Category, Representation, and by Type (if necessary). There are now classes for:

• SampleValueObservation
• SampleTimeSeriesObservation
• SampleDataSetObservation
• SampleTableObservation
• EventValueObservation
• EventDataSetObservation
• EventTableObservation
• ConditionObservation

Each one of the above classes only has properties based on the representation. For example, a SampleDataSetObservation has Entries and not CDATA. All values are accessible through the Observation base class. This is to allow a DataSet to still access CDATA in order to determine UNAVAILABLE. This also allows a client application to read all Observations as the Observation base class and still have access to all of the Values using the GetValue(string valueKey) method. Each value is stored using a "ValueKey" (this was previously called "ValueType"). For example a value for a DataSet may be "DATA_SET[var1]" to reference the DataSet value at key = var1.

Each class above now implements a corresponding Interface which is read only. This way a client will receive the read only Interface since they wouldn't need to change the values when reading.

Project Structure

The project is now divided up into several more assemblies.

• The MTConnect.NET-Common assembly is intended to contain all of the models and Agent logic but none of the Protocol or Document Formatting classes.
• The MTConnect.NET-HTTP assembly contains all of the code necessary to implement the HTTP Protocol for both the Agent and Clients.
• The MTConnect.NET-XML assembly contains all of the code necessary to create XML documents using the data retrieved by an IMTConnectAgent or client.
• The MTConnect.NET-SHDR assembly contains all of the code necessary to implement the SHDR protocol for adapters.
• The MTConnect.NET-MQTT assembly contains all of the code neccesary to implement an MQTT broker (still in development)
• The MTConnect.NET-JSON assembly contains all of the code neccesary to format the response documents in JSON (still in development)

This structure should provide more flexibility as well as make development/maintenance easier.

Path Query Parameter

The "Path" http query parameter is now implemented to use XPath to filter results from the IMTConnectAgent. There is an MTConnect.Formatters.PathFormatter class responsible for handling this and is left generic in order to possibly interpret the Path parameter as something other than XPath (example would be JsonPath for a Json document response).

At Query Parameter

The "At" http query parameter should be working properly now. I still need to do some final testing to insure it is working properly with DataSets and Tables though.

Xml Serializer

I went through and rewrote much of the way I was previously serializing/deserializing XML. This is almost all custom serializers for Streams and I will most likely write all custom serializers for Devices and Assets as well. This allows to fully separate the XML from the Device classes in MTConnect.NET-Common and will hopefully allow me to remove all of the Xml related attributes for classes and members.

One benefit to writing custom serializers is the ability to output Xml Comments using the descriptions in the Devices classes. This is partially working as of now. I still need to add comments for some of the classes in Devices but will need to write the custom serializers for them as mentioned above.

Applications

I added a few new Agent applications, MQTT Agent and Gateway Agent (both are still under development).

The MQTT agent is just working to the point that it creates an MQTT broker and sends Devices and Streams data as topics. The message payload still needs to be figured out though. My plan is to create several JSON document format assemblies (if needed) such as a simple JSON that follows the current model as well as possibly a JSON-SparkplugB that would format the message payload according to the Sparkplug-B specification. This seems to make the most sense since it would be compatible with other MQTT applications.

The Gateway Agent is an agent that runs MTConnect clients on the backend and pushes the response documents to the gateway agent. This provides a central access point for multiple clients as well as other use cases such as being able to run the primary agent on a server with more resources than the device the source agent is running on. Also this agent can run on IIS which would allow more security options.

ToDo

• Further Assets development (mostly working but needs to be refined)
• Implement SHDR agent commands (used for configuring the agent through SHDR)

[PatrickRitchie]

Support

I wanted to give an update and say that I appreciate the support received so far for this project. Those that have given feedback, created Issues, and downloaded and used this library is very much appreciated.

We are looking at getting this ready for testing in production by the start of May 2022. If anyone is interested in performance testing or testing on different runtimes/devices (ex. Raspberry Pi), the results of those tests would be much appreciated.

Thanks again for the support and if you like this project and found it helpful then please Star our repository here in GitHub as that helps us show up in search results.

Thank you,

-Patrick

[PatrickRitchie]

Agent Sequence Numbers

@SergeyGulik added a Pull Request to fix an issue with the way the Agent and Observation Buffer were handling the sequence numbers. Essentially the sequence number was getting incremented when it shouldn't have been and so it was skipping a sample in certain situations. Thanks to @SergeyGulik for the contribution as well as adding a project for Integration Tests which I will look to use and build onto going forward. Please don't hesitate to create Issues or Pull Requests as that is very helpful and will hopefully make the project better and more useful for everyone.

Http PUT Request

Added functionality to accept a PUT request in the MTConnectHttpServer class. The MTConnectHttpServer class has a virtual OnObservationInput() method that is meant to be overridden by a derived class. This was to separate the SHDR library from the Http library as well as support future implementations that may have a different protocol/format.

The ShdrMTConnectHttpServer class is used to implement this and processes each URL query parameter value as an SHDR line. Supports Samples, Events, and Conditions as well as Value, TimeSeries, DataSet, and Table representations.

I still need to add POST functionality to add Assets. Since they don't use SHDR and will have to be handled differently, I didn't include it with these most recent changes.

Linux Testing

I started testing the MTConnect-Agent-Http application on an Ubuntu x64 box and everything seems to be working correctly. I would also like to test with ARM devices at some point as I have compiled binaries and included them in the latest Releases. I will put together a short tutorial on how to get it running on Ubuntu, but it was pretty simple, just install .NET 5 or .NET 6 and run "dotnet MTConnect-Agent-Http.dll" in a terminal. This also worked for the MTConnect-Agent-Http-Gateway application.

ToDo

• Add PUT functionality to the MTConnect-Http-AspNetCore library
• Update Models to get that working again since the latest update to Observations

[PatrickRitchie]

v0.5.0 Release

I just published the new 0.5.0 release which is hopefully one of the last beta versions before the first official release. There was some new functionality added in this release so I would like to test that out before an official release.

Windows Installer

One of the first things added was a new Windows Installer that will hopefully make setup of Agents easier. I will be including an installer with any new releases from now on. The installer unpacks the files in the selected directory and then installs the Agent as a Windows Service.

Windows Service

I added functionality to install and run the Agents (and Adapters) as a Windows Service using command line parameters. The parameters use the same format as traditional agents (install/run/debug) but I also added several options to Start, Stop, and Remove the Service. I also added an optional parameter to set the Http Port to use. This was helpful for testing as you can launch several agents at different ports without having to create new configuration files for each.

File System Buffers

One of the things added was an Agent project to use the new File Buffers that backup the Observation and Asset buffers to make them persistent. This retains the state of the Agent between restarts. I have run into issues in the past where data is missing due to a machine/PC being restarted and this should fix that issue.

This took some work to get to work correctly as both the Observations and Assets need to be stored in the sequence they were recorded by the Agent so that upon restart, the sequence numbers are the same. I ended up writing my own page file structure that creates a page file with a file name of the largest sequence stored in that file (ex. 100 would store 0-100). This allows for correct ordering as well as you can search for specific sequence numbers based on the filename. Assets are stored in a similar way except that each file represents a single asset and the filename includes the asset type.

All files allow compression (gzip) and 150,000 buffered observations took up about 1.5 MB on the file system. The file contents for Observations is a JSON array of the Observation. This is the most minimal structure I could come up with while still maintaining all of the necessary information.

Asset files are stored as a JSON object. This caused me to finally separate all of the Asset models from XML as before I was still storing the original XML in the object. This was originally to support non standard assets but I think the best approach is to just create an extension library if custom assets are needed.

Assets

The in-memory Asset buffer was updated and now works in the correct circular FIFO pattern described in the MTConnect Standard. I apparently misread that part originally as the previous method was just a key-value store without an order.

I added the ability to add Assets using an Http POST request as well as I added functionality to Remove and Remove All Assets to the SHDR adapter using the @REMOVE_ASSET@ and @REMOVE_ALL_ASSETS@ commands.

XML Validation

I added the ability to validate against XSD schemas and to log any validation errors in both the Log file as well as within an MTConnectError response document. I also added a ValidationLevel configuration parameter to set whether validation errors should be ignored completely, warnings logged but still send the document as normal, or to log the errors and return an MTConnectError document.

Validation is handled internally using the individual Type classes so, in theory, XML validation shouldn't be needed in most cases as the MTConnectAgent class should only respond with a valid document.

Using the "version" Http query parameter, you should be able to request a document from the Agent using any version of MTConnect and the Agent will remove any entities that are not compatible. I tested this the best I could but would appreciate any feedback if anyone runs into any issues. This is mostly to be able to update your MTConnect Agent while not effecting any client applications that might not support MTConnect 2.0 yet.

Tests

I added a few new Unit Test projects to the repo. The only one I have updated so far is the MTConnect.NET-SHDR library. I added tests for DataItem SHDR output formats. I will look to add to this project as well as other tests. Any contributions to this would be appreciated.

Configuration File Watcher

I added some configuration file watchers to automatically restart the Agent when the configuration file changes. There is also a watcher for the Devices file that will not restart the Agent but will update the Device.

Agent Device

I updated the "Agent Device" to function better as well as to retain the UUID between restarts. Adapter components are added for each Adapter that is set in the configuration file

Configuration File Changes

I changed some of the wording of some of the configuration file parameters. Previously I was just using the wording used in traditional agents but I decided to deviate from that as some of the parameters did not apply. The main changes are using the term "DeviceKey" as opposed to "Device" or "DeviceName". This is used throughout the source code as a word to mean that it can be either the Device "Name " or "UUID". I also changed the name of the SHDR adapter "Host" to "Hostname" to be more uniform.

Documentation

I'm working on better documentation for both the library and the Agent applications. I will look to have better documentation by the time of the first official release.

Notes

If anyone is interested in testing out the Persistent Agent then please let me know. I didn't include a compiled version in the release since I haven't gotten a change to test it out much yet. I'm not sure how I'm going to package it yet, whether it should be an configuration option to enable "Persistence" or if it should be a separate application. Any feedback on this would be welcome.

Thanks,

-Patrick

[PatrickRitchie]

v0.5.1 Release

Linux

I updated some of the Program.cs classes for the Agent applications to skip over the ServiceProcess related stuff in order for everything to work correctly on a Linux system.

Docker

I created some new Docker images located Here. The images are to run the Agent applications in a Docker container that exposes the default 5000 port for the Agent and the 7878 port for the SHDR Adapter. I built images for most of the dotnet runtime images that were listed. If anyone has a specific image they would like then please let me know and I will add it.

Agent Version in Nuget Packages

I updated the Agent version that shows up in the MTConnect Headers to use the version of the MTConnect.NET-Common assembly. I was a little conflicted on whether this should be the version of the application or the library but I think the version of the library gives better context on the functionality of the Agent. This would mean that the "version" sent in the MTConnect header would be 0.5.1 (latest version) regardless of the application it is built with.

[PatrickRitchie]

Upcoming Releases

Version 0.6

I'm looking to publish the Release for version 0.6 by the end of this week.

This version is focused on updates to the Agent functionality and performance. There were some issues that I had found in earlier versions that caused excess CPU and memory usage. I completely redid how the buffered observations were being stored and now the performance is much better. I've also done some work to improve the File system backed buffers.

There was also some missing functionality for the Agent that I added. Along with the MTConnect standardized functionality, I added a few new features including a "deviceType" URL query parameter to exclude the "Agent Device", the ability to load Device files from a directory instead of a single devices.xml file, the Device files can now support just the node (excluding the header and nodes), and a few other features.

One of the major updates is to the HttpServer. I found an issue with a memory leak which caused me to rethink how the server should function. The issue is now fixed and will also support a separate thread pool for streaming requests in order to not tie up the single request threads. This is an issue I have seen in the past where if you have, for example 5 streams connected, and the default max is 5, then single Http requests (ex. Probe) will get blocked. This is now fixed by using a separate pool.

Version 0.7

For version 0.7, I will be focusing on the SHDR adapter to review the functionality and improve performance. Some of the SHDR protocol commands are processed using Regex and some of those patterns look like they could be optimized along with a few other areas. I will be looking to solidify a better design layout for how the adapter actually sends data. I initially created a Queue that would send data at an interval but I think I need a few additional options. I will most likely make an ShdrAdapter class that sends at an interval, a class that sends on demand, and a class that has a buffer (sends everything). I'm also looking at a file system backed buffer.

Version 0.7 should be released during the week of August 12th.

Version 0.8

Version 1.8 will focus on reviewing the functionality of the Client classes. I've done a lot of work on the Agent and will need to go back and make sure that all of that functionality is updated in the Clients. Two major items that I can think of right now are supporting response compression and supporting JSON.

Version 0.8 should be released during the week of August 19th.

Version 1.0 (Official Release)

The first official release, v1.0, should be released September 1st.

This will contain v1.0 for the Agent Applications and v4.0 for the Nuget packages. I am working on a documentation site to better document how to use both the Agent and libraries and should have that available by the release date. I'm hoping to have at least a few examples and a tutorial to step through how to use MTConnect.NET to embed an MTConnect Agent in an application.

My goal is to have Windows Installers for each Agent application, an APT repo for Linux distros (ex. apt-get install mtconnect-agent-http), Docker images for each Linux distro, and Nuget packages for each library.

Future

After the v1.0 release, I will be looking to add on other protocols such as MQTT and looking at a Protobuf document format.

I am also looking to release the first beta version of my main TrakHound project around the first of September. This has been my main project (MTConnect.NET was kind of a side project) that I've been working on for essentially a few years. I will release more information over the next month but the main focus is on how to store data such as MTConnect and how to allow that data to live in the same space and interact with other data. I'm very excited about this project and am anxious to get it released as I've had to delay the release for several months to get things just right. The first specification that will be supported is MTConnect and so that is what led me to further development on this MTConnect.NET project.

Thanks for all of the support from everyone and stay tuned for the upcoming updates!

-Patrick

[PatrickRitchie]

Updated Release Schedule

I found some issues with performance and delayed the release of v0.6. After a lot of profiling and testing, I rewrote the buffer as well as wrote a new Xml serializer and am getting ~3 ms responses for Probe, Current, and Asset requests. I also noticed some issues with high memory usage so I addressed that and it now seems to be stable and uses much less (around 60 MB). This library is intended to be used on machine HMIs and embedded systems so I wanted to make sure the resource usage was minimal.

I'm looking to have v0.6 released by the end of this week, combine v0.7 and 0.8 to be released by the end of the next week, and still look to release v1.0 the first week of September.

[PatrickRitchie]

Version 0.6.1 Release

I just published the new 0.6.1 release that fixed a few issues that were noticed in 0.6.0 that I released on Friday. The main focus was to fix issues seem with HTTP streaming.

Http Streaming

There ended up being several issues with streaming, some on the server side and some on the client side. Most of the issues were with spacing (whitespace characters) and compression encoding. I also added logic to parse the leading and trailing whitespace along with any encoding marks (BOM) when deserializing the response documents to XML.

Observation Buffer

In the 0.6.0 release I had gone through and totally rewritten the observation buffer to increase performance but there was one line that used Array.Copy() when the buffer was full to shift the array over. This was pretty inefficient because it shifted the entire array so I was able to find some examples of circular buffers which I based the new CircularBuffer class on. The new CircularBuffer class no longer needs to shift the array over and instead just keeps up with an internal index that can overflow/wrap. This makes it much faster and I immediately saw performance gains.

Release Schedule

I ended up combining the Agent, Adapter, and Client updates into the 0.6 releases so I may just end up going straight from 0.6 to 1.0. I'll look to publish the 1.0 release later next week as long as I don't come across anything before then. I'll be working on some better examples and better documentation after 1.0 is released.

Thanks again to everyone who has given feedback and has used this project! Please let me know if anyone find any issues and has any feature requests.

Thanks
-Patrick