R5.1.0

Thunder Release Notes R5.1

introduction

This document describes the new features and changes introduced in Thunder R5.1 (compared to the latest R5.0 release).
See here for the release notes of Thunder R5.0.
This document describes the changes in Thunder, ThunderTools and ThunderInterfaces as they are related.

Thunder

Process Changes and new Features

Feature: Github actions extensions

The Github actions were extended to have better build coverage, they are ran on every Pull Request commit. More components are now included in the builds. Also the 32bit build and Process Containers are now included as target.

Change: Unit test improvements

The existing Thunder unit tests were improved again and new tests were added. These are also triggered from a GitHub action on each commit added to a Thunder Pull Request.

Change: Thunder Documentation

The Thunder documentation was extended with new content, please find the documentation here. More content still to come!

Changes and new Features

New Feature: Whole plugin OOP (out of process)

It so now possible to run the full plugin out of process (OOP). Before this feature was added a plugin had to be specifically designed to be run OOP and part of it was always in process. The "whole plugin OOP" feature is added mainly as a development/debug option, for example to test leaks and crashes with plugins only designed to run in process. There is also the prerequisite that the plugin must have a JSON-RPC interface, a prerequisite that may be removed in a future Thunder version.
You use this feature by adding a root section including mode="Local" outside the config part of the plugin (like you normally would have inside the plugin config). Any other modes should work as well but have not been tested.

New Feature: PluginSmartInterfaceType

Next to the already existing SmartInterfaceType that can be used to more easily communicate with plugins exposing interfaces over a COM-RPP connection from native code a new PluginSmartInterfaceType was added that enables the same but then from plugin to plugin. Of course this could already be done but the new PluginSmartInterfaceType makes this much easier and requires less boilerplate code.
Find the code here and an example on how to use this feature can be found here.

New Feature: Concurrent Process Containers

On request of an external partner it is now possible to have multiple container flavours (container providers) active at the same time.
This feature was added fully backwards compatible regarding the Thunder and plugin config files, meaning if only one provider is active no changes to the config files are needed. However a change is required to the Container implementation as well and this has been done for runC and LXC. For Dobby and AWC we are discussing this with the owners of these implementations.
To use the feature first enable multiple container providers in your build. Then in the Thunder config file add the following section to set the default container implementation to be used for a plugin:

"processcontainers" : {
    "default" : "lxc"
  }

Other valid options are: "runc", "crun", "dobby" and "awc".
If in a plugin you want to use another container flavour than the default one you can do so by adding this to the config file of the container (of course the "mode : Container" should always be there when you want to run a plugin inside a container):

"root":{
   "mode":"Container",
   "configuration" : {
      "containertype":"runc"
   }
}

More detailed documentation on this feature and process containers in general will be added to the Thunder documentation in the near future.

New Feature: Flowcontrol

To mitigate issues with plugins that have methods that have a relatively long processing time while at the same time have an interface that for example invite the user of the interface to do a lot of calls to the plugin and/or the plugin emits a lot of events Thunder 5.1 now has a FlowControl feature.
Per channel (e.g. websocket) there can now be only one JSON-RPC call at a time, the next call received in parallel while the first one has not been handled will only be executed once the first one has been completed. Next to the channel flow control also a feature is added that will only allow one JSON-RPC call at a time to a certain plugin, so even if they come from different channels.
In a future Thunder version the number of allowed parallel calls (both for a channel as to a plugin) will be made configurable.

New Feature: Destroy plugin option

Next to the already existing possibility to Clone a plugin it is now also possible to Destroy a (cloned) plugin using the Controller COM-RPC or JSON-RPC interface.

Change: General bug fixes

As Thunder R5.0 was quite a change compared to Thunder R4 a lot of issue found in the mean time in Thunder 5.0 have been fixed in Thunder 5.1. If you are using Thunder 5.0 we would strongly advice to upgrade to Thunder 5.1. As Thunder 5.1 is released relatively short after Thunder 5.0 it does not contain that many new features and changes but more importantly does contain quite some fixes for issues found in Thunder 5.0.

Change: Message Engine improvements

There were quite some improvements to the message engine. UDP support for exporting messages was enhanced greatly.

Breaking Changes

Although Thunder 5.1 is a minor upgrade and therefore should not contain breaking changes there is one however. This because of the request to the Thunder team to include this fix and only after consulting Thunder user representatives.

Change: JSON-RPC over HTTP error code

We were made aware that the HTTP 202 result code that was returned in case there was an error handling a JSON-RPC call over HTTP was not as one would expect.
We did some investigation and although there is no official RFC for JSON-RPC over HTTP the consensus is indeed a 200 code should be returned, also when there is an error handling the JSON-RPC call. So Thunder 5.1 will return a HTTP 200 result code in this case.
Note in case this does cause issues in certain situations the old behaviour can be simply brought back by setting the LEGACY_JSONRPCOVERHTTP_ERRORCODE build flag when building Thunder.

Thunder Tools

Changes and new Features

Case handling improvements

There were may changes in the code generators to better support the different Case options for the JSON-RPC methods, parameters, POD members etc.

camelCase COM-RPC methods

camelCase COM-RPC methods names are now allowed and supported by the generators (please note this is not the standard used in the Thunder repositories)

Enhanced fixed array support

Fixed arrays are now also allowed as method parameters and POD members in interfaces. Please note only primitive types are allowed as already was the case with dynamic arrays. Thunder documentation will be updated soon with more information on this.

OptionalType enhancements

OptionalType is now also allowed for property indexes and iterators in interfaces. The OptionalType for the property index is for example used in the Thunder Controller interface: if filled it indicates the callsign of the plugin for which info is requested, if omitted (so empty) information for all plugins will be returned.

null returned for property setter

In case a property setter was used with an array or object it might return something else than null in previous versions of Thunder. The generated documentation however did (correctly) specify null would be returned. This is now also actually the case for the generated code.

Thunder Interfaces

Changes and new Features

IDolby

IDolby was made backwards compatible again (this was broken in ThunderInterfaces R5.0)

Test Report

R5_1-TestReport.zip

R5.0.0

Thunder Release Notes R5.0

Introduction

This document describes the new features and changes introduced in Thunder R5.0 (compared to the latest R4 release).
See here for the release notes of Thunder R4.4.
This document describes the changes in Thunder and ThunderTools as they are related.

WARNING

Thunder R5.0 is a major step up from R4. For the new features quite a lot of the internal code was changed or rewritten. Although it was very carefully tested by QA it will most probably contain some issues that have not been found yet as the Thunder framework can be used in a lot of different ways. Therefore our advise will be to use R5.0 only to do an early integration as it also contains some breaking changes (see the Braking Changes paragraph below). Please report any issues you find so these can be fixed. Our advise will be to use R5.1 for production usage as it will have all the combined fixes for the issues found with R5.0.

Thunder

Process Changes and new Features

New Feature: Issue Template

An issue template was added to the documentation for reporting Thunder issues. Please use the template that can be found here when you want to report a Thunder issue.

Change: Unit test improvements

The existing Thunder unit tests were improved and new tests were added. These are also triggered from a GitHub action on each merge of a Pull Request.

Change: Thunder Documentation

The Thunder documentation was extended with new content, please find the documentation here. More content still to come!

Change: QA interfaces

To make it clear what interfaces are specifically intended for QA purposes these have been put into a separate folder and namespace. They can be found here

Major Changes and new Features

New Feature: JSON-RPC non happy day scenarios

One of the main focus points of Thunder R5 was improved support for JSON-RPC non happy day scenario's. When a JSON-RPC interface was for example "session based" there was little automated support for handling connection issues and correct session management based on that (one would need to write the complete interface by hand). In R5 it is now possible to specify a separate JSON-RPC interface for these purposes and this will bring also much more options when the JSON-RPC interface deviates from the COM-RPC interface. An example can be found here but this will be fully documented in the Thunder documentation asap.

New Feature: Private COM-RPC

A feature was added where the communicator used to access a plugin via COM-RPC can be changed from the default communicator (that can be used to access all plugins) to a specific port so that for example applications in a process container can only be given access to that port and will not be able to access other containers.

New Feature: Off host plugin support

Infrastructure was added to enable running plugins on different devices/SOCs. A second instance of Thunder can be run on a different device handling the actual remote plugin but the plugin is exposed as a normal plugin in the local instance of Thunder (so can be activated/deactivated, communicated with via JSON-RPC and COM-RPC etc.)

New Feature: Aggregated plugin support

It is now possible to expose multiple plugins within a parent plugin, so to have an aggregated plugin. These can then be access and communicated with via the parent plugin. This is for example convenient when a group of plugins share a common responsibility or to expose multiple plugins running on a different host.

New Feature: Thundershark [Application]

A plugin was developed for the popular WireShark network analysis tool that enables analyzing Thunder COM-RPC in detail. All message exchange can be seen in detail (both online and offline) with details like which method is called on what interface using what parameters.
Details can be found here

New Feature: proxy leakage detection and reporting

Thunder now has improved detection of proxies being leaked when an out of process plugin is deactivated/crashed. These can now also be reported in more detail, for example via a JSON-RPC interface for QA purposes.

New Feature: Mac-OS support

Next to Linux and Microsoft Windows Thunder can now also be build and run on Mac-OS.

Change: IController suspend en resume via IController ILifetime

Managing suspend and resume plugin lifetime now can be done from the IController ILifetime interface. Most importantly this means that there are no longer both a JSON-RPC and COM-RPC interface file but all is specified in one IDL header file. The JSON-RPC interface has not changed because of this.

New Feature: Delegated release configurable.

Unbalanced reference counted objects are for the non-happy day scenarios now automatically destructed. This can lead to unexpected crashes in case the plugin has an incorrectly implementation of the refcount handling of objects. This automatic cleanup feature can be turned off by using the "ccdr" (COM-RPC channel delegated releases) boolean option in the Thunder configuration file (default is on off course)

Minor Changes and new Features

New Feature: Installation and Cryptography subsystems

Two new subsystems were added to the available subsystems: Installation and Cryptography.

New Feature: Messaging enhancement

The messaging engine now by default also reroutes all stdin and stdout messages to the message control plugin (where it then like all other messages can be printed, saved to file, forwarded to a websocket etc.). This behaviour can be turned of using the -f or -F startup parameter.

Change: Messaging efficiency improvements

The internal handling of messages in the messaging engine has been improved to reduce the duplication of messages and therefore improving the resource usage of the message engine in both memory as CPU usage.

Change: Messaging -F -f behaviour

When the message engine -f or -F option is used no longer the messages (tracing, syslog, waring reporting etc.) are forwarded to the message control plugin, they are now only locally outputted improving performance in this situation.

Change: Yocto meta layers

The Yocto meta layers for Thunder and its components have been refactored to comply with the latest CMake standards.

Change: Plugin/interface versioning improved

The plugin MetaData (holding the version information) is now also inserted through the COM-RPC definitions.

Change: JSON serialization/deserialization improvements

There have been multiple improvements to the JSON serialization and deserialization to improve compliance with the JSON standard (e.g. utf-8 handling, escaping, type support, hex characters have been improved)
These changes have also been backported to R4.4.

Change: Websocket improvements

WebSocket: Masking handling has been improved.

Change: Compliance higher waring level

CMake as well as Thunder code has been changed to handle higher warning levels and prevent false positives. Also the default is now that a warning is treated as an error.
Also compliance with GCC12 warning generation was achieved.

Change: Verify SSL certificates by default

In the secure websocket implementation the SNI is now always validated against the root-certificates installed on the box.

General bug fixes

A lot of issues were fixed in Thunder R5.0 improving stability and resource usage.

Breaking Changes

WPEFramework renamed to Thunder

All references to WPEFramework have been renamed to Thunder, this goes for example for the namespace in code, names of binaries, name of build files etc.
For the namespace changes mitigation measures have been put in place to have old code still using the WPEFramework namespace still compile but due to the complexity of the C++ language this will not cover all cases. Also make sure to adapt your script referring to Thunder binary and/or make files.

Plugin/interface versioning

In the JSON-RPC interface the plugin version is now embedded in a MetaData structure.

JSONRPC error codes

The JSON-RPC error codes are now better aligned with the JSON-RPC specification so error codes can have different numbers compared to older Thunder versions.

Old config meta files

The old config meta files are removed and the new config.in files are now the default. For plugins the old meta files are still supported.

AutoStart

In plugin config files the "AutoStart" : boolean has been changed into "Mode" : [Unavailable,Activated,Deactivated,Resumes,Suspend] . Reason for this change was the addition of the Unavailable mode.
Note that tools will still handle plugin configuration meta files correctly that use "AutoStart", it will generate a correct config will (where "mode" will be used).

Metadata

The "Metadata" structure in plugins has been renamed to "MetaData".

Plugin status via Controller JSON-RPC

The plugin status that can be retrieved via the controller JSON-RPC will now always return an array. You can still request one plugin status but it will be enclosed in an array

Addref has return value

The IReferenceCounted AddRef virtual method now requires a return value. This should not cause any issues as it is not expected that there would be a need to implement this outside the Thunder framework itself.
This was also backport...

Wait for the WebSocket to be open (i.s.o. the TCP connection)

Now the SmartLinkType<> waits till the WebSocket included the upgrade exchange before reporting that it is open.

In the past the Open would already return if the TCP connection was established, however, the socket is than not yet fully open, it first has to request the upgrade. If, before the socket was fully opened, the first messages where to be exchanged, they got lost.

QuickFix

In R2, the LinkType<> template to communicate with Thunder over JSONRPC, used a maxium timeout of 100ms to wait for the channel to open before to continue. This was set to 0ms as the assumption was that the SmartLinkType would be used for this purpose and the SmartLinkType waits for the Opened Event before submitting notifications/invokes.

However some application still use theLinkType<> and expect the Channel to be open after the construction (trickey, not advised but it is code out there). Restored the waittime now to at least 1S.

Bug fixes on R2

Minor bug fixes on R2-v1.6
Introduction of Unavailable state

Bug fixes on R2

Major focus of this release is verbosity. Try to report as much as possible if exceptional shutdowns occure.

Minor bug fixes on R2

Minor bug fixes on R2-v1.4

Minor bug fixes on R2

Minor bug fixes on R2-1.3

Minor bug fixes on R2

See commit log for changes.

Fixes on R2

This is a maintenance release that has no new functionality but has several fixes found during integration testing at a customer site.