Define a way for consumers to advertise data they are interested in

[amrc-benmorrow]

We need a mechanism for data consumers to advertise the data they want, so the infrastructure can work out how to provide it.

Ideally this will not involve requiring every consumer to become a Sparkplug Node; we've been down that road before and the only way to achieve it would be some sort of dynamic registration of a Node address for a particular instance of an application. This then breaks the assumption that a Node is a security principal; we could have multiple dynamic Nodes operated by the same user account.

All our security is per-user, so if one user account is running two consuming applications at the same time it is reasonable for them to able to see each others' data, and for both sets of adverts to end up in the same place. However it must be possible for each application to work out what subscriptions it needs to make to get the data it needs, and only that data, and it must be possible for one application to withdraw its adverts without affecting another.

[amrc-benmorrow]

There are basically four possibilities here:

• (R) Consumers make REST requests directly to the reconciliation service.
• (A) Consumers make REST requests to an advertisement service.
• (M) Another MQTT namespace (outside spBv1.0) which potential consumers can publish to.
• (S) A dedicated Sparkplug Node which publishes adverts on consumers' behalf.

With option (R), consumption requests are tied rather directly to a particular reconciliation service. It would be better to support multiple reconciliation services with different capabilities, for example services providing automatic translation.

With options (R) and (A) it will be necessary to define a path for reconciliation results to be pushed back to the consumers. These results will be asynchronous and are likely to change over time as producer devices come and go. In principle this means that for Factory+ they should travel over Sparkplug.

Option (M) leaves open the question of how we ensure any reconciliation service(s) have seen all available adverts. Even if we use MQTT's reliability features to the full we would still need some kind of 'everybody rebirth' facility for when a reconciler loses their MQTT session. We run the strong risk here of trying to redesign Sparkplug; while that might be a good plan it is not what we should be doing here.

Option (S) requires a means of control: assuming we create a Device per user or per application session we will need a way to request Device creation, to publish adverts to the Device, and to publish responses back which the consumer can use. This could all be reasonably be accomplished via CMDs, with the consumer(s) and the reconciler(s) using the Device as a rendezvous point, except perhaps for the initial Device creation request. That could either use a custom REST API on the rendezvous service, or we could standardise that CMDs convey the original principal somehow. In either case the rendezvous Node would need the right to grant specific access to its child Devices; this is not currently a permission that exists in the Auth service.

[amrc-benmorrow]

While it's attractive to perform rendezvous entirely over Sparkplug, there may be too much data. There is also the perennial problem of Sparkplug not providing atomic updates. This may devolve into another mostly-REST service with Sparkplug change-notify.