Code generation

[sleipnir]

Although we rely on serializable data types via Protobuf and our user experience in the development of our APIs so they are contract-first approaches the fact is that we do little with regard to generating code for the user. If we could generate code in different languages ​​from protobuf declarations including the rpc type we would bring a better experience for our developer users.

The positive side effect of this approach is that by declaring service methods via protobuf's service rpc we could generate the methods of the actor's functions in the supported languages ​​and we could guarantee the consistency of the names of these methods for the different languages. Currently we make invocations to actors passing the method name as an argument, but developers of different languages ​​have different ways of naming their methods/functions, for example a python developer would think of a method name based on the pattern using underscore while a java developer would use the camel case pattern.

If the methods were generated from the protobuf rpc source then the command names of the actors would be declared consistently between actors regardless of the way adopted to declare their method or function in the different sdks.

[marcellanz]

I found so far to be quite liberating not to provide too much tooling for the user for this. I meant the protoc compiler plugins already produce idiomatic code mapping to protobuf definitions then. protobuf itself has its own rules how to name things I think.

I'd be not against to provide more tooling for users, but we also could just let them use what they think is best, and our interface towards Actor invocations is just method name and a buffer holding the serialised message; in our case proto.

Regarding tooling; I've followed Lightbend a bit with their Java and JS SDKs, they seem to invest quite so resources to produce the "service objects" and therefore code for the service contracts their users describe services in – hiding code generation otherwise than the API contract in proto.

[sleipnir]

Hi @marcellanz

I agree with the issue of effort and it was not my intention here to prioritize this type of code generation, as I said before my concern was more subtle and one of the ways to solve the issue would be with the use of code generators, but we have a solution that requires less effort and therefore should be prioritized, in my view, to the detriment of the solution based on generators.

But it's good to discuss and understand everyone's views and that's why I preferred to open a general discussion.

My opinion, and I could be wrong, is that we should expose the Spawn core protocol at all times.
What I mean by exposing is that if the user wants to have their own implementation without getting stuck there is one of our sdks then they should be able to do that.

Regarding data types, as long as the user travels a valid type, that is, a message that the internal protocol of the proxy understands, it can use the real type it wants stored under our standard message. In other words, our protocol is a wrapper for all types of data that the user wants, as long as all participants in the Spawn mesh are able to pack and unpack the information. The proxy has little to do with the actual data type all it expects is an Any and even expecting that it does nothing with that information other than passing it along, but it has to be under a google.protobuf.Any.

In theory any types can handle anything as long as it is provided as a byte array and has a type identifier. In practice it is actually a little less flexible as many languages ​​are unable to pack a type into Any other than through a Protobuf type of the native language. It is worth remembering that the data arrives as raw bytes on the other side of a request, so without an adequate mechanism to know the type of data of the type of request, very little the user can do.

I think Cloudstate solved this by adding a type convention for casting. But today we don't have that specified and porting the support languages ​​just use the native mechanisms in their languages ​​to pack and unpack the message.

[marcellanz]

I agree, well said. The proxy does not care for the InvocationRequest, it is orthogonal to the protocol and the duties of the proxy. I liberates from what we care about, to no care at all. Let's see how far we can get with this if we really think we don't have to care.

[ralphlaude]

You put it well @sleipnir so let go with the option with less effort

[sleipnir]

I agree, well said. The proxy does not care for the InvocationRequest, it is orthogonal to the protocol and the duties of the proxy. I liberates from what we care about, to no care at all. Let's see how far we can get with this if we really think we don't have to care.

For now the proxy doesn't need to care, but maybe specifying the expected behaviors in the supporting languages is a good thing to document. Or perhaps specify less common use cases, such as those that traverse non-protobuf data inside Any. Or even more creating CODECS to support different types of serialization. I haven't thought much about these things yet but it sure is possible to do a lot.

[sleipnir]

You put it well @sleipnir so let go with the option with less effort

It is planned for our second release as existing support languages will have to adapt to the protocol changes.