Guidance for OPAL deployment in kubernetes - 1 server + 1 client in same pod, separate server deployment, etc?

[phil-lee-kb]

Hi there,

I figured I'd start a separate thread as this one is a topic in its own right....

We're looking at deploying OPAL into k8s, and I'm interested in recommendations for how to configure it in terms of server vs client instances. Essentially, I'm thinking of having a standalone OPAL deployment (where deployment means a group of OPAL pods running the same configuration, in their own namespace), where each pod has an OPAL server container + the OPAL client container. This is slightly different from the helm example provided, which has the OPAL server deployed separately to the client, and which in theory would mean one server could have multiple clients, and potentially have clients in different namespaces. Hence my question as to whether this is a standard deployment model for OPAL? It would result in more chat between the server instances via the backbone as the number of pods increases, but that doesn't seem like a major issue to me.

We may also have a use case where we want to deploy OPAL sidecars as part of application pods. I.e. 1 pod = application container + OPAL server container + OPAL client container.

I guess I'm interested in whether having the OPAL server sit alongside the OPAL client in the same pod is common practice? This also has implications for releases such as rolling deployments, so I'm keen for any guidance or examples from the wild.

Thanks,
Phil

[orweis]

Hi @phil-lee-kb

Yes running multiple servers for multiple clients is fine - and not much different from running one server with multiple workers (they too need to connect via the backbone pubsub)

would mean one server could have multiple clients, and potentially have clients in different namespaces
That's very common - defiently a good way to use OPAL.

You can serve different clients with different needs from one Server (or cluster of servers) using different topics for policy and/or data ; or using different OPAL scopes.

We may also have a use case where we want to deploy OPAL sidecars as part of application pods. I.e. 1 pod = application container + OPAL server container + OPAL client container.

That would work. Though you don't need to scale the servers linearly with the clients.

I guess I'm interested in whether having the OPAL server sit alongside the OPAL client in the same pod is common practice? This also has implications for releases such as rolling deployments, so I'm keen for any guidance or examples from the wild.

It's not common practice, but it's okay.

[phil-lee-kb]

Excellent, thank you! For the case where the clients and servers are deployed independently, it's likely that the server instances would live in a cluster/deployment behind a k8s service/ingress and hence be accessible via either a load-balanced address, or cluster-local service address. Is that ok for when the clients register themselves with the server? I.e. instead of registering with an individual server, they'll just use the load-balanced url, and resolve to one of the server instances which will then establish the socket connection.

I'm also thinking of situations where the server instance is recycled and the client needs to re-register. Presumably this is handled automatically within the OPAL client code?

[orweis]

it's likely that the server instances would live in a cluster/deployment behind a k8s service/ingress and hence be accessible via either a load-balanced address, or cluster-local service address. Is that ok for when the clients register themselves with the server?

Yes, very much okay - just make sure the load-balancer is websocket (long lived connection) friendly.

I'm also thinking of situations where the server instance is recycled and the client needs to re-register. Presumably this is handled automatically within the OPAL client code?

Yes. every time a client loses connection and reconnects - it assumes it needs to fetch data from scratch. and does so according to to the servers OPAL_DATA_SOURCES_CONFIG