Availability concern with Postgres persistence

[atararaksin]

Hi team, I just completed reading your Medium post about Conductor scalability.

It briefly mentioned that Conductor servers can be scaled out horizontally and the production setup should include at least 2-3 instances in different availability zones, which makes sense to me. The question is, since this setup is using Postgres for persistence layer, and Postgres is a non-distributed database, I understand there's no way we can provide similar redundancy at persistence level. Unless we setup some cluster with master-slave replication, I guess.

What's your recommendation for enhancing Postgres server availability? Altogether, do you recommend using Postgres at all for production?

[boney9]

In this version of Conductor we are using two data stores postgres and redis. Both of these needs to be setup with replication across zones and required backups. The postgres is only vertically scalable in this edition, but if you need larger volumes we have an enterprise edition that you can look into. But the vertical scaling will get you really far in terms of scale.

The same goes for Redis too, replication + backups is what is powering this edition - and it can be vertically scaled to a certain limit easily. Beyond that we need some level of sharding support to scale. But let us know what are your requirements and we can meet to discuss what would be a good production configuraiton.