Horizontally scalable Dragonfly Cluster

[fdsmax]

Does Dragonfly support configuring multi-node horizontally scalable cluster (similar to Redis Cluster) for data intensive in-memory cache? Any references on how to configure it with any external tools?

[chakaz]

@thulyatech I'm sorry to say that, at the moment, we do not directly support multi-node clusters. If you'd like to set one up, you'll have to program your clients to know which node to connect to for which keys.
That is of course not to say that we do not plan to support clusters in the future, but we're just not there yet.

[tittuvarghese]

How Dragonfly is taking care of availability ?

[chakaz]

@tittuvarghese Dragonfly supports replication, that is having a master node that streams all of its data (continuously) to another replica node. Upon failures, the replica can be promoted to a master (which means it will stop receiving data from the previous-master node, and will allow writes).
This is the same as Redis, and in fact we also support Dragonfly as a replica of a Redis master.
Does this answer your question?

[alexted]

@chakaz Do you plan to expand fault tolerance and horizontal scaling capabilities in the near future?

[ashotland]

Hi, so a couple of notes regarding high availability:

• Dragonfly supports Redis Sentinel - so you can get HA with a master + replica + sentinel deployment
• We also released a k8s operator https://github.com/dragonflydb/dragonfly-operator so you can get HA on k8s
• I'll let @chakaz chime in on the timeline for cluster, however is important to note that it won't provide HA out of the box. We hope the community will contribute a control plane component that manages a Dragonfly cluster.

Hope that helps :)

[alexted]

Sorry, I don't get it. What's the point of this cluster if it doesn't support HA?

[brandondrew]

@alexted I think the key point there was that HA was not provided out of the box, and not that it wouldn't support high availability at all.

[githubzhangfei]

Can Redis Sentinel be used to manage dragonflydb? That is to say, he supports master-slave replication, but does not support fail over and requires the use of proxies to ensure high availability. Is my understanding correct?

[ashotland]

#1498 (comment)

[cyppe]

On this topic - is it supported and/or recommended when running dragonfly operator with for example "replicas: 3", to use the passive "slaves" as read only replicas?

I am not sure if it makes sense, maybe the master will never become a bottleneck in read performance. But just curious if it's recommended or not and if so if there is any service/hostname exposed by the operator that prioritizes connection to slaves in similar fashion as it created a svc for the current master.

Just feels like a little bit of potentially wasted resources if they just sit and wait for master to go down if we can use them for reading.

[chakaz]

My recommendation would be to use the master for both reads and writes as much as possible. Dragonfly can do millions of QPS. Consider using the replica for reads only if the master truly and verifiably is overloaded.