Push is a Real Time Reactive Push Messaging API highly inspired by Netflix 's Zuul Push, but with some differences. It was written in Kotlin with the Spring framework and it uses Redis as a message broker. Push provides WebSocket and Server-Sent Events endpoints that can be used to subscribe to real-time messages in a web environment. It also provides an HTTP endpoint to publish messages.
Push was designed with the goal of being scalable and possibly used in a distributed environment, it uses Redis as message broker.
The following diagram demonstrates the architecture behind Push in a Kubernetes environment, where we can create a cluster for Redis with master and slave nodes in order to have horizontal scalability and resiliency and also multiple instances of the application itself for the same purpose.
This section describes the architecture of Push through a component diagram.
Push takes advantage of the "Deployment" workload API from Kubernetes. This section describes how it works under the hood. A Deployment provides declarative updates for Pods and ReplicaSets and is used to tell Kubernetes how to create or modify instances of the pods that hold a containerized application. Deployments can scale the number of replica pods, enable rollout of updated code in a controlled manner, or roll back to an earlier deployment version if necessary.
Push makes use of the "StatefulSet" workload API to manage the Redis cluster. This section describes how a kubernetes StatefulSet works under the hood. StatefulSet is the workload API object used to manage stateful applications. It manages the deployment and scaling of a set of Pods, and provides guarantees about the ordering and uniqueness of these Pods. Similar to a Deployment, a StatefulSet manages Pods that are based on an identical container spec.
Required software:
If you also want to compile the code directly on your machine you'll also need:
To compile the code simply run
$ gradle clean build -x test -x compileAotMainJava$ gradle clean build -x compileAotMainJava$ docker-compose -f docker-compose.test.yml build unit-tests && docker-compose -f docker-compose.test.yml run unit-tests- Starting the needed dependencies:
$ docker-compose -f docker-compose.test.yml up -d redis$ gradle clean build integrationTest -x compileAotMainJava$ docker-compose -f docker-compose.test.yml build integration-tests && docker-compose -f docker-compose.test.yml run integration-tests- Starting the needed dependencies:
$ docker-compose up -d redis- Booting the application
$ gradle clean build bootRun -x compileAotMainJava- Booting
$ docker-compose up --build -d push- Terminating
$ docker-compose down- Booting
$ ./k8s/init-cluster.sh && kubectl port-forward svc/push 8080:8080 8000:8000- Terminating
$ kubectl delete -f ./k8s/cluster.yml- Publishing messages to channel
qwerty
$ while true; do curl -H 'Content-Type: application/json' --request POST --data '{"channel":"qwerty","message":"a message to channel 'qwerty'"}' http://localhost:8080/messages ; done
# {"subscribers":0}- Subscribing to channel
qwertyand channelxyzby SSE (Server-Sent Events)
$ curl 'http://localhost:8080/sse/messages?channels=xyz,qwerty'
# event:heartbeat
# data:{}
# event:message
# data:{"channel":"qwerty","message":"a message to channel qwerty"}- Subscribing to channel
qwertyand channelxyzby WS (WebSockets)
$ websocat -v 'ws://localhost:8080/ws/messages?channels=qwerty,xyz'
# [INFO websocat::lints] Auto-inserting the line mode
# [INFO websocat::stdio_threaded_peer] get_stdio_peer (threaded)
# [INFO websocat::ws_client_peer] get_ws_client_peer
# [INFO websocat::ws_client_peer] Connected to ws
# {"event":"ping","data":{}}
# {"event":"message","data":{"channel":"qwerty","message":"a message to channel 'qwerty'"}}| Name | Description | Default value |
|---|---|---|
push.reconnect.dither.min.duration |
Minimum value for a randomization window for each client's max connection lifetime. Helps in spreading subsequent client reconnects across time. | 120s (120 seconds / 2 minutes) |
push.reconnect.dither.max.duration |
Maximum value for a randomization window for each client's max connection lifetime. Helps in spreading subsequent client reconnects across time. | 180s (180 seconds / 3 minutes) |
push.client.close.grace.period.duration |
Time the server will wait for the client to close the connection or respond to a "ping" before it closes it forcefully from its side | 4s (4 seconds) |
push.heartbeat.interval.duration |
Interval for when the server will emit heartbeats or pings to the client | 30s (Every 30 seconds) |
Both Server-Sent Events and WebSockets require persistent connections, so there are additional challenges that need to be addressed in order to provide scalability to the application.
// TODO complete
- Add unit and integration tests
- Write documentation
- Dockerize application
- Add observability and monitoring
- Add heartbeats, timeouts and reconnection events to the SSE endpoint
- Add ping-pongs, timeouts and reconnection events to the WS endpoint
- Create Kubernetes cluster with redundancy
- Automatically configure cluster for Redis
- Configure horizontal pod autoscaler for Redis
- Configure horizontal pod autoscaler for Push based on custom metrics (e.g. throughput, number of concurrent connections, etc)
- Add Sharded Pub/Sub capacity
- Use Terraform to deploy to Kubernetes
- Add "acks" to websockets messages
- Configure native compilation
- https://github.com/Netflix/zuul/wiki/Push-Messaging
- https://www.youtube.com/watch?v=IdR6N9B-S1E
- https://github.com/rustudorcalin/deploying-redis-cluster
- https://www.vmware.com/topics/glossary/content/kubernetes-deployment.html
- https://kubernetes.io/docs/concepts/workloads/controllers/deployment/
- https://congdonglinux.com/rolling-updates-and-rollbacks-in-kubernetes/
- https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/
- https://jayendrapatil.com/kubernetes-components/