pause-resume

Pause/Resume

In this recipe we'll learn how to pause and resume consumption from a data stream.

Pinot Version	1.0.0
Schema	config/schema.json
Real-Time Table Config	config/table.json

---

git clone git@github.com:startreedata/pinot-recipes.git
cd pinot-recipes/recipes/pause-resume

Spin up a Pinot cluster using Docker Compose:

docker-compose up

Add tables and schema:

docker run \
   --network pauseresume \
   -v $PWD/config:/config \
   apachepinot/pinot:1.0.0 AddTable \
     -schemaFile /config/schema.json \
     -tableConfigFile /config/table.json \
     -controllerHost "pinot-controller-pauseresume" \
    -exec

Import messages into Kafka:

python datagen.py --sleep 0.0001 2>/dev/null |
jq -cr --arg sep ø '[.uuid, tostring] | join($sep)' |
kcat -P -b localhost:9092 -t events -Kø

Query Pinot:

select count(* FROM events

Pause consumption:

curl -X POST \
  "http://localhost:9000/tables/events/pauseConsumption" \
  -H "accept: application/json"

Update table config:

docker run \
   --network pauseresume \
   -v $PWD/config:/config \
   apachepinot/pinot:1.0.0 AddTable \
     -schemaFile /config/schema.json \
     -tableConfigFile /config/table-fixed.json \
     -controllerHost "pinot-controller-pauseresume" \
    -exec -update

Resume consumption

curl -X POST \
  "http://localhost:9000/tables/events/resumeConsumption?consumeFrom=smallest" \
  -H "accept: application/json"