kafka-to-es-demo.adoc

Pre-Flight Setup

Start the environment

cd docker-compose
./scripts/setup.sh

Get a prompt

docker run --rm -it --network docker-compose_default confluentinc/cp-kafkacat \
  kafkacat -b kafka:29092 -t jfokus19 -P

Launch Kibana dashboard

Demo

Show Kafka Connect config

curl -X "POST" "http://kafka-connect:18083/connectors/" \
     -H "Content-Type: application/json" \
     -d '{
    "name": "es_sink_jfokus19",
    "config": {
    "connector.class": "io.confluent.connect.elasticsearch.ElasticsearchSinkConnector",
    "key.converter": "org.apache.kafka.connect.storage.StringConverter",
    "value.converter": "org.apache.kafka.connect.json.JsonConverter",
    "value.converter.schemas.enable": false,
    "topics": "jfokus19",
    "key.ignore": "true",
    "schema.ignore": "true",
    "type.name": "type.name=kafkaconnect",
    "connection.url": "http://elasticsearch:9200"
  }
}'

From kafkacat prompt

docker run --rm -it --network docker-compose_default confluentinc/cp-kafkacat \
  kafkacat -b kafka:29092 -t jfokus19 -P

send some data:

{"hello":"world"}
{"hello":"jfokus19"}
{"jfokus19":"is great!"}