Skip to content

Simple Camel Quarkus app that scrapes a website for a particular piece of information, checks if it has changed and in that case sends a notification to a Telegram channel. It also demonstrates how to deploy the app to Openshift and use secrets and configmaps without further configuration

License

Notifications You must be signed in to change notification settings

kdubois/datascraper

Repository files navigation

Bruvax datascraper project

This Camel Quarkus project scrapes a website (Bruvax) to see if a value in the content has changed, and in that case it sends a Telegram notification.

Watch a recorded walkthrough of this demo here: https://youtu.be/t9Z75ccqpws

There are 2 components:

- A DataScraper bean and service (DataScraper.java / ScraperService.java)
- A Camel Route (BruvaxBot.java) that consumes the datascraper website, compares with an existing value stored in a config map, and if the value has changed, it will send a Telegram notification 

Create a Telegram bot

Go to https://core.telegram.org/bots#6-botfather and follow the instructions to create a Telegram Bot. Add it to a group chat (or create one).
You will need to retrieve the bot's token, and the chat id of the chat where you want the message to appear.

Store Telegram token and chat id

Either store them directly in the application.properties file, or (preferably) store them in a Kubernetes secret. To do this, add the values to a kubefiles/secrets.yml (there's a secrets-example.yaml you can use as a template)

kubectl apply -f kubefiles/configmap.yaml -f kubefiles/secrets.yaml -f kubefiles/rbac.yaml

Running the application in dev mode

You can run your application in dev mode that enables live coding using:

./mvnw compile quarkus:dev

NOTE: Quarkus now ships with a Dev UI, which is available in dev mode only at http://localhost:8080/q/dev/.

Build & deploy the application

If you're running Openshift, you can build the app as a native binary trigger a knative serverless deployment easy-peasy with the following maven command:

./mvnw clean package -Pnative -Dquarkus.kubernetes.deploy=true

(make sure you have installed the Openshift Serverless operator!)

The camel route will run once and then scale down to 0, so if you want to trigger it on a regular interval, you could add a Pingsource to the app:

kubectl apply -f kubefiles/pingsource.yaml

About

Simple Camel Quarkus app that scrapes a website for a particular piece of information, checks if it has changed and in that case sends a notification to a Telegram channel. It also demonstrates how to deploy the app to Openshift and use secrets and configmaps without further configuration

Topics

Resources

License

Stars

Watchers

Forks

Packages

No packages published