This service is responsible to manage the Vaccine Order entity. It is done with Smallrye microprofile and reactive messaging with Kafka, hibernate ORM with panache for DB2 database, Quarkus stack and Tekton pipeline.
Visit detail implementation approach, design and different deployment model, read explanations of this service in the main solution documentation.
The goals of this project are:
- Quarkus app with Debezium outbox extension
- Reactive REST APP with Mutiny
- JPA with Hibernate and Panache
- Debezium Postgres connector to publish OrderEvents to Kafka topic
As the environment is using a DB2 database configured for change data capture, a specific Db2 image is necessary. We have integrated a dockerfile and asn configuration to build such image. The details for this configuration are in the Debezium Db2 connector documentation.
If you want to build this image you can do the following command.
# under environment/db2image
docker build -t ibmcase/db2orders .
# push the image to a registry
docker push ibmcase/db2ordersThen start db2 to prepare the database for the first time only.
docker-compose -f strimzi-docker-compose.yaml up db2 -dIn the logs you should see the execution of the cdcsetup.sh script which maps to the steps described in this Debezium note
To run locally we have defined a docker-compose file with one Kafka broker, one zookeeper, one DB2 container for the persistence, one Kafka Connect with the Debezium code with the DB2 Jdbc driver and one vaccine-order-service to support the demonstration.
- To validate DB2 settings you can do one of the following troubleshooting commands:
# connect to DB2 server
docker exec -ti db2 bash -c "su - db2inst1"
# Access the database
db2 connect to TESTDB user db2inst1
# list existing schemas
db2 "select * from syscat.schemata"
# list tables
db2 list tables
# this is the outcomes if the order services was started
Table/View Schema Type Creation time
------------------------------- --------------- ----- --------------------------
ORDERCREATEDEVENT DB2INST1 T 2020-11-12-01.50.10.400490
ORDEREVENTS DB2INST1 T 2020-11-12-01.50.10.650172
ORDERUPDATEDEVENT DB2INST1 T 2020-11-12-01.50.10.796566
VACCINEORDERENTITY DB2INST1 T 2020-11-12-01.50.10.874172
# Verify the content of the current orders
db2 "select * from vaccineorderentity"
# List the table for the change data capture schema
db2 list tables for schema asncdc- To deploy this DB2 image on OpenShift cluster.
oc login ...
oc new-project eda-db2
# Authorize default user using security constraint
oc adm policy add-scc-to-user anyuid -z default
# Deploy statefulset for the db2 image
oc apply -f environment/db2image/statefulset-db2orders.yaml
# be sure to have packaged the order app first with
mvn package -Dui.deps -Dui.dev -D -DskipTests
# build the image
docker build -f src/main/docker/Dockerfile.jvm -t ibmcase/vaccineorderms .
# can also build with docker compose
cd environment
# with the option to build the db2, and debezium cdc container images
docker-compose -f strimzi-docker-compose.yaml up -d --build
# or with pre-existing images coming from dockerhub or your local registry if images already built
docker-compose -f strimzi-docker-compose.yaml up -dIf you want to start everything in development mode, the vaccine order service is executed via a maven container which starts quarkus:dev. Therefore the command is using another compose file:
cd environment
docker-compose -f dev-docker-compose.yaml up -d [--build]When started for the first time, the DB2 container may take some time to complete the TESTDB creation with the change data capture tables. (To do need to find a health check way to assess db2 is running)
- Define Kafka topics
Some topics are created by the Kafka Connector.
# Under environment folder
./createTopic.sh
# validate topics created
./listTopics.sh
__consumer_offsets
db_history_vaccine_orders
src_connect_configs
src_connect_offsets
src_connect_statuses
vaccine_shipmentsThe db_history_vaccine_orders is the topic used to include database schema change on the vaccine orders table.
- Deploy and start the Debezium DB2 connector. The connector definition is register-db2.json
# under environment/cdc
curl -i -X POST -H "Accept:application/json" -H "Content-Type:application/json" http://localhost:8083/connectors/ -d @cdc/register-db2.json- Verify Topics created by the connector:
./listTopics.sh
vaccine_lot_db
vaccine_lot_db.DB2INST1.ORDEREVENTSThe newly created vaccine_lot_db topic includes definition of the database and the connector. It does not aim to be used by application. The one to be used to get business events is vaccine_lot_db.DB2INST1.ORDEREVENTS.
The connector is doing a snapshot of the DB2INST1.ORDEREVENTS table to send existing records to the topic.
- Start a consumer on the CDC topic for the order events
docker-compose exec kafka /opt/kafka/bin/kafka-console-consumer.sh --bootstrap-server kafka:9092 --from-beginning --property print.key=true --topic vaccine_lot_db.DB2INST1.ORDEREVENTS-
Add new order from the user interface: http://localhost:8080/#/Orders, or...
-
Post an order using the API: http://localhost:8080/swagger-ui/#/default/post_orders. Use the following JSON
{
"deliveryDate": "2021-07-25",
"deliveryLocation": "Milano",
"askingOrganization": "Italy gov",
"priority": 1,
"quantity": 100,
"type": "COVID-19"
}- The expected result should have the following records in the Kafka topic:
{"ID":"lvz4gYs/Q+aSqKmWjVGMXg=="}
{"before":null,"after":{"ID":"lvz4gYs/Q+aSqKmWjVGMXg==","AGGREGATETYPE":"VaccineOrderEntity","AGGREGATEID":"21","TYPE":"OrderCreated","TIMESTAMP":1605304440331350,"PAYLOAD":"{\"orderID\":21,\"deliveryLocation\":\"London\",\"quantity\":150,\"priority\":2,\"deliveryDate\":\"2020-12-25\",\"askingOrganization\":\"UK Governement\",\"vaccineType\":\"COVID-19\",\"status\":\"OPEN\",\"creationDate\":\"13-Nov-2020 21:54:00\"}"},"source":{"version":"1.3.0.Final","connector":"db2","name":"vaccine_lot_db","ts_ms":1605304806596,"snapshot":"last","db":"TESTDB","schema":"DB2INST1","table":"ORDEREVENTS","change_lsn":null,"commit_lsn":"00000000:0000150f:0000000000048fca"},"op":"r","ts_ms":1605304806600,"transaction":null}If you want to build each images manually:
- Db2 with ASN table and scripts for CDC
cd environment/db2images
docker build -t ibmcase/db2orders .- Build the Debezium Kafka Connector
cd environment/db2images
docker build -t ibmcase/cdc-connector .- Build the vaccine order service
# from project folder
mvn package -Dui.deps -Dui.dev -DskipTests
docker build -f src/main/docker/Dockerfile.jvm -t ibmcase/vaccineorderms .Unit and integration tests are done with Junit 5 and Test Container when needed or mockito to avoid backend access for CI/CD.
For end to end testing the e2e folder includes some python scripts to test new order creation.
cd e2e
./post-order.shFor UI development start the components with docker-compose -f dev-docker-compose.yaml up -d, then under the ui folder, do the following:
yarn install
yarn serve
Use the web browser and developer console to the address http://localhost:4545. The Vue app is configured to proxy to localhost:8080.
To deploy on OpenShift cluster see instructions in the main documentation.
# microservice logs:
To delete the CDC connector:
curl -i -X DELETE http://localhost:8083/connectors/orderdb-connector
DB2 may not have finished its setup, specially using docker compose where the image may be built with Debezium and change data capture settings. Restarting the order service later should help.
This error may happen on OpenShift deployment and it is linked to a connection to the DB. Verify JDBC URL settings, user, password or network connection.
This repository includes a Github workflow to build the app and push a new docker image to public registry. To do that we need to define 4 secrets in the github repository:
- DOCKER_IMAGE_NAME the image name to build. Here it is
vaccineorderms - DOCKER_USERNANE: user to access docker hub
- DOCKER_PASSWORD: and its password.
- DOCKER_REPOSITORY for example the organization we use is
ibmcase