insolvency-delta-consumer

insolvency-delta-consumer is responsible for transforming insolvency data from the insolvency-delta kafka as part of chips and chs data sync

Development

Common commands used for development and running locally can be found in the Makefile, each make target has a description which can be listed by running make help

Target               Description
------               -----------
all                  Calls methods required to build a locally runnable version, typically the build target
build                Pull down any dependencies and compile code into an executable if required
clean                Reset repo to pre-build state (i.e. a clean checkout state)
deps                 Install dependencies
docker/kafka         Run kafka and create topics within docker
docker/kafka-create-topics Create kafka topics within docker
docker/kafka-start   Run kafka within docker
docker/kafka-stop    Stop kafka within docker
package              Create a single versioned deployable package (i.e. jar, zip, tar, etc.). May be dependent on the build target being run before package
sonar                Run sonar scan
test                 Run all test-* targets (convenience method for developers)
test-integration     Run integration tests
test-unit            Run unit tests

Running kafka locally

From root folder of this project run docker-compose up -d

Once containers up, run docker-compose exec kafka bash to enter kafka bash to create topics

Create kafka topics locally

kafka-topics.sh --create --zookeeper zookeeper:2181 --replication-factor 1 --partitions 1 --topic insolvency-delta

kafka-topics --create --topic insolvency-delta --bootstrap-server localhost:9092

kafka-topics.sh --create --zookeeper zookeeper:2181 --replication-factor 1 --partitions 1 --topic retry-topic

kafka-topics.sh --create --zookeeper zookeeper:2181 --replication-factor 1 --partitions 1 --topic error-topic

Create kafka topics locally

kafka-topics.sh --list --zookeeper zookeeper:2181

Produce kafka test messages locally

kafka-console-producer.sh --topic delta-topic --broker-list localhost:9092

Terraform ECS

What does this code do?

The code present in this repository is used to define and deploy a dockerised container in AWS ECS. This is done by calling a module from terraform-modules. Application specific attributes are injected and the service is then deployed using Terraform via the CICD platform 'Concourse'.

Application specific attributes	Value	Description
ECS Cluster	data-sync	ECS cluster (stack) the service belongs to
Load balancer	N/A consumer service	The load balancer that sits in front of the service
Concourse pipeline	Pipeline link Pipeline code	Concourse pipeline link in shared services

Contributing

• Please refer to the ECS Development and Infrastructure Documentation for detailed information on the infrastructure being deployed.

Testing

• Ensure the terraform runner local plan executes without issues. For information on terraform runners please see the Terraform Runner Quickstart guide.
• If you encounter any issues or have questions, reach out to the team on the #platform slack channel.

Vault Configuration Updates

• Any secrets required for this service will be stored in Vault. For any updates to the Vault configuration, please consult with the #platform team and submit a workflow request.

Useful Links

• ECS service config dev repository
• ECS service config production repository