Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move ATM fraud to examples repo #425

Merged
merged 16 commits into from
Jan 18, 2024
Merged
Show file tree
Hide file tree
Changes from 8 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .gitmodules
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
[submodule "examples"]
path = examples
url = https://github.com/bakdata/kpops-examples
27 changes: 12 additions & 15 deletions docs/docs/user/examples/atm-fraud-pipeline.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ postgresql bitnami/postgresql
??? example "PostgreSQL Example Helm chart values (`postgresql.yaml`)"
```yaml
auth:
database: app_db
database: app_db
enablePostgresUser: true
password: AppPassword
postgresPassword: StrongPassword
Expand Down Expand Up @@ -74,20 +74,21 @@ kubectl port-forward --namespace kpops service/k8kafka-cp-kafka-connect 8083:808

<!-- dprint-ignore-start -->

1. Export environment variables in your terminal:
1. Clone the [kpops-examples repository](https://github.com/bakdata/kpops-examples){target=_blank} and `cd` into the directory.

2. Install KPOps `pip install -r requirements.txt`.

3. Export environment variables in your terminal:

```shell
export DOCKER_REGISTRY=bakdata && \
export NAMESPACE=kpops
```

2. Deploy the pipeline
4. Deploy the pipeline

```shell
poetry run kpops deploy ./examples/bakdata/atm-fraud-detection/pipeline.yaml \
--pipeline-base-dir ./examples \
--config ./examples/bakdata/atm-fraud-detection/config.yaml \
--execute
kpops deploy atm-fraud/pipeline.yaml --execute
```

!!! Note
Expand Down Expand Up @@ -116,7 +117,7 @@ You should be able to see pipeline shown in the image below:
<!-- dprint-ignore-start -->

!!! Attention
Kafka Connect needs some time to set up the connector.
Kafka Connect needs some time to set up the connector.
Moreover, Streams Explorer needs a while to scrape the information from Kafka connect.
Therefore, it might take a bit until you see the whole graph.

Expand Down Expand Up @@ -146,11 +147,7 @@ helm --namespace kpops uninstall postgresql
2. Remove the pipeline

```shell
poetry run kpops clean ./examples/bakdata/atm-fraud-detection/pipeline.yaml \
--pipeline-base-dir ./examples \
--config ./examples/bakdata/atm-fraud-detection/config.yaml \
--verbose \
--execute
kpops clean atm-fraud/pipeline.yaml --verbose --execute
```

!!! Note
Expand All @@ -166,12 +163,12 @@ helm --namespace kpops uninstall postgresql

- `deploy` fails:
1. Read the error message.
2. Try to correct the mistakes if there were any. Likely the configuration is not correct or the port-forwarding is not working as intended.
2. Try to correct the mistakes if there were any. Likely the configuration is incorrect, or the port-forwarding is not working as intended.
3. Run `clean`.
4. Run `deploy --dry-run` to avoid havig to `clean` again. If an error is dropped, start over from step 1.
5. If the dry-run is succesful, run `deploy`.
- `clean` fails:
1. Read the error message.
2. Try to correct the indicated mistakes if there were any. Likely the configuration is not correct or the port-forwarding is not working as intended.
2. Try to correct the indicated mistakes if there were any. Likely the configuration is incorrect, or the port-forwarding is not working as intended.
3. Run `clean`.
4. If `clean` fails, follow the steps in [teardown](../getting-started/teardown.md).
9 changes: 2 additions & 7 deletions docs/docs/user/getting-started/quick-start.md
Original file line number Diff line number Diff line change
Expand Up @@ -83,9 +83,7 @@ kubectl port-forward --namespace kpops service/k8kafka-cp-kafka-connect 8083:808
4. Deploy the pipeline

```shell
kpops deploy word-count/pipeline.yaml \
--defaults word-count/defaults.yaml \
--execute
kpops deploy word-count/pipeline.yaml --execute
```

!!! Note
Expand Down Expand Up @@ -145,10 +143,7 @@ helm --namespace kpops uninstall redis
2. Remove the pipeline

```shell
kpops clean word-count/pipeline.yaml \
--defaults word-count/defaults.yaml \
--verbose \
--execute
kpops clean word-count/pipeline.yaml --verbose --execute
```

!!! Note
Expand Down
1 change: 1 addition & 0 deletions examples
disrupted marked this conversation as resolved.
Show resolved Hide resolved
Submodule examples added at 62fc37
19 changes: 0 additions & 19 deletions examples/bakdata/atm-fraud-detection/config.yaml

This file was deleted.

32 changes: 0 additions & 32 deletions examples/bakdata/atm-fraud-detection/defaults.yaml

This file was deleted.

104 changes: 0 additions & 104 deletions examples/bakdata/atm-fraud-detection/pipeline.yaml

This file was deleted.

Loading
Loading