0.4.0

v0.4.0 (2024-09-16)

Feat

• wait for schema registry to become available
• add Kafka Avro reader
• add Kafka Avro reader
• add avro parser
• add schema registry class

Fix

• start from earliest checkpoint
• use colima on mac
• wait for schema registry to be ready before running tests

Refactor

• schema type to config
• Switch to Poetry from PDM

[main 2cb5e1c] bump: version 0.3.1 → 0.4.0
2 files changed, 22 insertions(+), 1 deletion(-)

0.3.1

v0.3.1 (2024-09-11)

Fix

• spark: lock package version, to ensure they match plugins.
• typing: error "AnalysisException" has no attribute "desc"

Refactor

• test: run development services using devenv.
• switch to poetry.

[main 48627c8] bump: version 0.3.0 → 0.3.1
2 files changed, 13 insertions(+), 1 deletion(-)

0.3.0

v0.3.0 (2024-09-10)

Feat

• Add Kafka stream publisher
• add dataframe to kafka compatible dataframe transforner

[main 5ec6089] bump: version 0.2.0 → 0.3.0
2 files changed, 8 insertions(+), 1 deletion(-)

0.2.0

v0.2.0 (2024-09-06)

Feat

• add table reader

Refactor

• iceberg config name

[main 21ff032] bump: version 0.1.2 → 0.2.0
2 files changed, 11 insertions(+), 1 deletion(-)

0.1.2

v0.1.2 (2024-09-06)

Fix

• set minium pyspark version instead of locking to specific version
• set minimum python version instead of locking to exact version

[main 3e01e8f] bump: version 0.1.1 → 0.1.2
2 files changed, 8 insertions(+), 1 deletion(-)

0.1.1

v0.1.1 (2024-09-05)

Refactor

• devenv: downgrade Java to version 8.
• devenv: switch to nixpkgs-unstable.

[main 801fbad] bump: version 0.1.0 → 0.1.1
2 files changed, 8 insertions(+), 1 deletion(-)

0.1.0

v0.1.0 (2024-09-05)

Feat

• add Iceberg writer with tests
• add Sparkle class
• add pyspark to dependencies.
• sources: initialize the reader design.
• config: initialize the application structure.

Fix

• do not fail fast CI matrix
• python version
• devenv & python version
• config: config implemnetation errors.

Refactor

• ruff hints
• Read source on reader initialization
• Sparkle: to generate input configuration.
• Soruces: to preserve typing, documentation and autocompletion.

0.0.0 (2024-07-26)

[main 3503ea8] bump: version 0.0.0 → 0.1.0
2 files changed, 26 insertions(+), 1 deletion(-)
create mode 100644 CHANGELOG.md

0.0.0

This is the Inception 🎉