Skip to content

Releases: exasol/spark-connector

2.2.0: Fix Exasol-JDBC compatibility, CVE CVE-2024-25638

28 Aug 13:39
@Shmuma Shmuma
2.2.0
341fde9
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Learn about vigilant mode.
Compare
Choose a tag to compare
2.2.0: Fix Exasol-JDBC compatibility, CVE CVE-2024-25638 Latest
Latest

Summary

Fixed issue with Exasol-JDBC parameter change.
Fixed CVE-2024-25638 in dnsjava:dnsjava:jar:2.1.7:provided (hadoop-client dependency).

Security

  • #228: Fix vulnerability in dnsjava:dnsjava:jar:2.1.7:provided

Bugfixes

  • #227: Spark Connector is not compatible with exasol-jdbc V24+

Dependency Updates

Spark Exasol Connector With JDBC

Compile Dependency Updates

  • Updated com.exasol:exasol-jdbc:7.1.20 to 24.1.2
  • Updated com.exasol:spark-connector-common-java:2.0.1 to 2.0.5

Spark Exasol Connector With S3

Compile Dependency Updates

  • Updated com.exasol:spark-connector-common-java:2.0.1 to 2.0.5
Assets 67

2.1.6: Spark 3.5 support

11 Mar 14:06
@Shmuma Shmuma
2.1.6
4d09214
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Learn about vigilant mode.
Compare
Choose a tag to compare
2.1.6: Spark 3.5 support

Summary

Added support of spark 3.5.

Features

Assets 67
Loading

2.1.5: Fix CVE-2024-21634 and CVE-2023-33546

19 Jan 07:15
@kaklakariada kaklakariada
2.1.5
f922c6c
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Learn about vigilant mode.
Compare
Choose a tag to compare
2.1.5: Fix CVE-2024-21634 and CVE-2023-33546

Summary

This release fixes CVE-2024-21634 in transitive test dependency software.amazon.ion:ion-java.

Security

Dependency Updates

Spark Exasol Connector Parent POM

Plugin Dependency Updates

  • Updated org.apache.maven.plugins:maven-surefire-plugin:3.1.2 to 3.2.3
  • Added org.apache.maven.plugins:maven-toolchains-plugin:3.1.0
  • Updated org.codehaus.mojo:versions-maven-plugin:2.16.1 to 2.16.2

Spark Exasol Connector With JDBC

Plugin Dependency Updates

  • Added org.apache.maven.plugins:maven-toolchains-plugin:3.1.0

Spark Exasol Connector With S3

Test Dependency Updates

  • Updated com.amazonaws:aws-java-sdk-s3:1.12.518 to 1.12.639

Plugin Dependency Updates

  • Added org.apache.maven.plugins:maven-toolchains-plugin:3.1.0
Assets 51
Loading

2.1.4: PK back in config, CVE fixed in transitive deps.

21 Nov 13:08
@Shmuma Shmuma
2.1.4
045f133
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
2.1.4: PK back in config, CVE fixed in transitive deps.

Summary

Project keeper added to Maven configuration.
Vulnerability fixed in transitive dependency.

Features

  • #211: Brought PK plugin back to the Maven config
  • #212: Fix vulnerability in org.eclipse.parsson:parsson

Dependency Updates

Spark Exasol Connector Parent POM

Plugin Dependency Updates

  • Updated com.exasol:error-code-crawler-maven-plugin:1.3.0 to 1.3.1
  • Updated org.apache.maven.plugins:maven-enforcer-plugin:3.4.0 to 3.4.1
  • Updated org.codehaus.mojo:versions-maven-plugin:2.16.0 to 2.16.1
  • Updated org.jacoco:jacoco-maven-plugin:0.8.10 to 0.8.11
  • Updated org.sonarsource.scanner.maven:sonar-maven-plugin:3.9.1.2184 to 3.10.0.2594

Spark Exasol Connector With JDBC

Test Dependency Updates

  • Updated com.exasol:exasol-testcontainers:6.6.2 to 6.6.3

Spark Exasol Connector With S3

Test Dependency Updates

  • Updated com.exasol:exasol-testcontainers:6.6.2 to 6.6.3
Assets 51
Loading

2.1.3: More flexibility for AWS Credentials specification in spark-connector-s3

20 Oct 13:10
@Shmuma Shmuma
2.1.3
07bb831
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
2.1.3: More flexibility for AWS Credentials specification in spark-connector-s3

Summary

In addition to explicit AWS Credentials specification we now support environment variables and EC2 instance profiles.
Fixes CVE-2023-39410 in apache avro (transitive dependency).

Features

  • 192: Add support for AWS IAM Profile Credentials for s3 connector.

Dependency Updates

Spark Exasol Connector With JDBC

Compile Dependency Updates

  • Added org.apache.avro:avro:1.11.3

Spark Exasol Connector With S3

Compile Dependency Updates

  • Added org.apache.avro:avro:1.11.3

Test Dependency Updates

  • Added org.junit-pioneer:junit-pioneer:2.1.0
Assets 51
Loading

2.1.2: Transaction commit fixes, security fixes in dependencies

27 Sep 16:20
@Shmuma Shmuma
2.1.2
e3f997e
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
2.1.2: Transaction commit fixes, security fixes in dependencies

Summary

Fixes CVE-2023-42503 in org.apache.commons:commons-compress.
Fixes logic in transaction commit during the DB import.

Features

  • 194: Wrong name of Exasol JDBC format in documentation
  • 197: Committing transaction in the finally handler
  • 198: Remove unused Statement variable
  • 206: Fix CVE-2023-42503 and update dependencies

Dependency Updates

Spark Exasol Connector Parent POM

Plugin Dependency Updates

  • Updated org.apache.maven.plugins:maven-enforcer-plugin:3.3.0 to 3.4.0

Spark Exasol Connector With JDBC

Compile Dependency Updates

  • Updated com.exasol:spark-connector-common-java:2.0.0 to 2.0.1

Test Dependency Updates

  • Updated com.exasol:exasol-testcontainers:6.6.1 to 6.6.2
  • Updated com.exasol:hamcrest-resultset-matcher:1.6.0 to 1.6.1
  • Updated com.exasol:test-db-builder-java:3.4.2 to 3.5.1
  • Updated org.mockito:mockito-core:5.4.0 to 5.5.0
  • Updated org.mockito:mockito-junit-jupiter:5.4.0 to 5.5.0
  • Updated org.scalatestplus:scalatestplus-mockito_2.13:1.0.0-M2 to 1.0.0-SNAP5
  • Updated org.scalatest:scalatest_2.13:3.2.9 to 3.3.0-SNAP4

Spark Exasol Connector With S3

Compile Dependency Updates

  • Updated com.exasol:spark-connector-common-java:2.0.0 to 2.0.1
  • Updated org.apache.hadoop:hadoop-aws:3.3.4 to 3.3.6

Test Dependency Updates

  • Updated com.exasol:exasol-testcontainers:6.6.1 to 6.6.2
  • Updated com.exasol:hamcrest-resultset-matcher:1.6.0 to 1.6.1
  • Updated com.exasol:test-db-builder-java:3.4.2 to 3.5.1
  • Updated org.mockito:mockito-junit-jupiter:5.4.0 to 5.5.0
  • Updated org.testcontainers:junit-jupiter:1.18.3 to 1.19.0
  • Updated org.testcontainers:localstack:1.18.3 to 1.19.0
Assets 51
Loading

2.1.1: JDBC connection cache drop, `magic` s3 hadoop committer

29 Aug 09:37
@Shmuma Shmuma
2.1.1
a0732bf
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
2.1.1: JDBC connection cache drop, `magic` s3 hadoop committer

Summary

S3 connector uses faster hadoop committer, fix possible race condition in RDD operations

Features

  • #163: Switched to magic hadoop committer for intermediate s3 files
  • #200: Dropped JDBC connection cache in ExasolConnectionManager
Assets 51
Loading

2.1.0: Added filter pushdown and column selection for `S3` variant

02 Aug 08:45
@morazow morazow
2.1.0
0a34451
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
2.1.0: Added filter pushdown and column selection for `S3` variant

Summary

This release adds predicate pushdown and column selection for the S3 variant. Similarly, uses the latest API changes from the spark-connector-common-java library.

Features

  • #190: Added predicate pushdown and column selection for S3 variant

Dependency Updates

Spark Exasol Connector With JDBC

Compile Dependency Updates

  • Updated com.exasol:spark-connector-common-java:1.1.1 to 2.0.0

Plugin Dependency Updates

  • Updated com.diffplug.spotless:spotless-maven-plugin:2.37.0 to 2.38.0

Spark Exasol Connector With S3

Compile Dependency Updates

  • Updated com.exasol:spark-connector-common-java:1.1.1 to 2.0.0
  • Updated software.amazon.awssdk:s3:2.20.103 to 2.20.115

Test Dependency Updates

  • Updated com.amazonaws:aws-java-sdk-s3:1.12.506 to 1.12.518
  • Added com.exasol:java-util-logging-testing:2.0.3
  • Updated org.junit.jupiter:junit-jupiter-api:5.9.3 to 5.10.0
  • Updated org.junit.jupiter:junit-jupiter:5.9.3 to 5.10.0
Assets 51
Loading

2.0.0: Support S3 intermediate storage

20 Jul 11:41
@morazow morazow
2.0.0
93df67d
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
2.0.0: Support S3 intermediate storage

Summary

In this release we added support to use AWS S3 bucket as an intermediate storage layer when accessing Exasol database from Spark cluster.

With this release, we separated the connector into two variants, S3 and JDBC. We recommend to use this new S3 variant instead of JDBC variant. It improves the stability of the connector.

Please refer to the user guide for updated API changes and usage instructions.

Features

  • #149: Added s3 import query generator and runner
  • #150: Added S3 intermediate storage layer
  • #159: Added cleanup process to remove intermediate data after job finish
  • #160: Add support for writing to Exasol database using S3 as intermediate storage
  • #168: Refactored to add module setup

Security

  • #151: Fixed vulnerability CVE-2023-26048 coming with jetty-util transitive dependency

Bugs

  • #176: Fixed artifact upload of sha256sum files
  • #188: Fixed Maven Central deployment bug

Refactoring

  • #155: Unified user options
  • #158: Refactored common options class
  • #164: Validated that write directory is empty
  • #171: Refactored artifact packaging and releasing for module setup
  • #174: Refactored Github .github/workflow/ action files
  • #183: Updated user guide and prepared for release

Dependency Updates

Spark Exasol Connector Parent POM

Plugin Dependency Updates

  • Added com.exasol:error-code-crawler-maven-plugin:1.3.0
  • Added io.github.zlika:reproducible-build-maven-plugin:0.16
  • Added org.apache.maven.plugins:maven-clean-plugin:2.5
  • Added org.apache.maven.plugins:maven-compiler-plugin:3.11.0
  • Added org.apache.maven.plugins:maven-deploy-plugin:3.1.1
  • Added org.apache.maven.plugins:maven-enforcer-plugin:3.3.0
  • Added org.apache.maven.plugins:maven-install-plugin:2.4
  • Added org.apache.maven.plugins:maven-site-plugin:3.3
  • Added org.apache.maven.plugins:maven-surefire-plugin:3.1.2
  • Added org.basepom.maven:duplicate-finder-maven-plugin:2.0.1
  • Added org.codehaus.mojo:flatten-maven-plugin:1.5.0
  • Added org.codehaus.mojo:versions-maven-plugin:2.16.0
  • Added org.itsallcode:openfasttrace-maven-plugin:1.6.2
  • Added org.jacoco:jacoco-maven-plugin:0.8.10
  • Added org.sonarsource.scanner.maven:sonar-maven-plugin:3.9.1.2184
  • Added org.sonatype.ossindex.maven:ossindex-maven-plugin:3.2.0

Spark Exasol Connector With JDBC

Compile Dependency Updates

  • Added com.exasol:error-reporting-java8:1.0.1
  • Added com.exasol:exasol-jdbc:7.1.20
  • Added com.exasol:spark-connector-common-java:1.1.1
  • Added com.exasol:sql-statement-builder-java8:4.5.4

Test Dependency Updates

  • Added com.exasol:exasol-testcontainers:6.6.1
  • Added com.exasol:hamcrest-resultset-matcher:1.6.0
  • Added com.exasol:test-db-builder-java:3.4.2
  • Added org.apache.logging.log4j:log4j-1.2-api:2.20.0
  • Added org.apache.logging.log4j:log4j-api:2.20.0
  • Added org.apache.logging.log4j:log4j-core:2.20.0
  • Added org.mockito:mockito-core:5.4.0
  • Added org.mockito:mockito-junit-jupiter:5.4.0
  • Added org.scalatestplus:scalatestplus-mockito_2.13:1.0.0-M2
  • Added org.scalatest:scalatest_2.13:3.2.9

Plugin Dependency Updates

  • Added com.diffplug.spotless:spotless-maven-plugin:2.37.0
  • Added com.exasol:error-code-crawler-maven-plugin:1.3.0
  • Added io.github.evis:scalafix-maven-plugin_2.13:0.1.4_0.9.31
  • Added io.github.zlika:reproducible-build-maven-plugin:0.16
  • Added net.alchim31.maven:scala-maven-plugin:4.8.1
  • Added org.apache.maven.plugins:maven-clean-plugin:2.5
  • Added org.apache.maven.plugins:maven-compiler-plugin:3.11.0
  • Added org.apache.maven.plugins:maven-deploy-plugin:3.1.1
  • Added org.apache.maven.plugins:maven-enforcer-plugin:3.3.0
  • Added org.apache.maven.plugins:maven-failsafe-plugin:3.1.2
  • Added org.apache.maven.plugins:maven-gpg-plugin:3.1.0
  • Added org.apache.maven.plugins:maven-install-plugin:2.4
  • Added org.apache.maven.plugins:maven-jar-plugin:3.3.0
  • Added org.apache.maven.plugins:maven-javadoc-plugin:3.5.0
  • Added org.apache.maven.plugins:maven-resources-plugin:2.6
  • Added org.apache.maven.plugins:maven-shade-plugin:3.4.1
  • Added org.apache.maven.plugins:maven-site-plugin:3.3
  • Added org.apache.maven.plugins:maven-source-plugin:3.2.1
  • Added org.apache.maven.plugins:maven-surefire-plugin:3.1.2
  • Added org.basepom.maven:duplicate-finder-maven-plugin:2.0.1
  • Added org.codehaus.mojo:flatten-maven-plugin:1.5.0
  • Added org.codehaus.mojo:versions-maven-plugin:2.16.0
  • Added org.itsallcode:openfasttrace-maven-plugin:1.6.2
  • Added org.jacoco:jacoco-maven-plugin:0.8.10
  • Added org.scalatest:scalatest-maven-plugin:2.2.0
  • Added org.sonarsource.scanner.maven:sonar-maven-plugin:3.9.1.2184
  • Added org.sonatype.ossindex.maven:ossindex-maven-plugin:3.2.0
  • Added org.sonatype.plugins:nexus-staging-maven-plugin:1.6.13

Spark Exasol Connector With S3

Compile Dependency Updates

  • Added com.exasol:spark-connector-common-java:1.1.1
  • Added org.apache.hadoop:hadoop-aws:3.3.4
  • Added org.scala-lang:scala-library:2.13.11
  • Added org.wildfly.openssl:wildfly-openssl:2.2.5.Final
  • Added software.amazon.awssdk:s3:2.20.103

Test Dependency Updates

  • Added com.amazonaws:aws-java-sdk-s3:1.12.506
  • Added com.exasol:exasol-testcontainers:6.6.1
  • Added com.exasol:hamcrest-resultset-matcher:1.6.0
  • Added com.exasol:test-db-builder-java:3.4.2
  • Added org.junit.jupiter:junit-jupiter-api:5.9.3
  • Added org.junit.jupiter:junit-jupiter:5.9.3
  • Added org.mockito:mockito-junit-jupiter:5.4.0
  • Added org.testcontainers:junit-jupiter:1.18.3
  • Added org.testcontainers:localstack:1.18.3

Plugin Dependency Updates

  • Added com.exasol:error-code-crawler-maven-plugin:1.3.0
  • Added io.github.zlika:reproducible-build-maven-plugin:0.16
  • Added org.apache.maven.plugins:maven-clean-plugin:2.5
  • Added org.apache.maven.plugins:maven-compiler-plugin:3.11.0
  • Added org.apache.maven.plugins:maven-deploy-plugin:3.1.1
  • Added org.apache.maven.plugins:maven-enforcer-plugin:3.3.0
  • Added org.apache.maven.plugins:maven-failsafe-plugin:3.1.2
  • Added org.apache.maven.plugins:maven-gpg-plugin:3.1.0
  • Added org.apache.maven.plugins:maven-install-plugin:2.4
  • Added org.apache.maven.plugins:maven-jar-plugin:2.4
  • Added org.apache.maven.plugins:maven-javadoc-plugin:3.5.0
  • Added org.apache.maven.plugins:maven-resources-plugin:2.6
  • Added org.apache.maven.plugins:maven-shade-plugin:3.4.1
  • Added org.apache.maven.plugins:maven-site-plugin:3.3
  • Added org.apache.maven.plugins:maven-source-plugin:3.2.1
  • Added org.apache.maven.plugins:maven-surefire-plugin:3.1.2
  • Added org.basepom.maven:duplicate-finder-maven-plugin:2.0.1
  • Added org.codehaus.mojo:flatten-maven-plugin:1.5.0
  • Added org.codehaus.mojo:versions-maven-plugin:2.16.0
  • Added org.itsallcode:openfasttrace-maven-plugin:1.6.2
  • Added org.jacoco:jacoco-maven-plugin:0.8.10
  • Added org.sonarsource.scanner.maven:sonar-maven-plugin:3.9.1.2184
  • Added org.sonatype.ossindex.maven:ossindex-maven-plugin:3.2.0
  • Added org.sonatype.plugins:nexus-staging-maven-plugin:1.6.13
Assets 51
Loading

1.4.0: Support Apache Spark 3.4.0

21 Apr 08:03
@morazow morazow
1.4.0
5942981
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
1.4.0: Support Apache Spark 3.4.0

Summary

In this release, we added support for the Apache Spark 3.4.0 version. With this release, users could use the Exasol connector in the latest Apache Spark clusters such as Azure Databricks.

We also fixed bugs due to the missing dependency classes when using the connector as a Maven dependency in the Spark applications.

Features

  • #147: Added support for Apache Spark 3.4.0 version

Bug Fixes

  • #143: Fixed broken link in Readme file
  • #145: Fixed missing class issue when using the connector as Maven dependency

Dependency Updates

Compile Dependency Updates

  • Updated com.exasol:error-reporting-java8:0.4.1 to 1.0.1
  • Updated com.exasol:exasol-jdbc:7.1.17 to 7.1.19
  • Updated com.exasol:sql-statement-builder-java8:4.5.0 to 4.5.4
  • Added com.fasterxml.woodstox:woodstox-core:6.5.1
  • Updated com.google.protobuf:protobuf-java:3.22.2 to 3.22.3

Test Dependency Updates

  • Updated com.exasol:exasol-testcontainers:6.5.1 to 6.5.2
  • Updated com.exasol:hamcrest-resultset-matcher:1.5.2 to 1.6.0
  • Updated org.mockito:mockito-core:5.2.0 to 5.3.0

Plugin Dependency Updates

  • Updated com.diffplug.spotless:spotless-maven-plugin:2.35.0 to 2.36.0
  • Updated com.exasol:error-code-crawler-maven-plugin:1.2.2 to 1.2.3
  • Updated com.exasol:project-keeper-maven-plugin:2.9.6 to 2.9.7
  • Updated org.apache.maven.plugins:maven-deploy-plugin:3.1.0 to 3.1.1
  • Updated org.apache.maven.plugins:maven-enforcer-plugin:3.2.1 to 3.3.0
  • Updated org.apache.maven.plugins:maven-failsafe-plugin:3.0.0-M8 to 3.0.0
  • Updated org.apache.maven.plugins:maven-javadoc-plugin:3.4.1 to 3.5.0
  • Updated org.apache.maven.plugins:maven-surefire-plugin:3.0.0-M8 to 3.0.0
  • Added org.basepom.maven:duplicate-finder-maven-plugin:1.5.1
  • Updated org.codehaus.mojo:flatten-maven-plugin:1.3.0 to 1.4.1
  • Updated org.codehaus.mojo:versions-maven-plugin:2.14.2 to 2.15.0
  • Updated org.itsallcode:openfasttrace-maven-plugin:1.6.1 to 1.6.2
  • Updated org.jacoco:jacoco-maven-plugin:0.8.8 to 0.8.9
Assets 27
Loading