This project creates bridge for PySpark to use StreamQueryListener class from java in python.
In this article we explain and show examples of how to use this jar with PySpark project:
Medium article
Tested it Spark 3.1.2 version, should also work for other Spark versions.
This project is based on this Stackoverflow comment: Stackoverflow link
Building command:
mvn clean install
After Building the project jar, we can use it like this with PySpark:
pyspark --jars StreamingQueryListenerGateway-{version}.jar