Add PySpark to the add-ons flow #1160

Job	Run time
lint (ubuntu-latest, 3.11) / lint	3m 12s
e2e-tests (ubuntu-latest, 3.10) / e2e-tests	7m 10s
e2e-tests (ubuntu-latest, 3.8) / e2e-tests	7m 0s
e2e-tests (ubuntu-latest, 3.11) / e2e-tests	7m 23s
e2e-tests (ubuntu-latest, 3.9) / e2e-tests	8m 17s
e2e-tests (windows-latest, 3.10) / e2e-tests	23m 13s
e2e-tests (windows-latest, 3.11) / e2e-tests	26m 34s
e2e-tests (windows-latest, 3.8) / e2e-tests	25m 21s
e2e-tests (windows-latest, 3.9) / e2e-tests	27m 57s
pip-compile (ubuntu-latest, 3.10) / pip-compile	2m 43s
pip-compile (ubuntu-latest, 3.11) / pip-compile	2m 45s
pip-compile (ubuntu-latest, 3.8) / pip-compile	2m 3s
pip-compile (ubuntu-latest, 3.9) / pip-compile	2m 21s
pip-compile (windows-latest, 3.10) / pip-compile	5m 16s
pip-compile (windows-latest, 3.11) / pip-compile	8m 22s
pip-compile (windows-latest, 3.8) / pip-compile	6m 39s
pip-compile (windows-latest, 3.9) / pip-compile	6m 59s
unit-tests (ubuntu-latest, 3.10) / unit-tests	8m 45s
unit-tests (ubuntu-latest, 3.11) / unit-tests	10m 55s
unit-tests (ubuntu-latest, 3.8) / unit-tests	10m 17s
unit-tests (ubuntu-latest, 3.9) / unit-tests	7m 56s
unit-tests (windows-latest, 3.10) / unit-tests	18m 32s
unit-tests (windows-latest, 3.8) / unit-tests	20m 15s
unit-tests (windows-latest, 3.9) / unit-tests	19m 26s
unit-tests (windows-latest, 3.11) / unit-tests	22m 33s
	4h 51m 54s

Provide feedback