[Chore] CI/CD overhaul (#403)

* moved dependencies to requirements subdir

* added delta helper func to configure spark session

* cleaned up comments

* added DBR 12.2 and 14.2

* changed conditional to use isinstance

* added DBR 11.3 to tox.ini

* added DBR 10.4 to tox.ini

* set ingore_metadata=True

* simplified tox.ini and moved doc requirements to docs dir

* added some TODOs

* added DBR 9.1

* updated CONTRIBUTING.md

* fixed test to check for appropriate delta-spark version

* removed useDeltaOpt conditional in io.write

* formatting

* linting

* removed event_time assumption and made zorder contingent on optimizationCols

* moved requirements and removed DBR 7 testing

* fixed tox.ini and added pyenv to test.yml

* manual trigger

* readded push trigger

* split into two jobs

* typo

* modified pyenv install cmd

* restart shell cmd

* added cmds to .profile

* .bash_profile

* combined steps

* added print statement

* removed shell restart

* pulled pyenv action from mlflow

* renamed job

* forgot installation cmd

* moved python installation into matrix

* tox type checking now works

* ignore imports for mypy

* added pyenv local cmd

* trying gabrielfalcao/pyenv-action@v18

* removed default

* trying setup-python action

* changed sequence to strings

* removed pyenv cmd

* only trying two env

* tox-gh-actions plugin

* fetch tags true

* testing git

* testing git describe

* fetch-depth = 0

* uncomment

* removed unneccessary tox envs

* removed dupe codecov step

* fixed coverage cmds in tox.ini

* removed coverage-init

* moved erase cmd

* PR template

* issue templates

* dbr 14.3 support

* removed lint and type-check from default envlist

* made reusable ci action

* forgot version ref

* moved to workflows subdir

* removed version ref

* fixed job name mismatch

* convert int to str

* forgot -e flag

* split push and release actions

* testing new push action

* try again

* added code cov token

* fixed tox build env

* testing release action

* fixed checkout depth

* removed requirements copy step

* fixed bad docstrings

* got tox build-docs working

* toxified release

* formatting

* forgot checkout step

* black check and diff

* stupid black version mismatch

* updated doc requirements

* downgraded to py3.9 on docs job

* relaxing all constraints

* switched to newer tox gh plugin

* tox gh extension is no good

* comma

* fixme reminders

* removed extra workflows

* remove build requirements for dbr91 env

* changes to buld locally on arm64 arch

* simplify shell for dbr104 setup

* simplify shell for dbr104 setup

* no build deps for packages that need numpy

* remove dbr104 env support

* removed support for python 3.8 DBRs

* updated contribution docs

* set fail-fast to false

* checkpoint commit of some updates to the code that converts nano-second precision timestamps

* refactored test code data format to allow for better separation of DF creation from TSDF constructor args

* formatting

* We should just check for style compliance with black, not apply them

* Revert "We should just check for style compliance with black, not apply them"

This reverts commit 4c90318.

* moving dbr version specifiers to use compatibility syntax

* updating 2 test cases

* renamed test action

* testing json refactor

* found new error

* still broken

* variable name refactoring

* remove re import as it was unused

* remove get_data_as_tsdf

* remove get_data_as_sdf

* fix typo in docstring

* refactor schema comparison for test helper

* general refactor for TSDFBaseTests

* remove schema check since that is performed by chispa.asssert_df_equality

* existing tests refactored and passing

* interpol test case work

* WIP for test_tsdf_interpolate

* add idf getter to test dataframe builder

* tests for tsdf refactored

* remove test_tsdf_interpolate

* refactor interpol_tests complete

* additional checks for idf dataframe equality

* refactor intervals_tests

* refactor io_tests

* reindent with 2 spaces for consistency

* refactor resample_2_tests

* refactor resample_tests

* safe delete and refactor to remove resample_2_tests

was identical test cases to resample_tests

* refactor utils_tests

* chore: tox lint

* fix missing ts_convert keys in json

* adjust build release to only trigger when tag is pushed to master

* Revert "adjust build release to only trigger when tag is pushed to master"

This reverts commit 1ac3391.

* need to update CodeQL action version

* updating to latest actions versions

---------

Co-authored-by: Lorin <lorin@databricks.com>
Co-authored-by: Lorin <lorin.dawson@databricks.com>
Co-authored-by: Tristan Nixon <tristan.nixon@databricks.com>

.github/ISSUE_TEMPLATE/bug.yml

@@ -0,0 +1,66 @@
+# See https://docs.github.com/en/communities/using-templates-to-encourage-useful-issues-and-pull-requests/syntax-for-issue-forms
+# and https://docs.github.com/en/communities/using-templates-to-encourage-useful-issues-and-pull-requests/syntax-for-githubs-form-schema
+name: Bug Report
+description: Something is not working with Tempo
+title: "[BUG]: "
+labels: ["bug", "needs-triage"]
+projects: ["databrickslabs/12"]
+body:
+  - type: checkboxes
+    attributes:
+      label: Is there an existing issue for this?
+      description: Please search to see if an issue already exists for the bug you encountered.
+      options:
+      - label: I have searched the existing issues
+        required: true
+  - type: textarea
+    attributes:
+      label: Current Behavior
+      description: |
+        A concise description of what you're experiencing.
+        **Do not paste links to attachments with logs and/or images, as all issues will attachments will get deleted.**
+        Use the `Relevant log output` field to paste redacted log output without personal identifying information (PII).
+        You can Ctrl/Cmd+V the screenshot, which would appear as a rendered image if it doesn't contain any PII.
+    validations:
+      required: false
+  - type: textarea
+    attributes:
+      label: Expected Behavior
+      description: A concise description of what you expected to happen.
+    validations:
+      required: false
+  - type: textarea
+    attributes:
+      label: Steps To Reproduce
+      description: Steps to reproduce the behavior.
+      placeholder: |
+        1. In this environment...
+        1. With this config...
+        1. Run '...'
+        1. See error...
+    validations:
+      required: false
+  - type: dropdown
+    id: cloud
+    attributes:
+      label: Cloud
+      description: What cloud are you using?
+      options:
+        - AWS
+        - Azure
+        - GCP
+    validations:
+      required: true
+  - type: textarea
+    id: version
+    attributes:
+      label: Version
+      description: What version of our software are you running?
+    validations:
+      required: true
+  - type: textarea
+    id: logs
+    attributes:
+      label: Relevant log output
+      description: Please copy and paste any relevant log output. This will be automatically formatted into code, so no need for backticks.
+      render: shell

.github/ISSUE_TEMPLATE/config.yml

@@ -0,0 +1,9 @@
+blank_issues_enabled: false
+contact_links:
+  - name: General Databricks questions
+    url: https://help.databricks.com/
+    about: Issues related to Databricks and not related to UCX
+
+  - name: UCX Documentation
+    url: https://databrickslabs.github.io/tempo/
+    about: Documentation about Tempo

.github/ISSUE_TEMPLATE/feature.yml

@@ -0,0 +1,33 @@
+# See https://docs.github.com/en/communities/using-templates-to-encourage-useful-issues-and-pull-requests/syntax-for-issue-forms
+# and https://docs.github.com/en/communities/using-templates-to-encourage-useful-issues-and-pull-requests/syntax-for-githubs-form-schema
+name: Feature Request
+description: Something new needs to happen with Tempo
+title: "[FEATURE]: "
+labels: ["enhancement", "needs-triage"]
+projects: ["databrickslabs/13"]
+body:
+  - type: checkboxes
+    attributes:
+      label: Is there an existing issue for this?
+      description: Please search to see if an issue already exists for the feature request you're willing to submit
+      options:
+      - label: I have searched the existing issues
+        required: true
+  - type: textarea
+    attributes:
+      label: Problem statement
+      description: A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
+    validations:
+      required: true
+  - type: textarea
+    attributes:
+      label: Proposed Solution
+      description: A clear and concise description of what you want to happen.
+    validations:
+      required: true
+  - type: textarea
+    attributes:
+      label: Additional Context
+      description: Add any other context, references or screenshots about the feature request here.
+    validations:
+      required: false

.github/pull_request_template.md

@@ -0,0 +1,29 @@
+## Changes
+<!-- Summary of your changes that are easy to understand. Add screenshots when necessary -->
+
+### Linked issues
+<!-- DOC: Link issue with a keyword: close, closes, closed, fix, fixes, fixed, resolve, resolves, resolved. See https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword -->
+
+Resolves #..
+
+### Functionality 
+
+- [ ] added relevant user documentation
+- [ ] added a new Class method
+- [ ] modified existing Class method: `...`
+- [ ] added a new function
+- [ ] modified existing function: `...`
+- [ ] added a new test
+- [ ] modified existing test: `...`
+- [ ] added a new example
+- [ ] modified existing example: `...`
+- [ ] added a new utility
+- [ ] modified existing utility: `...`
+
+### Tests
+<!-- How is this tested? Please see the checklist below and also describe any other relevant tests -->
+
+- [ ] manually tested
+- [ ] added unit tests
+- [ ] added integration tests
+- [ ] verified on staging environment (screenshot attached)

.github/workflows/build-release.yml

@@ -0,0 +1,78 @@
+name: build-release
+
+on:
+  pull_request:
+    types: [opened, synchronize]
+  push:
+    branches: ['master']
+
+jobs:
+  release:
+    runs-on: ubuntu-latest
+    environment: release
+    permissions:
+      # Used to authenticate to PyPI via OIDC and sign the release's artifacts with sigstore-python.
+      id-token: write
+      # Used to attach signing artifacts to the published release.
+      contents: write
+
+    steps:
+      - uses: actions/checkout@v4
+        with:
+          fetch-depth: 0
+          fetch-tags: true
+
+      - name: Set up Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: '3.10'
+
+      - name: Install dependencies
+        run: |
+          python -m pip install --upgrade pip
+          python -m pip install tox
+
+      - name: Build dist
+        working-directory: ./python
+        run: tox -e build-dist
+
+      - name: Publish a Python distribution to PyPI
+        uses: pypa/gh-action-pypi-publish@release/v1
+        if: startsWith(github.ref, 'refs/tags/v')
+        with:
+          packages-dir: python/dist/
+
+  docs:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+        with:
+          fetch-depth: 0
+          fetch-tags: true
+
+      - name: Set up Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: '3.9'
+
+      - name: Install dependencies
+        run: |
+          python -m pip install --upgrade pip
+          python -m pip install tox
+
+      - name: Build docs
+        working-directory: ./python
+        run: tox -e build-docs
+
+      - name: Upload artifacts
+        uses: actions/upload-artifact@v4
+        with:
+          name: html-docs
+          path: docs/_build/html/
+
+      - name: Deploy 🚀
+        uses: peaceiris/actions-gh-pages@v3
+        if: startsWith(github.ref, 'refs/tags/v')
+        with:
+          github_token: ${{ secrets.GITHUB_TOKEN }}
+          publish_dir: docs/_build/html