-
Notifications
You must be signed in to change notification settings - Fork 53
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
CI/CD overhaul #403
Merged
Merged
CI/CD overhaul #403
Changes from all commits
Commits
Show all changes
141 commits
Select commit
Hold shift + click to select a range
1b5766e
moved dependencies to requirements subdir
jtisbell4 42516b0
added delta helper func to configure spark session
jtisbell4 5133f9d
cleaned up comments
jtisbell4 002040f
added DBR 12.2 and 14.2
jtisbell4 e89df24
changed conditional to use isinstance
jtisbell4 fdaf6ec
added DBR 11.3 to tox.ini
jtisbell4 4812caf
added DBR 10.4 to tox.ini
jtisbell4 9b5b650
set ingore_metadata=True
jtisbell4 2a5c33d
simplified tox.ini and moved doc requirements to docs dir
jtisbell4 14d29ec
added some TODOs
jtisbell4 7e5a5de
added DBR 9.1
jtisbell4 cf1abe0
updated CONTRIBUTING.md
jtisbell4 f3297a9
fixed test to check for appropriate delta-spark version
jtisbell4 16d29ab
removed useDeltaOpt conditional in io.write
jtisbell4 fc0b357
formatting
jtisbell4 e11fbb2
linting
jtisbell4 c557fc1
removed event_time assumption and made zorder contingent on optimizat…
jtisbell4 e316127
moved requirements and removed DBR 7 testing
jtisbell4 d7fb52c
fixed tox.ini and added pyenv to test.yml
jtisbell4 506d088
manual trigger
jtisbell4 d465f3a
readded push trigger
jtisbell4 941855f
split into two jobs
jtisbell4 51399c3
typo
jtisbell4 c0a908e
modified pyenv install cmd
jtisbell4 4a465cc
restart shell cmd
jtisbell4 fedabd3
added cmds to .profile
jtisbell4 e752015
.bash_profile
jtisbell4 e1f606f
combined steps
jtisbell4 33d8b02
added print statement
jtisbell4 7865d37
removed shell restart
jtisbell4 545065f
pulled pyenv action from mlflow
jtisbell4 10ff339
renamed job
jtisbell4 15d0c02
forgot installation cmd
jtisbell4 fe89957
moved python installation into matrix
jtisbell4 b513b85
tox type checking now works
jtisbell4 de56c14
ignore imports for mypy
jtisbell4 5eb5ac7
added pyenv local cmd
jtisbell4 fab1343
trying gabrielfalcao/pyenv-action@v18
jtisbell4 2358319
removed default
jtisbell4 35d47bd
trying setup-python action
jtisbell4 9eab483
changed sequence to strings
jtisbell4 30d4436
removed pyenv cmd
jtisbell4 5e2554a
only trying two env
jtisbell4 f29e5b7
tox-gh-actions plugin
jtisbell4 124f93a
fetch tags true
jtisbell4 36f633c
testing git
jtisbell4 acba106
testing git describe
jtisbell4 d5e60cb
fetch-depth = 0
jtisbell4 13efc50
uncomment
jtisbell4 8ceb750
removed unneccessary tox envs
jtisbell4 ed756bb
removed dupe codecov step
jtisbell4 4b30e4e
fixed coverage cmds in tox.ini
jtisbell4 74352ea
removed coverage-init
jtisbell4 f1479e7
moved erase cmd
jtisbell4 14c3030
PR template
eeaea58
issue templates
8d8f95f
Merge branch 'lorin/tempo/cicd-updates' into tox-refactor
47870bb
dbr 14.3 support
jtisbell4 3c7f188
removed lint and type-check from default envlist
jtisbell4 39e49c4
made reusable ci action
jtisbell4 c7a509a
forgot version ref
jtisbell4 4939ac5
moved to workflows subdir
jtisbell4 12ef690
removed version ref
jtisbell4 2eafd0e
fixed job name mismatch
jtisbell4 9eb5fb0
convert int to str
jtisbell4 5dd768a
forgot -e flag
jtisbell4 3a5b4a2
split push and release actions
jtisbell4 96db776
testing new push action
jtisbell4 b5089e2
try again
jtisbell4 eff62de
added code cov token
jtisbell4 d24d0f4
fixed tox build env
jtisbell4 457dea8
testing release action
jtisbell4 97e860c
fixed checkout depth
jtisbell4 6a97cb9
removed requirements copy step
jtisbell4 59b2c65
fixed bad docstrings
jtisbell4 357abe6
got tox build-docs working
jtisbell4 6383c1b
toxified release
jtisbell4 5bcee97
formatting
jtisbell4 68e0c39
forgot checkout step
jtisbell4 706fdc9
black check and diff
jtisbell4 7703b7e
stupid black version mismatch
jtisbell4 837e806
updated doc requirements
jtisbell4 59e9fc1
downgraded to py3.9 on docs job
jtisbell4 bc837c0
relaxing all constraints
jtisbell4 09db68c
switched to newer tox gh plugin
jtisbell4 ee09bd5
tox gh extension is no good
jtisbell4 c47a48e
comma
jtisbell4 0f0144d
fixme reminders
jtisbell4 dd1407f
removed extra workflows
jtisbell4 3b50f8c
Merge branch 'master' into tox-refactor
R7L208 b4e3cfe
remove build requirements for dbr91 env
R7L208 3b4a825
changes to buld locally on arm64 arch
R7L208 4f18dc0
simplify shell for dbr104 setup
R7L208 e4e8f6a
simplify shell for dbr104 setup
R7L208 59171a5
no build deps for packages that need numpy
R7L208 a0cc65b
remove dbr104 env support
R7L208 550d4e0
removed support for python 3.8 DBRs
jtisbell4 333975a
updated contribution docs
jtisbell4 4d02b80
set fail-fast to false
jtisbell4 b5a9a0c
checkpoint commit of some updates to the code that converts nano-seco…
tnixon 8a7eb5e
Merge branch 'fix_nanos_conversion' into tox-refactor
tnixon 7469a50
refactored test code data format to allow for better separation of DF…
tnixon 0e1c3ef
formatting
jtisbell4 4c90318
We should just check for style compliance with black, not apply them
tnixon 81f0239
Revert "We should just check for style compliance with black, not app…
tnixon fc55de2
moving dbr version specifiers to use compatibility syntax
tnixon 8e42fb4
updating 2 test cases
tnixon dbf08c0
renamed test action
jtisbell4 e543b69
testing json refactor
jtisbell4 8d5a583
found new error
jtisbell4 35e2b50
still broken
jtisbell4 afb1175
variable name refactoring
R7L208 c24db4c
remove re import as it was unused
R7L208 4ea9e22
remove get_data_as_tsdf
R7L208 9b0094f
remove get_data_as_sdf
R7L208 645114f
fix typo in docstring
R7L208 d6e48f6
refactor schema comparison for test helper
R7L208 2414027
general refactor for TSDFBaseTests
R7L208 f6f3520
remove schema check since that is performed by chispa.asssert_df_equa…
R7L208 a46dd39
existing tests refactored and passing
R7L208 1bb8383
interpol test case work
R7L208 582dac5
WIP for test_tsdf_interpolate
R7L208 3d33a68
add idf getter to test dataframe builder
R7L208 1a5add8
tests for tsdf refactored
R7L208 ab5210a
remove test_tsdf_interpolate
R7L208 09e6423
refactor interpol_tests complete
R7L208 981f1ab
additional checks for idf dataframe equality
R7L208 55e65ad
refactor intervals_tests
R7L208 f1958b2
refactor io_tests
R7L208 21405b8
reindent with 2 spaces for consistency
R7L208 79390e7
refactor resample_2_tests
R7L208 8558e90
refactor resample_tests
R7L208 fc60934
safe delete and refactor to remove resample_2_tests
R7L208 4f2e8f9
refactor utils_tests
R7L208 a095585
chore: tox lint
R7L208 3cb8cd3
fix missing ts_convert keys in json
R7L208 1ac3391
adjust build release to only trigger when tag is pushed to master
R7L208 8ffdce7
Revert "adjust build release to only trigger when tag is pushed to ma…
tnixon fb17a8b
Merge branch 'master' into tox-refactor
tnixon 60f9fc8
need to update CodeQL action version
tnixon 06f93ac
updating to latest actions versions
tnixon File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,66 @@ | ||
# See https://docs.github.com/en/communities/using-templates-to-encourage-useful-issues-and-pull-requests/syntax-for-issue-forms | ||
# and https://docs.github.com/en/communities/using-templates-to-encourage-useful-issues-and-pull-requests/syntax-for-githubs-form-schema | ||
name: Bug Report | ||
description: Something is not working with Tempo | ||
title: "[BUG]: " | ||
labels: ["bug", "needs-triage"] | ||
projects: ["databrickslabs/12"] | ||
body: | ||
- type: checkboxes | ||
attributes: | ||
label: Is there an existing issue for this? | ||
description: Please search to see if an issue already exists for the bug you encountered. | ||
options: | ||
- label: I have searched the existing issues | ||
required: true | ||
- type: textarea | ||
attributes: | ||
label: Current Behavior | ||
description: | | ||
A concise description of what you're experiencing. | ||
**Do not paste links to attachments with logs and/or images, as all issues will attachments will get deleted.** | ||
Use the `Relevant log output` field to paste redacted log output without personal identifying information (PII). | ||
You can Ctrl/Cmd+V the screenshot, which would appear as a rendered image if it doesn't contain any PII. | ||
validations: | ||
required: false | ||
- type: textarea | ||
attributes: | ||
label: Expected Behavior | ||
description: A concise description of what you expected to happen. | ||
validations: | ||
required: false | ||
- type: textarea | ||
attributes: | ||
label: Steps To Reproduce | ||
description: Steps to reproduce the behavior. | ||
placeholder: | | ||
1. In this environment... | ||
1. With this config... | ||
1. Run '...' | ||
1. See error... | ||
validations: | ||
required: false | ||
- type: dropdown | ||
id: cloud | ||
attributes: | ||
label: Cloud | ||
description: What cloud are you using? | ||
options: | ||
- AWS | ||
- Azure | ||
- GCP | ||
validations: | ||
required: true | ||
- type: textarea | ||
id: version | ||
attributes: | ||
label: Version | ||
description: What version of our software are you running? | ||
validations: | ||
required: true | ||
- type: textarea | ||
id: logs | ||
attributes: | ||
label: Relevant log output | ||
description: Please copy and paste any relevant log output. This will be automatically formatted into code, so no need for backticks. | ||
render: shell |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change | ||||
---|---|---|---|---|---|---|
@@ -0,0 +1,9 @@ | ||||||
blank_issues_enabled: false | ||||||
contact_links: | ||||||
- name: General Databricks questions | ||||||
url: https://help.databricks.com/ | ||||||
about: Issues related to Databricks and not related to UCX | ||||||
|
||||||
- name: UCX Documentation | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||
url: https://databrickslabs.github.io/tempo/ | ||||||
about: Documentation about Tempo |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,33 @@ | ||
# See https://docs.github.com/en/communities/using-templates-to-encourage-useful-issues-and-pull-requests/syntax-for-issue-forms | ||
# and https://docs.github.com/en/communities/using-templates-to-encourage-useful-issues-and-pull-requests/syntax-for-githubs-form-schema | ||
name: Feature Request | ||
description: Something new needs to happen with Tempo | ||
title: "[FEATURE]: " | ||
labels: ["enhancement", "needs-triage"] | ||
projects: ["databrickslabs/13"] | ||
body: | ||
- type: checkboxes | ||
attributes: | ||
label: Is there an existing issue for this? | ||
description: Please search to see if an issue already exists for the feature request you're willing to submit | ||
options: | ||
- label: I have searched the existing issues | ||
required: true | ||
- type: textarea | ||
attributes: | ||
label: Problem statement | ||
description: A clear and concise description of what the problem is. Ex. I'm always frustrated when [...] | ||
validations: | ||
required: true | ||
- type: textarea | ||
attributes: | ||
label: Proposed Solution | ||
description: A clear and concise description of what you want to happen. | ||
validations: | ||
required: true | ||
- type: textarea | ||
attributes: | ||
label: Additional Context | ||
description: Add any other context, references or screenshots about the feature request here. | ||
validations: | ||
required: false |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,29 @@ | ||
## Changes | ||
<!-- Summary of your changes that are easy to understand. Add screenshots when necessary --> | ||
|
||
### Linked issues | ||
<!-- DOC: Link issue with a keyword: close, closes, closed, fix, fixes, fixed, resolve, resolves, resolved. See https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword --> | ||
|
||
Resolves #.. | ||
|
||
### Functionality | ||
|
||
- [ ] added relevant user documentation | ||
- [ ] added a new Class method | ||
- [ ] modified existing Class method: `...` | ||
- [ ] added a new function | ||
- [ ] modified existing function: `...` | ||
- [ ] added a new test | ||
- [ ] modified existing test: `...` | ||
- [ ] added a new example | ||
- [ ] modified existing example: `...` | ||
- [ ] added a new utility | ||
- [ ] modified existing utility: `...` | ||
|
||
### Tests | ||
<!-- How is this tested? Please see the checklist below and also describe any other relevant tests --> | ||
|
||
- [ ] manually tested | ||
- [ ] added unit tests | ||
- [ ] added integration tests | ||
- [ ] verified on staging environment (screenshot attached) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,78 @@ | ||
name: build-release | ||
|
||
on: | ||
tnixon marked this conversation as resolved.
Show resolved
Hide resolved
|
||
pull_request: | ||
types: [opened, synchronize] | ||
push: | ||
branches: ['master'] | ||
|
||
jobs: | ||
release: | ||
runs-on: ubuntu-latest | ||
environment: release | ||
permissions: | ||
# Used to authenticate to PyPI via OIDC and sign the release's artifacts with sigstore-python. | ||
id-token: write | ||
# Used to attach signing artifacts to the published release. | ||
contents: write | ||
|
||
steps: | ||
- uses: actions/checkout@v4 | ||
with: | ||
fetch-depth: 0 | ||
fetch-tags: true | ||
|
||
- name: Set up Python | ||
uses: actions/setup-python@v5 | ||
with: | ||
python-version: '3.10' | ||
|
||
- name: Install dependencies | ||
run: | | ||
python -m pip install --upgrade pip | ||
python -m pip install tox | ||
|
||
- name: Build dist | ||
working-directory: ./python | ||
run: tox -e build-dist | ||
|
||
- name: Publish a Python distribution to PyPI | ||
uses: pypa/gh-action-pypi-publish@release/v1 | ||
if: startsWith(github.ref, 'refs/tags/v') | ||
with: | ||
packages-dir: python/dist/ | ||
|
||
docs: | ||
runs-on: ubuntu-latest | ||
steps: | ||
- uses: actions/checkout@v4 | ||
with: | ||
fetch-depth: 0 | ||
fetch-tags: true | ||
|
||
- name: Set up Python | ||
uses: actions/setup-python@v5 | ||
with: | ||
python-version: '3.9' | ||
|
||
- name: Install dependencies | ||
run: | | ||
python -m pip install --upgrade pip | ||
python -m pip install tox | ||
|
||
- name: Build docs | ||
working-directory: ./python | ||
run: tox -e build-docs | ||
|
||
- name: Upload artifacts | ||
uses: actions/upload-artifact@v4 | ||
with: | ||
name: html-docs | ||
path: docs/_build/html/ | ||
|
||
- name: Deploy 🚀 | ||
uses: peaceiris/actions-gh-pages@v3 | ||
if: startsWith(github.ref, 'refs/tags/v') | ||
with: | ||
github_token: ${{ secrets.GITHUB_TOKEN }} | ||
publish_dir: docs/_build/html |
This file was deleted.
Oops, something went wrong.
This file was deleted.
Oops, something went wrong.
This file was deleted.
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.