Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixed updating the library version in the CD workflow #970

Merged
merged 2 commits into from
Feb 25, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 17 additions & 11 deletions .github/workflows/npm-cd.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,11 +3,16 @@
name: Continuous Deployment

on:
pull_request:
paths:
- .github/workflows/npm-cd.yml
- .github/workflows/build-node-wrapper/action.yml
push:
tags:
- "v*.*"
jobs:
publish-binaries:
if: github.repository_owner == 'aws'
name: Publish packages to NPM
runs-on: ${{ matrix.build.RUNNER }}
strategy:
Expand All @@ -20,15 +25,13 @@ jobs:
RUNNER: ubuntu-latest,
ARCH: x64,
TARGET: x86_64-unknown-linux-gnu,
NPM_PUBLISH: true,
}
- {
OS: ubuntu-latest,
NAMED_OS: linux,
RUNNER: [self-hosted, Linux, ARM64],
ARCH: arm64,
TARGET: aarch64-unknown-linux-gnu,
NPM_PUBLISH: true,
CONTAINER: "2_28",
}
- {
Expand All @@ -37,15 +40,13 @@ jobs:
RUNNER: macos-latest,
ARCH: x64,
TARGET: x86_64-apple-darwin,
NPM_PUBLISH: true,
}
- {
OS: macos-latest,
NAMED_OS: darwin,
RUNNER: macos-13-xlarge,
arch: arm64,
TARGET: aarch64-apple-darwin,
NPM_PUBLISH: true,
}
steps:
- name: Checkout
Expand All @@ -56,10 +57,10 @@ jobs:
- name: Set the release version
shell: bash
run: |
echo "RELEASE_VERSION=${GITHUB_REF:11}" >> $GITHUB_ENV
export version=`if ${{ github.event_name == 'pull_request' }}; then echo '255.255.255'; else echo ${GITHUB_REF:11}; fi`
echo "RELEASE_VERSION=${version}" >> $GITHUB_ENV

- name: Setup node
if: matrix.build.NPM_PUBLISH == true
uses: actions/setup-node@v3
with:
node-version: "16"
Expand All @@ -68,9 +69,14 @@ jobs:
scope: "${{ vars.NPM_SCOPE }}"
always-auth: true
token: ${{ secrets.NPM_AUTH_TOKEN }}


- name: Update package version in config.toml
uses: ./.github/workflows/update-glide-version
with:
folder_path: "${{ github.workspace }}/node/rust-client/.cargo"
named_os: ${{ matrix.build.NAMED_OS }}

- name: Build Node wrapper
if: matrix.build.NPM_PUBLISH == true
uses: ./.github/workflows/build-node-wrapper
with:
os: ${{ matrix.build.OS }}
Expand All @@ -82,7 +88,7 @@ jobs:
github-token: ${{ secrets.GITHUB_TOKEN }}

- name: Publish to NPM
if: matrix.build.NPM_PUBLISH == true
if: github.event_name != 'pull_request'
shell: bash
working-directory: ./node
run: |
Expand All @@ -101,7 +107,6 @@ jobs:
NODE_AUTH_TOKEN: ${{ secrets.NPM_AUTH_TOKEN }}

- name: Pack the Node package
if: matrix.build.NPM_PUBLISH == true
shell: bash
working-directory: ./node
run: |
Expand All @@ -115,7 +120,7 @@ jobs:
NODE_AUTH_TOKEN: ${{ secrets.NPM_AUTH_TOKEN }}

- name: Upload the Node package
if: matrix.build.NPM_PUBLISH == true
if: github.event_name != 'pull_request'
uses: actions/upload-artifact@v3
barshaul marked this conversation as resolved.
Show resolved Hide resolved
with:
name: ${{ matrix.build.TARGET }}
Expand Down Expand Up @@ -190,6 +195,7 @@ jobs:
done

- name: Publish the base package
if: github.event_name != 'pull_request'
shell: bash
working-directory: ./node/npm/glide
run: |
Expand Down
38 changes: 22 additions & 16 deletions .github/workflows/pypi-cd.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,11 +3,16 @@
name: Continuous Deployment

on:
pull_request:
paths:
- .github/workflows/pypi-cd.yml
- .github/workflows/build-python-wrapper/action.yml
push:
tags:
- "v*.*"
jobs:
publish-binaries:
if: github.repository_owner == 'aws'
name: Publish packages to PyPi
runs-on: ${{ matrix.build.RUNNER }}
strategy:
Expand All @@ -20,15 +25,13 @@ jobs:
RUNNER: ubuntu-latest,
ARCH: x64,
TARGET: x86_64-unknown-linux-gnu,
PYPI_PUBLISH: true,
}
- {
OS: ubuntu-latest,
NAMED_OS: linux,
RUNNER: [self-hosted, Linux, ARM64],
ARCH: arm64,
TARGET: aarch64-unknown-linux-gnu,
PYPI_PUBLISH: true,
CONTAINER: "2_28",
}
- {
Expand All @@ -37,15 +40,13 @@ jobs:
RUNNER: macos-latest,
ARCH: x64,
TARGET: x86_64-apple-darwin,
PYPI_PUBLISH: true,
}
- {
OS: macos-latest,
NAMED_OS: darwin,
RUNNER: macos-13-xlarge,
arch: arm64,
TARGET: aarch64-apple-darwin,
PYPI_PUBLISH: true,
}
steps:
- name: Checkout
Expand All @@ -56,10 +57,10 @@ jobs:
- name: Set the release version
shell: bash
run: |
echo "RELEASE_VERSION=${GITHUB_REF:11}" >> $GITHUB_ENV
export version=`if ${{ github.event_name == 'pull_request' }}; then echo '255.255.255'; else echo ${GITHUB_REF:11}; fi`
echo "RELEASE_VERSION=${version}" >> $GITHUB_ENV

- name: Set the package version for Python
if: matrix.build.PYPI_PUBLISH == true
working-directory: ./python
run: |
SED_FOR_MACOS=`if [[ "${{ matrix.build.OS }}" =~ .*"macos".* ]]; then echo "''"; fi`
Expand All @@ -68,25 +69,30 @@ jobs:
cat Cargo.toml

- name: Set up Python
if: matrix.build.PYPI_PUBLISH == true && !contains(matrix.build.RUNNER, 'self-hosted')
if: ${{ !contains(matrix.build.RUNNER, 'self-hosted') }}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

minor: why do you need the curly braces here and not in lines 78 / 83?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The linter gets angry when it's with ! (not) :(

uses: actions/setup-python@v4
with:
python-version: "3.10"

- name: Set up Python older versions for MacOS
if: matrix.build.PYPI_PUBLISH == true && startsWith(matrix.build.NAMED_OS, 'darwin')
if: startsWith(matrix.build.NAMED_OS, 'darwin')
run: |
brew install python@3.8 python@3.9

- name: Setup Python for self-hosted Ubuntu runners
if: matrix.build.PYPI_PUBLISH == true && contains(matrix.build.OS, 'ubuntu') && contains(matrix.build.RUNNER, 'self-hosted')
if: contains(matrix.build.OS, 'ubuntu') && contains(matrix.build.RUNNER, 'self-hosted')
run: |
sudo apt update -y
sudo apt upgrade -y
sudo apt install python3 python3-venv python3-pip -y

- name: Update package version in config.toml
uses: ./.github/workflows/update-glide-version
with:
folder_path: "${{ github.workspace }}/python/.cargo"
named_os: ${{ matrix.build.NAMED_OS }}

- name: Build Python wrapper
if: matrix.build.PYPI_PUBLISH == true
uses: ./.github/workflows/build-python-wrapper
with:
os: ${{ matrix.build.OS }}
Expand All @@ -96,20 +102,19 @@ jobs:

- name: Include protobuf files in the package
working-directory: ./python
if: matrix.build.PYPI_PUBLISH == true
run: |
SED_FOR_MACOS=`if [[ "${{ matrix.build.OS }}" =~ .*"macos".* ]]; then echo "''"; fi`
sed -i $SED_FOR_MACOS '/pb2.py/d' .gitignore
# Log the edited .gitignore file
cat .gitignore

- name: Build Python wheels (linux)
if: matrix.build.PYPI_PUBLISH == true && startsWith(matrix.build.NAMED_OS, 'linux')
if: startsWith(matrix.build.NAMED_OS, 'linux')
uses: PyO3/maturin-action@v1
with:
working-directory: ./python
target: ${{ matrix.build.TARGET }}
args: --release --strip --out wheels -i python3.8 python3.9 python3.10 python3.11 python3.12
args: --release --strip --out wheels -i ${{ github.event_name != 'pull_request' && 'python3.8 python3.9 python3.10 python3.11 python3.12' || 'python3.10' }}
manylinux: auto
container: ${{ matrix.build.CONTAINER != '' && matrix.build.CONTAINER || '2014' }}
before-script-linux: |
Expand All @@ -125,22 +130,23 @@ jobs:
fi

- name: Build Python wheels (macos)
if: matrix.build.PYPI_PUBLISH == true && startsWith(matrix.build.NAMED_OS, 'darwin')
if: startsWith(matrix.build.NAMED_OS, 'darwin')
uses: PyO3/maturin-action@v1
with:
working-directory: ./python
target: ${{ matrix.build.TARGET }}
args: --release --strip --out wheels -i python3.8 python3.9 python3.10 python3.11 python3.12
args: --release --strip --out wheels -i ${{ github.event_name != 'pull_request' && 'python3.8 python3.9 python3.10 python3.11 python3.12' || 'python3.10' }}

- name: Upload Python wheels
if: matrix.build.PYPI_PUBLISH == true
if: github.event_name != 'pull_request'
uses: actions/upload-artifact@v3
with:
name: wheels
path: python/wheels
if-no-files-found: error

publish-to-pypi:
if: github.event_name != 'pull_request'
name: Publish the base PyPi package
runs-on: ubuntu-latest
needs: publish-binaries
Expand Down
26 changes: 26 additions & 0 deletions .github/workflows/update-glide-version/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
name: Update GLIDE version in the config.toml file

inputs:
folder_path:
description: "The folder path of the config.toml file"
required: true
type: string
named_os:
description: "The name of the current operating system"
required: false
default: "linux"
type: string
options:
- linux
- darwin
runs:
using: "composite"
steps:
- name: Update package version in config.toml
working-directory: ${{ inputs.folder_path }}
shell: bash
run: |
SED_FOR_MACOS=`if ${{ inputs.named_os == 'darwin' }}; then echo "''"; fi`
sed -i $SED_FOR_MACOS "s|unknown|${{ env.RELEASE_VERSION }}|g" ./config.toml
# log the config.toml file
cat ./config.toml
2 changes: 1 addition & 1 deletion benchmarks/rust/.cargo/config.toml
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
[env]
GLIDE_NAME = "Glide" # This should be overwritten by each wrapper library.
GLIDE_VERSION = "0.1.0"
GLIDE_VERSION = "unknown"
2 changes: 1 addition & 1 deletion csharp/lib/.cargo/config.toml
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
[env]
GLIDE_NAME = { value = "GlideC#", force = true }
GLIDE_VERSION = "0.1.0"
GLIDE_VERSION = "unknown"
2 changes: 1 addition & 1 deletion glide-core/.cargo/config.toml
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
[env]
GLIDE_NAME = "Glide" # This should be overwritten by each wrapper library.
GLIDE_VERSION = "0.1.0"
GLIDE_VERSION = "unknown"
2 changes: 1 addition & 1 deletion java/.cargo/config.toml
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
[env]
GLIDE_NAME = { value = "GlideJava", force = true }
GLIDE_VERSION = "0.1.0"
GLIDE_VERSION = "unknown"
2 changes: 1 addition & 1 deletion node/rust-client/.cargo/config.toml
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
[env]
GLIDE_NAME = { value = "GlideJS", force = true }
GLIDE_VERSION = "0.1.0"
GLIDE_VERSION = "unknown"
2 changes: 1 addition & 1 deletion node/tests/SharedTests.ts
Original file line number Diff line number Diff line change
Expand Up @@ -85,7 +85,7 @@ export function runBaseTests<Context>(config: {
const result = await client.customCommand(["CLIENT", "INFO"]);

expect(result).toContain("lib-name=GlideJS");
expect(result).toContain("lib-ver=0.1.0");
expect(result).toContain("lib-ver=unknown");
}, protocol);
},
config.timeout
Expand Down
2 changes: 1 addition & 1 deletion python/.cargo/config.toml
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
[env]
GLIDE_NAME = { value = "GlidePy", force = true }
GLIDE_VERSION = "0.1.0"
GLIDE_VERSION = "unknown"
4 changes: 2 additions & 2 deletions python/python/tests/test_async_client.py
Original file line number Diff line number Diff line change
Expand Up @@ -116,7 +116,7 @@ async def test_register_client_name_and_version(self, redis_client: TRedisClient
info = await redis_client.custom_command(["CLIENT", "INFO"])
assert type(info) is str
assert "lib-name=GlidePy" in info
assert "lib-ver=0.1.0" in info
assert "lib-ver=unknown" in info

@pytest.mark.parametrize("cluster_mode", [True, False])
@pytest.mark.parametrize("protocol", [ProtocolVersion.RESP2, ProtocolVersion.RESP3])
Expand Down Expand Up @@ -981,7 +981,7 @@ async def test_expireat_pexpireat_ttl_with_positive_timeout(
# set command clears the timeout.
assert await redis_client.set(key, "bar") == OK
current_time_ms = int(time.time() * 1000)
if not check_if_server_version_lt(redis_client, "7.0.0"):
if not await check_if_server_version_lt(redis_client, "7.0.0"):
assert (
await redis_client.pexpireat(
key, current_time_ms + 50000, ExpireOptions.HasExistingExpiry
Expand Down
8 changes: 4 additions & 4 deletions utils/get_licenses_from_ort.py
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@ def __str__(self):
]

all_licenses_set: Set = set()
unkown_licenses: List[PackageLicense] = []
unknown_licenses: List[PackageLicense] = []

for ort_result in ort_results_per_lang:
with open(ort_result.analyzer_result_file, "r") as ort_results, open(
Expand All @@ -93,7 +93,7 @@ def __str__(self):
final_licenses = [license]
for license in final_licenses:
if license not in APPROVED_LICENSES:
unkown_licenses.append(
unknown_licenses.append(
PackageLicense(package["id"], ort_result.name, license)
)
all_licenses_set.add(license)
Expand All @@ -108,6 +108,6 @@ def __str__(self):
for license in all_licenses_set:
print(f"{license}")

print("\n\n#### Unkown / Not Pre-Approved Licenses #####\n")
for package in unkown_licenses:
print("\n\n#### unknown / Not Pre-Approved Licenses #####\n")
for package in unknown_licenses:
print(str(package))
Loading