Skip to content

Commit

Permalink
Revert "remove incorrect end-to-end implementation"
Browse files Browse the repository at this point in the history
This reverts commit ebc5e31 from instructlab#76.

Here is my comment on instructlab#76 that better explains how this workflow fits
in to the bigger picture:

> Testing the user end-to-end instructlab workflow is the point of
> this workflow. It will give you an early indication if your changes to
> the library break the CLI. Sometimes that might be on purpose, but in
> case it's not, it will let you know before you publish a release.
>
> This is not intended to replace other functional testing aimed at the
> library more directly, but the full end-to-end workflow is important.
>
> I'm OK if you want to disable it for the moment, but it needs to go
> back on at some point. A better way to do that is to just edit the
> workflow to turn it off instead of removing it from the repo.

Issue instructlab#63

Signed-off-by: Russell Bryant <rbryant@redhat.com>
  • Loading branch information
russellb committed Jun 26, 2024
1 parent 60dc322 commit f9bfcee
Show file tree
Hide file tree
Showing 2 changed files with 137 additions and 0 deletions.
92 changes: 92 additions & 0 deletions .github/workflows/e2e.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,92 @@
# SPDX-License-Identifier: Apache-2.0

name: E2E test

on:
push:
branches:
- "main"
- "release-**"
paths:
- '**.py'
- 'pyproject.toml'
- 'requirements*.txt'
- '.github/workflows/e2e.yml'
pull_request:
branches:
- "main"
- "release-**"
paths:
- '**.py'
- 'pyproject.toml'
- 'requirements*.txt'
- '.github/workflows/e2e.yml'

jobs:
e2e:
runs-on: ubuntu-gpu

permissions:
pull-requests: write

steps:
# No step-security/harden-runner since this is a self-hosted runner
- name: Checkout
uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
with:
# https://github.com/actions/checkout/issues/249
fetch-depth: 0

- name: Install Packages
run: |
sudo apt-get install -y cuda-toolkit git cmake build-essential virtualenv
nvidia-smi
ls -l /dev/nvidia*
- name: Setup Python 3.11
uses: actions/setup-python@82c7e631bb3cdc910f68e0081d67478d79c6982d # v5.1.0
with:
python-version: 3.11
cache: pip
cache-dependency-path: |
**/pyproject.toml
**/requirements*.txt
- name: Remove llama-cpp-python from cache
run: |
pip cache remove llama_cpp_python
- name: Cache huggingface
uses: actions/cache@0c45773b623bea8c8e75f6c82b208c3cf94ea4f9 # v4.0.2
with:
path: ~/.cache/huggingface
# config contains DEFAULT_MODEL
key: huggingface-${{ hashFiles('src/instructlab/configuration.py') }}

- name: Install instructlab and libraries
run: |
export PATH="/home/runner/.local/bin:/usr/local/cuda/bin:$PATH"
python3 -m venv venv
. venv/bin/activate
git clone https://github.com/instructlab/instructlab.git
cd instructlab
sed 's/\[.*\]//' requirements.txt > constraints.txt
python3 -m pip cache remove llama_cpp_python
CMAKE_ARGS="-DLLAMA_CUBLAS=on" python3 -m pip install --no-binary llama_cpp_python -c constraints.txt llama_cpp_python
# needed for --4-bit-quant option to ilab train
python3 -m pip install bitsandbytes
# install instructlab
python3 -m pip install .
cd ..
# Install instructlab-sdg
python3 -m pip install .
- name: Run e2e test
run: |
. venv/bin/activate
./instructlab/scripts/basic-workflow-tests.sh -cm
- name: Remove llama-cpp-python from cache
if: always()
run: |
pip cache remove llama_cpp_python
45 changes: 45 additions & 0 deletions .github/workflows/spellcheck.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
# SPDX-License-Identifier: Apache-2.0

name: Spellcheck

on:
push:
branches:
- "main"
paths:
- '**.md'
- '.github/workflows/spellcheck.yml' # This workflow
pull_request:
branches:
- "main"
paths:
- '**.md'
- '.github/workflows/spellcheck.yml' # This workflow

env:
LC_ALL: en_US.UTF-8

defaults:
run:
shell: bash

permissions:
contents: read

jobs:
spellcheck:
name: Spellcheck (en_US)
runs-on: ubuntu-latest
steps:
- name: "Harden Runner"
uses: step-security/harden-runner@17d0e2bd7d51742c71671bd19fa12bdc9d40a3d6 # v2.8.1
with:
egress-policy: audit # TODO: change to 'egress-policy: block' after couple of runs

- name: "Checkout"
uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
with:
fetch-depth: 0

- name: Spellcheck
uses: rojopolis/spellcheck-github-actions@d354a4dc525c8067555c7481b60416cedb0060ff # v0.38.0

0 comments on commit f9bfcee

Please sign in to comment.