Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Workbench session image promotion #876

Merged
merged 32 commits into from
Nov 27, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
ab27c77
add workbench-session-init target for release builds
skyeturriff Nov 1, 2024
3f4580d
add github workflow jobs
skyeturriff Nov 1, 2024
7f2f366
Merge branch 'dev' into workbench-session-init-release-build
bschwedler Nov 6, 2024
f354d51
Fixup for gha job indentation
bschwedler Nov 7, 2024
00cb98c
Use env var for WORKBENCH_SESSION_INIT_VERSION or default to `get-ver…
ianpittwood Nov 15, 2024
deb3a6e
Merge pull request #870 from rstudio/workbench-session-init-release-b…
ianpittwood Nov 15, 2024
f2049d9
Merge pull request #863 from rstudio/workbench-session-init-release-b…
skyeturriff Nov 15, 2024
fa19299
Add workbench-session, starting with r-session-complete
zachhannum Nov 15, 2024
f2db0fa
Update build-manual.yaml
skyeturriff Nov 15, 2024
2487ad4
Merge pull request #872 from rstudio/skyeturriff-patch-1
skyeturriff Nov 15, 2024
f184da8
Remove RSW from Workbench Session
zachhannum Nov 15, 2024
5d9c081
Add workbench session target/matrix
zachhannum Nov 19, 2024
0a988be
Put Quarto on path in earlier layer
zachhannum Nov 19, 2024
016e1da
Set QUARTO_VERSION as arg
zachhannum Nov 19, 2024
ec39cda
Fix goss tests
zachhannum Nov 20, 2024
22c0011
Update quarto on PATH
zachhannum Nov 20, 2024
644d2d4
Merge versions job into workbench-session-init job
ianpittwood Nov 20, 2024
b04cd5d
Remove versions as dependency job
ianpittwood Nov 20, 2024
6808d62
Merge pull request #875 from rstudio/merge-versions-job
ianpittwood Nov 20, 2024
6363aef
Add Go-based entrypoint for Workbench session init (#871)
skyeturriff Nov 20, 2024
cf85df2
Don't install tinytex
zachhannum Nov 21, 2024
d6fc4d2
Quarto isn't in /usr/local/bin
zachhannum Nov 21, 2024
2944d3b
Merge branch 'dev' into feature/workbench-session
zachhannum Nov 21, 2024
dcfdba1
Update workbench-session/README.md
zachhannum Nov 22, 2024
b318c93
Use the correct build matrix
zachhannum Nov 22, 2024
22974a2
Add check for pip installation in goss tests
zachhannum Nov 22, 2024
c1fed68
COPY positron.extensions.conf
zachhannum Nov 26, 2024
a0bf31f
Remove Snyk policy file from workbench-session
zachhannum Nov 26, 2024
27864ce
Fix missing colons in goss tests
zachhannum Nov 26, 2024
3277b3d
Merge pull request #874 from rstudio/feature/workbench-session
zachhannum Nov 26, 2024
7b0d352
Update build-bake.yaml
skyeturriff Nov 27, 2024
1d62002
Merge pull request #878 from rstudio/skyeturriff-patch-1
bschwedler Nov 27, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
106 changes: 106 additions & 0 deletions .github/workflows/build-bake.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -290,6 +290,112 @@ jobs:
snyk-org: ${{ secrets.SNYK_ORG }}
snyk-token: '${{ secrets.SNYK_TOKEN }}'

workbench-session:
needs: [setup]
name: Workbench Session
runs-on: ubuntu-latest-8x

concurrency:
group: bake-workbench-session-${{ github.ref }}
cancel-in-progress: true

env:
target: workbench-session
GIT_SHA: ${{ needs.setup.outputs.GIT_SHA }}

steps:
- name: Checkout
if: github.event_name == 'schedule'
uses: actions/checkout@v4
with:
ref: 'main'

- name: Checkout
if: github.event_name != 'schedule'
uses: actions/checkout@v4

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
id: setup-buildx
with:
buildkitd-config: ./share/buildkitd.toml

- name: Build, Test, and Push
uses: ./.github/actions/bake-test-push
with:
target: ${{ env.target }}
push-image: ${{ github.ref == 'refs/heads/main' || github.event_name == 'schedule' }}
ghcr-token: ${{ secrets.GITHUB_TOKEN }}
dockerhub-username: ${{ secrets.DOCKER_HUB_USERNAME }}
dockerhub-token: ${{ secrets.DOCKER_HUB_ACCESS_TOKEN }}
gcp-json: '${{ secrets.GCP_ARTIFACT_REGISTRY_JSON }}'
snyk-org: ${{ secrets.SNYK_ORG }}
snyk-token: '${{ secrets.SNYK_TOKEN }}'

workbench-session-init:
if: false # TODO: Re-enable this job after next release
needs: [setup]
name: Workbench Session Init
runs-on: ubuntu-latest-8x

concurrency:
group: bake-workbench-session-init-${{ github.ref }}
cancel-in-progress: true

env:
target: workbench-session-init
GIT_SHA: ${{ needs.setup.outputs.GIT_SHA }}

steps:
- name: Checkout
if: github.event_name == 'schedule'
uses: actions/checkout@v4
with:
ref: 'main'

- name: Checkout
if: github.event_name != 'schedule'
uses: actions/checkout@v4

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
id: setup-buildx
with:
buildkitd-config: ./share/buildkitd.toml

- name: Set up Just
uses: extractions/setup-just@v2
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

- uses: actions/setup-python@v5
with:
python-version: '3.12'

- name: Install Python dependencies
run: |
pip install requests

- name: Get Version
id: get-version
run: |
WORKBENCH_SESSION_INIT_VERSION=$(just -f ci.Justfile get-version workbench --type=daily --local)
echo "WORKBENCH_SESSION_INIT_VERSION=$WORKBENCH_SESSION_INIT_VERSION" >> $GITHUB_OUTPUT

- name: Build, Test, and Push
env:
WORKBENCH_SESSION_INIT_VERSION: ${{ steps.get-version.outputs.WORKBENCH_SESSION_INIT_VERSION }}
uses: ./.github/actions/bake-test-push
with:
target: ${{ env.target }}
push-image: ${{ github.ref == 'refs/heads/main' || github.event_name == 'schedule' }}
ghcr-token: ${{ secrets.GITHUB_TOKEN }}
dockerhub-username: ${{ secrets.DOCKER_HUB_USERNAME }}
dockerhub-token: ${{ secrets.DOCKER_HUB_ACCESS_TOKEN }}
gcp-json: '${{ secrets.GCP_ARTIFACT_REGISTRY_JSON }}'
snyk-org: ${{ secrets.SNYK_ORG }}
snyk-token: '${{ secrets.SNYK_TOKEN }}'

workbench:
needs: [setup]
name: Workbench
Expand Down
2 changes: 2 additions & 0 deletions .github/workflows/build-manual.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -141,6 +141,8 @@ jobs:
product="CONNECT"
elif [[ "$product" == "package-manager" ]]; then
product="PACKAGE_MANAGER"
elif [[ "$product" == "workbench-session-init" ]]; then
product="WORKBENCH_SESSION_INIT"
else
product="WORKBENCH"
fi
Expand Down
13 changes: 11 additions & 2 deletions Justfile
Original file line number Diff line number Diff line change
Expand Up @@ -49,13 +49,17 @@ delete-builder:
alias build := bake
# just bake workbench-images
bake target="default":
#!/bin/bash
just -f {{justfile()}} create-builder || true
if [ -z "$WORKBENCH_SESSION_INIT_VERSION" ]; then
WORKBENCH_SESSION_INIT_VERSION=$(just -f ci.Justfile get-version workbench --type=daily --local)
fi
GIT_SHA=$(git rev-parse --short HEAD) \
WORKBENCH_SESSION_INIT_VERSION=${WORKBENCH_SESSION_INIT_VERSION} \
docker buildx bake --builder=posit-builder -f docker-bake.hcl {{target}}

# just preview-bake workbench-images dev
preview-build:
just preview-bake "default"
alias preview-build := preview-bake
preview-bake target branch="$(git branch --show-current)":
just -f {{justfile()}} create-builder || true
WORKBENCH_DAILY_VERSION=$(just -f ci.Justfile get-version workbench --type=daily --local) \
Expand Down Expand Up @@ -89,7 +93,12 @@ preview-plan branch="$(git branch --show-current)":

# just test workbench
test target="default" file="docker-bake.hcl":
#!/bin/bash
if [ -z "$WORKBENCH_SESSION_INIT_VERSION" ]; then
WORKBENCH_SESSION_INIT_VERSION=$(just -f ci.Justfile get-version workbench --type=daily --local)
fi
GIT_SHA=$(git rev-parse --short HEAD) \
WORKBENCH_SESSION_INIT_VERSION=${WORKBENCH_SESSION_INIT_VERSION} \
python3 {{justfile_directory()}}/tools/test_bake_artifacts.py --target "{{target}}" --file "{{file}}"

# just preview-test connect dev
Expand Down
60 changes: 60 additions & 0 deletions docker-bake.hcl
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,10 @@ variable WORKBENCH_VERSION {
default = "2024.09.1+394.pro7"
}

variable WORKBENCH_SESSION_INIT_VERSION {
default = ""
}

variable DRIVERS_VERSION {
default = "2024.03.0"
}
Expand Down Expand Up @@ -181,6 +185,18 @@ variable WORKBENCH_BUILD_MATRIX {
}
}

variable WORKBENCH_SESSION_MATRIX {
default = PRO_BUILD_MATRIX
}

variable WORKBENCH_SESSION_INIT_BUILD_MATRIX {
default = {
builds = [
{os = "ubuntu2204"},
]
}
}

variable WORKBENCH_GOOGLE_CLOUD_WORKSTATION_BUILD_MATRIX {
default = {
builds = [
Expand Down Expand Up @@ -209,6 +225,8 @@ group "default" {
"package-manager",
"r-session-complete",
"workbench",
"workbench-session",
"workbench-session-init",
bschwedler marked this conversation as resolved.
Show resolved Hide resolved
]
}

Expand Down Expand Up @@ -428,6 +446,31 @@ target "r-session-complete" {
}
}

target "workbench-session" {
inherits = ["base"]
name = "workbench-session-${builds.os}-r${replace(builds.r_primary, ".", "-")}_${replace(builds.r_alternate, ".", "-")}-py${replace(builds.py_primary, ".", "-")}_${replace(builds.py_alternate, ".", "-")}"

tags = [
"ghcr.io/rstudio/workbench-session:${builds.os}-r${builds.r_primary}_${builds.r_alternate}-py${builds.py_primary}_${builds.py_alternate}",
"docker.io/rstudio/workbench-session:${builds.os}-r${builds.r_primary}_${builds.r_alternate}-py${builds.py_primary}_${builds.py_alternate}",
]

dockerfile = "Dockerfile.${builds.os}"
context = "workbench-session"
contexts = {
product-base-pro = "target:product-base-pro-${builds.os}-r${replace(builds.r_primary, ".", "-")}_${replace(builds.r_alternate, ".", "-")}-py${replace(builds.py_primary, ".", "-")}_${replace(builds.py_alternate, ".", "-")}"
}

matrix = WORKBENCH_SESSION_MATRIX
args = {
R_VERSION = builds.r_primary
R_VERSION_ALT = builds.r_alternate
PYTHON_VERSION = builds.py_primary
PYTHON_VERSION_ALT = builds.py_alternate
JUPYTERLAB_VERSION = DEFAULT_JUPYTERLAB_VERSION
}
}

target "workbench" {
inherits = ["base"]

Expand All @@ -453,6 +496,23 @@ target "workbench" {
}
}

target "workbench-session-init" {
inherits = ["base"]
target = "build"

name = "workbench-session-init-${builds.os}-${replace(tag_safe_version(WORKBENCH_SESSION_INIT_VERSION), ".", "-")}"
tags = get_tags(builds.os, "workbench-session-init", WORKBENCH_SESSION_INIT_VERSION)

dockerfile = "Dockerfile.${builds.os}"
context = "workbench-session-init"

matrix = WORKBENCH_SESSION_INIT_BUILD_MATRIX

args = {
RSW_VERSION = WORKBENCH_SESSION_INIT_VERSION
}
}

### Workbench for Google Cloud Workstations targets ###
target "workbench-for-google-cloud-workstations" {
inherits = ["base"]
Expand Down
39 changes: 33 additions & 6 deletions workbench-session-init/Dockerfile.ubuntu2204
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
FROM ubuntu:22.04 AS build
FROM ubuntu:22.04 AS builder

# Install required tools:
# - ca-certificates installs necessary certificates to use cURL with HTTPS websites
Expand All @@ -9,17 +9,44 @@ RUN apt-get update && \
rm -rf /var/lib/apt/lists/*

ARG RSW_VERSION=2024.09.1+394.pro7
ARG GO_VERSION=1.22.2

# Download the RStudio Workbench session components and install Go
SHELL ["/bin/bash", "-o", "pipefail", "-c"]
RUN mkdir -p /pwb-staging && \
RSW_VERSION_URL=$(echo -n "${RSW_VERSION}" | sed 's/+/-/g') && \
echo "Downloading https://s3.amazonaws.com/rstudio-ide-build/session/multi/x86_64/rsp-session-multi-linux-${RSW_VERSION_URL}-x86_64.tar.gz" && \
curl -fsSL -o /pwb-staging/rsp-session-multi-linux.tar.gz "https://s3.amazonaws.com/rstudio-ide-build/session/multi/x86_64/rsp-session-multi-linux-${RSW_VERSION_URL}-x86_64.tar.gz" && \
mkdir -p /opt/session-components && \
tar -C /opt/session-components -xf /pwb-staging/rsp-session-multi-linux.tar.gz && \
chmod -R 755 /opt/session-components && \
tar -C /opt/session-components -xpf /pwb-staging/rsp-session-multi-linux.tar.gz && \
chmod 755 /opt/session-components && \
curl -fsSL -o /pwb-staging/go.tar.gz "https://go.dev/dl/go${GO_VERSION}.linux-amd64.tar.gz" && \
tar -C /usr/local -xf /pwb-staging/go.tar.gz && \
rm -rf /pwb-staging

COPY --chmod=755 run.sh /usr/local/bin/run.sh
# Add Go binary to PATH
ENV PATH="/usr/local/go/bin:$PATH"

# Set the Go workspace
WORKDIR /workspace

# Copy the Go source code and download dependencies
COPY entrypoint/go.mod entrypoint/go.sum ./
RUN go mod download

# Copy the Go source code and build the binary
COPY entrypoint/main.go .
RUN CGO_ENABLED=0 GOOS=linux go build -ldflags '-s -w' -o entrypoint main.go

# Create the final image
FROM ubuntu:22.04 AS build

ARG DEBIAN_FRONTEND=noninteractive
RUN apt-get update && \
apt-get install -y --no-install-recommends ca-certificates curl && \
rm -rf /var/lib/apt/lists/*

# Copy the compiled Go binary and session components from the builder stage
COPY --from=builder --chmod=755 /workspace/entrypoint /usr/local/bin/entrypoint
COPY --from=builder --chmod=755 /opt/session-components /opt/session-components

ENTRYPOINT ["/usr/local/bin/run.sh"]
ENTRYPOINT ["/usr/local/bin/entrypoint"]
16 changes: 11 additions & 5 deletions workbench-session-init/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,16 +7,22 @@ This directory contains a Dockerfile and script that will create an init contain
* Maintained by: [the Posit Docker team](https://github.com/rstudio/rstudio-docker-products)
* Where to get help: [our Github Issues page](https://github.com/rstudio/rstudio-docker-products/issues)
* Posit Workbench image: [Docker Hub](https://hub.docker.com/r/rstudio/rstudio-workbench)
* RStudio r-session-complete image: [Docker Hub](https://hub.docker.com/r/rstudio/r-session-complete)
* Workbench Session Init image (Daily/Preview): [Docker Hub](https://hub.docker.com/r/rstudio/rstudio-workbench-session-init-preview)
* Posit Workbench session image: [Docker Hub](https://hub.docker.com/r/rstudio/workbench-session)
* Posit Workbench session init image: [Docker Hub](https://hub.docker.com/r/rstudio/workbench-session-init)

## Supported tags and respective Dockerfile links

* [`jammy-daily`, `ubuntu2204-daily`, `jammy-2024.11.0`, `ubuntu2204-2024.11.0`](https://github.com/rstudio/rstudio-docker-products/blob/main/workbench-session-init/Dockerfile.2204)

## Building

Currently daily builds are supported. To build the image, run:
Just will build an image using a default Connect distribution.

```console
just build
```

Daily builds are also supported. To build the daily image, run:

```console
just preview-bake workbench-session-init-daily
Expand All @@ -28,14 +34,14 @@ You can observe what gets copied by the container:

```console
mkdir init
docker run --rm -v $(pwd)/init:/mnt/init rstudio/workbench-session-init-preview:workbench-session-init-jammy-2024.11.0-daily-328.pro3
docker run --rm -v $(pwd)/init:/mnt/init rstudio/workbench-session-init:jammy-2024.11.0
# The init directory has been populated with the Workbench session runtime components.
```

You can also test using GOSS:

```console
just preview-test workbench-session-init-daily
just test workbench-session-init
```

## Licensing
Expand Down
9 changes: 9 additions & 0 deletions workbench-session-init/entrypoint/go.mod
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
module entrypoint

go 1.22.2

require (
github.com/otiai10/copy v1.14.0 // indirect
golang.org/x/sync v0.3.0 // indirect
golang.org/x/sys v0.0.0-20220715151400-c0bba94af5f8 // indirect
)
6 changes: 6 additions & 0 deletions workbench-session-init/entrypoint/go.sum
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
github.com/otiai10/copy v1.14.0 h1:dCI/t1iTdYGtkvCuBG2BgR6KZa83PTclw4U5n2wAllU=
github.com/otiai10/copy v1.14.0/go.mod h1:ECfuL02W+/FkTWZWgQqXPWZgW9oeKCSQ5qVfSc4qc4w=
golang.org/x/sync v0.3.0 h1:ftCYgMx6zT/asHUrPw8BLLscYtGznsLAnjq5RH9P66E=
golang.org/x/sync v0.3.0/go.mod h1:FU7BRWz2tNW+3quACPkgCx/L+uEAv1htQ0V83Z9Rj+Y=
golang.org/x/sys v0.0.0-20220715151400-c0bba94af5f8 h1:0A+M6Uqn+Eje4kHMK80dtF3JCXC4ykBgQG4Fe06QRhQ=
golang.org/x/sys v0.0.0-20220715151400-c0bba94af5f8/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
Loading
Loading