Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/workbench session #877

Closed
wants to merge 23 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
8ad7c35
add workbench-session-init target for release builds
skyeturriff Nov 1, 2024
a55a0a7
add github workflow jobs
skyeturriff Nov 1, 2024
b6eb895
Fixup for gha job indentation
bschwedler Nov 7, 2024
2015b18
Use env var for WORKBENCH_SESSION_INIT_VERSION or default to `get-ver…
ianpittwood Nov 15, 2024
8eb8964
Add workbench-session, starting with r-session-complete
zachhannum Nov 15, 2024
0ecedaa
Remove RSW from Workbench Session
zachhannum Nov 15, 2024
323250f
Add workbench session target/matrix
zachhannum Nov 19, 2024
e5090aa
Put Quarto on path in earlier layer
zachhannum Nov 19, 2024
e1ebcea
Set QUARTO_VERSION as arg
zachhannum Nov 19, 2024
3657dc6
Fix goss tests
zachhannum Nov 20, 2024
3a5b990
Update quarto on PATH
zachhannum Nov 20, 2024
298aaf3
Don't install tinytex
zachhannum Nov 21, 2024
7d533e6
Quarto isn't in /usr/local/bin
zachhannum Nov 21, 2024
7871f1c
Update build-manual.yaml
skyeturriff Nov 15, 2024
cd7fe09
Merge versions job into workbench-session-init job
ianpittwood Nov 20, 2024
afed1ba
Remove versions as dependency job
ianpittwood Nov 20, 2024
773522f
Add Go-based entrypoint for Workbench session init (#871)
skyeturriff Nov 20, 2024
4720f10
Update workbench-session/README.md
zachhannum Nov 22, 2024
7179f13
Use the correct build matrix
zachhannum Nov 22, 2024
da27613
Add check for pip installation in goss tests
zachhannum Nov 22, 2024
cb4f0aa
COPY positron.extensions.conf
zachhannum Nov 26, 2024
bd0ae19
Remove Snyk policy file from workbench-session
zachhannum Nov 26, 2024
f46c966
Fix missing colons in goss tests
zachhannum Nov 26, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
105 changes: 105 additions & 0 deletions .github/workflows/build-bake.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -290,6 +290,111 @@ jobs:
snyk-org: ${{ secrets.SNYK_ORG }}
snyk-token: '${{ secrets.SNYK_TOKEN }}'

workbench-session:
needs: [setup]
name: Workbench Session
runs-on: ubuntu-latest-8x

concurrency:
group: bake-workbench-session-${{ github.ref }}
cancel-in-progress: true

env:
target: workbench-session
GIT_SHA: ${{ needs.setup.outputs.GIT_SHA }}

steps:
- name: Checkout
if: github.event_name == 'schedule'
uses: actions/checkout@v4
with:
ref: 'main'

- name: Checkout
if: github.event_name != 'schedule'
uses: actions/checkout@v4

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
id: setup-buildx
with:
buildkitd-config: ./share/buildkitd.toml

- name: Build, Test, and Push
uses: ./.github/actions/bake-test-push
with:
target: ${{ env.target }}
push-image: ${{ github.ref == 'refs/heads/main' || github.event_name == 'schedule' }}
ghcr-token: ${{ secrets.GITHUB_TOKEN }}
dockerhub-username: ${{ secrets.DOCKER_HUB_USERNAME }}
dockerhub-token: ${{ secrets.DOCKER_HUB_ACCESS_TOKEN }}
gcp-json: '${{ secrets.GCP_ARTIFACT_REGISTRY_JSON }}'
snyk-org: ${{ secrets.SNYK_ORG }}
snyk-token: '${{ secrets.SNYK_TOKEN }}'

workbench-session-init:
needs: [setup]
name: Workbench Session Init
runs-on: ubuntu-latest-8x

concurrency:
group: bake-workbench-session-init-${{ github.ref }}
cancel-in-progress: true

env:
target: workbench-session-init
GIT_SHA: ${{ needs.setup.outputs.GIT_SHA }}

steps:
- name: Checkout
if: github.event_name == 'schedule'
uses: actions/checkout@v4
with:
ref: 'main'

- name: Checkout
if: github.event_name != 'schedule'
uses: actions/checkout@v4

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
id: setup-buildx
with:
buildkitd-config: ./share/buildkitd.toml

- name: Set up Just
uses: extractions/setup-just@v2
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

- uses: actions/setup-python@v5
with:
python-version: '3.12'

- name: Install Python dependencies
run: |
pip install requests

- name: Get Version
id: get-version
run: |
WORKBENCH_SESSION_INIT_VERSION=$(just -f ci.Justfile get-version workbench --type=daily --local)
echo "WORKBENCH_SESSION_INIT_VERSION=$WORKBENCH_SESSION_INIT_VERSION" >> $GITHUB_OUTPUT

- name: Build, Test, and Push
env:
WORKBENCH_SESSION_INIT_VERSION: ${{ steps.get-version.outputs.WORKBENCH_SESSION_INIT_VERSION }}
uses: ./.github/actions/bake-test-push
with:
target: ${{ env.target }}
push-image: ${{ github.ref == 'refs/heads/main' || github.event_name == 'schedule' }}
ghcr-token: ${{ secrets.GITHUB_TOKEN }}
dockerhub-username: ${{ secrets.DOCKER_HUB_USERNAME }}
dockerhub-token: ${{ secrets.DOCKER_HUB_ACCESS_TOKEN }}
gcp-json: '${{ secrets.GCP_ARTIFACT_REGISTRY_JSON }}'
snyk-org: ${{ secrets.SNYK_ORG }}
snyk-token: '${{ secrets.SNYK_TOKEN }}'

workbench:
needs: [setup]
name: Workbench
Expand Down
2 changes: 2 additions & 0 deletions .github/workflows/build-manual.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -141,6 +141,8 @@ jobs:
product="CONNECT"
elif [[ "$product" == "package-manager" ]]; then
product="PACKAGE_MANAGER"
elif [[ "$product" == "workbench-session-init" ]]; then
product="WORKBENCH_SESSION_INIT"
else
product="WORKBENCH"
fi
Expand Down
13 changes: 11 additions & 2 deletions Justfile
Original file line number Diff line number Diff line change
Expand Up @@ -49,13 +49,17 @@ delete-builder:
alias build := bake
# just bake workbench-images
bake target="default":
#!/bin/bash
just -f {{justfile()}} create-builder || true
if [ -z "$WORKBENCH_SESSION_INIT_VERSION" ]; then
WORKBENCH_SESSION_INIT_VERSION=$(just -f ci.Justfile get-version workbench --type=daily --local)
fi
GIT_SHA=$(git rev-parse --short HEAD) \
WORKBENCH_SESSION_INIT_VERSION=${WORKBENCH_SESSION_INIT_VERSION} \
docker buildx bake --builder=posit-builder -f docker-bake.hcl {{target}}

# just preview-bake workbench-images dev
preview-build:
just preview-bake "default"
alias preview-build := preview-bake
preview-bake target branch="$(git branch --show-current)":
just -f {{justfile()}} create-builder || true
WORKBENCH_DAILY_VERSION=$(just -f ci.Justfile get-version workbench --type=daily --local) \
Expand Down Expand Up @@ -89,7 +93,12 @@ preview-plan branch="$(git branch --show-current)":

# just test workbench
test target="default" file="docker-bake.hcl":
#!/bin/bash
if [ -z "$WORKBENCH_SESSION_INIT_VERSION" ]; then
WORKBENCH_SESSION_INIT_VERSION=$(just -f ci.Justfile get-version workbench --type=daily --local)
fi
GIT_SHA=$(git rev-parse --short HEAD) \
WORKBENCH_SESSION_INIT_VERSION=${WORKBENCH_SESSION_INIT_VERSION} \
python3 {{justfile_directory()}}/tools/test_bake_artifacts.py --target "{{target}}" --file "{{file}}"

# just preview-test connect dev
Expand Down
60 changes: 60 additions & 0 deletions docker-bake.hcl
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,10 @@ variable WORKBENCH_VERSION {
default = "2024.09.1+394.pro7"
}

variable WORKBENCH_SESSION_INIT_VERSION {
default = ""
}

variable DRIVERS_VERSION {
default = "2024.03.0"
}
Expand Down Expand Up @@ -181,6 +185,18 @@ variable WORKBENCH_BUILD_MATRIX {
}
}

variable WORKBENCH_SESSION_MATRIX {
default = PRO_BUILD_MATRIX
}

variable WORKBENCH_SESSION_INIT_BUILD_MATRIX {
default = {
builds = [
{os = "ubuntu2204"},
]
}
}

variable WORKBENCH_GOOGLE_CLOUD_WORKSTATION_BUILD_MATRIX {
default = {
builds = [
Expand Down Expand Up @@ -209,6 +225,8 @@ group "default" {
"package-manager",
"r-session-complete",
"workbench",
"workbench-session",
"workbench-session-init",
]
}

Expand Down Expand Up @@ -428,6 +446,31 @@ target "r-session-complete" {
}
}

target "workbench-session" {
inherits = ["base"]
name = "workbench-session-${builds.os}-r${replace(builds.r_primary, ".", "-")}_${replace(builds.r_alternate, ".", "-")}-py${replace(builds.py_primary, ".", "-")}_${replace(builds.py_alternate, ".", "-")}"

tags = [
"ghcr.io/rstudio/workbench-session:${builds.os}-r${builds.r_primary}_${builds.r_alternate}-py${builds.py_primary}_${builds.py_alternate}",
"docker.io/rstudio/workbench-session:${builds.os}-r${builds.r_primary}_${builds.r_alternate}-py${builds.py_primary}_${builds.py_alternate}",
]

dockerfile = "Dockerfile.${builds.os}"
context = "workbench-session"
contexts = {
product-base-pro = "target:product-base-pro-${builds.os}-r${replace(builds.r_primary, ".", "-")}_${replace(builds.r_alternate, ".", "-")}-py${replace(builds.py_primary, ".", "-")}_${replace(builds.py_alternate, ".", "-")}"
}

matrix = WORKBENCH_SESSION_MATRIX
args = {
R_VERSION = builds.r_primary
R_VERSION_ALT = builds.r_alternate
PYTHON_VERSION = builds.py_primary
PYTHON_VERSION_ALT = builds.py_alternate
JUPYTERLAB_VERSION = DEFAULT_JUPYTERLAB_VERSION
}
}

target "workbench" {
inherits = ["base"]

Expand All @@ -453,6 +496,23 @@ target "workbench" {
}
}

target "workbench-session-init" {
inherits = ["base"]
target = "build"

name = "workbench-session-init-${builds.os}-${replace(tag_safe_version(WORKBENCH_SESSION_INIT_VERSION), ".", "-")}"
tags = get_tags(builds.os, "workbench-session-init", WORKBENCH_SESSION_INIT_VERSION)

dockerfile = "Dockerfile.${builds.os}"
context = "workbench-session-init"

matrix = WORKBENCH_SESSION_INIT_BUILD_MATRIX

args = {
RSW_VERSION = WORKBENCH_SESSION_INIT_VERSION
}
}

### Workbench for Google Cloud Workstations targets ###
target "workbench-for-google-cloud-workstations" {
inherits = ["base"]
Expand Down
39 changes: 33 additions & 6 deletions workbench-session-init/Dockerfile.ubuntu2204
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
FROM ubuntu:22.04 AS build
FROM ubuntu:22.04 AS builder

# Install required tools:
# - ca-certificates installs necessary certificates to use cURL with HTTPS websites
Expand All @@ -9,17 +9,44 @@ RUN apt-get update && \
rm -rf /var/lib/apt/lists/*

ARG RSW_VERSION=2024.09.1+394.pro7
ARG GO_VERSION=1.22.2

# Download the RStudio Workbench session components and install Go
SHELL ["/bin/bash", "-o", "pipefail", "-c"]
RUN mkdir -p /pwb-staging && \
RSW_VERSION_URL=$(echo -n "${RSW_VERSION}" | sed 's/+/-/g') && \
echo "Downloading https://s3.amazonaws.com/rstudio-ide-build/session/multi/x86_64/rsp-session-multi-linux-${RSW_VERSION_URL}-x86_64.tar.gz" && \
curl -fsSL -o /pwb-staging/rsp-session-multi-linux.tar.gz "https://s3.amazonaws.com/rstudio-ide-build/session/multi/x86_64/rsp-session-multi-linux-${RSW_VERSION_URL}-x86_64.tar.gz" && \
mkdir -p /opt/session-components && \
tar -C /opt/session-components -xf /pwb-staging/rsp-session-multi-linux.tar.gz && \
chmod -R 755 /opt/session-components && \
tar -C /opt/session-components -xpf /pwb-staging/rsp-session-multi-linux.tar.gz && \
chmod 755 /opt/session-components && \
curl -fsSL -o /pwb-staging/go.tar.gz "https://go.dev/dl/go${GO_VERSION}.linux-amd64.tar.gz" && \
tar -C /usr/local -xf /pwb-staging/go.tar.gz && \
rm -rf /pwb-staging

COPY --chmod=755 run.sh /usr/local/bin/run.sh
# Add Go binary to PATH
ENV PATH="/usr/local/go/bin:$PATH"

# Set the Go workspace
WORKDIR /workspace

# Copy the Go source code and download dependencies
COPY entrypoint/go.mod entrypoint/go.sum ./
RUN go mod download

# Copy the Go source code and build the binary
COPY entrypoint/main.go .
RUN CGO_ENABLED=0 GOOS=linux go build -ldflags '-s -w' -o entrypoint main.go

# Create the final image
FROM ubuntu:22.04 AS build

ARG DEBIAN_FRONTEND=noninteractive
RUN apt-get update && \
apt-get install -y --no-install-recommends ca-certificates curl && \
rm -rf /var/lib/apt/lists/*

# Copy the compiled Go binary and session components from the builder stage
COPY --from=builder --chmod=755 /workspace/entrypoint /usr/local/bin/entrypoint
COPY --from=builder --chmod=755 /opt/session-components /opt/session-components

ENTRYPOINT ["/usr/local/bin/run.sh"]
ENTRYPOINT ["/usr/local/bin/entrypoint"]
16 changes: 11 additions & 5 deletions workbench-session-init/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,16 +7,22 @@ This directory contains a Dockerfile and script that will create an init contain
* Maintained by: [the Posit Docker team](https://github.com/rstudio/rstudio-docker-products)
* Where to get help: [our Github Issues page](https://github.com/rstudio/rstudio-docker-products/issues)
* Posit Workbench image: [Docker Hub](https://hub.docker.com/r/rstudio/rstudio-workbench)
* RStudio r-session-complete image: [Docker Hub](https://hub.docker.com/r/rstudio/r-session-complete)
* Workbench Session Init image (Daily/Preview): [Docker Hub](https://hub.docker.com/r/rstudio/rstudio-workbench-session-init-preview)
* Posit Workbench session image: [Docker Hub](https://hub.docker.com/r/rstudio/workbench-session)
* Posit Workbench session init image: [Docker Hub](https://hub.docker.com/r/rstudio/workbench-session-init)

## Supported tags and respective Dockerfile links

* [`jammy-daily`, `ubuntu2204-daily`, `jammy-2024.11.0`, `ubuntu2204-2024.11.0`](https://github.com/rstudio/rstudio-docker-products/blob/main/workbench-session-init/Dockerfile.2204)

## Building

Currently daily builds are supported. To build the image, run:
Just will build an image using a default Connect distribution.

```console
just build
```

Daily builds are also supported. To build the daily image, run:

```console
just preview-bake workbench-session-init-daily
Expand All @@ -28,14 +34,14 @@ You can observe what gets copied by the container:

```console
mkdir init
docker run --rm -v $(pwd)/init:/mnt/init rstudio/workbench-session-init-preview:workbench-session-init-jammy-2024.11.0-daily-328.pro3
docker run --rm -v $(pwd)/init:/mnt/init rstudio/workbench-session-init:jammy-2024.11.0
# The init directory has been populated with the Workbench session runtime components.
```

You can also test using GOSS:

```console
just preview-test workbench-session-init-daily
just test workbench-session-init
```

## Licensing
Expand Down
9 changes: 9 additions & 0 deletions workbench-session-init/entrypoint/go.mod
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
module entrypoint

go 1.22.2

require (
github.com/otiai10/copy v1.14.0 // indirect
golang.org/x/sync v0.3.0 // indirect
golang.org/x/sys v0.0.0-20220715151400-c0bba94af5f8 // indirect
)
6 changes: 6 additions & 0 deletions workbench-session-init/entrypoint/go.sum
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
github.com/otiai10/copy v1.14.0 h1:dCI/t1iTdYGtkvCuBG2BgR6KZa83PTclw4U5n2wAllU=
github.com/otiai10/copy v1.14.0/go.mod h1:ECfuL02W+/FkTWZWgQqXPWZgW9oeKCSQ5qVfSc4qc4w=
golang.org/x/sync v0.3.0 h1:ftCYgMx6zT/asHUrPw8BLLscYtGznsLAnjq5RH9P66E=
golang.org/x/sync v0.3.0/go.mod h1:FU7BRWz2tNW+3quACPkgCx/L+uEAv1htQ0V83Z9Rj+Y=
golang.org/x/sys v0.0.0-20220715151400-c0bba94af5f8 h1:0A+M6Uqn+Eje4kHMK80dtF3JCXC4ykBgQG4Fe06QRhQ=
golang.org/x/sys v0.0.0-20220715151400-c0bba94af5f8/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
Loading
Loading