Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/workbench session #874

Merged
merged 16 commits into from
Nov 26, 2024
Merged
Show file tree
Hide file tree
Changes from 13 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
42 changes: 42 additions & 0 deletions .github/workflows/build-bake.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -290,6 +290,48 @@ jobs:
snyk-org: ${{ secrets.SNYK_ORG }}
snyk-token: '${{ secrets.SNYK_TOKEN }}'

workbench-session:
needs: [setup]
name: Workbench Session
runs-on: ubuntu-latest-8x

concurrency:
group: bake-workbench-session-${{ github.ref }}
cancel-in-progress: true

env:
target: workbench-session
GIT_SHA: ${{ needs.setup.outputs.GIT_SHA }}

steps:
- name: Checkout
if: github.event_name == 'schedule'
uses: actions/checkout@v4
with:
ref: 'main'

- name: Checkout
if: github.event_name != 'schedule'
uses: actions/checkout@v4

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
id: setup-buildx
with:
buildkitd-config: ./share/buildkitd.toml

- name: Build, Test, and Push
uses: ./.github/actions/bake-test-push
with:
target: ${{ env.target }}
push-image: ${{ github.ref == 'refs/heads/main' || github.event_name == 'schedule' }}
ghcr-token: ${{ secrets.GITHUB_TOKEN }}
dockerhub-username: ${{ secrets.DOCKER_HUB_USERNAME }}
dockerhub-token: ${{ secrets.DOCKER_HUB_ACCESS_TOKEN }}
gcp-json: '${{ secrets.GCP_ARTIFACT_REGISTRY_JSON }}'
snyk-org: ${{ secrets.SNYK_ORG }}
snyk-token: '${{ secrets.SNYK_TOKEN }}'

workbench-session-init:
needs: [setup]
name: Workbench Session Init
Expand Down
30 changes: 30 additions & 0 deletions docker-bake.hcl
Original file line number Diff line number Diff line change
Expand Up @@ -185,6 +185,10 @@ variable WORKBENCH_BUILD_MATRIX {
}
}

variable WORKBENCH_SESSION_MATRIX {
default = PRO_BUILD_MATRIX
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just curious, what's the reasoning behind inheriting all the base image builds?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Similar to content-pro, I didn't see a reason to not offer a few different combinations of R and Python versions for images. I'm happy to be overridden on that decision if it doesn't make sense, though.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, I can understand that. I think this is fine for now. It's something we can take into account for future planning.

}

variable WORKBENCH_SESSION_INIT_BUILD_MATRIX {
default = {
builds = [
Expand Down Expand Up @@ -221,6 +225,7 @@ group "default" {
"package-manager",
"r-session-complete",
"workbench",
"workbench-session",
"workbench-session-init",
]
}
Expand Down Expand Up @@ -441,6 +446,31 @@ target "r-session-complete" {
}
}

target "workbench-session" {
inherits = ["base"]
name = "workbench-session-${builds.os}-r${replace(builds.r_primary, ".", "-")}_${replace(builds.r_alternate, ".", "-")}-py${replace(builds.py_primary, ".", "-")}_${replace(builds.py_alternate, ".", "-")}"

tags = [
"ghcr.io/rstudio/workbench-session:${builds.os}-r${builds.r_primary}_${builds.r_alternate}-py${builds.py_primary}_${builds.py_alternate}",
"docker.io/rstudio/workbench-session:${builds.os}-r${builds.r_primary}_${builds.r_alternate}-py${builds.py_primary}_${builds.py_alternate}",
]

dockerfile = "Dockerfile.${builds.os}"
context = "workbench-session"
contexts = {
product-base-pro = "target:product-base-pro-${builds.os}-r${replace(builds.r_primary, ".", "-")}_${replace(builds.r_alternate, ".", "-")}-py${replace(builds.py_primary, ".", "-")}_${replace(builds.py_alternate, ".", "-")}"
}

matrix = WORKBENCH_SESSION_MATRIX
args = {
R_VERSION = builds.r_primary
R_VERSION_ALT = builds.r_alternate
PYTHON_VERSION = builds.py_primary
PYTHON_VERSION_ALT = builds.py_alternate
JUPYTERLAB_VERSION = DEFAULT_JUPYTERLAB_VERSION
}
}

target "workbench" {
inherits = ["base"]

Expand Down
4 changes: 2 additions & 2 deletions workbench-session-init/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,8 @@ This directory contains a Dockerfile and script that will create an init contain
* Maintained by: [the Posit Docker team](https://github.com/rstudio/rstudio-docker-products)
* Where to get help: [our Github Issues page](https://github.com/rstudio/rstudio-docker-products/issues)
* Posit Workbench image: [Docker Hub](https://hub.docker.com/r/rstudio/rstudio-workbench)
* RStudio r-session-complete image: [Docker Hub](https://hub.docker.com/r/rstudio/r-session-complete)
* Workbench Session Init image: [Docker Hub](https://hub.docker.com/r/rstudio/workbench-session-init)
* Posit Workbench session image: [Docker Hub](https://hub.docker.com/r/rstudio/workbench-session)
* Posit Workbench session init image: [Docker Hub](https://hub.docker.com/r/rstudio/workbench-session-init)

## Supported tags and respective Dockerfile links

Expand Down
3 changes: 3 additions & 0 deletions workbench-session/.env
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
R_VERSION=4.1.0
PYTHON_VERSION=3.9.5
DRIVERS_VERSION=2024.03.0-1
20 changes: 20 additions & 0 deletions workbench-session/.snyk
zachhannum marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
# Snyk (https://snyk.io) policy file, patches or ignores known vulnerabilities.
version: v1.25.0
# ignores vulnerabilities until expiry date; change duration by modifying expiry date
ignore:
SNYK-GOLANG-GITHUBCOMCREWJAMSAML-5971016:
- '*':
reason: >-
Reported upstream in
https://github.com/rstudio/rstudio-pro/issues/6529
expires: 2024-08-31T00:00:00.000Z
created: 2024-07-02T20:33:30.847Z
SNYK-GOLANG-GITHUBCOMGOJOSEGOJOSEV3-6070737:
- '*':
reason: >-
Confirmed fixed upstream in
https://github.com/rstudio/rstudio-pro/issues/6635. Patch will be
ingested in Workbench 2024.08.0 (expected within 1 week).
expires: 2024-08-07T00:00:00.000Z
created: 2024-07-31T17:46:24.852Z
patch: {}
46 changes: 46 additions & 0 deletions workbench-session/Dockerfile.ubuntu2204
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
FROM product-base-pro as build

ARG DEBIAN_FRONTEND=noninteractive
ARG R_VERSION=4.4.0
ARG R_VERSION_ALT=4.3.3
ARG PYTHON_VERSION=3.9.17
ARG PYTHON_VERSION_ALT=3.8.17
ARG JUPYTERLAB_VERSION=3.6.5
ARG SCRIPTS_DIR=/opt/positscripts

ENV WORKBENCH_JUPYTER_PATH=/usr/local/bin/jupyter

SHELL ["/bin/bash", "-o", "pipefail", "-c"]
RUN apt-get update \
&& apt-get install --no-install-recommends -y \
krb5-user \
libcurl4-gnutls-dev \
libuser \
libuser1-dev \
libpq-dev \
rrdtool \
subversion \
&& apt-get autoremove -y \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/*

# Add Jupyter, Python, and Quarto to the PATH
ENV PATH="/opt/python/jupyter/bin:/opt/python/bin:/usr/lib/rstudio-server/bin/quarto/bin:${PATH}"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does the init container install tinytex now?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It does not. The discussion happened in #workbench-dev so I can't link it here, but I'll re-post my notes here:

I don't think the TinyTex installation works at all on the existing r-session-complete images:

zach.hannum@session-0da293c9ab5a7f73c50c8-zachhannum---vs-code-sessionqxv4f:~$ quarto tools
[✓] Inspecting tools

Tool         Status            Installed     Latest  
chromium     Not installed     ---           869685  
tinytex      Not installed     ---           v2024.11

From root:

root@session-0da293c9ab5a7f73c50c8-zachhannum---vs-code-sessionqxv4f:~# which tlmgr
/usr/local/bin/tlmgr

root@session-0da293c9ab5a7f73c50c8-zachhannum---vs-code-sessionqxv4f:~# ls -al /usr/local/bin/tlmgr
lrwxrwxrwx 1 root root 37 Nov 21 12:18 /usr/local/bin/tlmgr -> /root/.TinyTeX/bin/x86_64-linux/tlmgr
zach.hannum@session-0da293c9ab5a7f73c50c8-zachhannum---vs-code-sessionqxv4f:~$ echo $PATH
/usr/lib/rstudio-server/bin/pwb-code-server/bin/remote-cli:/opt/python/jupyter/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin

Back in the session:

zach.hannum@session-0da293c9ab5a7f73c50c8-zachhannum---vs-code-sessionqxv4f:~$ stat /root/.TinyTeX/bin/x86_64-linux/tlmgr
stat: cannot statx '/root/.TinyTeX/bin/x86_64-linux/tlmgr': Permission denied

Okay, so it's on the PATH but we don't actually have the correct permissions to use it.
tinytex doesn't really seem to want to behave as a system dependency, it's installer sticks it in the user's home directory by default.

IMO we should just leave tinytex off the base installation and let users install it to their home directory via quarto if they need it.


RUN /opt/python/"${PYTHON_VERSION}"/bin/python -m venv /opt/python/jupyter \
&& /opt/python/jupyter/bin/python -m pip install --upgrade pip \
&& /opt/python/jupyter/bin/python -m pip install --upgrade setuptools \
&& /opt/python/jupyter/bin/python -m pip install \
jupyterlab~=4.2.4 \
notebook \
pwb_jupyterlab~=1.0 \
&& ln -s /opt/python/jupyter/bin/jupyter /usr/local/bin/jupyter \
&& /opt/python/${PYTHON_VERSION}/bin/python -m pip install ipykernel \
&& /opt/python/${PYTHON_VERSION_ALT}/bin/python -m pip install ipykernel \
&& /opt/python/${PYTHON_VERSION}/bin/python -m ipykernel install --name py${PYTHON_VERSION} --display-name "Python ${PYTHON_VERSION}" \
&& /opt/python/${PYTHON_VERSION_ALT}/bin/python -m ipykernel install --name py${PYTHON_VERSION_ALT} --display-name "Python ${PYTHON_VERSION_ALT}" \
&& /opt/python/jupyter/bin/python -m pip cache purge

COPY vscode.extensions.conf /etc/rstudio/vscode.extensions.conf

EXPOSE 8788/tcp
4 changes: 4 additions & 0 deletions workbench-session/NEWS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
# 2024-11-15

- Add NEWS.md
- Add daily builds
78 changes: 78 additions & 0 deletions workbench-session/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
# Quick reference

* Maintained by: [the Posit Docker team](https://github.com/rstudio/rstudio-docker-products)
* Where to get help: [our Github Issues page](https://github.com/rstudio/rstudio-docker-products/issues)
* Posit Workbench image: [Docker Hub](https://hub.docker.com/r/rstudio/rstudio-workbench)
* Posit Workbench session image: [Docker Hub](https://hub.docker.com/r/rstudio/workbench-session)
* Posit Workbench session init image: [Docker Hub](https://hub.docker.com/r/rstudio/workbench-session-init)

# Supported tags and respective Dockerfile links

* [`ubuntu2204-r4.4.1_4.3.3-py3.12.6_3.11.10`, `ubuntu2204-r4.4.1_4.3.3-py3.11.10_3.10.15`, `ubuntu2204-r4.4.0_4.3.3-py3.12.1_3.11.7`](https://github.com/rstudio/rstudio-docker-products/blob/main/workbench-session/Dockerfile.ubuntu2204)
bschwedler marked this conversation as resolved.
Show resolved Hide resolved

# What are the r-session-complete images?

Images for R and Python sessions and jobs to be used RStudio Workbench, Launcher, and Kubernetes.

# Notice for support

1. This image may introduce **BREAKING** changes; as such we recommend:
- Avoid using the `{operating-system}` tags to avoid unexpected version changes, and
- Always read through the [NEWS](./NEWS.md) to understand the changes before updating.
1. Outdated images will be removed periodically from DockerHub as product version updates are made. Please make plans to
update at times or use your own build of the images.
1. These images are meant as a starting point for your needs. Consider creating a fork of this repo, where you can
continue to merge in changes we make while having your own security scanning, base OS in use, or other custom
changes. We
provide [instructions for how to build and use](#how-to-use-these-docker-images)
for these cases.
1. **Security Note:** These images are provided AS IS based on the build environment at the time their product version was released/updated. They should be reviewed and updated before production use. If your organization has a specific set of security requirements related to CVE/Vulnerability severity levels, you should plan to use the [instructions for building](https://github.com/rstudio/rstudio-docker-products#instructions-for-building) to clone this repository, and rebuild these images to your specific internal security standards.

# How to use these images

The Docker images built from these Dockerfiles are intended to be used for R and
Jupyter sessions and jobs with Posit Workbench (PWB), Launcher, and
Kubernetes.

Note: These Docker images are not equipped or intended to be used to run Posit
Workbench within a Docker container. Visit the
[rstudio/rstudio-worbench Docker Hub page](https://hub.docker.com/r/rstudio/rstudio-workbench)
for images built for that purpose.

Note: These images do not include the Posit Workbench Session Components. To use these images with Posit Workbench, the [session init container](https://hub.docker.com/r/rstudio/workbench-session-init) must be enabled within the Posit Workbench configuration. For more information, refer to the [Posit Workbench documentation](https://docs.rstudio.com/ide/server-pro/launcher/).

For more information about Posit Workbench and Launcher, refer to the
[Launcher Overview](https://solutions.rstudio.com/launcher/overview/) on the
RStudio Solutions website.

For more information about how to use these images with RStudio Workbench and
Launcher, refer to the RStudio support article on [Using Docker images with
RStudio Workbench, Launcher, and Kubernetes](https://support.rstudio.com/hc/en-us/articles/360019253393-Using-Docker-images-with-RStudio-Server-Pro-Launcher-and-Kubernetes).

We provide simple ways to extend and build the Dockerfiles. After you have cloned the repo, you can create your own containers fairly simply with the provided Justfile.

## Overview

Built images are available from the
[rstudio/workbench-session](https://hub.docker.com/r/rstudio/workbench-session)
repository on Docker Hub.

These images include the following layers:

* Base OS
* System packages required for R, R packages, and RStudio Professional Drivers
* Two versions of R
* Two versions of Python
* Jupyter Notebooks, JupyterLab, and RSW/RSC notebook extensions
* RStudio Professional Drivers

# Licensing

The license associated with the RStudio Docker Products repository is located [in LICENSE.md](https://github.com/rstudio/rstudio-docker-products/blob/main/LICENSE.md).

As is the case with all container images, the images themselves also contain other software which may be under other
licenses (i.e. bash, linux, system libraries, etc., along with any other direct or indirect dependencies of the primary
software being contained).

It is an image user's responsibility to ensure that use of this image (and any of its dependent layers) complies with
all relevant licenses for the software contained in the image.
15 changes: 15 additions & 0 deletions workbench-session/docker-compose.test.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
version: '2.3'
services:

sut:
image: $IMAGE_NAME
command: /run_tests.sh
entrypoint: []
environment:
# uses .env by default
- R_VERSION
- PYTHON_VERSION
volumes:
- "./test/run_tests.sh:/run_tests.sh"
- "./test/goss.yaml:/tmp/goss.yaml"
- "./test/goss_vars.yaml:/tmp/goss_vars.yaml"
2 changes: 2 additions & 0 deletions workbench-session/positron.extensions.conf
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
posit.shiny
posit.publisher
46 changes: 46 additions & 0 deletions workbench-session/test/goss.yaml
zachhannum marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
file:
/usr/local/bin/jupyter:
exists: true
/opt/rstudio-drivers:
exists: true
filetype: directory
/var/lib/rstudio-server/r-versions:
exists: false

command:
"echo '{ \"cells\": [], \"metadata\": {}, \"nbformat\": 4, \"nbformat_minor\": 2}' | /opt/python/jupyter/bin/jupyter nbconvert --to notebook --stdin --stdout":
title: jupyter_works
timeout: 60000
exit-status: 0

# Ensure correct R version
"/opt/R/{{.Env.R_VERSION}}/bin/R --version":
title: r_version_match
exit-status: 0
stdout: [
"{{.Env.R_VERSION}}"
]

# Ensure correct python version
"/opt/python/{{.Env.PYTHON_VERSION}}/bin/python3 --version":
title: python_version_matches
exit-status: 0
stdout: [
"{{ .Env.PYTHON_VERSION }}"
]

"python3 --version":
title: python_in_path_var
exit-status: 0
stdout: [
"{{ .Env.PYTHON_VERSION }}"
]

"jupyter --version":
title: jupyter_in_path_var
timeout: 60000
exit-status: 0

"pip --version":
title: pip_installed
exit-status: 0
19 changes: 19 additions & 0 deletions workbench-session/test/run_tests.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
#!/bin/bash

GOSS_FILE=${GOSS_FILE:-/test/goss.yaml}
GOSS_VERSION=${GOSS_VERSION:-0.4.6}
GOSS_MAX_CONCURRENT=${GOSS_MAX_CONCURRENT:-50}

if [ -f /etc/debian_version ]; then
OS="ubuntu"
else
echo "OS not supported. Exiting"
exit 1
fi

# install goss to tmp location and make executable
curl -fsSL https://github.com/aelsabbahy/goss/releases/download/v$GOSS_VERSION/goss-linux-amd64 -o /tmp/goss \
&& chmod +x /tmp/goss \
&& GOSS=/tmp/goss

OS=$OS GOSS_FILE=$GOSS_FILE $GOSS v --format documentation --max-concurrent $GOSS_MAX_CONCURRENT
5 changes: 5 additions & 0 deletions workbench-session/vscode.extensions.conf
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
quarto.quarto
REditorSupport.r@2.8.2
ms-python.python
posit.shiny
ms-toolsai.jupyter
Loading