Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove support for deprecated Julia REQUIRE, Julia < 1.3 #1350

Merged
merged 3 commits into from
May 29, 2024
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/source/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -87,7 +87,7 @@ the `build` method of the `BuildPack` base class.

The **assemble** stage builds the specific environment that is requested by the repository.
This usually means installing required libraries specified in a format native to the language
(`requirements.txt`, `environment.yml`, `REQUIRE`, `install.R`, etc).
(`requirements.txt`, `environment.yml`, `Project.toml`, `install.R`, etc).

Most of this work is done in `get_assemble_scripts` method. It can return arbitrary bash script
lines that can be run as different users, and has access to the repository contents (unlike
Expand Down
10 changes: 2 additions & 8 deletions docs/source/config_files.rst
Original file line number Diff line number Diff line change
Expand Up @@ -102,14 +102,8 @@ of the Julia packages that are installed.
``REQUIRE`` - Install a Julia environment (legacy)
==================================================

A ``REQUIRE`` file can specify both the version of Julia to be used and
which Julia packages should be used. The use of ``REQUIRE`` is only
recommended for pre 1.0 Julia versions. The recommended way of installing
a Julia environment that uses Julia 1.0 or newer is to use a ``Project.toml``
file. If both a ``REQUIRE`` and a ``Project.toml`` file are detected,
the ``REQUIRE`` file is ignored. To see an example of a Julia repository
with ``REQUIRE`` and ``environment.yml``, visit
`binder-examples/julia-python <https://github.com/binder-examples/julia-python>`_.
``REQUIRE`` files no longer work, and are no longer supported.
The recommended way of installing a Julia environment is to use a ``Project.toml`` file.


.. _install.R:
Expand Down
5 changes: 3 additions & 2 deletions docs/source/faq.rst
Original file line number Diff line number Diff line change
Expand Up @@ -55,9 +55,10 @@ We **strongly** recommend specifying a Python version (in environment.yml, runti
Julia
~~~~~

All Julia versions since Julia 0.7.0 are supported via a :ref:`Project.toml <Project.toml>`
All Julia versions since Julia 1.3 are supported via a :ref:`Project.toml <Project.toml>`
file, and this is the recommended way to install Julia environments.
Julia versions 0.6.x and earlier are supported via a :ref:`REQUIRE <REQUIRE>` file.

Julia < 1.3 and the older Julia REQUIRE file is no longer supported because required infrastructure has been removed.

R
~
Expand Down
4 changes: 2 additions & 2 deletions docs/source/specification.rst
Original file line number Diff line number Diff line change
Expand Up @@ -5,14 +5,14 @@ The Reproducible Execution Environment Specification
====================================================

repo2docker scans a repository for particular :ref:`config-files`, such
as ``requirements.txt`` or ``REQUIRE``. The collection of files, their contents,
as ``requirements.txt`` or ``Project.toml``. The collection of files, their contents,
and the resulting actions that repo2docker takes is known
as the **Reproducible Execution Environment Specification** (or REES).

The goal of the REES is to automate and encourage existing community best practices
for reproducible computational environments. This includes installing pacakges using
community-standard specification files and their corresponding tools,
such as ``requirements.txt`` (with ``pip``), ``REQUIRE`` (with Julia), or
such as ``requirements.txt`` (with ``pip``), ``Project.toml`` (with Julia), or
``apt.txt`` (with ``apt``). While repo2docker automates the
creation of the environment, a human should be able to look at a REES-compliant
repository and reproduce the environment using common, clear steps without
Expand Down
36 changes: 0 additions & 36 deletions repo2docker/buildpacks/julia/install-repo-dependencies.jl

This file was deleted.

198 changes: 11 additions & 187 deletions repo2docker/buildpacks/julia/julia_require.py
Original file line number Diff line number Diff line change
@@ -1,205 +1,29 @@
"""Generates a Dockerfile based on an input matrix with REQUIRE for legacy Julia"""
"""
DEPRECATED - Dependencies of REQUIRE have been removed
"""

import os
from functools import lru_cache

from ...semver import parse_version as V
from ..python import PythonBuildPack


class JuliaRequireBuildPack(PythonBuildPack):
"""
Julia build pack which uses conda and REQUIRE.
"""

minor_julias = {"0.6": "0.6.4", "0.7": "0.7.0", "1.0": "1.0.4", "1.1": "1.1.1"}
major_julias = {"1": "1.1.1"}

@property
def python_version(self):
# IJulia doesn't build on julia 0.6
# due to old incompatibilities with Jupyter-core >= 4.5,
# so use the similarly-old Python 3.5 base environment
if V(self.julia_version) < V("0.7"):
return "3.5"
else:
return super().python_version

@property
def julia_version(self):
require = self.binder_path("REQUIRE")
try:
with open(require) as f:
julia_version_line = (
f.readline().strip()
) # First line is optionally a julia version
except FileNotFoundError:
julia_version_line = ""

if not julia_version_line.startswith("julia "):
# not a Julia version line.
# use the default Julia.
self._julia_version = self.minor_julias["0.6"]
return self._julia_version

julia_version_info = julia_version_line.split(" ", 1)[1].split(".")
julia_version = ""
if len(julia_version_info) == 1:
julia_version = self.major_julias[julia_version_info[0]]
elif len(julia_version_info) == 2:
# get major.minor
julia_version = self.minor_julias[".".join(julia_version_info)]
else:
# use supplied julia version
julia_version = ".".join(julia_version_info)
self._julia_version = julia_version
return self._julia_version

@lru_cache()
def get_build_env(self):
"""Get additional environment settings for Julia and Jupyter

Returns:
an ordered list of environment setting tuples

The tuples contain a string of the environment variable name and
a string of the environment setting:
- `JULIA_PATH`: base path where all Julia Binaries and libraries
will be installed
- `JULIA_HOME`: path where all Julia Binaries will be installed
- `JULIA_PKGDIR`: path where all Julia libraries will be installed
- `JULIA_DEPOT_PATH`: path where Julia libraries are installed.
Similar to JULIA_PKGDIR, used in 1.x.
- `JULIA_VERSION`: default version of julia to be installed
- `JULIA_ARCH`: machine architecture used in Julia download URLs
- `JULIA_ARCH_SHORT`: machine architecture used in Julia download URLs
- `JUPYTER`: environment variable required by IJulia to point to
the `jupyter` executable

For example, a tuple may be `('JULIA_VERSION', '0.6.0')`.

"""
if self.platform == "linux/arm64":
julia_arch = julia_arch_short = "aarch64"
else:
julia_arch = "x86_64"
julia_arch_short = "x64"

if V(self.julia_version) < V("0.7"):
# IJulia with Julia 0.6 isn't compatible with more recent jupyter-core
# point it to the one in the kernel env
# I _think_ this is only relevant during installation
jupyter = "${KERNEL_PYTHON_PREFIX}/bin/jupyter"
else:
jupyter = "${NB_PYTHON_PREFIX}/bin/jupyter"
return super().get_build_env() + [
("JULIA_PATH", "${APP_BASE}/julia"),
("JULIA_HOME", "${JULIA_PATH}/bin"), # julia <= 0.6
("JULIA_BINDIR", "${JULIA_HOME}"), # julia >= 0.7
("JULIA_PKGDIR", "${JULIA_PATH}/pkg"),
("JULIA_DEPOT_PATH", "${JULIA_PKGDIR}"), # julia >= 0.7
("JULIA_VERSION", self.julia_version),
("JULIA_ARCH", julia_arch),
("JULIA_ARCH_SHORT", julia_arch_short),
("JUPYTER", jupyter),
]

@lru_cache()
def get_path(self):
"""Adds path to Julia binaries to user's PATH.

Returns:
an ordered list of path strings. The path to the Julia
executable is added to the list.

"""
return super().get_path() + ["${JULIA_HOME}"]

@lru_cache()
def get_build_scripts(self):
"""
Return series of build-steps common to "ALL" Julia repositories

All scripts found here should be independent of contents of a
particular repository.

This creates a directory with permissions for installing julia packages
(from get_assemble_scripts).

"""
return super().get_build_scripts() + [
(
"root",
r"""
mkdir -p ${JULIA_PATH} && \
curl -sSL "https://julialang-s3.julialang.org/bin/linux/${JULIA_ARCH_SHORT}/${JULIA_VERSION%[.-]*}/julia-${JULIA_VERSION}-linux-${JULIA_ARCH}.tar.gz" | tar -xz -C ${JULIA_PATH} --strip-components 1
""",
),
(
"root",
r"""
mkdir -p ${JULIA_PKGDIR} && \
chown ${NB_USER}:${NB_USER} ${JULIA_PKGDIR}
""",
),
(
"${NB_USER}",
# HACK: Can't seem to tell IJulia to install in sys-prefix
# FIXME: Find way to get it to install under /srv and not $HOME?
r"""
julia -e 'if (VERSION > v"0.7-") using Pkg; else Pkg.init(); end; Pkg.add("IJulia"); using IJulia;' && \
mv ${HOME}/.local/share/jupyter/kernels/julia-${JULIA_VERSION%[.-]*} ${NB_PYTHON_PREFIX}/share/jupyter/kernels/julia-${JULIA_VERSION%[.-]*}
""",
),
]

@lru_cache()
def get_assemble_scripts(self):
"""
Return series of build-steps specific to "this" Julia repository

Precompile all Julia libraries found in the repository's REQUIRE
file. The parent, CondaBuildPack, will add the build steps for
any needed Python packages found in environment.yml.

"""
require = self.binder_path("REQUIRE")
return super().get_assemble_scripts() + [
(
"${NB_USER}",
# Install and pre-compile all libraries if they've opted into it.
# In v0.6, Pkg.resolve() installs all the packages, but in v0.7+, we
# have to manually Pkg.add() each of them (since the REQUIRES file
# format is deprecated).
# The precompliation is done via `using {libraryname}`.
r"""
julia /tmp/install-repo-dependencies.jl "%(require)s"
"""
% {"require": require},
# TODO: For some reason, `rm`ing the file fails with permission denied.
# && rm /tmp/install-repo-dependencies.jl
)
]
Now just an informative error message.
"""

@lru_cache()
def get_build_script_files(self):
files = {
"julia/install-repo-dependencies.jl": "/tmp/install-repo-dependencies.jl"
}
files.update(super().get_build_script_files())
return files
def build(self, *args, **kwargs):
raise ValueError(
"Julia REQUIRE no longer supported due to removed infrastructure. Use Project.toml."
)

def detect(self):
"""
Check if current repo should be built with the Julia Legacy Build pack

super().detect() is not called in this function - it would return
false unless an `environment.yml` is present and we do not want to
require the presence of a `environment.yml` to use Julia.

Instead we just check if the path to `REQUIRE` exists and that there is
no julia 1.0 style environment
Check if current repo exects tp be built with the Julia Legacy Build pack

This no longer works, but try to raise an informative error.
"""
return os.path.exists(self.binder_path("REQUIRE")) and not (
os.path.exists(self.binder_path("Project.toml"))
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,4 @@
IteratorInterfaceExtensions = "82899510-4779-5014-852e-03e436cf321d"

[compat]
julia = "=1.0.2"
julia = "=1.3"
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
#!/usr/bin/env julia

if VERSION != v"1.0.2"
println("Julia version should be 1.0.2")
if VERSION != v"1.3.1"
println("Julia version should be 1.3.1")
exit(1)
end

Expand Down
5 changes: 0 additions & 5 deletions tests/julia/require-1-requirements-file/REQUIRE

This file was deleted.

1 change: 0 additions & 1 deletion tests/julia/require-1-requirements-file/requirements.txt

This file was deleted.

34 changes: 0 additions & 34 deletions tests/julia/require-1-requirements-file/verify

This file was deleted.

2 changes: 0 additions & 2 deletions tests/julia/require/REQUIRE

This file was deleted.

19 changes: 0 additions & 19 deletions tests/julia/require/verify

This file was deleted.

Loading