Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

chore(common): Allow to build offline #12439

Merged
merged 1 commit into from
Sep 20, 2024
Merged

chore(common): Allow to build offline #12439

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions resources/build/builder.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -327,6 +327,8 @@ The following parameters are pre-defined and should not be overridden:
* `--no-color`: forces off ANSI color output for the script
* `--verbose`, `-v`: verbose mode, sets the [`$builder_verbose`] variable
* `--debug`, `-d`: debug build; see [`builder_is_debug_build`] for more detail
* `--offline`: allow to build while offline. This might fail if not all
dependencies are cached.

--------------------------------------------------------------------------------

Expand Down
25 changes: 25 additions & 0 deletions resources/builder.inc.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,12 @@ function _builder_init() {
else
builder_use_color false
fi

# Set shared Meson package cache (works with Meson >= 1.3)
if [[ -z ${MESON_PACKAGE_CACHE_DIR:-} ]]; then
export MESON_PACKAGE_CACHE_DIR="${XDG_CACHE_HOME:-${HOME}/.cache}/keyman/builder"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is ~/.cache a safe path? We have already a ~/.keyman path used for nvm (see

keyman/docs/build/node.md

Line 33 in aa50672

### Caveats on macOS/Linux
) so maybe it'd be better to share that?

Copy link
Contributor Author

@ermshiperete ermshiperete Sep 20, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

~/.cache is the default location for caches on Linux. In contrast to the path for nvm (where the directory contains links to the actual executable). MESON_PACKAGE_CACHE_DIR serves as a real cache that caches the files we have to download. If the files don't exist they'll get downloaded (again). In either case they get extracted/copied to the subprojects folder in e.g. core/src. This means it's safe to delete the files from ~/.cache/keyman/builder, whereas I think the build will fail if ~/.keyman/node doesn't exist. So I'd prefer to put the files under ~/.cache/....

mkdir -p "${MESON_PACKAGE_CACHE_DIR}"
fi
}

function _builder_findRepoRoot() {
Expand Down Expand Up @@ -416,6 +422,7 @@ _builder_execute_child() {
${child_options[@]} \
$builder_verbose \
$builder_debug \
$_builder_offline \
&& (
if $_builder_debug_internal; then
builder_echo success "## $action$target completed successfully"
Expand Down Expand Up @@ -1267,6 +1274,7 @@ _builder_parse_expanded_parameters() {
_builder_chosen_options=()
_builder_current_action=
_builder_is_child=1
_builder_offline=

local n=0

Expand Down Expand Up @@ -1399,6 +1407,9 @@ _builder_parse_expanded_parameters() {
# internal reporting function, ignores all other parameters
_builder_report_dependencies
;;
--offline)
_builder_offline=--offline
;;
*)
# script does not recognize anything of action or target form at this point.
if [[ $key =~ ^: ]]; then
Expand Down Expand Up @@ -1565,6 +1576,7 @@ builder_display_usage() {
_builder_pad $width " --debug, -d" "Debug build"
_builder_pad $width " --color" "Force colorized output"
_builder_pad $width " --no-color" "Never use colorized output"
_builder_pad $width " --offline" "Try to build while being offline"
if builder_has_dependencies; then
_builder_pad $width " --deps" "Build dependencies if required (default)"
_builder_pad $width " --no-deps" "Skip build of dependencies"
Expand Down Expand Up @@ -1731,6 +1743,7 @@ _builder_do_build_deps() {
"$REPO_ROOT/$dep/build.sh" "configure$dep_target" "build$dep_target" \
$builder_verbose \
$builder_debug \
$_builder_offline \
$_builder_build_deps \
--builder-dep-parent "$THIS_SCRIPT_IDENTIFIER" && (
if $_builder_debug_internal; then
Expand Down Expand Up @@ -1761,6 +1774,18 @@ builder_is_child_build() {
return $_builder_is_child
}

#
# return `1` for a regular build, `0` to try to build while being offline.
# The offline build might fail if not all dependencies are cached.
#
builder_try_offline() {
if [[ "${_builder_offline}" == "--offline" ]]; then
return 0
else
return 1
fi
}

#
# returns `0` if we should attempt to do quick builds in a dependency build, for
# example skipping `tsc -b` where a parent may also do it; corresponds to the
Expand Down
7 changes: 6 additions & 1 deletion resources/shellHelperFunctions.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -267,7 +267,12 @@ verify_npm_setup() {

pushd "$KEYMAN_ROOT" > /dev/null

try_multiple_times npm ci
offline_param=
if builder_try_offline; then
builder_echo "Trying offline build"
offline_param=--prefer-offline
fi
try_multiple_times npm ${offline_param} ci
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could consider to always pass --prefer-offline which would probably help if nodejs.org has an outage, but I don't know if that has unwanted side-effects.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know if that's a good idea for build agents?


popd > /dev/null
}
Expand Down
Loading