Improvement: smooth installation on non-root systems

[edmcman]

I am installing dorothy on a machine that I don't have root on. It begins to install the required packages, and basically prompts me to enter a sudo password many times for bash, moreutils, llvm, openssl, and so on. This is pretty annoying. I can press Ctrl-D to make sudo bail and dorothy continues, but it's definitely not a great experience.

Since dorothy can't upgrade the bash package, it also downloads and installs a new version of bash from source, which I really didn't want it to do.

[balupton]

Hrmmm. Yeah, def could be improved.

Are you wishing for those utilities to be installed, but just via a method that doesn't require sudo? Such would still let all the Dorothy commands function as expected. Let's call this the non-root complete install option; this might be easy enough to do, such as updating setup-util to filter methods based on root/non-root with a flag.

Or do you wish to not have them installed at all, and use only a minimal set of Dorothy commands that do not need them? If so, is this use of Dorothy only for Dorothy's cross-shell configuration loading but not for Dorothy's own commands? Let's call this non-root minimal install; this will require some thinking on the best approach.

For bash, which bash version did you wish to keep it on? Or are you okay with Dorothy getting a newer bash, as long as Dorothy only has access to it? Bash is definitely a dependency in which older version support varies… here is a list of issues that I've encountered with older bash versions: https://github.com/bevry/dorothy/blob/master/docs/bash/versions.md the bash.bash file provides shims of various compromises: https://github.com/bevry/dorothy/blob/master/sources/bash.bash

What operating system and architecture was this? Does it have a docker container? If so, we can add CI testing.

[edmcman]

I think this issue is about the "non-root complete install". If I hadn't been prompted many times by sudo, the process would have been fine.

The complaint about bash was more of an afterthought. My thought as it was happening was basically "Do I really need a new version of bash?" and I felt a little out of control during the process. This machine is fairly old and outdated: RHEL 7.9 on x86-64. The built-in version of /bin/bash is 4.2.46. Maybe all that is needed here is a prompt or message saying that bash is being upgraded because it is older than a certain version, or making it an interactive decision.

It does look like there is a docker container for RHEL 7.9, but it's deprecated so it's probably not even worth your time to worry about that version specifically. (We are upgrading soon..... hopefully).

[balupton]

I think this issue is about the "non-root complete install". If I hadn't been prompted many times by sudo, the process would have been fine.

Interesting, so did the deps manage to install via an alternative method? Or did Dorothy, or that dep installation, end up failing in the end? Do you still have the output of this session and are able to share, if so, it'll be easier for me to know which methods to filter as non-root compatible.

The complaint about bash was more of an afterthought.

Good feedback. It does this check internally, but never communicates it externally if it can just do the upgrade:

dorothy/sources/bash.bash

Lines 60 to 74 in a982b1d

	# any v5 version is supported by dorothy
	if test "$BASH_VERSION_MAJOR" -eq 5; then
	IS_BASH_VERSION_OUTDATED='no'
	function require_upgraded_bash {
	true
	}
	else
	IS_BASH_VERSION_OUTDATED='yes'
	function require_upgraded_bash {
	echo-style \
	--code="$0" ' ' --error='is incompatible with' ' ' --code="bash $BASH_VERSION" $'\n' \
	'Run ' --code='setup-util-bash' ' to upgrade capabilities, then run the prior command again.' >/dev/stderr
	return 45 # ENOTSUP 45 Operation not supported
	}
	fi

dorothy/commands/is-bash-version-outdated

Lines 86 to 90 in a982b1d

	if test "$IS_BASH_VERSION_OUTDATED" = 'yes'; then
	return 0
	else
	return 1
	fi

dorothy/commands/setup-util-bash

Lines 92 to 99 in a982b1d

	# check if still outdated, if so, install via building instead
	if is-bash-version-outdated --quiet; then
	options+=(
	--upgrade
	--order='download ...'
	)
	setup_util "${options[@]}"
	fi

Will update it accordingly so that it is mentioned why it is upgrading.

[edmcman]

Interesting, so did the deps manage to install via an alternative method? Or did Dorothy, or that dep installation, end up failing in the end?

Some of them, such as bash, managed to install via an alternate method. It seemed like some of the others, like openssl, failed but then the process moved on. It's a bit unclear what happened to the dorothy install as a whole. The install did seem to fail, but it wasn't completely clear why. (It might have been my fault -- I have a submodule on my dotfiles repo as an added layer of protection).

Do you still have the output of this session and are able to share, if so, it'll be easier for me to know which methods to filter as non-root compatible.

I didn't save it, but I will recreate it.

[balupton]

Some of them, such as bash, managed to install via an alternate method. It seemed like some of the others, like openssl, failed but then the process moved on. It's a bit unclear what happened to the dorothy install as a whole. The install did seem to fail, but it wasn't completely clear why. (It might have been my fault -- I have a submodule on my dotfiles repo as an added layer of protection).

Hrmm, will need to see then in the recreation.

I didn't save it, but I will recreate it.

Ok cool, thank you. Could you record as well if not too inconvenient?

---

I think the next step here is finding an up to date container that can reproduce the non-root experience. Then I can develop the adjustments against it, and add CI support for the adjustments. Would you be able to post back with such a container?

I will schedule any adjustments for after #229 lands, which is my current task. So have 2 weeks before I start on this.

[edmcman]

Here's a script log. dorothy.log

A couple notes:

• I forgot to remove the user local bash so it doesn't try to install newer bash via yum.
• The behavior with bash is still odd...

ed@banana /tmp> cat dorothy.log | fgrep bash
[eschwartz@pd4 ~]$ bash -ic "$(curl -fsSL https://dorothy.bevry.me/install)"
< 'yum' '-y' '--best' '--quiet' 'install' 'bash-completion' >
</ 'yum' '-y' '--best' '--quiet' 'install' 'bash-completion' >[1]
Your login shell is an outdated version of bash.
Preference available on this system: /home/eschwartz/.local/bin/bash
┌  Selected [bash] at [/home/eschwartz/.local/bin/bash]  ┐
Preference available on this system: /usr/bin/bash
┌  Selected [bash] at [/usr/bin/bash]  ┐
Preference available on this system: /bin/bash
┌  Selected [bash] at [/bin/bash]  ┐
└  Selected [bash] at [/bin/bash]  ┘

Not sure what was going on there, I didn't "select" any of them.

• I think my dotfiles repo is the reason why the install process failed. Not worried about that.

I'll try in the container and see if the same thing happens.

[edmcman]

I replicated this in a Docker container here: https://github.com/edmcman/dorothy-issue233

I also created a video:

[balupton]

Great, thank you.

So there's two approaches that I'm thinking of:

1. we proceed the installation as normal, however if the root prompt fails, we ask the user if they want to (a) try again, (b) skip this invocation but still try for other invocations, or (c) skip root altogether. The latter will write a $DOROTHY/state/no-root file, which sudo-helper and setup-util will check in the future, to adapt their behaviour accordingly.

2. same as the first option, but we prompt the user prior to trying sudo; I'm leaning away from this, as there are too many install pathways that could avoid a direct prompt.

for the actual dorothy installation:

1. the login shell enablement will need to be skipped, as it requires root
2. will need to document exactly which commands require which of those required deps
3. explain why bash is being upgraded

I've scheduled this in my Dorothy batch of work and for completion it'll probably be sometime next month. If someone wants to beat me to it, that's good too.

depending on how it all goes, it may make sense to introduce a few different installation options:

1. Clone the git repo and expose the dorothy command only via an alias
2. Clone the git repo and add its bins to path
3. Clone the git repo and load init.sh
4. Clone the git repo, install deps, load init.sh, ask about user config, etc - current method

however alternative installation options will be significant increase in complexity and support burden, so should only be done if justified and sustainable

[balupton]

Besides the few fixes on the issue tracker, this is up next.

[balupton]

Okay, so finally got those fixes finished with #250. Will release this week. Then will finish #221, which may take next week. Then this. Then #195