ksh 93u+m/1.0.10

Here is the tenth ksh 93u+m/1.0 bugfix release – exactly two years after 93u+m/1.0.0 and twelve years after 93u+. Unfortunately, we're not done fixing bugs yet, but progress continues steadily.

Main changes between ksh 93u+m/1.0.9 and 93u+m/1.0.10:

• Fixed a serious and longstanding bug in the arithmetic subsystem that was triggered on non-Intel processors (such as ARM): any division of an integer by a negative integer threw a spurious "divide by zero" error. This bug has been in ksh since 2005.
• Fixed a regression where a broken pipe signal (SIGPIPE), when occurring in a pipe construct within a subshell, caused incorrect signal handling in the parent/main shell, in some cases causing a script to abort.
• Fixed a bug where printf %T, after having printed the time in UTC once with the TZ variable set to "UTC", would always print the time in UTC from then on, even if the TZ variable was changed to another time zone.
• The history expansion character (! by default) is now not processed when immediately following ${. This makes it possible to use expansion syntax like ${!varname} and ${!prefix@} on the interactive command line with the histexpand option on; these no longer trigger an "event not found" error.
• The shell is now capable of handling more than 32767 simultaneous background jobs, subject to system limitations.

Full Changelog: v1.0.9...v1.0.10

ksh 93u+m/1.0.9

Here is the ninth ksh 93u+m/1.0 bugfix release. This release contains many fixes of old and not-so-old bugs (including a couple of regressions in the printf built-in introduced in 93u+m/1.0.5), adds Android/Termux as a supported platform, and reintroduces the ability to build a dynamically linked ksh (with libast, libdll, libcmd, and libshell available for other applications to use) on most supported platforms.

Main changes between ksh 93u+m/1.0.8 and 93u+m/1.0.9:

• Android/Termux is now a supported platform. Build dependencies: binutils, clang, getconf. Runtime dependencies (optional): ncurses-utils, getconf.
• Reintroduced support for building a dynamically linked ksh(1)/shcomp(1), with libast, libdll, libcmd, and libshell available to other programs as dynamic libraries. bin/package install /your/basepath will install development headers. The dynamically linked version is built in a dyn subdirectory; there are no changes to the statically linked version. Dynamic linking is currently tested and supported on Linux, Android, macOS, all the BSDs, illumos, Solaris, QNX, and Haiku.
• On systems where the external printf(1) utility supports deprecated pre-POSIX syntax for formatters starting with -, ksh now adapts its built-in printf to match, for compatibility with system scripts. However, ksh's built-in printf options such as -v or --man are not affected.
• Fixed a regression in the printf built-in, introduced in 93u+m/1.0.5, where each instance of \0 or %Z in the format operand caused a string argument to be incorrectly skipped.
• Fixed a regression, introduced in 93u+m/1.0.5, in ordinal specifiers in printf %T date specifications. For example, printf '%(%F)T\n' '4th tuesday in march 2016' wrongly printed '2016-04-09' and now again correctly prints '2016-03-22'.
• Fixed a regression of return within traps, reintroduced in 93u+m/1.0.8 after being fixed in 93u+m/1.0.0. The regression caused a return or exit with no arguments to assume the before-trap exit status instead of that of the last-run command. This broke the shipped autocd function.
• Fixed a longstanding bug in shell arithmetic: the representation of negative integers with a base other than 10 was incorrectly treated as unsigned long. For example, typeset -i16 n=-12; echo $n now correctly outputs -16#c and no longer ouputs 16#fffffffffffffff4.
• Fixed a bug, introduced in ksh93q+ 2005-05-22, that stopped an append assignment from working together with a declaration command. For example,typeset var+=value or export var+=value now again work as expected.
• Fixed a longstanding bug where the default terminal width for typeset -L, -R, or -Z, if not given, was miscalculated for multibyte or control characters.
• Fixed: expansions of name references in loops were incorrectly treated as invariant so they yielded the wrong values.
• If a .get or .getn discipline function is set for a variable, it is no longer incorrectly triggered when performing an arithmetic assignment on that variable; only the .set discipline is now triggered (as documented).
• Many other bug fixes (see the NEWS file).

Full Changelog: v1.0.8...v1.0.9

ksh 93u+m/1.0.8

Here is the eighth ksh 93u+m/1.0 bugfix release. It fixes a bug that caused an incorrect default exit status for 'exit' within a trap action, as well as a race condition (introduced in 1.0.7) occurring on some systems when running an external command with a redirection from a command substitution.

Main changes between ksh 93u+m/1.0.7 and 93u+m/1.0.8:

• Fixed a regression in the behaviour of exit in a trap action. The exit status used when no argument is given to exit is now once again the exit status of the last command executed before the trap action.
• Fixed a race condition, introduced in 1.0.7, that occurred on some systems when running an external command with a standard output redirection from a command substitution.
• Fixed an init-time crash on failure to trim the shell command history file due to a non-writeable parent directory; ksh now prints a warning instead.
• The kill built-in command now correctly refuses to issue SIGSTOP to the shell's own process if the shell is a login shell.

Full Changelog: v1.0.7...v1.0.8

ksh 93u+m/1.0.7

Here is the seventh ksh 93u+m/1.0 bugfix release. It fixes a hang in command substitutions when combined with 'exec' and certain redirections.

Main changes between ksh 93u+m/1.0.6 and 93u+m/1.0.7:

• Fixed a hang in command substitutions (introduced in 93u+m/1.0.0) that was triggered when redirecting standard output within a command substitution, in combination with other factors. E.g., the following no longer hangs:
  { v=$(redirect 2>&1 1>&9); } 9>&1
• Fixed a crash on trying to append an indexed array value to an unset name reference, e.g.: nameref unsetref; unsetref+=(foo bar). This now produces a "removing nameref attribute" warning before performing the assignment.
• Fixed: assignments like name=(...) to arrays did not preserve the array and variable types; similarly, assigning an empty set () to a compound indexed array caused the -C attribute to be lost.
• Fixed incorrect rejection of the tab key while reading input using the read built-in command.
• Fixed a bug in printf %T: when using dates and times in the past, time zones for the present were incorrectly used, ignoring historical changes.

Full Changelog: v1.0.6...v1.0.7

ksh 93u+m/1.0.6

Sooner than expected after the fifth, here is the sixth ksh 93u+m/1.0 bugfix release. 93u+m/1.0.5 brought a wide range of bugfixes and robustness enhancements, but also introduced a serious regression in pathname expansion (#660), so that previous release is WITHDRAWN and should not be used.

Main changes between ksh 93u+m/1.0.5 and 93u+m/1.0.6:

• Fixed a serious regression in pathname expansion where quoted wildcard characters were incorrectly expanded if a pattern contains both a brace expansion and a variable expansion.
• Fixed a bug where the command to launch a full-screen editor (^X^E in emacs and v in vi) could cause the wrong command line to be edited if two shell sessions share a .sh_history file.

Main changes between ksh 93u+m/1.0.4 and 93u+m/1.0.5:

• Fixed various bugs causing crashes.
• Fixed many bugs in the emacs and vi line editors, in command completion, and in file name completion.
• Fixed various bugs in the handling of quotes, backslash escapes and braces when processing shell glob patterns (e.g. in pathname expansion and case).
• ksh now throws a panic and exits if a read error (such as an I/O error) occurs while trying to read the next command(s) from a running script.
• Fixed many bugs in printf and print -f built-in commands, including:
  • Multiple bugs causing incorrect output for relative date specifications, e.g., printf %T\\n 'exactly 20 months ago' now outputs a correct result.
  • More printf bugs with mix and match of % and %x$.
  • A data corruption bug when using %B with printf -v varname.
  • A bug causing double evaluation of arithmetic expressions.
• Fixed a bug where unset -f commandname, executed in a subshell, hides any built-in command by the same name for the duration of that subshell.
• Fixed ${var/#/string} and ${var/%/string} (with anchored empty pattern) to work as on mksh, bash and zsh; these are no longer ineffective.
• Fixed incorrect result of array slicing ${array[@]:offset:length} where length is a nested expansion involving an array.
• Command names can now end in : as they can on other shells.
• Fixed a spurious syntax error in compound assignments upon encountering a pair of repeated opening parentheses ((.
• Fixed spurious syntax error in ${parameter:offset:length}: the arithmetic expressions offset and length may now contain the operators ( ) & |.
• Fixed a parsing bug in the declaration of .sh.math.* arithmetic functions.
• Fixed nameref self-reference loop detection for more than two namerefs.
• Several improvements to the POSIX compatibility mode.
• Many more minor and/or esoteric bugfixes.

Full Changelog: v1.0.4...v1.0.6

ksh 93u+m/1.0.4

Main changes between 1.0.3 and 1.0.4:

• Fixed multiple scoping-related bugs in the += additive assignment operator.
• A number of crashing bugs have been fixed.
• Various fixes for the Haiku operating system, notably ulimit -a now works.
• Fixed the expansion of out-of-range \n back references in the string part of ${parameter//pattern/string}. For example:
  v=AB; echo "${v/@(A)B/\0:\1:\2}"
  now yields AB:A: instead of AB:A:\2.
• Fixed quoted !, ^ and - within [bracket] expressions in glob patterns; single or double quotes failed to disable their operator behaviour.
• Fixed a bug introduced on 2021-04-04 that incorrectly allowed typeset to turn off the readonly and export attributes on a readonly variable.
• In the emacs line editor, the Ctrl+R reverse-search prompt is now visually distinct from a literal control character (^R: instead of ^R).
• In the vi line editor, fixed the behaviour of C, c$ and I to be consistent with standard vi(1) and with Bolsky & Korn (1995, p. 121).
• Aliases for many GNU long options have been added to the /opt/ast/bin built-in commands. Additionally, kill -s now has a --signal long option alias compatible with the util-linux option.
• Backported support for print -u p from ksh 93v- for compatibility with scripts written for 93v-/ksh2020 (this is equivalent to 'print -p').

Full Changelog: v1.0.3...v1.0.4

ksh 93u+m/1.0.3

This point release mainly fixes the following:

• A bug in history expansion (set -H) where any use of the history comment character caused processing to be aborted as if it were an invalid history expansion. Affected e.g. echo ${#v}.
• A bug in command line options processing that caused short-form option equivalents on some built-in commands to be ignored after one use, e.g., the new read -a equivalent of read -A.
• Ksh freezing or using excessive memory if HISTSIZE is assigned a pathologically large value.
• A bug that caused ksh in the vi editor mode to crash or produce invalid completions if ESC = was used at the beginning of a line.

Full Changelog: v1.0.2...v1.0.3

ksh 93u+m/1.0.2

This bugfix release fixes the interactive shell crashing when one of the predefined aliases (currently 'history' and 'r') is redefined, whether from a profile/kshrc script or manually. This crash occurred in two scenarios:

1. when redefining and then unsetting a predefined alias;
2. when redefining a predefined alias and then executing a shell script that does not begin with a #! path.

Both are fixed now.

ksh 93u+m/1.0.1

This is an urgent bugfix release that removes an incorrect exec optimization that was capable of terminating the execution of scripts prematurely in certain corner cases. It is known to make the build scripts of GNU binutils produce corrupted results if ksh is used as /bin/sh. See #507 for more information.

No other breakage resulting from this bug is known yet, but other breakage probably exists. Every 1.0.0 user should update to 1.0.1 ASAP.

Full Changelog: v1.0.0...v1.0.1

ksh 93u+m/1.0.0

 _        _        ___ _____                          ___   ___   ___
| | _____| |__    / _ \___ / _   _   _   _ __ ___    / / | / _ \ / _ \
| |/ / __| '_ \  | (_) ||_ \| | | |_| |_| '_ ` _ \  / /| || | | | | | |
|   <\__ \ | | |  \__, |__) | |_| |_   _| | | | | |/ / | || |_| | |_| |
|_|\_\___/_| |_|    /_/____/ \__,_| |_| |_| |_| |_/_/  |_(_)___(_)___/

It may have been exactly a decade since the last one, but here it is at last: a proper new ksh release. :) Many thanks to all contributors for their hard work! Compared to an unpatched ksh 93u+ 2012-08-01, this release has roughly a thousand bugs fixed. It incorporates a fair number of enhancements as well.

This being an initial release, not all known bugs have been worked out yet. Let's hope this release will rekindle interest and attract more bug hunters.

Download the source code: https://github.com/ksh93/ksh/archive/refs/tags/v1.0.0.tar.gz
Full Changelog: reboot...v1.0.0

CONTRIBUTORS

Work on ksh 93u+m started in May 2020.
Main developers: Martijn Dekker, Johnothan King, hyenias

Direct contributors: Andy Fiddaman, Anuradha Weeraman, atheik, Chase, Govind Kamat, Harald van Dijk, K. Eugene Carlson, Lev Kujawski, Marc Wilson, Ryan Schmidt, Sterling Jensen, Trey Valenta, Vincent Mihalkovic

Also includes backported contributions by: David Korn, Glenn Fowler, Lefteris Koutsofios, Siteshwar Vashisht, Kurtis Rader, Roland Mainz, Finnbarr P. Murphy, Lijo George, OpenSUSE ksh 93u+ patch authors, Red Hat ksh 93u+ path authors, Solaris ksh 93u+ patch authors, Debian ksh 93u+ patch authors, Apple ksh 93u+ patch authors

Many fixes have also been backported from the AT&T 93v- beta as well as the former ksh2020 project lead by Kurtis Rader and Siteshwar Vashisht; we appreciate and benefit from their work. Many thanks also to Siteshwar for graciously donating his 'ksh93' GitHub organisation account!

HOW TO GET IT

Please download the source code tarball below.
To build, follow the instructions in README.md or src/cmd/ksh93/README.

Or ask your distribution package manager to upgrade ksh93 to this version.

HOW TO GET INVOLVED

To report a bug, please open an issue at our GitHub page (see above). Alternatively, email me at martijn@inlv.org with your report. To get involved in development, read the brief policy information in README.md and then jump right in with a pull request or email a patch. See the TODO file in the top-level directory for a to-do list.

MAIN CHANGES between ksh 93u+ 2012-08-01 and 93u+m/1.0.0

Roughly a thousand bugs have been fixed, including many serious/critical bugs. See the NEWS file for more information, and the git commit log for complete documentation of every fix. Incompatible changes have been minimised, but not at the expense of fixing bugs. For a list of potentially incompatible changes, see src/cmd/ksh93/COMPATIBILITY.

Though there was a "no new features, bugfixes only" policy, some new features were found necessary, either to fix serious design flaws or to complete functionality that was evidently intended, but not finished. Below is a summary of these new features.

New command line editor features:

- The forward-delete and End keys are now handled as expected in the
  emacs and vi built-in line editors.

- In the vi and emacs line editors, repeat counts can now also be used for
  arrow keys and the forward-delete key, e.g., <ESC> 7 <left-arrow> works.

- Various keys on extended PC keyboards are now handled as expected in the
  emacs and vi built-in line editors.

New shell language features:

- Pathname expansion (a.k.a. globbing) now never matches the special names
  '.' (current directory) and '..' (parent directory). This change makes a
  pattern like .* useful; it now matches all hidden files (dotfiles) in the
  current directory, without the harmful inclusion of '.' and '..'.

- Tilde expansion can now be extended or modified by defining a .sh.tilde.get
  or .sh.tilde.set discipline function. See the manual for details.

- The &>file redirection shorthand (for >file 2>&1) is now available for all
  scripts and interactive sessions and not only for profile/login scripts.

- Arithmetic expressions in native ksh mode no longer interpret a number
  with a leading zero as octal in any context. Use 8#octalnumber instead
  (e.g. 8#400 == 256). Arithmetic expressions now also behave identically
  within and outside ((...)) and $((...)). If the POSIX mode is turned on,
  a leading zero now denotes an octal number in all arithmetic contexts.

New features in built-in commands:

- Usage error messages now show the --help/--man self-documentation options.

- Path-bound built-ins (such as /opt/ast/bin/cat) can now be executed by
  invoking the canonical path, so the following will now work as expected:
        $ /opt/ast/bin/cat --version
          version         cat (AT&T Research) 2012-05-31

- 'cd' now supports an -e option that, when combined with -P, verifies
  that $PWD is correct after changing directories; this helps detect
  access permission problems. See:
  https://www.austingroupbugs.net/view.php?id=253

- 'command -x' now looks for external commands only, skipping built-ins.
  In addition, its xargs-like functionality no longer freezes the shell on
  Linux and macOS, making it effectively a new feature on these systems.

- 'printf' now supports a -v option as in bash. This assigns formatted
  output directly to variables, which is very fast and will not strip
  final newline (\n) characters.

- 'redirect' now checks if all arguments are valid redirections before
  performing them. If an error occurs, it issues an error message instead
  of terminating the shell.

- 'return', when used to return from a function, can now return any
  status value in the 32-bit signed integer range, like on zsh. However,
  due to a traditional Unix kernel limitation, $? is still trimmed to its
  least significant 8 bits whenever a shell or subshell exits.

- 'suspend' now refuses to suspend a login shell, as there is probably no
  parent shell to return to and the login session would freeze.

- 'test'/'[' now supports all the same operators as [[ (including =~,
  \<, \>) except for the different 'and'/'or' operators. Note that
  'test'/'[' remains deprecated due to its unfixable pitfalls;
  [[ ... ]] is recommended instead.

- 'times' now gives high precision output in a POSIX compliant format.

- 'type'/'whence': Two bash-like flags were backported from ksh 93v-:
  - 'whence -P/type -P' is an alias to the existing -p flag.
  - 'whence -t/type -t' will print only the type of a command in a
    simple format that is designed to be easy to use for scripts.

- 'typeset' has a new '-g' flag that forces variables to be created or
  modified at the global scope regardless of context, as on bash 4.2+.

- 'typeset' now gives an informative error message if an incompatible
  combination of options is given.

- 'ulimit': Added three options inspired by bash:
  - 'ulimit -k' sets the maximum number of kqueues.
  - 'ulimit -P' sets the maximum number of pseudo-terminals.
  - 'ulimit -R' sets the maximum time in microseconds a real-time process
    can run before blocking.
  Note that not all operating systems support the limits set by these options.

- 'whence -v/-a' now reports the location of autoloadable functions.

New features in shell options:

- When the -b/--notify shell option is on and the vi or emacs/gmacs shell
  line editor is in use, 'Done' and similar notifications from completed
  background jobs are now inserted directly above the line you're typing,
  without affecting your command line display.

- A new --functrace long-form shell option causes the -x/--xtrace option's
  state and the DEBUG trap action to be inherited by function scopes instead
  of being reset to default. Changes made to them within a function scope
  still do not propagate back to the parent scope. Similarly, this option
  also causes the DEBUG trap action to be inherited by subshells.

- A new --globcasedetect shell option is added on operating systems where
  we can check for a case-insensitive file system (currently Linux, macOS,
  QNX 7.0+, and Windows/Cygwin). When this option is turned on, pathname
  expansion (globbing), as well as tab completion on interactive shells,
  automatically become case-insensitive depending on the file system.
  This is separately determined for each pathname component.

- Enhancement to -G/--globstar: symbolic links to directories are now
  followed if they match a normal (non-**) glob pattern. For example, if
  '/lnk' is a symlink to a directory, '/lnk/**' and '/l?k/**' now work as
  you would expect.

- The new --histreedit and --histverify options modify history expansion
  (--histexpand). If --histreedit is on and a history expansion fails, the
  command line is reloaded into the next prompt's edit buffer, allowing
  corrections. If --histverify is on, the results of a history expansion are
  not immediately executed but instead loaded into the next prompt's edit
  buffer, allowing further changes.

- A new --nobackslashctrl shell option disables the special escaping
  behaviour of the backslash character in the emacs and vi built-in editors.
  Particularly in the emacs editor, this makes it much easier to go back,
  insert a forgotten backslash into a command, and then continue editing
  without having your next arrow key replace your backslash with garbage.

- A new --posix shell option has been added to ksh 93u+m that makes the
  ksh language more compatible with other shells by following the POSIX
  standard more closely. See the manual page for details. It is enabled by
  default if ksh is invoked as sh, otherwise it is disabled by default.
...