diff --git a/.github/dependencies.yml b/.github/dependencies.yml index b2440fa7e..96d9991a3 100644 --- a/.github/dependencies.yml +++ b/.github/dependencies.yml @@ -46,9 +46,10 @@ dependencies: plugins/z: branch: master repo: agkozak/zsh-z - version: acd0e1984df350c189f8f9c4956ec586b6c73fca + version: 102fb78036ed76feedf623907483691777a1d510 precopy: | set -e + rm -r tests test -e README.md && mv -f README.md MANUAL.md postcopy: | set -e diff --git a/plugins/z/MANUAL.md b/plugins/z/MANUAL.md index 420535b27..1c8a1d222 100644 --- a/plugins/z/MANUAL.md +++ b/plugins/z/MANUAL.md @@ -2,20 +2,26 @@ [![MIT License](img/mit_license.svg)](https://opensource.org/licenses/MIT) ![Zsh version 4.3.11 and higher](img/zsh_4.3.11_plus.svg) +![Version](https://img.shields.io/github/v/release/agkozak/zsh-z) +![Workflow Status](https://github.com/agkozak/zsh-z/actions/workflows/test.yml/badge.svg) [![GitHub stars](https://img.shields.io/github/stars/agkozak/zsh-z.svg)](https://github.com/agkozak/zsh-z/stargazers) ![Zsh-z demo](img/demo.gif) Zsh-z is a command-line tool that allows you to jump quickly to directories that you have visited frequently or recently -- but most often a combination of the two (a concept known as ["frecency"](https://en.wikipedia.org/wiki/Frecency)). It works by keeping track of when you go to directories and how much time you spend in them. Based on this data, it predicts where you want to go when you type a partial string. For example, `z src` might take you to `~/src/zsh`. `z zsh` might also get you there, and `z c/z` might prove to be even more specific -- it all depends on your habits and how long you have been using Zsh-z to build up a database. After using Zsh-z for a little while, you will get to where you want to be by typing considerably less than you would need to if you were using `cd`. -Zsh-z is a native Zsh port of [`rupa/z`](https://github.com/rupa/z), a tool written for `bash` and Zsh that uses embedded `awk` scripts to do the heavy lifting. `rupa/z` was my most used command-line tool for a couple of years. I decided to translate it, `awk` parts and all, into pure Zsh script, to see if by eliminating calls to external tools (`awk`, `sort`, `date`, `sed`, `mv`, `rm`, and `chown`) and reducing forking through subshells I could make it faster. The performance increase is impressive, particularly on systems where forking is slow, such as Cygwin, MSYS2, and WSL. I have found that in those environments, switching directories using Zsh-z can be over 100% faster than it is using `rupa/z`. +Zsh-z is a native Zsh port of [`rupa/z`](https://github.com/rupa/z), a tool written for `bash` and Zsh that uses embedded `awk` scripts to do the heavy lifting. `rupa/z` was my most used command-line tool for a couple of years. I decided to translate it, `awk` parts and all, into pure Zsh script, to see if by eliminating calls to external tools (`awk`, `sort`, `date`, `sed`, `mv`, `rm`, and `chown`) and reducing forking through subshells I could make it faster. The performance increase is impressive, particularly on systems where forking is slow, such as Cygwin, MSYS2, and WSL. -There is also a significant stability improvement. Race conditions have always been a problem with `rupa/z`, and users of that utility occasionally lose their `~/.z` databases. By having Zsh-z only use Zsh (`rupa/z` uses a hybrid shell code standard that works on `bash` as well), I have been able to implement a `zsh/system`-based file-locking mechanism similar to [the one @mafredri once proposed for `rupa/z`](https://github.com/rupa/z/pull/199). It is now nearly impossible to crash the database. +There is also a significant stability improvement. Race conditions have always been a problem with `rupa/z`, and users of that utility occasionally lose their `~/.z` databases. By having Zsh-z only use Zsh (`rupa/z` uses a hybrid shell code standard that works on `bash` as well), I have been able to implement a `zsh/system`-based file-locking mechanism. It is now nearly impossible to crash the database. -There are other, smaller improvements which I document below in [Improvements and Fixes](#improvements-and-fixes). For instance, tab completions are now sorted by frecency by default rather than alphabetically (the latter behavior can be restored if you like it -- [see below](#settings)). +There are other, smaller improvements which I document below in [Other Improvements to the Original Functionality of `rupa/z`](#other-improvements-to-the-original-functionality-of-rupaz). For instance, tab completions are now sorted by frecency by default rather than alphabetically (the latter behavior can be restored if you like it -- [see below](#settings)). Zsh-z is a drop-in replacement for `rupa/z` and will, by default, use the same database (`~/.z`, or whatever database file you specify), so you can go on using `rupa/z` when you launch `bash`. +> ### Zsh-z v2.0 +> +> **v2.0 is the most significant release in the project's history.** It is faster than ever at adding, searching, and listing on modern Zsh; it never makes your prompt wait on database writes -- on any platform; it hardens those writes against corruption and against prying eyes; and it makes tab completion "just work" even under `setopt COMPLETE_ALIASES`. See [**v2.0**](#v20) in the News below for the full rundown, and [Performance](#performance) for the benchmarks. + ## Table of Contents - [News](#news) - [Installation](#installation) @@ -23,16 +29,45 @@ Zsh-z is a drop-in replacement for `rupa/z` and will, by default, use the same d - [Settings](#settings) - [Case Sensitivity](#case-sensitivity) - [`ZSHZ_UNCOMMON`](#zshz_uncommon) +- [`ZSHZ_OWNER`](#zshz_owner) - [Making `--add` work for you](#making---add-work-for-you) -- [Other Improvements to the Original Functionality of `rupa/z`](#other-improvements-to-the-original-functionality-of-rupa-z) +- [Performance](#performance) +- [Other Improvements to the Original Functionality of `rupa/z`](#other-improvements-to-the-original-functionality-of-rupaz) - [Migrating from Other Tools](#migrating-from-other-tools) - [`COMPLETE_ALIASES`](#complete_aliases) ## News -
- Here are the latest features and updates. +### v2.0 (August 14, 2026) +Version **2.0** is a major step forward, and these are the changes most worth knowing about: + +- **Zsh-z is even faster than before.** Version 2.0 builds on Zsh-z's past successes in optimizing adding to the datafile -- something it does at virtually every prompt -- by streamlining searching and listing, as well. See [Performance](#performance) for the numbers. +- **Database writes never block your prompt** -- on any platform. The per-prompt `--add` has long run in the background on most systems, but Cygwin and MSYS2 did the write in the foreground, because backgrounding there cost a wrapper subshell plus a job. `--add` now runs as a single disowned job (`&!`) everywhere: one fork, no wrapper subshell, no job-control noise. On Cygwin and MSYS2 that turns a foreground write whose cost grows with your datafile (~30 ms at 300 entries, ~300 ms at 1,000) into a flat ~10-12 ms fork. Elsewhere it halves the forks per prompt. +- **Safer, crash-resistant concurrent writes.** Writes are now guarded by a dedicated, stable lockfile using `zsh/system` file locking, with a bounded wait for lock acquisition (the new [`ZSHZ_LOCK_TIMEOUT`](#settings), default `1` second). Write errors are handled gracefully, and locks are always released even if a write is interrupted. On Cygwin and MSYS2 a write is also retried briefly if Windows refuses it: a virus scanner or the search indexer that opens the database in the instant between its being written and its being moved into place makes the move fail, which used to lose that one directory silently. Zsh-z now retries the move briefly. +- **Your database file now has `600` permissions** -- readable and writable only by you -- so that other users on a shared system cannot read your directory history ([#92](https://github.com/agkozak/zsh-z/issues/92)). On Zsh 5+ this uses the in-process `zf_chmod` builtin; on Zsh 4.3.11 it uses a `umask`-in-a-subshell technique that avoids the fork-and-exec of an external `chmod`. +- **`COMPLETE_ALIASES` works automatically in normal setups.** Tab completion no longer breaks when you have `setopt COMPLETE_ALIASES` enabled. Zsh-z registers the alias automatically on the first Tab press, so the manual `compdef` line that earlier versions required is no longer necessary. [See below](#complete_aliases). +- **Fixed a `can't clobber parameter tmpfd` error on some Zsh builds.** On certain Zsh builds, every database write could fail with `can't clobber parameter tmpfd containing file descriptor 0`, leaving an error at each new prompt. The file descriptor used for the temporary database file is now held in an unset scalar rather than one seeded with `0`, so the write never trips Zsh's file-descriptor-clobber guard ([#81](https://github.com/agkozak/zsh-z/issues/81)). +- **A misconfigured database file no longer closes your shell -- or nags you at every prompt.** When `ZSHZ_DATA` points at a directory, or names a file without a directory, Zsh-z now reports the problem and returns instead of calling `exit`. The per-prompt `--add` stays quiet about it, so you are told once, when you actually run `z`, rather than at every prompt ([#103](https://github.com/agkozak/zsh-z/issues/103); props @ahjota). +- **`z -x` can now remove the entry for a directory that no longer exists -- and can no longer crash Zsh 4.3.11.** The removal target used to have to exist on disk, so a database entry whose directory had been deleted -- exactly the entry you most want gone -- could not be removed. `z -x /deleted/dir` (and `z -xR`) now canonicalizes the argument without requiring it to exist, resolving symlinks in as much of the path as is still present. The same change fixes a crash on Zsh 4.3.11, where an upstream bug makes `${x:A}` segfault when the top-level component of the path is missing: `z -x /gone/sub` -- or a `ZSHZ_DATA` pointing into a missing top-level directory -- could kill the shell there. +- **More robust startup and operation.** A version check on an unsupported Zsh no longer risks exiting your interactive shell. + +#### Smaller changes you may notice + +None of these should need any action from you, but they are the things a v1 user is most likely to spot: + +- **`-r` and `-t` can no longer be combined.** They name mutually exclusive sort orders -- rank versus recency -- so `z -rt foo` was always contradictory, and `-t` silently took precedence. It is now rejected with an error message instead of silently picking one. +- **A lockfile appears next to your database.** Writes are serialized through `~/.z.lock` (or `$ZSHZ_DATA` plus `.lock`). It is created `600`, it stays empty, and it is deliberately never deleted -- removing a lockfile that another shell has already opened would reintroduce the very race it exists to prevent. Where `zsh/system` is missing and `zsystem flock` is therefore unavailable -- MobaXterm's cut-down Cygwin is the case in practice -- you will see a short-lived `~/.z.lock.d` **directory** instead, which is created and removed for each write. See [`ZSHZ_LOCK_TIMEOUT`](#settings) for how that fallback behaves. +- **An existing database gets tightened to `600` on the next write.** If your `~/.z` predates this release and is `644`, the first write will re-mode it. This is the [#92](https://github.com/agkozak/zsh-z/issues/92) fix applied to databases you already have, not just newly created ones. Setting the mode is treated as part of the write rather than a courtesy afterwards: if it cannot be done, the write is abandoned and reports failure instead of publishing a database that anyone on the machine could read. + +The dated entries below remain the historical record of changes leading up to v2.0. + +
+ Here are the older features and updates. + +- May 6, 2026 + + Zsh-z will now handle paths with dollar signs (`$`) in them. + + Workaround for Zsh emulating `sh`/`bash`/`ksh`. - May 1, 2026 + Various tab completion bugs resolved. - April 27, 2026 @@ -90,7 +125,7 @@ Zsh-z is a drop-in replacement for `rupa/z` and will, by default, use the same d - February 15, 2021 + Ranks are displayed the way `rupa/z` now displays them, i.e. as large integers. This should help Zsh-z to integrate with other tools. - January 31, 2021 - + Zsh-z is now efficient enough that, on MSYS2 and Cygwin, it is faster to run it in the foreground than it is to fork a subshell for it. + + Zsh-z is now efficient enough that, on MSYS2 and Cygwin, it is faster to run it in the foreground than it is to fork a subshell for it. (Behavior superseded in v2.0.) + `_zshz_precmd` simply returns if `PWD` is `HOME` or in `ZSHZ_EXCLUDE_DIRS`, rather than waiting for `zshz` to do that. - January 17, 2021 + Made sure that the `PUSHD_IGNORE_DUPS` option is respected. @@ -121,11 +156,11 @@ This plugin can be installed simply by putting the various files in a directory source /path/to/zsh-z.plugin.zsh -For tab completion to work, `_zshz` *must* be in the same directory as `zsh-z.plugin.zsh`, and you will want to have loaded `compinit`. The frameworks handle this themselves. If you are not using a framework, put +Tab completion requires `compinit`. `_zshz` *must* also be in the same directory as `zsh-z.plugin.zsh`. The frameworks handle both of these requirements, but if you are not using a framework, put autoload -U compinit; compinit -in your `.zshrc` somewhere below where you source `zsh-z.plugin.zsh`. +in your `.zshrc` somewhere below where you source `zsh-z.plugin.zsh` -- the plugin adds its directory to `fpath` at source time, and `compinit` needs to see it there in order to find `_zshz`. If you add @@ -195,7 +230,7 @@ Add a backslash to the end of the last line and add `'zsh-z'` to the list, e.g., Then relaunch `zsh`. ### For [zcomet](https://github.com/agkozak/zcomet) users - + Simply add zcomet load agkozak/zsh-z @@ -235,7 +270,7 @@ Add the line to your `.zshrc`. -Zsh-z supports `zinit`'s `unload` feature; just run `zinit unload agkozak/zsh-z` to restore the shell to its state before Zsh-z was loaded. +Zsh-z supports `zinit`'s `unload` feature; just run `zinit unload agkozak/zsh-z` to restore the shell to its state before Zsh-z was loaded. It gives back only what it took: the `fpath` entry is removed only if Zsh-z was the one that added it, so a directory your plugin manager put there is left alone, and the Tab-completion mapping is dropped only while it still points at Zsh-z's own completer. ### For [Znap](https://github.com/marlonrichert/zsh-snap) users @@ -274,6 +309,7 @@ to install Zsh-z. Zsh-z has environment variables (they all begin with `ZSHZ_`) that change its behavior if you set them. You can also keep your old ones if you have been using `rupa/z` (whose environment variables begin with `_Z_`). +* `ZSHZ_CASE` can be `ignore`, for case-insensitive matching, or `smart`, for Vim-like `smartcase` matching; [see below](#case-sensitivity) (default: empty, i.e., a case-sensitive match is tried first, then a case-insensitive one) * `ZSHZ_CMD` changes the command name (default: `z`) * `ZSHZ_CD` specifies the default directory-changing command (default: `builtin cd`) * `ZSHZ_COMPLETION` can be `'frecent'` (default) or `'legacy'`, depending on whether you want your completion results sorted according to frecency or simply sorted alphabetically @@ -281,9 +317,10 @@ Zsh-z has environment variables (they all begin with `ZSHZ_`) that change its be * `ZSHZ_ECHO` displays the new path name when changing directories (default: `0`) * `ZSHZ_EXCLUDE_DIRS` is an array of directories to keep out of the database (default: empty) * `ZSHZ_KEEP_DIRS` is an array of directories that should not be removed from the database, even if they are not currently available (useful when a drive is not always mounted) (default: empty) +* `ZSHZ_LOCK_TIMEOUT` is the number of seconds to wait for the database lock before giving up on a write (default: `1`). Where `zsh/system` is unavailable and `zsystem flock` cannot be used -- MobaXterm's cut-down Cygwin, for instance -- Zsh-z serializes writes with an atomic `mkdir` on `~/.z.lock.d` instead, and this setting bounds the wait the same way. That fallback has no equivalent of the kernel releasing a lock when its holder dies, so a lock directory older than 30 seconds is treated as abandoned and cleared; a write takes milliseconds, so nothing legitimate is ever that old. A write that times out is dropped silently -- the automatic `precmd` add is best-effort -- so if the database seems to stop updating, the lock is probably contended: run `z --add .` by hand and check `$?`. A `2` is a lock-acquisition timeout, which confirms contention -- look for a stale process holding `~/.z.lock` (or a leftover `~/.z.lock.d` directory on the fallback path), or raise this setting. A `1` is something else entirely, usually a permissions or ownership problem, such as a root-owned `~/.z` or `~/.z.lock` left behind by an earlier `sudo -s` session. * `ZSHZ_MAX_SCORE` is the maximum combined score the database entries can have before they begin to age and potentially drop out of the database (default: 9000) * `ZSHZ_NO_RESOLVE_SYMLINKS` prevents symlink resolution (default: `0`) -* `ZSHZ_OWNER` allows usage when in `sudo -s` mode (default: empty) +* `ZSHZ_OWNER` is the username the database belongs to; set it to your own login name to keep `z` working in a root shell, such as under `sudo -E -s`; [see below](#zshz_owner) (default: empty) * `ZSHZ_TILDE` displays the name of the `HOME` directory as a `~` (default: `0`) * `ZSHZ_TRAILING_SLASH` makes it so that a search pattern ending in `/` can match the final element in a path; e.g., `z foo/` can match `/path/to/foo` (default: `0`) * `ZSHZ_UNCOMMON` changes the logic used to calculate the directory jumped to; [see below](#zshz_uncommon) (default: `0`) @@ -319,6 +356,12 @@ then there is no common prefix. In this case, `z code` will simply send you to t You may enable an alternate, experimental behavior by setting `ZSHZ_UNCOMMON=1`. If you do that, Zsh-z will not jump to a common prefix, even if one exists. Instead, it chooses the highest-ranking match -- but it drops any subdirectories that do not include the search term. So if you type `z bat` and `/home/me/code/bat` is the best match, that is exactly where you will end up. If, however, you had typed `z code` and the best match was also `/home/me/code/bat`, you would have ended up in `/home/me/code` (because `code` was what you had searched for). This feature is still in development, and feedback is welcome. +## `ZSHZ_OWNER` + +If you are using `root` privileges while keeping your personal home directory as `HOME` (as is the case with `sudo -E -s`), conflicts with file ownership arise. You can resolve them by using `ZSHZ_OWNER`. If you set this variable to your username, Zsh-z will be able to use your personal datafile, restoring its proper ownership with every write. + +Setting `ZSHZ_OWNER` makes one thing stricter, because a privileged shell is then writing to a path that an unprivileged user controls: Zsh-z follows a symlink on the way to the datafile -- the file itself, or any parent directory -- only if `root` owns the link, and reports an error instead of writing otherwise. Symlinked system directories, such as `/home` → `/usr/home` on the BSDs or `/var` → `/private/var` on macOS, belong to root and still resolve. With `ZSHZ_OWNER` unset none of this applies: a symlinked `~/.z` is followed exactly as before, so pointing it at synced storage keeps working. + ## Making `--add` Work for You Zsh-z internally uses the `--add` option to add paths to its database. @zachriggle pointed out to me that users might want to use `--add` themselves, so I have altered it a little to make it more user-friendly. @@ -331,9 +374,33 @@ A good example might involve a directory tree that has Git repositories within i (As a Zsh user, I tend to use `**` instead of `find`, but it is good to see how deep your directory trees go before doing that.) +## Performance + +One of the goals of the rewrite that culminated in v2.0 was to make Zsh-z simultaneously more stable and faster. On modern Zsh, Zsh-z outpaces `rupa/z`'s `z.sh` at adding, searching, and listing. Representative figures (N = 200 database entries, medians of seven interleaved runs on a Core i7-12700 desktop under WSL2): + +**Modern Zsh (5.9) -- Zsh-z vs. `rupa/z`:** + +| Operation | `rupa/z` (`z.sh`) | Zsh-z | Winner | +| --------- | ----------------- | ----------- | ----------------- | +| `add` | 6.25 ms/op | 1.99 ms/op | **Zsh-z** ~3.14x | +| `search` | 6.51 ms/op | 3.62 ms/op | **Zsh-z** ~1.80x | +| `list` | 8.93 ms/op | 4.36 ms/op | **Zsh-z** ~2.05x | + +**Zsh 4.3.11 (the oldest supported release) -- Zsh-z vs. `rupa/z`:** + +| Operation | `rupa/z` (`z.sh`) | Zsh-z | Winner | +| --------- | ----------------- | ----------- | ----------------- | +| `add` | 5.00 ms/op | 2.92 ms/op | **Zsh-z** ~1.71x | +| `search` | 4.93 ms/op | 4.47 ms/op | **Zsh-z** ~1.10x | +| `list` | 7.92 ms/op | 6.08 ms/op | **Zsh-z** ~1.30x | + +Removal is absent from these tables: in `rupa/z`, the per-prompt hook re-adds the current directory right after `z -x` removes it, so the two implementations' `-x` are not doing comparable work. + +Relative to the previous generation of Zsh-z, the v2.0 read path is dramatically faster -- on modern Zsh, listing the whole database is about 2.4x faster and searching about 1.6x faster. + ## Other Improvements to the Original Functionality of `rupa/z` -* `z -x` works, with the help of `chpwd_functions`. +* `z -x` works: a directory you remove stays removed. * Zsh-z is compatible with Solaris. * Zsh-z uses the "new" `zshcompsys` completion system instead of the old `compctl` one. * No error message is displayed when the database file has not yet been created. @@ -355,12 +422,10 @@ If you are coming to Zsh-z (or even to the original `rupa/z`, for that matter) f ## `COMPLETE_ALIASES` -`z`, or any alternative you set up using `$ZSHZ_CMD` or `$_Z_CMD`, is an alias. `setopt COMPLETE_ALIASES` divorces the tab completion for aliases from the underlying commands they invoke, so if you enable `COMPLETE_ALIASES`, tab completion for Zsh-z will be broken. You can get it working again, however, by adding under +`z`, or any alternative you set up using `$ZSHZ_CMD` or `$_Z_CMD`, is an alias. `setopt COMPLETE_ALIASES` divorces the tab completion for aliases from the underlying commands they invoke, which historically broke Zsh-z's tab completion. Zsh-z now handles this automatically: the first Tab press registers the alias name with `_zshz` so completion works under `COMPLETE_ALIASES` without any extra setup. - setopt COMPLETE_ALIASES - -the line +That registration happens inside Zsh-z's own Tab widget. If a plugin loaded after Zsh-z replaces the Tab binding without invoking the previous widget, Zsh-z's widget never runs and the registration does not happen. Under `COMPLETE_ALIASES`, you would then have no completion for `z`. Once `compinit` has run, adding compdef _zshz ${ZSHZ_CMD:-${_Z_CMD:-z}} -That will re-bind `z` or the command of your choice to the underlying Zsh-z function. +below `setopt COMPLETE_ALIASES` in your `.zshrc` fixes that, and it is harmless if the automatic registration has already run. diff --git a/plugins/z/_z b/plugins/z/_z index a109841d6..09e70af43 100644 --- a/plugins/z/_z +++ b/plugins/z/_z @@ -1,6 +1,6 @@ #compdef zshz ${ZSHZ_CMD:-${_Z_CMD:-z}} # -# Zsh-z - jump around with Zsh - A native Zsh version of z without awk, sort, +# Zsh-z - jump around with Zsh - A native Zsh version of rupa/z without awk, sort, # date, or sed # # https://github.com/agkozak/zsh-z @@ -24,11 +24,7 @@ # LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE # SOFTWARE. -# -# z (https://github.com/rupa/z) is copyright (c) 2009 rupa deadwyler and -# licensed under the WTFPL license, Version 2.a -# -# shellcheck shell=ksh +################################################################################ ############################################################ # Zsh-z COMPLETIONS diff --git a/plugins/z/z.plugin.zsh b/plugins/z/z.plugin.zsh index 6df49822e..d922fa77c 100644 --- a/plugins/z/z.plugin.zsh +++ b/plugins/z/z.plugin.zsh @@ -1,6 +1,6 @@ ################################################################################ -# Zsh-z - jump around with Zsh - A native Zsh version of z without awk, sort, -# date, or sed +# Zsh-z - jump around with Zsh - A native Zsh version of rupa/z without awk, +# sort, date, or sed # # https://github.com/agkozak/zsh-z # @@ -24,9 +24,6 @@ # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE # SOFTWARE. # -# z (https://github.com/rupa/z) is copyright (c) 2009 rupa deadwyler and -# licensed under the WTFPL license, Version 2. -# # Zsh-z maintains a jump-list of the directories you actually use. # # INSTALL: @@ -57,22 +54,45 @@ # ZSHZ_COMPLETION -> completion method (default: 'frecent'; 'legacy' for # alphabetic sorting) # ZSHZ_DATA -> name of datafile (default: ~/.z) +# ZSHZ_DEBUG -> if set, turn on debugging aids: WARN_CREATE_GLOBAL while the +# command runs and per-function warnings (functions -W) at load time +# (default: unset) +# ZSHZ_ECHO -> if 1, print the directory name after jumping to it (default: 0) # ZSHZ_EXCLUDE_DIRS -> array of directories to exclude from your database # (default: empty) # ZSHZ_KEEP_DIRS -> array of directories that should not be removed from the # database, even if they are not currently available (default: empty) +# ZSHZ_LOCK_TIMEOUT -> seconds to wait for the lockfile before giving up +# (default: 1) # ZSHZ_MAX_SCORE -> maximum combined score the database entries can have # before beginning to age (default: 9000) # ZSHZ_NO_RESOLVE_SYMLINKS -> '1' prevents symlink resolution # ZSHZ_OWNER -> your username (if you want use Zsh-z while using sudo -s) +# ZSHZ_TILDE -> if 1, display ~ in place of the full $HOME path in output +# (default: 0) +# ZSHZ_TRAILING_SLASH -> if 1, a query ending in / matches at the end of a +# directory path (default: 0) # ZSHZ_UNCOMMON -> if 1, do not jump to "common directories," but rather drop # subdirectories based on what the search string was (default: 0) ################################################################################ -autoload -U is-at-least +# Minimalistic solution to allow this plugin to keep running under sh/bash/ksh +# emulation while continuing to use Zsh-only syntax features. `emulate zsh -c' +# evaluates its argument as code, so the script's own path -- `${(%):-%N}' -- +# must be `${(q)}'-quoted; otherwise an install directory containing spaces or +# other shell-special characters (common on Cygwin/MSYS2 and macOS, where a +# home directory can be "C:\Users\John Smith" or "/Users/John Smith") would be +# word-split and the plugin would silently fail to re-source. +if [[ -o KSH_ARRAYS || -o SH_WORD_SPLIT ]]; then + emulate zsh -c "source ${(q)${(%):-%N}}" + return $? +fi + +autoload -Uz is-at-least if ! is-at-least 4.3.11; then - print "Zsh-z requires Zsh v4.3.11 or higher." >&2 && exit + print "Zsh-z requires Zsh v4.3.11 or higher." >&2 + return 1 2> /dev/null || exit 1 fi ############################################################ @@ -99,6 +119,56 @@ With no ARGUMENT, list the directory history in ascending rank. fold -s -w $(( COLUMNS > 0 ? COLUMNS : 80 )) >&2 } +############################################################ +# Canonicalize a path in the manner of `:A' -- normalize it +# lexically as `:a' does, then resolve symlinks -- without +# requiring any of the path to exist. +# +# `${x:A}' itself cannot be trusted with a missing path on +# Zsh 4.3.11: when the top-level component of $x does not +# exist (`/gone/sub'), the realpath machinery segfaults the +# shell (upstream bug, 4.3.11 only; deeper missing +# components are handled correctly on every version). So +# apply `:A' only to the deepest ancestor of the path that +# exists -- `:A' on an existing path is safe everywhere -- +# and reattach the missing components verbatim. That +# reproduces `:A' exactly: `:A' resolves the symlinks in the +# existing prefix and carries the nonexistent tail +# unchanged, and the tail cannot contain live symlinks +# precisely because it does not exist. (A broken symlink +# stops the ancestor walk without being resolved -- `-e' +# fails on one -- which also matches `:A', which leaves +# broken symlinks unresolved.) +# +# Arguments: +# $1 The path to canonicalize +# +# Returns the canonical path in $REPLY. +############################################################ +_zshz_realpath() { + local dir=${1:a} + local -a tail + + # `:h' at its fixed point (`/', or `//' where the OS treats that as + # distinct) can climb no higher; if even that much of the path does not + # exist, settle for the lexical normalization rather than hand `:A' + # something dangerous. + while [[ ! -e $dir && $dir != "${dir:h}" ]]; do + tail=( "${dir:t}" "${tail[@]}" ) + dir=${dir:h} + done + [[ -e $dir ]] && dir=${dir:A} + + # `typeset -g': REPLY belongs to the caller by design. A plain assignment + # would trip WARN_NESTED_VAR under `ZSHZ_DEBUG', since _zshz_realpath is a + # top-level function and thus one of the ones `functions -W' marks. + if (( ${#tail} )); then + typeset -g REPLY=${dir%/}/${(j:/:)tail} + else + typeset -g REPLY=$dir + fi +} + # Load zsh/datetime module, if necessary (( ${+EPOCHSECONDS} )) || zmodload zsh/datetime @@ -106,16 +176,26 @@ With no ARGUMENT, list the directory history in ascending rank. typeset -gA ZSHZ # Fallback utilities in case Zsh lacks zsh/files (as is the case with MobaXterm) +ZSHZ[CHMOD]='chmod' ZSHZ[CHOWN]='chown' ZSHZ[MV]='mv' ZSHZ[RM]='rm' -# Try to load zsh/files utilities + +# Try to load zsh/files. zf_chown, zf_mv, and zf_rm are usually present in Zsh +# 4.3.11. zf_chmod only became available in Zsh 5.0, so we load it separately +# below. If zsh/files is not available at all, we silently fall back to the +# external utilities chmod, chown, mv, and rm. if [[ ${builtins[zf_chown]-} != 'defined' || ${builtins[zf_mv]-} != 'defined' || ${builtins[zf_rm]-} != 'defined' ]]; then zmodload -F zsh/files b:zf_chown b:zf_mv b:zf_rm &> /dev/null fi -# Use zsh/files, if it is available + +[[ ${builtins[zf_chmod]-} == 'defined' ]] || + zmodload -F zsh/files b:zf_chmod &> /dev/null + +# Use zsh/files, if it is available. +[[ ${builtins[zf_chmod]-} == 'defined' ]] && ZSHZ[CHMOD]='zf_chmod' [[ ${builtins[zf_chown]-} == 'defined' ]] && ZSHZ[CHOWN]='zf_chown' [[ ${builtins[zf_mv]-} == 'defined' ]] && ZSHZ[MV]='zf_mv' [[ ${builtins[zf_rm]-} == 'defined' ]] && ZSHZ[RM]='zf_rm' @@ -130,8 +210,44 @@ fi # Determine if zsystem flock is available zsystem supports flock &> /dev/null && ZSHZ[USE_FLOCK]=1 -# Determine if `print -v' is supported -is-at-least 5.3.0 && ZSHZ[PRINTV]=1 +# Windows only: how many times to retry a datafile rename that fails. +# +# On Cygwin and MSYS2, rename() fails with EBUSY or EACCES whenever another +# process holds the tempfile or the datafile open without FILE_SHARE_DELETE -- +# which is precisely what a virus scanner or the search indexer does to a file +# in the moments after it is created. Since the write path below creates the +# tempfile and renames it over the datafile microseconds later, that window is +# wide open. The rename's stderr is discarded there, so a scan that lands in +# the window silently loses an `--add' or a `-x': no message, no delay, just a +# directory that never made it into the database. The condition clears in +# milliseconds, so make a few more attempts before giving up. +# +# Everywhere else a failed rename means something real -- ENOSPC, EPERM, a +# cross-device move -- that retrying cannot fix and would only add latency to, +# so ZSHZ[MV_RETRIES] stays unset and the loops below make a single attempt, +# exactly as before. +# +# zsh/zselect provides the sub-second delay between attempts without forking +# /bin/sleep, whose fractional-seconds support is not portable in any case. +# MobaXterm's cut-down Cygwin does not ship zsh/zselect, so there +# ZSHZ[MV_RETRY_DELAY] stays unset and the retries happen back to back -- still +# worth making, since the scanner's handle is often gone by the next attempt. +# +# Four retries at 50ms is deliberately modest rather than generous. The rename +# runs while the lockfile is held, so every millisecond spent retrying is a +# millisecond other writers spend waiting, and they give up after +# ZSHZ_LOCK_TIMEOUT (1s by default) -- silently, since their adds are +# best-effort too. A budget that outlasts a large fraction of that timeout +# would trade one process's lost write for several others'. Measured on MSYS2 +# against a handle held open with FILE_SHARE_READ, this recovers renames +# blocked for up to ~0.3s, comfortably more than a scan of a file this small +# takes. +if [[ $OSTYPE == (cygwin|msys) ]]; then + ZSHZ[MV_RETRIES]=4 + [[ ${modules[zsh/zselect]-} == 'loaded' ]] || zmodload zsh/zselect &> /dev/null + # In hundredths of a second, per `zselect -t' + [[ ${builtins[zselect]-} == 'defined' ]] && ZSHZ[MV_RETRY_DELAY]=5 +fi ############################################################ # The Zsh-z Command @@ -145,6 +261,7 @@ is-at-least 5.3.0 && ZSHZ[PRINTV]=1 # ZSHZ_DEBUG # ZSHZ_EXCLUDE_DIRS # ZSHZ_KEEP_DIRS +# ZSHZ_LOCK_TIMEOUT # ZSHZ_MAX_SCORE # ZSHZ_OWNER # @@ -163,42 +280,136 @@ zshz() { # Allow the user to specify a custom datafile in $ZSHZ_DATA (or legacy $_Z_DATA) local custom_datafile="${ZSHZ_DATA:-$_Z_DATA}" + # $_zshz_quiet_add marks the automatic bookkeeping add that _zshz_precmd + # runs in a `&!' fork before every prompt (_zshz_precmd declares it `local', + # so it is visible here only through that one call). A fork cannot + # record anything in the parent shell, so it has no way to warn just once: + # an unusable $ZSHZ_DATA would otherwise put the same diagnostic on the + # terminal at every prompt for the life of the shell. Stay quiet on that + # path and leave the complaining to the entry points the user actually + # invoked -- including a hand-typed `z --add', which is not marked and so + # still reports. + local quiet + [[ -n ${_zshz_quiet_add-} ]] && quiet=1 + # If a datafile was provided as a standalone file without a directory path - # print a warning and exit + # print a warning and return if [[ -n ${custom_datafile} && ${custom_datafile} != */* ]]; then - print "ERROR: You configured a custom Zsh-z datafile (${custom_datafile}), but have not specified its directory." >&2 - exit + (( quiet )) || + print "ERROR: You configured a custom Zsh-z datafile (${custom_datafile}), but have not specified its directory." >&2 + return 1 + fi + + # Refuse a symlinked datafile while $ZSHZ_OWNER is set, rather than + # following it. That variable means root is acting for an unprivileged user + # -- the documented `sudo -s' setup -- and the resolution just below + # deliberately dereferences a link, so in that configuration Zsh-z would + # write the database wherever a name inside the user's own home points, with + # root's authority. Nothing has to be raced: the link is planted before the + # privileged shell ever starts. Unprivileged use crosses no such boundary and + # keeps the dereference, which is what makes pointing `.z' at synced storage + # work. + # + # Every component, not just the last. Resolution walks the whole path, so a + # symlinked *parent* redirects it just as effectively: with `link' -> `/etc' + # inside a user's home, a datafile of `~/link/passwd' resolves to + # `/etc/passwd' and root rewrites it. + # + # Judged by who owns each link rather than by its mere presence. Symlinked + # system directories are ordinary -- `/home' -> `/usr/home' on the BSDs, + # `/var' -> `/private/var' on macOS -- and refusing those would break Zsh-z + # under $ZSHZ_OWNER on those systems for nothing. Those are root's; what this + # has to reject is a link an unprivileged owner could have planted. `zstat + # -L' reports the link's own owner rather than its target's, which is the + # distinction `-O' cannot make. + if [[ -n ${ZSHZ_OWNER:-${_Z_OWNER}} ]]; then + local _zshz_df=${custom_datafile:-$HOME/.z} + [[ $_zshz_df == /* ]] || _zshz_df="$PWD/$_zshz_df" + zmodload -F zsh/stat b:zstat 2> /dev/null + local _zshz_pfx _zshz_part _zshz_luid + for _zshz_part in ${(s:/:)_zshz_df}; do + [[ -n $_zshz_part ]] || continue + _zshz_pfx+="/$_zshz_part" + [[ -L $_zshz_pfx ]] || continue + # Without zsh/stat there is no way to tell whose link this is, so refuse + # it rather than guess: this path is privileged by definition. + _zshz_luid='' + (( ${+builtins[zstat]} )) && + _zshz_luid=$(zstat -L +uid "$_zshz_pfx" 2> /dev/null) + if [[ $_zshz_luid != 0 ]]; then + (( quiet )) || + print "ERROR: Zsh-z will not follow the symlink ${_zshz_pfx} on the way to its datafile while ZSHZ_OWNER is set." >&2 + return 1 + fi + done fi # If the user specified a datafile, use that or default to ~/.z - # If the datafile is a symlink, it gets dereferenced - local datafile=${${custom_datafile:-$HOME/.z}:A} + # If the datafile is a symlink, it gets dereferenced (except under + # $ZSHZ_OWNER, refused just above). Canonicalized with + # _zshz_realpath rather than a bare `:A', which would segfault Zsh 4.3.11 + # on a $ZSHZ_DATA pointing into a missing top-level directory -- at every + # prompt, since this line runs in the backgrounded precmd add. + _zshz_realpath "${custom_datafile:-$HOME/.z}" + local datafile=$REPLY + # Clear REPLY as soon as it is captured: the matching machinery below + # relies on it staying empty until a common root or best match is put in + # it (_zshz_find_common_root only assigns REPLY when it finds a root), so + # a datafile path left in REPLY here would surface as a bogus match. + REPLY='' - # If the datafile is a directory, print a warning and exit + # If the datafile is a directory, print a warning and return if [[ -d $datafile ]]; then - print "ERROR: Zsh-z's datafile (${datafile}) is a directory." >&2 - exit + (( quiet )) || + print "ERROR: Zsh-z's datafile (${datafile}) is a directory." >&2 + return 1 fi # Make sure that the datafile exists before attempting to read it or lock it - # for writing - [[ -f $datafile ]] || { mkdir -p "${datafile:h}" && touch "$datafile" } + # for writing. Create it with 0600 permissions from the first instant (umask + # in a subshell) rather than chmodding it afterward: this creation runs + # before the lock is taken, and on Cygwin/MSYS2 a concurrent writer's rename + # passes through a window in which the datafile is unlinked or delete- + # pending, so any second syscall on the path (chmod) -- or even the creating + # open itself -- can fail spuriously. Append mode (>>) creates the file + # without truncating one that a concurrent writer has just renamed into + # place. The first attempt is silent; if the file still does not exist + # afterward (so no concurrent writer supplied it), retry loudly so that real + # failures (directory permissions, read-only filesystem) reach the user. + [[ -f $datafile ]] || { + mkdir -p "${datafile:h}" && + ( umask 077; : >> "$datafile" ) 2> /dev/null || + [[ -f $datafile ]] || + ( umask 077; : >> "$datafile" ) + # When $ZSHZ_OWNER is set (e.g. under `sudo -s'), hand the freshly created + # file off to that user immediately, so a query-only invocation can't leave + # behind a root-owned .z that the normal-user shell can't read. `-h' so a + # symlink that appeared since the check above is retitled itself rather + # than dereferenced onto its target. + local _owner=${ZSHZ_OWNER:-${_Z_OWNER}} + [[ -n $_owner ]] && + ${ZSHZ[CHOWN]} -h "${_owner}:$(id -ng "${_owner}")" "$datafile" + } + + # If the datafile still does not exist, the loud retry above has already + # said why; nothing below -- reading, locking, writing -- can succeed + # without it, and each failure would add its own noise. Bailing out here + # matters most on Zsh 4.3.11, where the failed `$(< $datafile)' reads + # below are fatal to a non-interactive shell. + [[ -f $datafile ]] || return 1 # Bail if we don't own the datafile and $ZSHZ_OWNER is not set [[ -z ${ZSHZ_OWNER:-${_Z_OWNER}} && -f $datafile && ! -O $datafile ]] && return - # Load the datafile into an array and parse it - lines=( ${(f)"$(< $datafile)"} ) - # Discard entries that are incomplete or incorrectly formatted - lines=( ${(M)lines:#/*\|[[:digit:]]##[.,]#[[:digit:]]#\|[[:digit:]]##} ) - ############################################################ # Add a path to or remove one from the datafile # # Globals: # ZSHZ # ZSHZ_EXCLUDE_DIRS + # ZSHZ_LOCK_TIMEOUT + # ZSHZ_NO_RESOLVE_SYMLINKS # ZSHZ_OWNER # # Arguments: @@ -206,17 +417,21 @@ zshz() { # $2 The path to add ############################################################ _zshz_add_or_remove_path() { - local action=${1} + local action=$1 shift if [[ $action == '--add' ]]; then - # TODO: The following tasks are now handled by _agkozak_precmd. Dead code? + # These $HOME / $ZSHZ_EXCLUDE_DIRS guards mirror the ones in + # _zshz_precmd, but they are not redundant: precmd filters $PWD as an + # early-out (skip the background fork), whereas --add is now a public + # entry point and must enforce the same policies as the precmd function. + # Keep both in sync. # Don't add $HOME [[ $* == $HOME ]] && return - # Don't track directory trees excluded in ZSHZ_EXCLUDE_DIRS + # Don't track directory trees excluded in $ZSHZ_EXCLUDE_DIRS local exclude for exclude in ${(@)ZSHZ_EXCLUDE_DIRS:-${(@)_Z_EXCLUDE_DIRS}}; do case $* in @@ -225,97 +440,382 @@ zshz() { done fi - # A temporary file that gets copied over the datafile if all goes well - local tempfile="${datafile}.${RANDOM}" - - # See https://github.com/rupa/z/pull/199/commits/ed6eeed9b70d27c1582e3dd050e72ebfe246341c - if (( ZSHZ[USE_FLOCK] )); then - - local lockfd - - # Grab exclusive lock (released when function exits) - zsystem flock -f lockfd "$datafile" 2> /dev/null || return - - fi - - integer tmpfd - case $action in - --add) - exec {tmpfd}>|"$tempfile" # Open up tempfile for writing - _zshz_update_datafile $tmpfd "$*" - local ret=$? - ;; - --remove) - local xdir # Directory to be removed - - if (( ${ZSHZ_NO_RESOLVE_SYMLINKS:-${_Z_NO_RESOLVE_SYMLINKS}} )); then - [[ -d ${${*:-${PWD}}:a} ]] && xdir=${${*:-${PWD}}:a} - else - [[ -d ${${*:-${PWD}}:A} ]] && xdir=${${*:-${PWD}}:A} - fi - - local -a lines_to_keep - if (( ${+opts[-R]} )); then - # Prompt user before deleting entire database - if [[ $xdir == '/' ]] && ! read -q "?Delete entire Zsh-z database? "; then - print && return 1 - fi - # All of the lines that don't match the directory to be deleted - lines_to_keep=( ${lines:#${xdir}\|*} ) - # Or its subdirectories - lines_to_keep=( ${lines_to_keep:#${xdir%/}/**} ) - else - # All of the lines that don't match the directory to be deleted - lines_to_keep=( ${lines:#${xdir}\|*} ) - fi - if [[ $lines != "$lines_to_keep" ]]; then - lines=( $lines_to_keep ) - else - return 1 # The $PWD isn't in the datafile - fi - exec {tmpfd}>|"$tempfile" # Open up tempfile for writing - print -u $tmpfd -l -- $lines - local ret=$? - ;; - esac - - if (( tmpfd != 0 )); then - # Close tempfile - exec {tmpfd}>&- - fi - - if (( ret != 0 )); then - # Avoid clobbering the datafile if the write to tempfile failed - ${ZSHZ[RM]} -f "$tempfile" - return $ret - fi - - local owner - owner=${ZSHZ_OWNER:-${_Z_OWNER}} - - if (( ZSHZ[USE_FLOCK] )); then - # An unusual case: if inside Docker container where datafile could be bind - # mounted - if [[ -f '/.dockerenv' || ( -r '/proc/1/cgroup' && "$(< '/proc/1/cgroup')" == *docker* ) ]]; then - print "$(< "$tempfile")" > "$datafile" 2> /dev/null - ${ZSHZ[RM]} -f "$tempfile" - # All other cases + # Resolve the directory to be removed, and confirm a full-database wipe, + # *before* taking the lock. Both are independent of the datafile, and the + # confirmation is interactive: holding the lock across a `read -q' the user + # might walk away from would make concurrent writers in other shells time + # out on ZSHZ_LOCK_TIMEOUT and silently drop their adds while the prompt + # sits open. A lock should wrap the read-modify-write, never a question. + local xdir # Directory to be removed + if [[ $action == '--remove' ]]; then + # The target is canonicalized without any existence test: an entry + # whose directory has since been deleted is exactly the one a user most + # wants out of the database. _zshz_realpath resolves a missing path the + # way `:A' resolves one -- and, unlike a bare `:A', cannot segfault Zsh + # 4.3.11 on a path whose top-level component is gone. (The old + # `[[ -d ${...:A} ]]' guard offered no protection there: the `:A' + # expands, and crashes, before `-d' ever sees it.) + if (( ${ZSHZ_NO_RESOLVE_SYMLINKS:-${_Z_NO_RESOLVE_SYMLINKS}} )); then + xdir=${${*:-${PWD}}:a} else - ${ZSHZ[MV]} "$tempfile" "$datafile" 2> /dev/null || - ${ZSHZ[RM]} -f "$tempfile" + _zshz_realpath "${*:-${PWD}}" + xdir=$REPLY fi - if [[ -n $owner ]]; then - ${ZSHZ[CHOWN]} ${owner}:"$(id -ng ${owner})" "$datafile" + # Both branches above yield a non-empty absolute path, and that + # matters: under `-R' an empty $xdir would collapse the subtree filter + # below into `${lines_to_keep:#/**}', which matches every line in the + # datafile and erases the lot -- silently, since the whole-database + # confirmation just below tests for `/' rather than for emptiness. Keep + # this guard in case a future change lets an empty resolution through. + [[ -n $xdir ]] || return 1 + + if (( ${+opts[-R]} )) && [[ $xdir == '/' ]]; then + if ! read -q "?Delete entire Zsh-z database? "; then + print && return 1 + fi fi - else - if [[ -n $owner ]]; then - ${ZSHZ[CHOWN]} "${owner}":"$(id -ng "${owner}")" "$tempfile" - fi - ${ZSHZ[MV]} -f "$tempfile" "$datafile" 2> /dev/null || - ${ZSHZ[RM]} -f "$tempfile" fi + # A temporary file that gets copied over the datafile if all goes well + local tempfile="${datafile}.${RANDOM}" lockfile="${datafile}.lock" + integer lockfd=0 + # The no-flock fallback's lock. Deliberately a *different* name from + # $lockfile: a plain file left behind by a flock-capable Zsh would make + # `mkdir' fail forever on the same path, deadlocking every later write. + local lockdir="${datafile}.lock.d" + integer lockdir_held=0 + + { + # Using zsystem flock + if (( ZSHZ[USE_FLOCK] )); then + + # Obtain an exclusive lock on the lockfile. + # + # Locking the datafile directly would not actually serialize concurrent + # writers, since the datafile gets replaced by mv and each new datafile + # has a new inode -- so a separate, stable lockfile is needed. + # + # Bound the lock acquisition (default 1s, override with ZSHZ_LOCK_TIMEOUT) + # so a stuck holder can't stall the backgrounded precmd add or freeze a + # user's foreground `z --add' / `z -x'. Once the holder dies, the kernel + # frees the lock and the next add succeeds automatically -- no manual + # `rm ~/.z.lock' needed. + # + # On timeout we return silently and on purpose: the precmd add is + # best-effort and runs backgrounded (`&!'), so there is nowhere useful + # to report to -- a message would land on the terminal asynchronously, + # mid-keystroke, possibly every prompt. To diagnose a database that has + # stopped updating, run a foreground `z --add .' and check `$?': a + # nonzero status means the write did not happen -- 2 is a lock- + # acquisition timeout (contention, or a raised ZSHZ_LOCK_TIMEOUT is + # still too low), 1 is a permissions or ownership problem (e.g. a stale + # root-owned lockfile left by an earlier `sudo -s' session, or a + # symlinked lockfile refused under $ZSHZ_OWNER). + # Create the lockfile 0600-from-birth and silently (umask in a + # subshell), mirroring the datafile creation above rather than a bare + # `touch' under the ambient umask with unsuppressed stderr. zsystem + # flock opens the lockfile O_RDWR, so under `sudo -s' with $ZSHZ_OWNER + # the unprivileged user must be able to open it: hand it off at + # creation, not only after a successful write -- a timed-out or failed + # first write by root would skip the post-write chown and leave a + # root-owned lockfile, turning every later user --add / -x into a + # silently-swallowed EACCES no-op. The lockfile is deliberately never + # removed: unlinking one a waiter has already opened reintroduces the + # two-inodes race the stable lockfile exists to prevent. + # Under $ZSHZ_OWNER all of this runs with root's authority on a path the + # unprivileged owner controls, and every step follows a symlink: `-f' + # tests the target, `>>' creates a dangling one, and flock opens it. + # $datafile survives a planted link only because the `mv' below replaces + # it outright; the lockfile is deliberately never removed, so a symlink + # here would persist and be acted on at every subsequent write. Refuse. + local _lock_owner=${ZSHZ_OWNER:-${_Z_OWNER}} + [[ -n $_lock_owner && -L $lockfile ]] && return 1 + if [[ ! -f $lockfile ]]; then + ( umask 077; : >> "$lockfile" ) 2> /dev/null + [[ -n $_lock_owner ]] && + ${ZSHZ[CHOWN]} -h "${_lock_owner}:$(id -ng "${_lock_owner}")" "$lockfile" + fi + zsystem flock -t ${ZSHZ_LOCK_TIMEOUT:-1} -f lockfd "$lockfile" 2> /dev/null || return + + else + + # No `zsystem flock' here. MobaXterm's cut-down Cygwin is the case that + # matters -- it ships no `zsh/system' at all -- and until now this path + # wrote with nothing serializing it: every writer read its own snapshot + # and the last `mv' won. Measured on MobaXterm, an entry added by one of + # four concurrent writers went missing in 7 runs out of 10. + # + # `mkdir' is the portable atomic primitive: it succeeds for exactly one + # caller and fails for the rest, with no module behind it. What it does + # not give us is the kernel's release-on-death, which is the whole + # reason `flock' is preferred where it exists -- so a holder that dies + # would wedge every later write. Hence the staleness sweep below. + # + # Failure to acquire returns 2, the same status the flock branch's + # timeout produces and the one the README documents for contention. + integer _zshz_deadline=$(( EPOCHSECONDS + ${ZSHZ_LOCK_TIMEOUT:-1} )) + local -a _zshz_stale + while :; do + if mkdir "$lockdir" 2> /dev/null; then + lockdir_held=1 + break + fi + # Break a lock nobody can still be holding. A write is a matter of + # milliseconds, so a lock directory older than 30 seconds means its + # owner died without releasing it. `mkdir' stamps the mtime at + # creation and no holder touches it afterwards, so the age is the + # hold time. `$lockdir' expands literally here -- only the qualifier + # is glob syntax -- so a datafile path containing `[' or `*' is safe. + _zshz_stale=( ${lockdir}(Nms+30) ) + if (( ${#_zshz_stale} )); then + rmdir "$lockdir" 2> /dev/null && continue + fi + (( EPOCHSECONDS >= _zshz_deadline )) && return 2 + # No `zselect' on the platforms that land here, so this costs a fork. + # It is the slow path already, and spinning would be worse. + sleep 0.05 2> /dev/null || : + done + + fi + + # Read the datafile only after obtaining the lock, so concurrent --add + # calls don't all act on the same stale snapshot. + lines=( ${(f)"$(< $datafile)"} ) + # Discard entries that are incomplete or incorrectly formatted + lines=( ${(M)lines:#/*\|[[:digit:]]##[.,]#[[:digit:]]#\|[[:digit:]]##} ) + + # Hold the fd in an *unset* scalar, not `integer tmpfd' (which seeds it + # with 0). On some Zsh builds, `exec {tmpfd}>|...' refuses to clobber a + # parameter already holding a number that names an open fd -- and 0 is + # stdin, always open -- yielding "can't clobber parameter tmpfd + # containing file descriptor 0". An empty scalar isn't a valid fd, so + # the guard never fires. See https://github.com/agkozak/zsh-z/issues/81 + local tmpfd + case $action in + --add) + # When zf_chmod isn't available (Zsh 4.3.11), avoid the + # ~900us fork+execve of external /usr/bin/chmod on every + # write. Create the tempfile with mode 0600 from the start + # via `umask 077' inside a subshell -- the umask change is + # contained to the forked child process and the OS prevents + # it from leaking back to the parent. Subshell fork without + # exec is ~50us, ~18x cheaper than the chmod fallback. + if [[ ${ZSHZ[CHMOD]} == 'zf_chmod' ]]; then + exec {tmpfd}>|"$tempfile" # Open up tempfile for writing + # Fail closed. The tempfile is born with the ambient umask (0666 + # under `umask 000'), and it is this inode -- not the datafile's -- + # that the rename below publishes, so a chmod whose failure went + # unnoticed would replace a 0600 datafile with a world-readable one + # and still report success. Nothing has been written yet, so + # there is no salvage: drop the tempfile and leave the database as + # it was. + if ! ${ZSHZ[CHMOD]} 600 "$tempfile"; then + exec {tmpfd}>&- + ${ZSHZ[RM]} -f "$tempfile" 2> /dev/null + return 1 + fi + _zshz_update_datafile $tmpfd "$*" + else + ( umask 077 + exec {tmpfd}>|"$tempfile" + _zshz_update_datafile $tmpfd "$*" ) + fi + local ret=$? + ;; + --remove) + # $xdir was resolved before the lock, and for `-xR /' the + # whole-database wipe was already confirmed there. + local -a lines_to_keep + if (( ${+opts[-R]} )); then + # All of the lines that don't match the directory to be deleted + lines_to_keep=( ${lines:#${xdir}\|*} ) + # Or its subdirectories + lines_to_keep=( ${lines_to_keep:#${xdir%/}/**} ) + else + # All of the lines that don't match the directory to be deleted + lines_to_keep=( ${lines:#${xdir}\|*} ) + fi + if [[ $lines != "$lines_to_keep" ]]; then + lines=( $lines_to_keep ) + else + return 1 # The $PWD isn't in the datafile + fi + # Same umask-subshell pattern as --add: avoid the external + # chmod when zf_chmod isn't available. + if [[ ${ZSHZ[CHMOD]} == 'zf_chmod' ]]; then + exec {tmpfd}>|"$tempfile" # Open up tempfile for writing + # Fail closed, exactly as on the --add path above. + if ! ${ZSHZ[CHMOD]} 600 "$tempfile"; then + exec {tmpfd}>&- + ${ZSHZ[RM]} -f "$tempfile" 2> /dev/null + return 1 + fi + # `-r': $lines are verbatim on-disk lines (the datafile stores + # literal paths), so they must be written back unchanged. Without + # `-r', print would collapse an escape -- e.g. a literal `\t' in a + # path into a tab -- silently corrupting bystander entries. + print -u $tmpfd -rl -- $lines + else + ( umask 077; print -rl -- $lines >| "$tempfile" ) + fi + local ret=$? + ;; + esac + + if [[ -n $tmpfd ]]; then + # Close tempfile + exec {tmpfd}>&- + fi + + if (( ret != 0 )); then + # Avoid clobbering the datafile if the write to tempfile failed + ${ZSHZ[RM]} -f "$tempfile" + return $ret + fi + + integer write_ret chown_ret mv_attempts + local owner + owner=${ZSHZ_OWNER:-${_Z_OWNER}} + + if (( ZSHZ[USE_FLOCK] )); then + # An unusual case: if inside Docker container where datafile could be bind + # mounted + if [[ -f '/.dockerenv' || ( -r '/proc/1/cgroup' && "$(< '/proc/1/cgroup')" == *docker* ) ]]; then + # Secure the datafile *before* its contents land. This branch writes + # in place instead of renaming an already-0600 tempfile over the + # path, so asserting the mode afterwards -- as this did -- leaves a + # bind-mounted datafile that arrived permissive readable for the + # length of the write, and leaves it readable for good if the chmod + # fails and nothing checks. The mode carries across the truncating + # write below, which reuses this same inode. + if ! ${ZSHZ[CHMOD]} 600 "$datafile" 2> /dev/null; then + ${ZSHZ[RM]} -f "$tempfile" 2> /dev/null + return 1 + fi + # This is the one write path where a symlink at $datafile redirects + # real database content: the sibling branch renames a finished + # tempfile over the path, and a rename *replaces* a link rather than + # writing through it, while `>|' follows one. Under $ZSHZ_OWNER that + # content goes out with root's authority to a path an unprivileged + # owner controls, so a `-L' test ahead of the write is not enough -- + # the path can be swapped in between. + # + # `sysopen -o nofollow' settles it atomically, at open time, and the + # write goes through that descriptor. If it is unavailable (Zsh + # 4.3.11 has `zsystem flock' but no `sysopen' at all, and O_NOFOLLOW + # is not universal) or it refuses the open, the privileged write is + # refused rather than retried by a following one: this degrades to + # failing closed, never to writing unsafely. Without an owner set no + # privilege is crossed and the plain redirection stands. + # + # `chmod' above stays path-based -- Zsh has no `fchmod' -- so a swap + # can still misdirect it. Setting the mode on the wrong file is a far + # smaller matter than writing the database into it, and the write is + # what this closes. + local _zshz_dfd + if [[ -n $owner ]]; then + if (( ${+builtins[sysopen]} )) && + sysopen -o trunc,nofollow -w -u _zshz_dfd "$datafile" 2> /dev/null + then + print -u $_zshz_dfd -r -- "$(< "$tempfile")" 2> /dev/null + write_ret=$? + exec {_zshz_dfd}>&- + else + ${ZSHZ[RM]} -f "$tempfile" 2> /dev/null + return 1 + fi + else + # `-r': re-emit the tempfile's already-literal contents byte-for-byte. + print -r -- "$(< "$tempfile")" >| "$datafile" 2> /dev/null + write_ret=$? + fi + ${ZSHZ[RM]} -f "$tempfile" 2> /dev/null + # All other cases + else + # Retry a rename that a Windows sharing violation turned away; see + # the ZSHZ[MV_RETRIES] comment at the top of this file. Off Windows + # this loop makes the same single attempt it always has. Retrying is + # safe here: the rename happens under the lock, so no other writer + # can slip in between attempts. + while :; do + if ${ZSHZ[MV]} "$tempfile" "$datafile" 2> /dev/null; then + write_ret=0 + else + write_ret=$? + fi + (( write_ret == 0 )) && break + (( mv_attempts++ >= ${ZSHZ[MV_RETRIES]:-0} )) && break + if (( ${+ZSHZ[MV_RETRY_DELAY]} )); then + zselect -t ${ZSHZ[MV_RETRY_DELAY]} || : + fi + done + (( write_ret != 0 )) && ${ZSHZ[RM]} -f "$tempfile" 2> /dev/null + fi + # Preserve the write failure itself; best-effort tempfile cleanup must not + # turn a failed persist into a successful return. + (( write_ret == 0 )) || return $write_ret + + if [[ -n $owner ]]; then + # Chown the lockfile alongside the datafile: zsystem flock opens it + # O_RDWR, so if root creates it first under sudo -s, the unprivileged + # $ZSHZ_OWNER user's flock attempts would fail with EACCES (silently + # swallowed), turning --add and -x into no-ops. + # `-h' on both: the lockfile is never replaced, so a symlink planted + # there outlives any one write, and $datafile can be relinked in the + # window between the `mv' above and this line. Retitling the link + # itself -- which the owner already owns -- costs nothing, while + # dereferencing hands root's authority to whatever it names. + ${ZSHZ[CHOWN]} -h "${owner}:$(id -ng "${owner}")" "$datafile" "$lockfile" + chown_ret=$? + # Surface post-write chown failures too: the current write landed, but a + # wrong owner can break the next locked write. + (( chown_ret == 0 )) || return $chown_ret + fi + else + if [[ -n $owner ]]; then + ${ZSHZ[CHOWN]} -h "${owner}:$(id -ng "${owner}")" "$tempfile" + chown_ret=$? + if (( chown_ret != 0 )); then + # In the no-flock path, chown happens before the move, so clean up the + # tempfile and leave the live database untouched. + ${ZSHZ[RM]} -f "$tempfile" 2> /dev/null + return $chown_ret + fi + fi + # Same Windows sharing-violation retry as the flock branch above. This + # path is the one MobaXterm's cut-down Cygwin takes, and it has neither + # zsystem flock nor zsh/zselect, so the retries there run back to back. + while :; do + if ${ZSHZ[MV]} -f "$tempfile" "$datafile" 2> /dev/null; then + write_ret=0 + else + write_ret=$? + fi + (( write_ret == 0 )) && break + (( mv_attempts++ >= ${ZSHZ[MV_RETRIES]:-0} )) && break + if (( ${+ZSHZ[MV_RETRY_DELAY]} )); then + zselect -t ${ZSHZ[MV_RETRY_DELAY]} || : + fi + done + if (( write_ret != 0 )); then + ${ZSHZ[RM]} -f "$tempfile" 2> /dev/null + return $write_ret + fi + fi + } always { + # zsystem flock -f opens a real fd; explicitly unlock it so repeated + # foreground `z --add' / `z -x' invocations in the interactive shell + # don't leak lock descriptors and stall peers. (A backgrounded precmd + # child releases its fd on exit regardless; this matters for the parent.) + (( lockfd != 0 )) && zsystem flock -u $lockfd 2> /dev/null + # Release the mkdir lock on every exit from the block above, including + # the early `return's -- unlike an fd, a directory outlives the process + # that made it, so a missed release here is a wedged database rather than + # a leaked descriptor. Only if this call is the one that took it. + (( lockdir_held )) && rmdir "$lockdir" 2> /dev/null + } + # In order to make z -x work, we have to disable zsh-z's adding # to the database until the user changes directory and the # chpwd_functions are run @@ -346,31 +846,35 @@ zshz() { # See https://github.com/rupa/z/issues/246 local add_path=${(q)2} - local -a existing_paths local now=$EPOCHSECONDS line dir local path_field rank_field time_field count x + local -i keep rank[$add_path]=1 time[$add_path]=$now - # Remove paths from database if they no longer exist for line in $lines; do - if [[ ! -d ${line%%\|*} ]]; then + path_field=${line%%\|*} + + # Filter non-existent paths (honoring ZSHZ_KEEP_DIRS) inline so + # we walk $lines once instead of twice. The `keep=1; break' also + # fixes a latent bug: the previous existence-check loop had no + # `break' after appending, so a non-existent path matching + # multiple ZSHZ_KEEP_DIRS patterns was processed more than once. + if [[ ! -d $path_field ]]; then + keep=0 for dir in ${(@)ZSHZ_KEEP_DIRS}; do - if [[ ${line%%\|*} == ${dir}/* || - ${line%%\|*} == $dir || - $dir == '/' ]]; then - existing_paths+=( $line ) + if [[ $path_field == ${dir}/* || $path_field == $dir || $dir == '/' ]]; then + keep=1 + break fi done - else - existing_paths+=( $line ) + (( keep )) || continue fi - done - lines=( $existing_paths ) - for line in $lines; do - path_field=${(q)line%%\|*} + # Quote in place: assoc-array keys need shell-special chars + # backslash-escaped (rupa/z#246). + path_field=${(q)path_field} rank_field=${${line%\|*}#*\|} time_field=${line##*\|} @@ -378,8 +882,13 @@ zshz() { (( rank_field < 1 )) && continue if [[ $path_field == $add_path ]]; then - rank[$path_field]=$rank_field - (( rank[$path_field]++ )) + # Compute the new rank with a scalar expression, not `(( rank[$key]++ ))'. + # The keys are `${(q)}'-quoted (rupa/z#246); a math-context subscript + # runs its key through the arithmetic lexer, which strips a backslash + # level and so misses any key containing `$ \ [ ] ( )' or a backtick -- + # incrementing a phantom raw-keyed entry and leaving the real one stuck. + # An assignment subscript expands the key literally, so it is safe. + rank[$path_field]=$(( rank_field + 1 )) time[$path_field]=$now else rank[$path_field]=$rank_field @@ -387,16 +896,33 @@ zshz() { fi (( count += rank_field )) done + local -a out if (( count > ${ZSHZ_MAX_SCORE:-${_Z_MAX_SCORE:-9000}} )); then # Aging for x in ${(k)rank}; do - print -u $fd -- "$x|$(( 0.99 * rank[$x] ))|${time[$x]}" || return 1 + # `${rank[$x]}', not a bare `rank[$x]' math subscript: the keys are + # `${(q)}'-quoted (rupa/z#246), and a math-context subscript would run + # the key through the arithmetic lexer, stripping a backslash level and + # missing any key with `$ \ [ ] ( )' or a backtick -- yielding 0, which + # the `rank_field < 1' drop above then erases on the next write. The + # expansion substitutes the numeric value before the math parser runs. + out+=( "$x|$(( 0.99 * ${rank[$x]} ))|${time[$x]}" ) done else for x in ${(k)rank}; do - print -u $fd -- "$x|${rank[$x]}|${time[$x]}" || return 1 + out+=( "$x|${rank[$x]}|${time[$x]}" ) done fi + # Deliberately NO `-r' here, unlike every other datafile write. The keys in + # $out are `${(q)}'-quoted (assoc-array keys need shell-special chars + # backslash-escaped -- rupa/z#246), and a plain `print' strips exactly one + # backslash level back off, so what lands on disk is the literal path the + # rest of the code expects. Adding `-r' would store the still-quoted form + # (e.g. `/foo\ bar'), which the read path -- it does not unquote -- would + # then fail to match. The verbatim-passthrough writes in + # `_zshz_add_or_remove_path' DO use `-r' because their input is already + # literal; this one is not. + print -u $fd -l -- $out || return 1 } ############################################################ @@ -418,21 +944,29 @@ zshz() { # Replace spaces in the search string with asterisks for globbing 1=${1//[[:space:]]/*} + # Hoist loop-invariants out of the per-line loop -- $1 and + # $ZSHZ_TRAILING_SLASH don't change inside the loop, so the + # lowercase comparison and the trailing-slash branch were pure + # waste when recomputed N times. `query_lower' lets the case- + # insensitive branch glob against a precompiled lowercase pattern. + local query_lower=${1:l} + local -i is_lowercase_query=0 + [[ $1 == $query_lower ]] && is_lowercase_query=1 + local -i trail=${ZSHZ_TRAILING_SLASH:-0} + for line in $lines; do path_field=${line%%\|*} path_field_normalized=$path_field - if (( ZSHZ_TRAILING_SLASH )); then - path_field_normalized=${path_field%/}/ - fi + (( trail )) && path_field_normalized=${path_field%/}/ # If the search string is all lowercase, the search will be case-insensitive - if [[ $1 == "${1:l}" && ${path_field_normalized:l} == *${~1}* ]]; then - print -- $path_field + if (( is_lowercase_query )) && [[ ${path_field_normalized:l} == *${~query_lower}* ]]; then + print -r -- $path_field # Otherwise, case-sensitive elif [[ $path_field_normalized == *${~1}* ]]; then - print -- $path_field + print -r -- $path_field fi done @@ -440,62 +974,18 @@ zshz() { # insensitively. } - ############################################################ - # `print' or `printf' to REPLY - # - # Variable assignment through command substitution, of the - # form - # - # foo=$( bar ) - # - # requires forking a subshell; on Cygwin/MSYS2/WSL1 that can - # be surprisingly slow. Zsh-z avoids doing that by printing - # values to the variable REPLY. Since Zsh v5.3.0 that has - # been possible with `print -v'; for earlier versions of the - # shell, the values are placed on the editing buffer stack - # and then `read' into REPLY. - # - # Globals: - # ZSHZ - # - # Arguments: - # Options and parameters for `print' - ############################################################ - _zshz_printv() { - # NOTE: For a long time, ZSH's `print -v' had a tendency - # to mangle multibyte strings: - # - # https://www.zsh.org/mla/workers/2020/msg00307.html - # - # The bug was fixed in late 2020: - # - # https://github.com/zsh-users/zsh/commit/b6ba74cd4eaec2b6cb515748cf1b74a19133d4a4#diff-32bbef18e126b837c87b06f11bfc61fafdaa0ed99fcb009ec53f4767e246b129 - # - # In order to support shells with the bug, we must use a form of `printf`, - # which does not exhibit the undesired behavior. See - # - # https://www.zsh.org/mla/workers/2020/msg00308.html - - if (( ZSHZ[PRINTV] )); then - builtin print -v REPLY -f %s $@ - else - builtin print -z $@ - builtin read -rz REPLY - fi - } - ############################################################ # If matches share a common root, find it, and put it in # REPLY for _zshz_output to use. # # Arguments: - # $1 Name of associative array of matches and ranks + # $@ Candidate paths ############################################################ _zshz_find_common_root() { local -a common_matches local x short - common_matches=( ${(@Pk)1} ) + common_matches=( "$@" ) for x in ${(@)common_matches}; do if [[ -z $short ]] || (( $#x < $#short )) || [[ $x != ${short}/* ]]; then @@ -509,7 +999,7 @@ zshz() { [[ $x != $short* ]] && return done - _zshz_printv -- $short + REPLY=$short } ############################################################ @@ -521,6 +1011,7 @@ zshz() { # 3) Put a common root or best match into REPLY # # Globals: + # ZSHZ_TILDE # ZSHZ_UNCOMMON # # Arguments: @@ -532,66 +1023,110 @@ zshz() { _zshz_output() { local match_array=$1 match=$2 format=$3 - local common k x + local common x v local -a descending_list output - local -A output_matches - output_matches=( ${(Pkv)match_array} ) - - _zshz_find_common_root $match_array + _zshz_find_common_root ${(@Pk)match_array} common=$REPLY + # Clear REPLY once the common root is captured: the caller reads REPLY as + # the jump target, so a value left over here would make `z -l ' + # change directory after listing. The default arm below overwrites REPLY + # deliberately; the completion and list arms must leave it empty. + REPLY='' + + # Iterate the caller's matches/imatches array as flat key-value + # pairs via ${(@Pkv)...} instead of copying into a local + # associative array. Avoids the hash-table allocation and K + # inserts that the copy required. + local -a kv + local -i i + kv=( ${(@Pkv)match_array} ) case $format in completion) - for k in ${(@k)output_matches}; do - _zshz_printv -f "%.2f|%s" ${output_matches[$k]} $k - descending_list+=( ${(f)REPLY} ) - REPLY='' + # Build "sortkey|path" rows, sort by the leading key descending, then + # strip the key+'|' prefix to keep just the paths (the key is never + # user-visible). The key MUST be an integer: `${(@On)}' numeric sort + # compares each run of digits on its own, so a raw float rank orders by + # its fractional digit-run rather than its value -- "100.5" would sort + # below "100.25" (5 < 25). Scale by 100 and drop the decimal so two + # digits of resolution survive (what the old `%.2f' rows preserved) as a + # single integer digit-run. (Negative `-t' ranks still sort by + # magnitude, since `n' ignores the sign -- unchanged from the `%.2f' + # rows, i.e. a pre-existing quirk, not introduced here.) + local sortkey + for ((i=1; i<=${#kv}; i+=2)); do + sortkey=$(( kv[i+1] * 100 )) + descending_list+=( "${sortkey%.*}|${kv[i]}" ) done descending_list=( ${${(@On)descending_list}#*\|} ) - print -l $descending_list + print -rl -- $descending_list ;; list) + # The bare `z -l' fast path (no query) inlines an equivalent + # formatting block straight on $lines to skip this pipeline -- + # keep the two list formatters in sync. local path_to_display - for x in ${(k)output_matches}; do - if (( ${output_matches[$x]} )); then - path_to_display=$x - (( ZSHZ_TILDE )) && - path_to_display=${path_to_display/#${HOME}/\~} - _zshz_printv -f "%-10d %s\n" ${output_matches[$x]} $path_to_display - output+=( ${(f)REPLY} ) - REPLY='' - fi + local -a displayed_paths + for ((i=1; i<=${#kv}; i+=2)); do + x=${kv[i]} v=${kv[i+1]} + (( v )) || continue + displayed_paths+=( $x ) + path_to_display=$x + (( ZSHZ_TILDE )) && + path_to_display=${path_to_display/#${HOME}/\~} + # Right-pad the integer rank to 10 chars, as `printf "%-10d %s\n"' + # used to, but in parameter expansion. The padding must be + # conditional: `%-10d' never shortened anything, but a bare + # `${(r:10:)}' *truncates* a rank longer than 10 characters -- an + # 11-character `-t' rank (sign + 10 digits, e.g. from a zeroed or + # hand-imported time field more than ~31.7 years old) or a frecency + # rank inflated by a raised $ZSHZ_MAX_SCORE would lose its last + # digits, garbling both the displayed figure and the numeric sort + # below. The `%.*' strip drops frecency's decimal tail + # ("30000.0" -> "30000") to match what `%-10d' produced. + v=${v%.*} + (( ${#v} < 10 )) && v=${(r:10:)v} + output+=( "$v $path_to_display" ) done + # Recompute the common root over the entries that survived the rank + # filter above: $common, computed at the top of this function, + # covers *every* match -- including rank-0 entries hidden from the + # listing -- so it could name a root the visible entries do not + # share. The bare `z -l' fast path filters rank-0 entries before + # looking for a root, and the two formatters must produce identical + # output. (The jump arm below still uses the full-match root: what + # `z foo' jumps to is a separate question from what a listing + # displays.) + common='' + if (( $#displayed_paths )); then + _zshz_find_common_root $displayed_paths + common=$REPLY + # A listing must never leave a jump target in REPLY. + REPLY='' + fi if [[ -n $common ]]; then (( ZSHZ_TILDE )) && common=${common/#${HOME}/\~} (( $#output > 1 )) && printf "%-10s %s\n" 'common:' $common fi - # -lt - if (( $+opts[-t] )); then - for x in ${(@On)output}; do - print -- $x - done - # -lr - elif (( $+opts[-r] )); then - for x in ${(@on)output}; do - print -- $x - done - # -l - else - for x in ${(@on)output}; do - print $x - done + if (( $#output )); then + # -lt: most-recent first (descending); -lr and default -l: + # ascending rank. + if (( $+opts[-t] )); then + print -rl -- ${(@On)output} + else + print -rl -- ${(@on)output} + fi fi ;; *) if (( ! ZSHZ_UNCOMMON )) && [[ -n $common ]]; then - _zshz_printv -- $common + REPLY=$common else - _zshz_printv -- ${(P)match} + REPLY=${(P)match} fi ;; esac @@ -606,63 +1141,85 @@ zshz() { # ZSHZ # ZSHZ_CASE # ZSHZ_KEEP_DIRS - # ZSHZ_OWNER + # ZSHZ_TRAILING_SLASH # # Arguments: - # #1 Pattern to match + # $1 Pattern to match # $2 Matching method (rank, time, or [default] frecency) # $3 Output format (completion, list, or [default] store - # in REPLY + # in REPLY) ############################################################ _zshz_find_matches() { setopt LOCAL_OPTIONS NO_EXTENDED_GLOB local fnd=$1 method=$2 format=$3 - local -a existing_paths - local line dir path_field rank_field time_field rank dx escaped_path_field + local line dir path_field rank_field time_field rank dx local -A matches imatches local best_match ibest_match hi_rank=-9999999999 ihi_rank=-9999999999 + local -i keep - # Remove paths from database if they no longer exist - for line in $lines; do - if [[ ! -d ${line%%\|*} ]]; then - for dir in ${(@)ZSHZ_KEEP_DIRS}; do - if [[ ${line%%\|*} == ${dir}/* || - ${line%%\|*} == $dir || - $dir == '/' ]]; then - existing_paths+=( $line ) - fi - done - else - existing_paths+=( $line ) - fi - done - lines=( $existing_paths ) + # Hoist loop-invariants. $fnd, $1, and $ZSHZ_TRAILING_SLASH don't + # change inside the per-line loop, so the space-to-glob + # substitution, the `${1:l} == $1' check, and the `:l' on $q were + # pure waste when recomputed N times. The `q_lower' precompute + # lets `${~q_lower}' replace `${~q:l}' in the case-insensitive + # branches: same expanded pattern, compiled once. + local q=${fnd//[[:space:]]/\*} + local q_lower=${q:l} + local -i is_lowercase_query=0 + [[ ${1:l} == $1 ]] && is_lowercase_query=1 + local -i trail=${ZSHZ_TRAILING_SLASH:-0} + local now=$EPOCHSECONDS + + # This flag is consumed by the ZSHZ_UNCOMMON trimming block, which must know + # whether the match it is about to trim was found case-insensitively. Clear + # it at the start of every search so a value left over from a previous call + # -- e.g. a tab-completion, which sets it but never runs the trimming block + # that would reset it -- can't steer this search into the wrong branch. The + # authoritative value is set below, from whichever match actually wins. + ZSHZ[CASE_INSENSITIVE]=0 for line in $lines; do path_field=${line%%\|*} + + # Filter non-existent paths (honoring ZSHZ_KEEP_DIRS) inline so we + # walk $lines once instead of twice. The `keep=1; break' inside the + # inner loop also fixes a latent bug: the previous existence-check + # loop had no `break' after appending, so a non-existent path that + # matched multiple ZSHZ_KEEP_DIRS patterns was processed more than + # once. + if [[ ! -d $path_field ]]; then + keep=0 + for dir in ${(@)ZSHZ_KEEP_DIRS}; do + if [[ $path_field == ${dir}/* || $path_field == $dir || $dir == '/' ]]; then + keep=1 + break + fi + done + (( keep )) || continue + fi + rank_field=${${line%\|*}#*\|} time_field=${line##*\|} case $method in rank) rank=$rank_field ;; - time) (( rank = time_field - EPOCHSECONDS )) ;; + time) (( rank = time_field - now )) ;; *) - # Frecency routine - (( dx = EPOCHSECONDS - time_field )) + # Frecency routine: weight a path's stored frequency (rank_field) + # by how recently it was visited (dx seconds ago). 10000 scales + # the result into integer-comparable territory; the 3.75 / (...) + # term decays from 3 (just now) toward 0 as dx grows, so older + # paths lose rank. This is the canonical copy; the bare `z -l' + # fast path inlines the same formula -- keep the two in sync. + (( dx = now - time_field )) rank=$(( 10000 * rank_field * (3.75/( (0.0001 * dx + 1) + 0.25)) )) ;; esac - # Use spaces as wildcards - local q=${fnd//[[:space:]]/\*} - - # If $ZSHZ_TRAILING_SLASH is set, use path_field with a trailing slash for matching. local path_field_normalized=$path_field - if (( ZSHZ_TRAILING_SLASH )); then - path_field_normalized=${path_field%/}/ - fi + (( trail )) && path_field_normalized=${path_field%/}/ # If $ZSHZ_CASE is 'ignore', be case-insensitive. # @@ -671,33 +1228,32 @@ zshz() { # # Otherwise, the default behavior of Zsh-z is to match case-sensitively if # possible, then to fall back on a case-insensitive match if possible. - if [[ $ZSHZ_CASE == 'smart' && ${1:l} == $1 && - ${path_field_normalized:l} == ${~q:l} ]]; then + # + # Track best_match / ibest_match directly from $rank in each branch so + # we never have to math-subscript matches[] / imatches[] -- the math + # parser interprets shell-special chars in associative-array keys as + # syntax (rupa/z#246), and the workaround used to be a seven-char + # escape pass on every line. Comparing the $rank scalar to the running + # max sidesteps the subscript entirely. + if [[ $ZSHZ_CASE == 'smart' ]] && (( is_lowercase_query )) && + [[ ${path_field_normalized:l} == ${~q_lower} ]]; then imatches[$path_field]=$rank + if (( rank > ihi_rank )); then + ibest_match=$path_field + ihi_rank=$rank + fi elif [[ $ZSHZ_CASE != 'ignore' && $path_field_normalized == ${~q} ]]; then matches[$path_field]=$rank - elif [[ $ZSHZ_CASE != 'smart' && ${path_field_normalized:l} == ${~q:l} ]]; then + if (( rank > hi_rank )); then + best_match=$path_field + hi_rank=$rank + fi + elif [[ $ZSHZ_CASE != 'smart' && ${path_field_normalized:l} == ${~q_lower} ]]; then imatches[$path_field]=$rank - fi - - # Escape characters that would cause "invalid subscript" errors - # when accessing the associative array. - escaped_path_field=${path_field//'\'/'\\'} - escaped_path_field=${escaped_path_field//'`'/'\`'} - escaped_path_field=${escaped_path_field//'('/'\('} - escaped_path_field=${escaped_path_field//')'/'\)'} - escaped_path_field=${escaped_path_field//'['/'\['} - escaped_path_field=${escaped_path_field//']'/'\]'} - - if (( matches[$escaped_path_field] )) && - (( matches[$escaped_path_field] > hi_rank )); then - best_match=$path_field - hi_rank=${matches[$escaped_path_field]} - elif (( imatches[$escaped_path_field] )) && - (( imatches[$escaped_path_field] > ihi_rank )); then - ibest_match=$path_field - ihi_rank=${imatches[$escaped_path_field]} - ZSHZ[CASE_INSENSITIVE]=1 + if (( rank > ihi_rank )); then + ibest_match=$path_field + ihi_rank=$rank + fi fi done @@ -707,6 +1263,10 @@ zshz() { if [[ -n $best_match ]]; then _zshz_output matches best_match $format elif [[ -n $ibest_match ]]; then + # The winning match is the case-insensitive one; tell the ZSHZ_UNCOMMON + # trimmer to count case-insensitively. A case-sensitive winner (the branch + # above) correctly leaves the flag at the 0 set at the top of the search. + ZSHZ[CASE_INSENSITIVE]=1 _zshz_output imatches ibest_match $format fi } @@ -736,6 +1296,17 @@ zshz() { return 1 fi + # -r (rank) and -t (recent) name different, mutually exclusive sort keys, so + # asking for both is contradictory. Reject it rather than letting an arbitrary + # one win -- the options loop below visits ${(k)opts} in hash order, so a + # silent winner would not even be predictable. Skipped when --complete is set: + # the completion widget always passes it, an error must not reach the terminal + # mid-completion, and the sort order is merely cosmetic for a completion list. + if (( ${+opts[-r]} && ${+opts[-t]} && ! ${+opts[--complete]} )); then + print "${ZSHZ_CMD:-${_Z_CMD:-z}}: options -r and -t cannot be combined." >&2 + return 1 + fi + local opt output_format method='frecency' fnd prefix req for opt in ${(k)opts}; do @@ -760,6 +1331,9 @@ zshz() { ;; --complete) if [[ -s $datafile && ${ZSHZ_COMPLETION:-frecent} == 'legacy' ]]; then + lines=( ${(f)"$(< $datafile)"} ) + # Discard entries that are incomplete or incorrectly formatted + lines=( ${(M)lines:#/*\|[[:digit:]]##[.,]#[[:digit:]]#\|[[:digit:]]##} ) _zshz_legacy_complete "$1" return fi @@ -771,7 +1345,10 @@ zshz() { _zshz_usage return ;; - -l) output_format='list' ;; + # --complete (completion mode) always wins over -l, independent of the + # order ${(k)opts} happens to visit them: completing `z -l ...' must still + # emit bare paths for compadd, never the rank-padded rows of a list. + -l) (( ${+opts[--complete]} )) || output_format='list' ;; -r) method='rank' ;; -t) method='time' ;; -x) @@ -785,6 +1362,12 @@ zshz() { ;; esac done + + # Load the datafile into an array and parse it + lines=( ${(f)"$(< $datafile)"} ) + # Discard entries that are incomplete or incorrectly formatted + lines=( ${(M)lines:#/*\|[[:digit:]]##[.,]#[[:digit:]]#\|[[:digit:]]##} ) + req="$*" fnd="$prefix$*" @@ -816,13 +1399,17 @@ zshz() { # If $ZSHZ_ECHO == 1, display paths as you jump to them. # If it is also the case that $ZSHZ_TILDE == 1, display # the home directory as a tilde. + # + # Globals: + # ZSHZ_ECHO + # ZSHZ_TILDE ######################################################### _zshz_echo() { if (( ZSHZ_ECHO )); then if (( ZSHZ_TILDE )); then - print ${PWD/#${HOME}/\~} + print -r -- ${PWD/#${HOME}/\~} else - print $PWD + print -r -- $PWD fi fi } @@ -832,11 +1419,88 @@ zshz() { [[ -d ${@: -1} ]] && zshz_cd ${@: -1} && _zshz_echo && return fi - # With option -c, make sure query string matches beginning of matches; - # otherwise look for matches anywhere in paths + # Fast path: bare `zshz -l' (no query, list format). Skip the + # `_zshz_find_matches' / `_zshz_output' pipeline -- there is nothing + # to match against, no `matches[]'/`imatches[]' to maintain, no + # case-mode branching, no `${(Pkv)...}' copy. Build the formatted + # output array directly, then sort and print. Mirrors the list arm + # of `_zshz_output' but operates straight on $lines. + if [[ $output_format == 'list' && -z $fnd ]]; then + local line path_field rank_field time_field rank dx path_to_display dir + local common now=$EPOCHSECONDS + local -a output paths + local -i keep - # zpm-zsh/colors has a global $c, so we'll avoid math expressions here - if [[ ! -z ${(tP)opts[-c]} ]]; then + for line in $lines; do + path_field=${line%%\|*} + + if [[ ! -d $path_field ]]; then + keep=0 + for dir in ${(@)ZSHZ_KEEP_DIRS}; do + if [[ $path_field == ${dir}/* || $path_field == $dir || $dir == '/' ]]; then + keep=1 + break + fi + done + (( keep )) || continue + fi + + rank_field=${${line%\|*}#*\|} + time_field=${line##*\|} + case $method in + rank) rank=$rank_field ;; + time) (( rank = time_field - now )) ;; + *) + # Frecency routine -- see _zshz_find_matches for the canonical + # copy and the constants' rationale; keep the two in sync. + (( dx = now - time_field )) + rank=$(( 10000 * rank_field * (3.75/( (0.0001 * dx + 1) + 0.25)) )) + ;; + esac + (( rank )) || continue + + paths+=( $path_field ) + path_to_display=$path_field + (( ZSHZ_TILDE )) && path_to_display=${path_to_display/#${HOME}/\~} + # Conditional padding, never a bare `${(r:10:)}' -- see the list arm + # of `_zshz_output' for why a rank must not be truncated. + rank=${rank%.*} + (( ${#rank} < 10 )) && rank=${(r:10:)rank} + output+=( "$rank $path_to_display" ) + done + + if (( $#paths )); then + _zshz_find_common_root $paths + common=$REPLY + REPLY= + fi + + if [[ -n $common ]]; then + (( ZSHZ_TILDE )) && common=${common/#${HOME}/\~} + (( $#output > 1 )) && printf "%-10s %s\n" 'common:' $common + fi + + if (( $#output )); then + if (( $+opts[-t] )); then + print -rl -- ${(@On)output} + else + print -rl -- ${(@on)output} + fi + return 0 + fi + return 1 + fi + + # With option -c, make sure query string matches beginning of matches; + # otherwise look for matches anywhere in paths. + # + # The `$PWD != /' guard mirrors the one where the prefix is set, above. At the + # root every path is already under $PWD, so no "$PWD " prefix is prepended and + # $fnd stays the bare query -- which, anchored, can never match a path + # beginning with `/'. Without the guard, `z -c foo' from `/' matches nothing + # whatever the query. Anchoring is still right in the other prefix-less case + # ($* is an absolute path under $PWD): there the query is itself anchored. + if (( ${+opts[-c]} )) && [[ $PWD != '/' ]]; then _zshz_find_matches "$fnd*" $method $output_format else _zshz_find_matches "*$fnd*" $method $output_format @@ -845,7 +1509,12 @@ zshz() { local ret2=$? local cd - cd=$REPLY + # Only the default (jump/echo) format communicates a destination through + # REPLY; list and completion print their results directly and leave REPLY + # empty. Checking the format here is a second line of defense: even if a + # future edit to `_zshz_output' lets a stray REPLY escape again, a listing + # must never turn into a directory change. + [[ -z $output_format ]] && cd=$REPLY # New experimental "uncommon" behavior # @@ -866,6 +1535,10 @@ zshz() { # Try dropping directory elements from the right; stop when it affects # how many times the search pattern appears until (( ( ${#cd:h} - ${#${${cd:h}//${~q}/}} ) != q_chars )); do + # ${cd:h} of `/' is `/', so without this guard the trim could spin + # forever once it reaches the root (e.g. `/' in the database with a + # pattern that matches zero characters there). + [[ ${cd:h} == $cd ]] && break cd=${cd:h} done @@ -873,6 +1546,9 @@ zshz() { else local q_chars=$(( ${#cd} - ${#${${cd:l}//${~${q:l}}/}} )) until (( ( ${#cd:h} - ${#${${${cd:h}:l}//${~${q:l}}/}} ) != q_chars )); do + # See the case-sensitive branch: guard against ${cd:h} no longer + # changing once the trim reaches the root. + [[ ${cd:h} == $cd ]] && break cd=${cd:h} done fi @@ -884,7 +1560,7 @@ zshz() { if (( ret2 == 0 )) && [[ -n $cd ]]; then if (( $+opts[-e] )); then # echo (( ZSHZ_TILDE )) && cd=${cd/#${HOME}/\~} - print -- "$cd" + print -r -- "$cd" else # cd if possible; echo the new path if $ZSHZ_ECHO == 1 [[ -d $cd ]] && zshz_cd "$cd" && _zshz_echo @@ -924,13 +1600,37 @@ _zshz_precmd() { esac done - # It appears that forking a subshell is so slow in Windows that it is better - # just to add the PWD to the datafile in the foreground - if [[ $OSTYPE == (cygwin|msys) ]]; then - zshz --add "$PWD" - else - (zshz --add "$PWD" &) - fi + # Add PWD to the datafile. Background the write so the prompt doesn't wait on + # read + tempfile + rename + chown -- which is tens of ms per prompt on + # 9P-bridged or VHD-backed paths. Backgrounding is safe under develop's + # lock design: the `always { zsystem flock -u $lockfd }' block in + # _zshz_add_or_remove_path guarantees the parent never holds an open + # lockfd between precmd invocations (so a `&!' fork can't inherit one), + # and ZSHZ_LOCK_TIMEOUT (default 1s) bounds contention so a stuck holder + # can't pile up writers. `&!' is zsh background + disown: no wrapper + # subshell, no job-table entry, no "Done" line at the next prompt. + # + # Do not restore the old foreground carve-out for Cygwin/MSYS2. It was + # right when backgrounding meant a subshell plus a job (two forks) and + # writes were line-by-line; with one disowned fork and batched writes, + # measurement (June 2026, Cygwin zsh 5.8 and MSYS2 zsh 5.9) shows ~10-12ms + # at the prompt for `&!' vs. ~30ms for a foreground add at 300 datafile + # entries -- and ~300ms at 1,000 entries, since the foreground cost grows + # with the datafile while the fork cost stays flat. + # + # `2> /dev/null' is what actually enforces the "stay quiet at every prompt" + # rule that $_zshz_quiet_add describes. That marker can only gate Zsh-z's own + # `print's; it cannot reach the external and builtin commands further down the + # --add path -- `mkdir -p', `id -ng', ${ZSHZ[CHOWN]}, the deliberately loud + # datafile-creation retry, or Zsh's own redirection diagnostics -- and any of + # those can fail when $ZSHZ_DATA sits on an unwritable or unmounted directory, + # or when $ZSHZ_OWNER names a user `id' can't resolve. Suppressing at the fork + # covers every such site at once, including ones added later, whereas + # suppressing site by site has to be kept in sync forever. Nothing actionable + # is lost: a foreground `z --add .' still reports in full, which is exactly + # the diagnostic the lock comment above tells the user to run. + local _zshz_quiet_add=1 + zshz --add "$PWD" 2> /dev/null &! # See https://github.com/rupa/z/pull/247/commits/081406117ea42ccb8d159f7630cfc7658db054b6 : $RANDOM @@ -961,11 +1661,33 @@ add-zsh-hook chpwd _zshz_chpwd # Standardized $0 handling # https://zdharma-continuum.github.io/Zsh-100-Commits-Club/Zsh-Plugin-Standard.html -0="${${ZERO:-${0:#$ZSH_ARGZERO}}:-${(%):-%N}}" +0="${${ZERO:-${0:#${ZSH_ARGZERO-}}}:-${(%):-%N}}" 0="${${(M)0:#/*}:-$PWD/$0}" -(( ${fpath[(ie)${0:A:h}]} <= ${#fpath} )) || fpath=( "${0:A:h}" "${fpath[@]}" ) +# Capture the plugin directory while $0 still names this file: inside the +# unload function, $0 is the function name (FUNCTION_ARGZERO), which `:A' +# would resolve against $PWD. +ZSHZ[PLUGIN_DIR]=${0:A:h} +# Add the plugin directory to $fpath only when nothing else has already put it +# there, and record having done so, so that unload can take back this entry +# and leave a plugin manager's alone. +# +# The record is only ever set, never cleared: on a re-source the directory is +# already present -- because this file added it the first time -- and clearing +# the record then would strand the entry in $fpath at unload. `typeset -gA' +# above preserves the value across that re-source. +if (( ${fpath[(ie)${ZSHZ[PLUGIN_DIR]}]} > ${#fpath} )); then + fpath=( "${ZSHZ[PLUGIN_DIR]}" "${fpath[@]}" ) + # Record the path itself, not a boolean. $ZSHZ[PLUGIN_DIR] is rewritten by + # every source, so a flag would end up describing whichever directory was + # sourced last: re-sourcing from a second, manager-owned installation would + # make unload drop *that* entry and strand the one this plugin actually + # added. Newline-separated, since a path may contain spaces, and split with + # `${(f)...}' at unload. + ZSHZ[ADDED_FPATH]="${ZSHZ[ADDED_FPATH]:+${ZSHZ[ADDED_FPATH]} +}${ZSHZ[PLUGIN_DIR]}" +fi # Save the existing Tab binding so that the completion widget can invoke it, # but being careful not to create a situation where the widget ends up calling @@ -992,6 +1714,24 @@ _zshz_zle_completion_widget() { local cmd=${ZSHZ_CMD:-${_Z_CMD:-z}} + # Ensure tab completion works under `setopt COMPLETE_ALIASES'. Under that + # option zsh looks up `_comps[$cmd]' verbatim rather than expanding the + # alias to `zshz' first; compinit's static `#compdef' tag in `_zshz' is + # parsed literally (no parameter expansion) and only covers the literal + # `zshz' command. Run once -- the guard short-circuits on subsequent Tabs. + # Record what was registered, so `zsh-z_plugin_unload' can take back exactly + # this entry and nothing else. Keyed on the effect rather than compdef's exit + # status: if the mapping did not land, there is nothing to take back. + if (( ! ${+_comps[$cmd]} )); then + compdef _zshz $cmd 2> /dev/null + # Append rather than overwrite. Re-sourcing with a changed $ZSHZ_CMD + # registers a second command while the first mapping is still live, and a + # single slot would forget the earlier one and strand it at unload. Space- + # separated, like $ZSHZ[FUNCTIONS], and split with `${=...}' there. + [[ ${_comps[$cmd]-} == '_zshz' ]] && + ZSHZ[COMPDEF]="${ZSHZ[COMPDEF]:+${ZSHZ[COMPDEF]} }$cmd" + fi + # If a trailing space was added after an already-completed absolute path # (e.g. `z /usr/local/bin '), a second Tab would otherwise re-trigger # completion on an empty word and insert a duplicate. Bail out early. @@ -1039,13 +1779,15 @@ fi # zsh-z functions ############################################################ ZSHZ[FUNCTIONS]='_zshz_usage + _zshz_realpath _zshz_add_or_remove_path _zshz_update_datafile _zshz_legacy_complete - _zshz_printv _zshz_find_common_root _zshz_output _zshz_find_matches + zshz_cd + _zshz_echo zshz _zshz_precmd _zshz_chpwd @@ -1095,9 +1837,49 @@ zsh-z_plugin_unload() { (( ${+functions[$x]} )) && unfunction $x done - unset ZSHZ + # The directory captured at source time -- $0 here is the function name, + # not the plugin file. Read it before ZSHZ is unset. + # + # Only when this plugin was the one that added it. A plugin manager that put + # the directory on $fpath owns that entry: taking it away would break + # autoloads for anything else living there and leave the manager believing + # its configuration is intact. And drop a single occurrence rather than + # filtering every match -- at most one of any duplicates can be ours. + # + # `(ie)', not `(i)': without the `e' the subscript treats the stored path as + # a *pattern*, so a plugin directory containing `[', `*' or `?' would not + # match itself and the entry would be left behind. The source-time lookup + # already uses `(ie)'; these two must agree. + local _zshz_dir + integer _zshz_fp + for _zshz_dir in ${(f)ZSHZ[ADDED_FPATH]-}; do + [[ -n $_zshz_dir ]] || continue + _zshz_fp=${fpath[(ie)$_zshz_dir]} + (( _zshz_fp <= ${#fpath} )) && fpath[$_zshz_fp]=() + done - fpath=( "${(@)fpath:#${0:A:h}}" ) + # Take back the completion mapping the widget installed on its first Tab. + # Without this the entry outlives the function it names -- `_zshz' is + # unfunctioned above and the plugin directory has just left $fpath, so a + # later completion on that command looks up something unloadable. + # + # Only this one entry. compinit's own registrations (`_comps[zshz]', from the + # static `#compdef' tag) are deliberately left in place: nothing re-runs + # compinit when the plugin is sourced again, so removing them would break + # completion for the literal `zshz' command until the user re-ran it by hand. + # This entry has no such problem -- the widget re-registers it on the next + # Tab after a reload. + # + # `$ZSHZ[COMPDEF]' is the ownership record: the registration above never + # overwrites an existing mapping, so one Zsh-z did not create must survive + # unload. Re-check the value too, in case it was repointed since. + local _zshz_compdef + for _zshz_compdef in ${=ZSHZ[COMPDEF]-}; do + [[ ${_comps[$_zshz_compdef]-} == '_zshz' ]] && + compdef -d "$_zshz_compdef" 2> /dev/null + done + + unset ZSHZ (( ${+aliases[${ZSHZ_CMD:-${_Z_CMD:-z}}]} )) && unalias ${ZSHZ_CMD:-${_Z_CMD:-z}}