I use vpn with a command openvpn. Now this command runs with sudo and then prompts me for my proton api username and password. I type it manually everytime. Instead I want to create a bash script that pipes the creds to the prompts. I tried using printf with \n but it didn't work.

A small trap that I think is easy to miss: declaring and assigning in one statement masks the exit status of the command substitution. I tested this on Bash 5.2.21:

f1() { local out=$(false); echo "f1 status: $?"; }
f2() { local out; out=$(false); echo "f2 status: $?"; }
f1
f2

Output:

f1 status: 0
f2 status: 1

In f1, $? is the status of local itself (which succeeded), not of false. In f2, the assignment is a separate statement, so $? is the status of the command substitution.

The same masking showed up in my test with export, declare and readonly:

export x=$(false);   echo $?   # 0
declare y=$(false);  echo $?   # 0
readonly z=$(false); echo $?   # 0

It matters with set -e too:

#!/usr/bin/env bash
set -e
g1() { local out=$(false); echo "g1: reached the next line"; }
g2() { local out; out=$(false); echo "g2: reached the next line"; }
g1
echo "after g1"
g2
echo "after g2 (should not print)"

Output on my machine:

g1: reached the next line
after g1

and the script exits with status 1 at g2. So g1 swallowed the failure, and g2 stopped the script as you’d expect.

The fix is to split the statement:

bash

local out
out=$(cmd) || return 1     # or handle the failure however you like

Notes and limits:

• This is about the combined local/export/declare/readonly forms. A plain out=$(cmd) keeps the status of cmd.

• set -e has its own corner cases (contexts like if conditions or || lists change its behavior), so I’d not rely on it alone to catch this.

• I only tested Bash 5.2.21; check your version if it matters.

Do you split declaration and assignment as a habit, or do you rely on a linter to flag it?

A classic one that still bites. This prints pipe: 0:

count=0
printf '%s\n' a b c | while IFS= read -r line; do
  count=$((count + 1))
done
echo "pipe: $count"

Each side of a pipe runs in its own subshell (by default), so the loop increments a copy of count and the parent never sees it. I tested the options below on Bash 5.2.21.

1. Process substitution (< <(cmd)): the loop runs in the current shell.

while IFS= read -r line; do
  count=$((count + 1))
done < <(printf '%s\n' a b c)

Prints 3. Bash-specific, not POSIX sh. Note that the exit status of the command inside <(...) isn’t checked by default.

2. shopt -s lastpipe**:** runs the last element of the pipeline in the current shell.

shopt -s lastpipe
printf '%s\n' a b c | while IFS= read -r line; do
  count=$((count + 1))
done

Prints 3 in a script. It only works when job control is off, which is the default for non-interactive scripts. In an interactive shell, or with set -m, I got 0 again.

3. Here-string with command substitution:

while IFS= read -r line; do
  count=$((count + 1))
done <<< "$(printf '%s\n' a b c)"

Prints 3. The whole output is captured in memory first, and trailing newlines are stripped, so it’s a poor fit for big outputs or when exact bytes matter.

4. Temp file or array: write the output to a file (or mapfile -t lines < <(cmd)) and loop over it. More lines, but no subshell surprises and easy to debug.

Which I’d reach for (opinion): process substitution when it’s Bash-only code, mapfile when I want the lines in an array anyway, and I avoid lastpipe in code that might get sourced interactively. If you need POSIX sh, process substitution and lastpipe aren’t available, and a temp file or restructuring is the way.

Related trap: the same applies to variables set inside ( ... ) or $( ... ), not just pipes.

Do you have a default for this, or does it depend on the script?

When I first became a dev, I was obsessed with doing things the way I could learn the most and have the highest ceiling for skill/efficiency. That meant going slower at first, but eventually coming out ahead of my team members with understanding and speed eventually. It all started with small vim motions and mini bash commands. I'm still early in my career so have a ton to learn from this community.

Which skills, tricks, or tools (i.e. using awk or ex commands like substitute) ended up becoming a staple in your everyday and do you wish you started practicing earlier?

I was benchmarking a pipeline in our regression scripts with `/usr/bin/time -f %e cmd1 | cmd2 | cmd3` and the numbers looked suspiciously good. The external binary only wraps `cmd1`. The bash `time` keyword is parsed by the shell and times the whole pipeline, which is why `time cmd1 | cmd2 | cmd3` gave me a much larger and correct wall time. If you want the keyword's output in a specific format, set `TIMEFORMAT='%R'` instead of reaching for `/usr/bin/time -f`. Also note that the keyword writes to the shell's stderr, not the pipeline's, so capture it with `{ time cmd1 | cmd2; } 2> timing.txt` rather than redirecting inside the pipeline.

The goal for this one is to take a package (eg. cmatrix) and its arguments and input them into the nix run command like this:

nix run nixpkgs#cmatrix -- -a -b ...

#!/usr/bin/env bash
PKG=$1
shift 1
ARGS=$@
nix run "nixpkgs#$PKG" -- "$ARGS"

This goal for this one is to take multiple packages (eg. cmatrix, sl) and put them into the nix shell command like this:

nix shell nixpkgs#cmatrix nixpkgs#sl

#!/usr/bin/env bash
ARGS=""
main() {
   args "$@"
   nix shell$ARGS
}
args() {
    while [[ $# -gt 0 ]]; do
        ARGS="$ARGS nixpkgs#$1"
        shift 1
    done
}

main "$@"

Any feedback or advice would be appreciated :)

EDIT: Here is my rewrite of the second one

#!/usr/bin/env bash
set -e
set -u
set -o pipefail

enter-temporary-shell() {
    if [[ $# -lt 1 ]]; then
        echo "USAGE: $0 PACKAGE [PACKAGE...]"
        exit 1
    fi

    local -a temporary_packages=${1:?}
    temporary_packages=$(prepend-nixpkgs "$@")

    echo "Entering temporary shell with""$temporary_packages" "enabled"
    echo "Press ^D to exit"
    nix shell $temporary_packages
}
prepend-nixpkgs () {
    local -a arguments=("$@")
    echo "${arguments[@]/#/ nixpkgs#}"
}

enter-temporary-shell "$@"

In case folks are interested in using Lua for bash automation, I wanted to cross-post this

This started as a joke after Sam Altman said on Cheeky Pint that the RAM in our PCs is "quite terrible" and "being used maybe 5% of the time". I wanted to see what it'd look like if your machine wrote back.

It's a small bash script that reads /proc/meminfo and works out used as MemTotal minus MemAvailable. Run it from cron, it logs a sample each time, and once a day it'll send an email if more than 10% is available.

The recipient is blank out of the box, so it does nothing until you set one. If you want to try it, point it at yourself or just use --dry-run, which prints the email without sending anything. --status shows your current figures.

I'm also interested in whether I've got the memory side right, so feedback on the script is welcome.

Repo: https://github.com/IainKay/ram-altman
Site: https://ram-altman.lol

Hi everyone! I don’t usually interact much—I mostly just browse posts on Reddit and X.

Anyway, to the point: at 3:16 AM, I put together this simple Fastfetch configuration inspired by ADO. I really like how it turned out, and I hope you do too!

For those interested, I’m using Bash extended with ble_sh (akinomyoga/ble.sh) to get Fish-like autosuggestions and syntax highlighting directly in Bash.

I don’t know about you, but I really liked it. I hope it serves as inspiration for you.

This is the repo https://github.com/SayoriLex/Ado-themed-Terminal.git

kbool adds what you expect from a high-level language to bash, and takes away none of what makes bash worth using: it is still a plain script, with no compiler, no runtime to install and no interpreter to ship. You source one file, and pipes, set -x and your ordinary commands work as before. Repo: https://github.com/Keksov/kbool (MIT, bash 5.1+). It was developed with AI assistance (I make the design decisions and review what goes in; the process is in the README).

The main part is kcl, the class library: 29 units, 39 classes and about 780 public methods (a class's own members, inherited ones not counted), plus about 150 properties. Most of it is not a new API made up for this project but a port of the Free Pascal and Delphi runtime libraries: interfaces that have been in daily use for years (TStringList since Delphi 1 in 1995) and have long been stable. The names, the behaviour and the edge cases come from a documented original, not from invention:

• Collections: TList, TStringList (sorting, duplicate policies, case-insensitive search), TObjectList, TDictionary / TObjectDictionary (24 methods: TryGetValue, GetValueDef, ForEach, KeysToArray, change notifications), THashSet (union, intersection, difference), TQueue / TStack and their owning variants, TArray (sort, binary search, min / max with a comparator)

• Text: TStringHelper (57 methods: trim, split, pad, startsWith, indexOfAny, …), TRegEx (isMatch, match with groups, matches, split, replace with a callback)

• Dates and numbers: DateUtils (185 methods: encode / decode, incMonth, daysBetween, week numbers, ISO 8601), Math (123 methods, including floating point, on any awk), TStopwatch

• Files: TPath (34), TFile (38: read / write / append, times, attributes, encrypt with HMAC), TDirectory (29), TIniFile / TMemIniFile

• Tools: TGrep, TSed, TAwk, TFind, THead, TTail (typed options instead of hand-assembled argv, results into arrays or callbacks in the same shell), TPipe (a stream into a callback, with the producer's exit status kept)

• Applications: TCustomApplication (an option parser ported line by line from FPC); there is also a small HTTP server, but it is a toy built to show off the class layer (one request at a time, about 10 requests per second), not something to put on a network

A small report: the top URLs of an access log, with settings from an INI file:

source kbool/kbool.sh || exit
kk.uses tinifile tfile tregex tdictionary tstringlist || exit   # five units, each loaded once

# INI parsing without a hand-written sed/awk parser. A missing file, section or
# key gives the default, and so does "top = abc": ReadInteger never hands you a
# non-number to put into $(( )).
TMemIniFile.new cfg report.ini
cfg.ReadString  report log access.log; log=$RESULT
cfg.ReadInteger report top 3;          top=$RESULT

# The whole file into an array in one call, no $(cat) and no while-read loop.
tfile.readAllLines "$log" lines || exit

TDictionary.new hits                   # a counter map with an API, not a bare declare -A
for line in "${lines[@]}"; do
    # A regex with capture groups that reads as one; the same pattern works with
    # glibc, musl, BSD, macOS and cygwin, and nothing forks.
    TRegEx.match "$line" '"[A-Z]+ ([^ ]+)' || continue
    url=${RESULT_GROUPS[0]}
    hits.GetValueDef "$url" 0          # the count so far, or 0 for a new URL
    hits.AddOrSetValue "$url" $(( RESULT + 1 ))
done

TStringList.new report
add_row() { printf -v row '%6d %s' "$2" "$1"; report.Add "$row"; }
hits.ForEach add_row                   # a callback per pair, in this shell: no subshell, no lost state
report.Sort                            # sorted in bash, no external sort, no fork
report.count; n=$RESULT
for (( i = n - 1; i >= n - top && i >= 0; i-- )); do
    report.Get "$i"; echo "$RESULT"    # every member answers in $RESULT
done

On a five-line sample log, with top = 2 in the INI file, it prints:

     2 /index.html
     2 /api/notes

Every unit follows one contract, so after the first one you can guess the rest: a member returns its value in RESULT rather than on stdout, answers with rc 0 (done), 1 (no / not found) or 2 (a usage error, with a message), and works under set -eu. The reason for RESULT is cost: one x=$(true) is a fork, about 750 µs on Linux and 16 ms on Windows, while a method call is about 0.2 ms. That is why the loop above forks nothing. When you prefer $( ), it still works.

Portability is part of the contract. The test suite has 9,694 tests in 30 suites, and the full run of 2026-10-31 passed all of them. The same suite runs on every push in 15 CI configurations:

• Linux, 8 rows: Alpine 3.22 with bash 5.1.16 and 5.2.37, Alpine 3.24 with bash 5.3.20 (musl, busybox tools), Debian 13, Ubuntu 22.04 (bash 5.1.16), Ubuntu 24.04 and Fedora 44 with their system bash, and Ubuntu 26.04 (the GitHub ubuntu-latest runner, bash 5.3.9, with uutils, the Rust rewrite of coreutils, as head, tail, od and ls)

• macOS (macos-latest, Homebrew bash), 2 rows: Apple's BSD tools only, and with GNU tools

• FreeBSD 15.1, pkg bash, 2 rows: without and with GNU packages

• Windows (windows-latest), 3 rows: Git-bash and MSYS2, both on the MSYS2 runtime (a fork of Cygwin), and Cygwin itself. All three report OSTYPE=cygwin, yet MSYS2's grep drops the CR of a CRLF line and Cygwin's keeps it

Locally the suite also runs in VMs (Ubuntu 24.04, Debian 13, Fedora 44, Alpine 3.24 and FreeBSD 15.1, plus Debian under a de_DE locale). In the Debian VM, bash 5.1.16 is built from the GNU sources for the minimum-version run, and kbool is checked to refuse bash 3.2, 4.4 and 5.0, also built from source, with one line.

The core units need no GNU tool and give the same results on every one of them: BSD stat and date, the spelling of \b in each libc, mawk / BWK awk / busybox awk, CRLF input and a decimal comma in EPOCHREALTIME are handled inside the library and pinned by tests. The tool wrappers give GNU behaviour everywhere: they find gsed, gawk, ggrep by name, or refuse with rc 2 and the package to install instead of quietly doing something else. The full list, with the test for each item: https://github.com/Keksov/kbool/blob/main/docs/portability.md

How a script grows is handled by uses, the module system. It plays the role of import in Python, require in Node or Ruby, or include_once in PHP, which plain bash does not have: bash has only source, which takes a path and runs the file again every time. A unit is any file with a two-line header. kk.uses NAME finds it by name, loads it once (a repeated kk.uses costs about 30 µs), and reports a missing unit, a failed load or a dependency cycle with rc 2 and one line. Your own units work the same way as the library's:

# app/lib/weblog.sh: a unit of your own
#!/usr/bin/env bash
[[ ${__KK_UNITS[@]@a} == A* ]] || source "${KBOOL_HOME-}/kbool.sh" || return 2
kk.unit weblog || return $__kk_unit_rc
kk.uses tregex || return                 # its dependencies, by name

weblog.url() { TRegEx.match "$1" '"[A-Z]+ ([^ ]+)' && RESULT=${RESULT_GROUPS[0]}; }

# app/main.sh
source kbool/kbool.sh || exit
kk.project app.conf || exit       # app.conf: "unitpath = lib", relative to the file itself
kk.uses weblog tstringlist || exit

No source "$(dirname "$0")/../lib/…" bookkeeping. The search path is the calling file's folder, then the project's and the user's unitpath, then the library. kk.uses --optional fancy || fallback handles optional dependencies, and kk.defined DEBUG reads defines from the project file. Each unit owns its namespace (weblog.*): a second unit with the same name, or a class that would take the name, is refused at load time, not discovered at run time. Paths in a project file are relative to the file itself, so the project folder can be moved as it is.

All of the library is built on kklass, the class layer, and your own classes can use it too. A class is declared Pascal-style, with the method bodies as plain bash functions (inheritance with inherited, properties, visibility sections, constructors and destructors):

kk.uses kklass_pascal || exit

class TCounter                         # the interface first, as in a Pascal unit
    public
        var           Name             # a field: a plain variable inside the bodies
        property      Count read GetCount write SetCount   # a property with a getter and a setter
        constructor   Create           # runs on TCounter.new
        destructor    Destroy          # runs on .delete
        proc          Add              # a procedure: no return value
        func          Report           # a function: answers in $RESULT, no fork
        func          GetCount
        proc          SetCount
        static var    Instances        # one value shared by the whole class
    private
        var           _count           # hidden: kklass warns when it is used from outside
end

TCounter.Create()   { Name="$1"; _count=0; Instances=$(( Instances + 1 )); }
TCounter.Destroy()  { Instances=$(( Instances - 1 )); }
TCounter.Add()      { _count=$(( _count + ${1:-1} )); }
TCounter.Report()   { RESULT="$Name: $_count"; }
TCounter.GetCount() { RESULT=$_count; }
TCounter.SetCount() { (( $1 >= 0 )) || return 2; _count=$1; }   # the setter validates
build TCounter                         # turns the bodies above into the class

class TLimitedCounter : TCounter       # inheritance
    public
        var           Limit
        constructor   Create
        override proc Add              # build fails if the parent has no Add: a typo guard
end

TLimitedCounter.Create() { inherited; Limit=${2:-10}; }   # the parent's constructor first
TLimitedCounter.Add() {
    inherited Add "$@"                 # the parent's Add, resolved statically, as in Pascal
    (( _count <= Limit )) || _count=$Limit
}
build TLimitedCounter

TLimitedCounter.new apples apples 5    # instance name, then the constructor's arguments
apples.Add 3
apples.Add 4                           # stops at the limit
apples.Report; echo "$RESULT"          # => apples: 5
apples.Count = 2                       # goes through SetCount
apples.Count; echo "$RESULT"           # => 2
TCounter.Instances                     # => 1
apples.delete                          # Destroy runs
TCounter.Instances                     # => 0

What kbool is good for: scripts that have outgrown functions and globals but must stay bash, such as deploy and build tooling, installers, admin utilities, CI glue and small services. These are scripts where the time goes into logic, structure and error handling, and that have to run on more than one system. What it is not for: a 20-line script, which loading would only slow down (kbool itself loads in 13 to 33 ms, a unit such as tstringlist adds about 150 ms, dateutils about 400 ms); crunching thousands of records, where a pipeline of grep, awk and sort is many times faster (bash reads a line into the shell in about 83 µs, with or without kbool); POSIX sh, and bash older than 5.1. The measurements behind these numbers: https://github.com/Keksov/kbool/blob/main/docs/benchmarks.md

If anyone is interested, I will publish a separate write-up on what kbool gives AI agents that write bash scripts.

Feedback is welcome, especially on the unit set: which class would you reach for first that is not there, and which platform difference has bitten your scripts and is not on the list?

Non-interactive script, a wrapper function that runs a command in the background and waits:

f() { "$@" > out & wait; }

f claude -p < "$in_file"

The command got nothing on stdin. In a non-interactive shell a background command gets /dev/null as stdin, unless the redirect is written on that command itself. A redirect on the function call doesn't reach it.

f() { "$@" < "$in_file" > out & wait; }

f claude -p

Easy to reproduce with cat instead of claude. The first one prints nothing, the second prints the file.

It was a small LLM check in my pre-commit hook. It exited 1 with "Input must be provided" on every commit from Aug 23 to Sep 14. The hook treated any failure as "skipped", so all those commits went through and looked normal.

A check that fails open has to print the error before it skips, otherwise "skipped" and "broken" look the same.

set -e looks like “stop on any error”, but the rules have exceptions. Five exhibits. Tested on Bash 5.x; behavior can differ in older versions.

Exhibit 1: ((i++)) from zero

set -e
i=0
((i++))
echo "never reached"

((i++)) evaluates to the old value, 0, and an arithmetic command with value 0 returns status 1. So the script exits. i=$((i+1)) avoids it.

Exhibit 2: local hides the failure

set -e
f() {
    local out=$(false)
    echo "still running"
}
f

The exit status is that of local, which succeeds. Declare first, assign second: local out; out=$(false).

Exhibit 3: set -e is ignored inside a condition

set -e
f() {
    false
    echo "this runs"
}
if f; then echo ok; fi

When a function is used as the condition of if, && or ||, set -e is disabled for everything inside it. The false doesn’t stop the function.

Exhibit 4: pipes only report the last command

set -e
false | cat
echo "this runs"

Without set -o pipefail, the status of a pipeline is the last command’s. Add pipefail so any failing stage counts.

Exhibit 5: set -u and empty arrays

set -u
arr=()
echo "${arr[@]}"

In Bash before 4.4 this fails with “unbound variable”. Newer versions accept it. If you support old systems, ${arr[@]+"${arr[@]}"} is the common workaround.

What I’d take from them:

• set -euo pipefail helps, but it isn’t a safety net. Some people avoid set -e and check statuses explicitly.

• ShellCheck catches several of these.

• Test the failure paths, not just the happy path.

What would you add to the museum?

When I trigger command completion, I’d like to see a brief description alongside each suggested command.

Is there a built-in Bash feature or plugin that can do this?

Hello! I need some help with a sed command for a GitHub (it's actually Forgejo but it's the same thing in practice) workflow I'm writing. I'm trying to align the version number in my Cargo.toml file with the release tag using the following command but it's not working: sed -i '3s/.\*/version: "$TAG"/' Cargo.toml.

Here is a bit more of the context if it is helpful, though I doubt it would be:

- name: Bump version, tag it, and push
id: bump
shell: bash
run: |
if ! cog bump --auto --dry-run >/dev/null 2>&1; then
echo "Nothing to release"
echo "released=false" >> "$GITHUB_OUTPUT"
exit 0
fi
cog bump --auto
TAG=$(git describe --tags --abbrev=0)
if ${{ inputs.align-cargo }} == 'true'; then
echo "Aligning cargo"
sed -i '3s/.*/version: "$TAG"/' Cargo.toml
echo "Updated Cargo.toml (first few lines):"
head Cargo.toml
git add Cargo.toml
echo "Resuming main flow"
fi
git push origin HEAD
git push origin "$TAG"
if [ "${{ inputs.move-major-tag }}" = 'true' ]; then
MAJOR="${TAG%%.*}"
git tag -f "$MAJOR" HEAD
git push -f origin "$MAJOR"
fi
echo "released=true" >> "$GITHUB_OUTPUT"
echo "tag=$TAG" >> "$GITHUB_OUTPUT"

Honestly I've always kind of avoided sed and just used vi or nano instead, so any help would be greatly appreciated.

EDIT: Solved, thank you all! This is the final solution I went with:

          # ----
          # Credit to users ropid, blobCabbage, OneTurnMore, hypnopixel,
          # Schreq, and jthill on reddit for helping to fix this section 
          echo "version = "\"$TAG\" | sed -i -e '2r /dev/fd/0' -e '3d' Cargo.toml
          # ----

I am writing a bunch of scripts for which I want to only use builtins and realised that dirname is not a builtin, I replaced it thusly:

SRC_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
SRC_DIR2="$({ cd "${BASH_SOURCE[0]%/*}" 2>/dev/null || :; } && pwd)"

I think these two lines are functionally equivalent, considering that BASH_SOURCE[n] will never contain a trailing /.

Then again I am often stupid. Does anyone see any issues?

I started solopreneuring a year ago, and my projects have a fair amount of Bash. My annoyances with Bats: I couldn't mock things efficiently, the assertion libraries are scattered over three repos with different naming, and table-driven tests are clumsy. I've written these scripts over the years in my career and thought about open-sourcing them, but never did. I'm using these libraries in my own frameworks and it helps me test my bash scripts in a more unified way, so I've chosen to productionize them and finally putting it out there:

bats-mock mocks and spies on commands and Bash functions. You set behavior with glob/regex rules, run your code, then assert on call counts, exact argv, stdin and call order across commands. Spies call through to the real thing. jasonkarns/bats-mock uses stubs on PATH, so it can't mock a function without unsetting it, and has no spies or stdin checks.

https://github.com/stealth-scale/bats-mock

bats-expect replaces bats-support, bats-assert and bats-file with one library. bats-assert's functions keep their names, options and failure format, so switching is a one-line change. Naming is consistent, and it adds assertions for variables, arrays, namerefs, numbers and JSON.

https://github.com/stealth-scale/bats-expect

bats-matrix does table-driven tests: inputs, expected exit status and expected output in one table, and a failure tells you which row broke and the exact command it ran. Bats' own bats_test_function works too, but needs a loop and a separate test body.

https://github.com/stealth-scale/bats-matrix

bats-test is the image I run it all in: every combination of Bash 4.4/5.1/5.2/5.3 and Bats 1.7.0/1.14.0, GNU tools, jq and a patched kcov.

https://github.com/stealth-scale/bats-test

I've been an engineer for a long time and I care about quality, so my libraries have a 100% line coverage gate in CI. That's how I ended up in a kcov rabbit hole: on Bash 5.3 one suite came out at 0%. kcov's Bash support had three bugs: a single quote inside [[ ]] made it drop every later hit, it counted every line of a multi-line command while Bash reports one, and its helper killed any bash -c child running with set -u. With the fixes, bats-mock went from 659 lines at 88% to 613 at 100%. The patches are in bats-test. I haven't sent them upstream yet, and I know I should.

If you don't mind switching frameworks, bashunit and ShellSpec already have mocks and data-driven tests. These repos are for staying on Bats.

The linked repo is a small release CLI whose test suite uses all four:

git clone --recurse-submodules https://github.com/stealth-scale/bats-example.git && cd bats-example && make test

It uses Podman (make test RUNTIME=docker for Docker) and runs without network. You need Bash 4.4+, and since the mocks are functions, /usr/bin/curl goes straight past them.

Everything is MIT (the image ships kcov, which is GPL-2.0). For me these tools are a big time-saver.

I'd love to hear what's still annoying to test in your Bash so I can further improve what I've used for years.

I'm writing to file $f using the snippet above, and the command after it also writes/appends to $f.

I'm using col because it's for fio --eta=always to display on terminal but not fill up the file with carriage returns or whatever commands that clutter the file and are intended control for the terminal for the purpose of live status updates.

Running it as in the title, the next command begins before col is done and their respective outputs mix a bit.

I believe the problem is that tee is being recognized as having finished/closed whereas >(col -b >> "$f") isn't being tracked and is still writing. I don't understand fully though I recognize, "The process... is run asynchronously."

I've found two solutions, but they don't quite feel right. Let me know if you can think of a better one or of which these two that you prefer.

{
fio --eta=always "${job}" | tee -a >(col -b >> "$f")
} &

wait $!

or

fio --eta=always "${job}" | tee /tmp/fio.log.tmp

cat /tmp/fio.log.tmp | col -b >> "$f"
rm /tmp/fio.log.tmp

The second one, I could probably add a mktemp and trap to both secure and delete the temp file.

I'm learning Bash via Codecademy(maybe, idk if I'll pay for the subscription) and LinkedIn Learning. The specific course I'm starting says macOS is okay but using a Linux VM with something like Virtual Box is preferred since Linux is industry standard.

I haven't worked with VMs before but it is also something I want to learn about, but I don't know what the curve is like for setting something like this up or if it's worth doing as a beginner.

What do y'all recommend?

Heyy i am new to Linux(currently using fedora silver blue) and exploring Linux, recently started exploring Bash for interest now i learn little bits of it like(basic commands , file permissions) from web3school , now what should i learn next ? From where(resource)

I had a cleanup script that always started with the usual set -euo pipefail. One day it reported success while half the work hadn’t happened. Here’s a stripped-down version of what bit me:

#!/usr/bin/env bash
set -euo pipefail

cleanup_old() {
  rm "$1/old.log"        # fails if the file is missing
  echo "cleaned $1"
}

if cleanup_old /tmp/nope; then
  echo "all good"
fi

Output:

rm: cannot remove '/tmp/nope/old.log': No such file or directory
cleaned /tmp/nope
all good

The rm fails, but the function keeps going, and the echo at the end gives it a zero exit status. The bash manual says -e is ignored for any command in a context where the exit status is being tested (if, while, &&/|| lists, !). That also applies to everything inside a function called from such a context. set -e at the top of the file doesn’t help, and nothing warns you.

What I do now:

cleanup_old() {
  rm "$1/old.log" || return 1
  echo "cleaned $1"
}

The explicit || return is noisier, but it works regardless of how the function gets called. For anything where partial success is dangerous, I also stopped using functions as if conditions and run them as plain statements, checking $? or using || on the call site deliberately.

Related one that bit me the same week: local out=$(some_cmd) always returns the exit status of local, not some_cmd. Splitting it into local out; out=$(some_cmd) fixes it.

I’ve been reading a lot of generated and copy-pasted scripts lately, and nearly all of them treat set -euo pipefail as a safety net. After this, I think of it as a default that works in the simple cases and quietly stops working in exactly the places where functions get composed.

For those who write larger bash scripts: do you rely on set -e at all, or do you handle errors explicitly everywhere and keep -e only as a backstop?

I maintain jc, which converts command output and common file types to JSON so you can query them with jq instead of grep/awk/cut. A couple of newer parsers turned out to be handy in Bash scripts, so I wrote up some examples.

Why not just awk?

authorized_keys looks like a whitespace-delimited file, but it isn't. Lines can start with an optional, comma-separated options field, and option values can be quoted strings with spaces in them. So the field positions shift from line to line. Here's a sample file:

ssh-ed25519 AAAAC3Nza... kelly@laptop
command="/usr/bin/rsync --server --sender -logDtpre.iLsfxC . /srv/backup",no-port-forwarding,no-pty ssh-rsa AAAAB3Nza... backup@nas
from="10.0.8.0/24",no-agent-forwarding ecdsa-sha2-nistp256 AAAAE2Vj... ci-runner@build01
ssh-ed25519 AAAAC3Nza...

The obvious awk one-liner to print the key type and comment:

$ awk '{print $1, $3}' ~/.ssh/authorized_keys
ssh-ed25519 kelly@laptop
command="/usr/bin/rsync --sender
from="10.0.8.0/24",no-agent-forwarding AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlz...
ssh-ed25519

Only the first line is right. Looking for RSA keys with awk '$1=="ssh-rsa"' returns nothing, because the one RSA key has options in front of it.

grep has the opposite problem: it matches text anywhere on the line. grep -v 'command=' to find unrestricted keys also drops a key whose comment says "no command= needed". And grep ssh-rsa matches a restricted ed25519 key whose forced command happens to contain the string ssh-rsa.

With jc --authorized-keys, every line comes out with the same fields, so there's no guessing:

$ jc --authorized-keys < ~/.ssh/authorized_keys | jq -r '.[] | "\(.type) \(.comment // "(no comment)")"'
ssh-ed25519 kelly@laptop
ssh-rsa backup@nas
ecdsa-sha2-nistp256 ci-runner@build01
ssh-ed25519 (no comment)

options is always an array, quoted values like command="echo ssh-rsa disabled; exit" stay intact, and multi-word comments are kept whole.

Tag every key with its owner, then query the whole box

A loop adds a user field to each key and emits JSON Lines:

for f in /home/*/.ssh/authorized_keys; do
    u=$(echo "$f" | cut -d/ -f3)
    jc --authorized-keys < "$f" | jq -c --arg u "$u" '.[] | . + {user: $u}'
done > keys.jsonl

Now each question is just a different jq filter. Which keys have no from= or command= restriction?

jq -r 'select(any(.options[]; startswith("from=") or startswith("command=")) | not)
       | "\(.user)  \(.type)  \(.comment // "(no comment)")"' keys.jsonl

Because this checks the options array instead of the whole line, comments and key data can't cause false matches.

Is the same key authorized for more than one user?

jq -s -r 'group_by(.key)[] | select(length > 1)
          | "\(.[0].comment // "(no comment)"): \(map(.user) | join(", "))"' keys.jsonl

Which legacy RSA or DSA keys are still around?

jq -r 'select(.type == "ssh-rsa" or .type == "ssh-dss") | "\(.user)  \(.comment)"' keys.jsonl

(Run the loop with sudo if you need to read other users' files.)

Bonus: a change-freeze guard from a shared calendar

.ics files are worse for line tools: long lines get folded onto the next line, and parameters are jammed into property names (DTSTART;TZID=America/Los_Angeles:20261019T090000). The --ics parser unfolds all of that and normalizes timestamps to ISO 8601. That means jq's fromdateiso8601 and now can check whether you're inside a freeze window, and jq -e turns the result into an exit code:

if jc --ics < freeze.ics | jq -e '
    [.[] | select(.categories[]? == "Freeze")
         | select((.dtstart | fromdateiso8601) <= now and now < (.dtend | fromdateiso8601))]
    | length > 0' > /dev/null
then
    echo "Change freeze in effect. Aborting deploy." >&2
    exit 1
fi

(This works with UTC timestamps, the ones ending in Z. Events with a TZID come through as local time with a separate dtstart_tzid field.)

Full write-up, which also covers headerless CSV/TSV files: JC Tips and Tricks – Part 3

What file formats are you still parsing by hand in your scripts?

I took your feedback on bash2json, (wasted another year in local testing), and made: shelljq, new JSON parser for shell
it's built on bash2json's json_to_arr function as core for most operations, as well as:
- new name
- ZSH support (+ macOS)
- deeper integration inside scripts (including autogenerated functions documentation)
- own storing format for nested objects
- more powerful typechecking and JSON validation
- completely reworked `json_trim` for the best performance
- testing suite

shelljq is something I personally use in my projects (e.g. upcoming bashcord v2) and had high hopes for reliable tool which can be integrated into any script

shjq is cross-shell for both bash and zsh

I'm open to any critique, this project is still alpha and may still contain many bugs

let me know if i should provide usage examples, i wasn't sure on where to put them