util-shell: make your shell script actually readable

4

(Note: I started typing this before LukeShu's comment, but I had a meeting before I could do a quick proofread. If I had seen his comment I probably wouldn't have bothered. But since I already wrote this...)

I actually find quite a few of these harder to read, or unnecessarily different. I don't want to discourage you, so I hope you'll read this as constructive criticism. If I've come across as unneccessarily harsh, I hope you understand that wasn't the intent.

There are some problems with the examples on the website. Your example of how to check for a substring in plain bash:

if [[ "$string" == *My* ]]
then
  echo "It's there!";
fi

Then your example of improvement with util-shell:

strings contains 'My string' 'My' && echo "yes" || "no"

But most of the readability benefits are due to using simpler looking syntax, not util-shell features. Also note that the util-shell example doesn't use variables, which again makes things appear simpler. The "plain" bash example can also be written as:

[[ "$string" == *My* ]] && echo "yes" || echo "no"

So right off the bat, you're cherry-picking your examples. Granted, this is typical of what you'd get from a Google search. But in fairness, googling "util-shell strings" gives me hadoop util.shell documentation. Ultimately, this example is a result of poor documentation, not necessarily poor functionality.

So lets look at some direct comparisons between bash and util-shell. First, plain bash:

# Compare numbers
[[ "$number1" <  "$number2" ]] && echo "yes" || echo "no"
[[ "$number1" >= "$number2" ]] && echo "yes" || echo "no"
[[ "$number1" =  "$number2" ]] && echo "yes" || echo "no"

# Compare strings
[[ "$string1" =  *$string2* ]] && echo "yes" || echo "no"
[[ "$string1" =  "$string2" ]] && echo "yes" || echo "no"

And now the same thing with util-shell:

# Compare numbers
math '<'  "$number1" "$number2" && echo yes || echo no
math '>=' "$number1" "$number2" && echo yes || echo no
math '==' "$number1" "$number2" && echo yes || echo no

# Compare strings
strings contains "$string1" "$string2" && echo "yes" || "no"
strings match    "$string1" "$string2" && echo "yes" || "no"

Some notes:

In plain bash, string and number comparisons are written the same way.
In util-shell, you need to know all the keywords. is it "equals"? No, it's "matches".
In util-shell, you need to quote or escape your comparison operators, otherwise bash will attempt to redirect output. Bash understands when you're doing a comparison with [[ ]], and you don't need to worry about it. Perhaps an unfair advantage due to being a builtin, though :)
I'll admit that I'm comparing my integers as strings, instead of using -eq. But it's readable, after all.

Now, I've cherry picked examples, too. There are some things that are seemingly easier to do with your library. For example:

math sqrt $number1

is nicer looking than piping through bc:

echo "sqrt($number)" | bc

But your library doesn't do sin, cos, etc. Which is understandable, and probably not something many bash scripts would ever need to do. But if one did, it would still have to shell out to bc. By abstracting away the complex tools and syntax for simple tasks, you're potentially unprepared to use them when a complex task calls for it. My ability to write readable bash should not be limited by my ability to write and compile code in go ;)

Now, there are some highlights of your efforts. Bash arrays are difficult to get the hang of (I've no real experience with hash maps). Again, though, I think that is a documentation problem, and not something that is solved by util-shell. I still need documentation, after all.

Besides, in some instances, util-shell's array syntax is less readable:

array get 1 "$arr"

versus bash's:

${array[1]}

Now, I don't want to discourage you. There are a lot of shitty things about doing programming in bash. And it would be great if bash's limited built-in arithmetic functions could handle decimals. I'm just not sure this current approach actually meets the goals of being easier, or more readable, or deployable in any sort of scale.

Perhaps an approach that used plain-bash functions (as interiot suggests) to improve some painful functionality? For example:

function calc () { echo "$@" | bc; }

calc "$number1 + $number2"

One could ship your util-shell.sh alongside their scripts and source it. Or embed it in their own scripts, or stick it in their ~/.bashrc. Without introducing an external dependency.

When writing your library, there are some important questions to keep in mind:

How easy is this to depend upon?
How easy is this to deploy?
Is it actually better than what we had before?

Generally shell scripts are used as it can be depended on always being there.

(As a side-note regarding { } in array syntax. I generally use the brace syntax for all variables now, so the use for arrays is not "weird" to me. This allows me to do things like cp "${filename}" "${filename}_backup", which wouldn't work without the braces).

-2
u/taylorchu Sep 07 '13 edited Sep 07 '13
I want to clarify few things:
util-shell uses consistent interface across these 4 components.
<category> <action> <subjects...>
example: array(category) get 1 (action) "$arr" (subject)
example: math(category) '+' (action) 5(subject) 4.4(subject)
this is not comparing number. see, somebody else will fall into trap.
Compare numbers
[[ "$number1" < "$number2" ]] && echo "yes" || echo "no"
this is still comparing string. to show this:
[[ "3.14" < "3.14e-22" ]] && echo wtf || echo ok   // wtf

math '<' 3.14 3.14e-22 && echo wtf || echo ok   // ok
match is "full-featured" regexp match; not just string. and it is not "equals". bash already does "equal" really well, and util-shell tries to solve the tricky part only
2

u/[deleted] Sep 07 '13

[removed] — view removed comment

0

u/sequentious Sep 07 '13

Thanks, I hadn't thought about $@ vs $*.
Regarding bash 4, I haven't really looked much at what it adds as I'm still supporting some systems that have bash 3.2. I've not seen that style of input redirection, but I'll definitely be investigating it next week.

2

u/Samus_ Sep 06 '13

It's written in go so I can't figure out by myself but it seems your arrays aren't compatible with bash

arr=$(array new 1 2 3 4)
array len "$arr"

that isn't an array, your array command writes something to stdout that you store as a plain variable.

-4

u/taylorchu Sep 06 '13

part of the design.
that provides a way to export/import a variable.
only string can be passed to a function

4

u/Samus_ Sep 06 '13

It's obvious that it's by design but that incompatibility is a big deal, it means to rewrite all scripts one may already have and also prevents interoperability with other scripts.

4

u/interiot Sep 06 '13 edited Sep 06 '13

I'd be happy if this became a standard part of Bash.

But usually the main reason I'll resort to using Bash when I need to do branching logic, instead of writing the script in Python/Perl, is that I want to improve cross-compatibility (eg. work on old AIX, old Solaris). Right now, this add-on is less likely to be found than Perl, so... I'll use that instead.

However, this could be reimplemented with Bash functions and Perl one-liners, making it possible to put in a Bash include file, and packaged with any script that needs to be cross-compatible.

2

u/neoice Sep 06 '13

once I learned some Perl basics, I've started thinking of it as a meta-bash. good variables and conditionals with all the ease-of-use of shell pipelines.

4

u/buried_treasure Sep 06 '13

That's exactly why Perl was born.
Larry Wall was becoming increasingly frustrated with trying to use sh + sed + awk + friends to do his job as a sysadmin, so went and created Perl in an attempt to make everything simpler.

0

u/taylorchu Sep 09 '13

finally, "math" gets some common expression: math "sin(PI/2)^2.2 + 4.5 * 3.32 - 7 / 8.9 * sqrt(3.21)".

vs. bc. 1. bc is programming language. you can define function. but it only has 6 defined function. util-shell does only math calculation(the ones on your scientific calculator).

2. bc can only raise integer power. i.e (number)^(int). util-shell can also raise float power. i.e (number)^float

0

u/[deleted] Sep 06 '13

[deleted]

-8

u/agumonkey Sep 06 '13

[ba]sh should retire. Python would be better, but whenever I write in it I tend to feel the same way about it and want to go in lisp.
/me waiting for a lisp to be the language of the system.

9

u/jalanb Sep 06 '13

[ba]sh should retire. Python would be better
As a daily user of both I must contend otherwise - each has it's own place in the greater scheme: For normal server maintenance bash is a lot quicker and easier than Python, whereas Python is comparatively for the win in serving www sites.

4

u/[deleted] Sep 06 '13

Agreed -- shell scripting is a light weight way to script procedures quickly and reliably, without worrying about libraries, dependencies, etc.
You can go into a box you've never seen before, with no permissions to speak of to install anything, and shell script using whatever is natively installed.
If you rely on Python, you'll have to have some modicum of control of the box, which hasn't been most of my professional experience.
PS _ This is why folks should learn vi too, but that's another thing.

2

u/agumonkey Sep 06 '13

I will look like a very unpragmatic alien but anyway. All these languages are just specific syntaxes on top of semantics. All in all, they lack extensibility and composability, hence my mention of lisp.
For instance, some clojure devs wrote an async library to write Go like concurrency code, no new language required, whether you wanna run concurrent processes, output html, monitor your FS or anything else.
</halfrant>

35

u/[deleted] Sep 06 '13 edited Sep 06 '13

[removed] — view removed comment

17

u/[deleted] Sep 06 '13

Thank you for typing out the whole rant I had in my head while reading this.

-4

u/taylorchu Sep 07 '13

now try to count number of substring :)
in util-shell, you know the style clearly: "<category> <action> <subjects...>"
for array, you have to write array literal in bash. in util-shell, you don't have to. ls | array new //like it? array new </etc/fstab
util-shell loses the readability war for "get/set/append", but might win the rest for simplicity reason
in bash, caller has to export, and callee has to eval. util-shell is more writable

2

u/[deleted] Sep 07 '13

[removed] — view removed comment

2

u/umbrum_senecae Sep 07 '13

An elegant demonstration. My hat is off to you, good sir.

util-shell: make your shell script actually readable

You are about to leave Redlib

Compare numbers