Cool Features of the zsh and bash2 Shells

These are the notes for my talk on 20 July 2000 at the UK (United Kingdom) UNIX User Group.

Summary: 45-minute talk highlighting some new and unique features of two shells for UNIX-like systems: zsh and bash version 2.

Note: the talk was written on a short deadline, so the notes are rough. Sorry; I hope this is more useful than no notes at all...

COOL FEATURES OF THE zsh AND bash2 SHELLS
Jerry Peek
20 July 2000

OVERVIEW: 30-minute talk highlighting some new and unique features of
two shells for UNIX-like systems: zsh and bash version 2.

PRESENTER: Jerry Peek, main author of "UNIX Power Tools":
jpeek@jpeek.com, http://www.jpeek.com/.  I'm writing a third edition
of the book and studying these two shells (among others) as I write... 

----------------------------------------------------------------------
----------------------------------------------------------------------

BACKGROUND:

I assume you've used a UNIX-like system and a common shell.  Quick review:

What's a shell?  It's the command-line user interface to UNIX systems:
- prints a prompt string
- waits for user to type command line (allows editing)
- parses command line
- starts/manages process(es)
- repeats cycle...

UNIX systems have many shells.
- First two major shells: Bourne, C
- Others: Korn, bash, tcsh, zsh, etc.
- bash (Bourne-again Shell) mainly superset of Bourne shell with some
  C shell features.  Version 2, out recently, has even more features.
- zsh (zed shell) a hybrid of Bourne, Korn and C shell.
  Can emulate other shells, adds many unique features.

----------------------------------------------------------------------
----------------------------------------------------------------------

BASH VERSION 2

----------------------------------------------------------------------
OVERVIEW:
- Probably(?) most-used shell, at least under Linux
- bash version 1 out for years; version 2 released ~2 years ago,
  now at 2.03
- FAQ: ftp://ftp.cwru.edu/pub/bash/FAQ has much more info

----------------------------------------------------------------------
SOME OF MY FAVOURITE NEW FEATURES OVER VERSION 1:
- one-dimensional arrays
- new expansions: substring extraction, pattern replacement,
  indirect variable expansion
- new variables, including DIRSTACK, PIPESTATUS and GLOBIGNORE
- new tilde prefixes that expand to directories from the directory stack
- builtin commands updated/expanded
- support for POSIX international characters (character classes,
  equivalence classes, collating symbols)
- egrep-style extended pattern matching operators
- case-insensitive globbing (filename expansion): "shopt -s nocaseglob"
- menu completion
- many more

----------------------------------------------------------------------
ARRAYS:
- zero-based
- Setting:
     name=(value0 value1 value2 ...)
     name[n]=valuen
     read -a name
- Expanding:
     ${name[value]}        (curly-braces required)

----------------------------------------------------------------------
NEW EXPANSIONS:

- SUBSTRING EXPANSION with ${param:offset} and ${param:offset:length}
  Expands to up to "length" (if any) characters of "parameter", starting
  at "offset".  "length" and "offset" are arithmetic expressions.
  Examples:
     $ echo $word
     supercalafragilisticexpialidocious
     $ echo ${word:20}
     expialidocious
     $ echo ${word:20:7}
     expiali
     $ echo ${word:20:10-3}
     expiali

- PATTERN REPLACEMENT: Expand a parameter, replacing a (glob-type)
  pattern with a string.  Replace either one match or all matches.
  This can do much more than these brief examples show:
     $ echo $adage
     Few women admit their age; Fewer men act it.
     $ echo ${adage/Few/Some}
     Some women admit their age; Fewer men act it.
     $ echo ${adage//Few/Dumb}
     Dumb women admit their age; Dumber men act it.
     $ echo ${adage//[ ;]/-}
     Few-women-admit-their-age--Fewer-men-act-it.
     $ echo ${adage/;*/ and that\'s the truth}
     Few women admit their age and that's the truth

- INDIRECT VARIABLE EXPANSION: If first character of parameter name is
  exclamation point (!), adds a level of indirection.  The value of the
  variable formed from the first expansion becomes the name of the
  variable which is then expanded again.  An example helps:
     $ leader=joe
     $ follower=ann
     $ which=leader
     $ echo ${which}
     leader
     $ echo ${!which}
     joe

----------------------------------------------------------------------
NEW VARIABLES:

- DIRSTACK: Array containing directory stack.  Example:
     $ dirs
     /foo/bar/baz/haha/hoho/rubbish /foo/bar/baz/backups
     $ pushd
     $ cp ${DIRSTACK[1]}/whatever .
     $ cp ~1/whatever-else .               (shorter version)
     $ pushd

- GLOBIGNORE: Colon-separated list of filenames NOT to match in wildcard
  expansion (globbing).
  NOTE: Automatically enables dotglob (wildcards will match filenames
  starting with a dot).
  Example:
     $ GLOBIGNORE='*~:.*'
     $ ls
     foo foo~
     $ ls -a
     .hmm .mmm .omm  foo  foo~
     $ echo *
     foo

----------------------------------------------------------------------
EXTENDED egrep-STYLE PATTERN MATCHING: If extglob option is set (using
  shopt builtin command), egrep-style GLOB (NOT regular expression)
  matching is allowed in pathname expansion.  Allows alternation
  (multiple matching patterns).  The five operations are:

  ?(pattern-list)   Zero or one occurrence of the given pattern(s)
  *(pattern-list)   Zero or more occurrences of the given pattern(s)
  +(pattern-list)   One or more occurrences of the given pattern(s)
  @(pattern-list)   Exactly one of the given pattern(s)
  !(pattern-list)   Anything except one of the given pattern(s)

  Examples:
     $ alias l='ls -d'
     $ l p*
     pam.d         passwd-     pine.conf        printcap      profile.d
     paper.config  passwd.OLD  pine.conf.fixed  printcap.bak  protocols
     passwd        pcmcia      ppp              profile       pwdb.conf
     $ l p@(ine|rint)*
     bash2: syntax error near unexpected token `p@(i'
     $ shopt -s extglob
     $ l p@(ine|rint)*
     pine.conf  pine.conf.fixed  printcap  printcap.bak
     $ l p!(*conf*|pp)
     pam.d   passwd-     pcmcia    printcap.bak  profile.d
     passwd  passwd.OLD  printcap  profile       protocols
     
     pine.conf        ppp       printcap.bak  profile.d  pwdb.conf
     pine.conf.fixed  printcap  profile       protocols
     $ l p!([ac])*    
     pam.d         passwd-     pine.conf        printcap      profile.d
     paper.config  passwd.OLD  pine.conf.fixed  printcap.bak  protocols
     passwd        pcmcia      ppp              profile       pwdb.conf

----------------------------------------------------------------------
MENU COMPLETION: Instead of listing all possible completions when you
  press TAB twice, shows the next possible completion each time you
  press TAB.  Off by default; enable it in readline setup file.
  Example:
     
     $ cat $INPUTRC
     TAB: menu-complete
     $ ch<TAB>
     $ chgrp <TAB>
     $ chmod <TAB>
     $ chown jpeek ~/.ba<TAB>
     $ chown jpeek ~/.bash_history <TAB>
     $ chown jpeek ~/.bash_logout <TAB>
     $ chown jpeek ~/.bash_profile <ENTER>

----------------------------------------------------------------------
AGAIN: THERE ARE MANY OTHER CHANGES IN bash2 THAT I HAVEN'T SHOWN.

----------------------------------------------------------------------
----------------------------------------------------------------------

zsh

OVERVIEW:
- Very flexible shell, under active development: "feeping creatureism?"
- FAQ: http://sunsite.auc.dk/zsh/FAQ/
- I won't try to cover all of it... I'll just give you some flavour.
  (In fact, I'm not sure *anyone* knows all of this stuff! ;-)

----------------------------------------------------------------------
PROGRAMMABLE COMPLETION:
- Completion of a command's arguments may be different for each command
  (the compctl command).  You also can specify the way that the command
  itself is completed (see compctl -C).
- New, more elegant completion system in version 3.1.6 not covered here.
- Example: the "mail" command has three default substring completions:
     zsh% compctl -s 'tim@foo.com ann@bar.com jpeek@jpeek.com' mail
     zsh% mail w<TAB>
     zsh% mail jpeek@jpeek.com t<TAB>
     zsh% mail jpeek@jpeek.com tim@foo.com 
- Example: "fg" command completes with with the first word
  of a job's command line (compctl -j).  When a word completes after
  "fg", add a "%" string before it (compctl -P "string"):
     zsh% compctl -j -P "%" fg
     zsh% jobs
     [1]  - suspended  man zshall
     [2]  + suspended  vi ukuug-talk
     zsh% fg m<TAB>
     zsh% fg %man 
- Many, many, many more.  See zshcompctl(1) manpage for details.

----------------------------------------------------------------------
COMMAND LINE EDITING:
- Multiline editing -- without invoking external editor.  Example --
  enter a for loop, get an error, use vi "up" key to recall entire loop:
     zsh% bindkey -v           (vi-mode editing)
     zsh% for f in *
     for> do expand $f | pr -h $f  
     for> done | lpz
     zsh: command not found: lpz
     zsh% <ESC><k>for f in *
     do expand $f | pr -h $f
     done | lpz<CURSOR>
- See zshzle(1) manpage for details

----------------------------------------------------------------------
GLOBBING (FILENAME GENERATION):
- Recursive globbing, like find(1): the pattern
     **/bar
  searches recursively, matching all pathnames ending with "bar"
  (you can also name specific starting directories).  So:
     zsh% cd /etc
     zsh% ls -l **/*sendmail*
     -rwxr-xr-x   1 root        1549 Sep  1  1999 rc.d/init.d/sendmail
     lrwxrwxrwx   1 root          18 Mar  8 12:28 rc.d/rc0.d/K30sendmail
         -> ../init.d/sendmail
     lrwxrwxrwx   1 root          18 Mar  8 12:28 rc.d/rc1.d/K30sendmail
         -> ../init.d/sendmail
     -rw-r--r--   1 root       34175 Sep  1  1999 sendmail.cf
     -rwxr-xr-x   1 root          20 Sep  1  1999 sysconfig/sendmail
- Attribute qualifiers: Wildcard patterns may end in a list of
  qualifiers in parentheses.  A few qualifiers:
     /      directories
     .      plain files
     @      symbolic links
     *      executable plain files (0100)
     r      owner-readable files (0400)
     w      owner-writable files (0200)
     x      owner-executable files (0100)
     A      group-readable files (0040)
     E      group-executable files (0010)
     X      world-executable files (0001)
       ...and so on...
  So, to list all symbolic links in the current directory:
     zsh% ls -l *(@)
     lrwxrwxrwx   1 root       11 Mar  8 12:12 rmt -> ../sbin/rmt
  The list matches if at least one qualifier matches (they're "OR'ed").
  So, to remove execute permission from all world-executable files:
     zsh% chmod o-x *(.X)

----------------------------------------------------------------------
MULTIPLE I/O REDIRECTION:
- The MULTIOS option allows multiple input and output redirections:
     zsh% setopt MULTIOS
- Multiple input redirections are handled in order, left-to-right.
  So this:
     zsh% cat file1 file2 | mail -s 'all results' joe@foo.ac.uk
  can be written as follows in zsh:
     zsh% mail -s 'all results' joe@foo.ac.uk <file1 <file2
- Multiple output redirections are all done at once, like tee(1):
     zsh% echo Hi mum >a >b
     zsh% cat a
     Hi mum
     zsh% cat b
     Hi mum
  This works with filename wildcards too.
  So, to append an "exit 0" to all shell scripts (names ending in .sh):
     zsh% echo exit 0 >> *.sh

----------------------------------------------------------------------
PATH EXPANSION:
- As in other shells, ~ and ~user are recognised.
  If ~dirname matches no usernames, it's looked up as a named directory
  (a shell variable whose value starts with "/").  Example:
     zsh% work=/etc/rc.d/rc0.d    
     zsh% cd ~work            
     zsh% pwd
     /etc/rc.d/rc0.d
- Command name prefixed with "=" is substituted with its full pathname:
     zsh% ls -l =perl
     -rwxr-xr-x   2 root     517052 Aug 31  1999 /usr/bin/perl

----------------------------------------------------------------------
SHELL EMULATION:
- In general, zsh understands both sh- and csh-type operations.
  For instance, it has both kinds of loops:
     zsh% for f in *.cc
     for> do g++ $f
     for> done
     zsh% foreach f (*.cc)
     foreach> g++ $f
     foreach> end
- Parameters like PATH, path, etc. supported in both sh and csh syntaxes:
     zsh% echo $PATH
     /bin:/usr/bin:/usr/X11R6/bin:/home/thomas/bin:/usr/local/bin:
     zsh% echo $path
     /bin /usr/bin /usr/X11R6/bin /home/thomas/bin /usr/local/bin .
- To totally emulate other shells (csh not emulated fully):
     zsh% emulate sh
       ...acts like Bourne shell...
     zsh% emulate ksh
       ...acts like Korn shell...
     ...etc...

----------------------------------------------------------------------
VARIABLE EDITING:
- Builtin "vared" command loads parameter value into internal editor
  buffer, then re-sets value after editor completes.
- Lots of clever uses.  Simple example -- modify PATH as you work:
     zsh% vared PATH
     /bin:/usr/bin:/usr/X11R6/bin:/home/thomas/bin:/usr/local/bin:

----------------------------------------------------------------------
ARRAY OPERATIONS:
- Many 

----------------------------------------------------------------------
CAUTIONS: zsh handles command line evaluation (including word splitting)
differently than other shells unless you set compatibility options.
- Word splitting: values of variables aren't split at spaces as in
  other shells.  (Use an array instead!)

     bash$ files='passwd passwd- passwd.OLD'
     bash$ ls -l $files
     -rw-r--r--   1 root     root          702 Jul 18 13:43 passwd
     -rw-r--r--   1 root     root          702 Jul 18 13:41 passwd-
     -rw-r--r--   1 root     root          696 Mar  8 12:32 passwd.OLD

     zsh% files='passwd passwd- passwd.OLD'
     zsh% ls $files
     ls: passwd passwd- passwd.OLD: No such file or directory
- Order of evaluation: Complex command lines that work in other shells
  may fail in zsh.

[Talks page] [Home page]

Contact Jerry Peek