Command Lines To Know: Time-saving tips at a shell prompt

by Jerry Peek
June 2010

There are times that a command line is much faster than a graphical application. This month, let's look into some tips for turbocharging the command line — to do a lot with little typing.

Remembering history

Before shells let you use Emacs and vi-style commands to edit a command line, the C shell introduced history. The shell keeps a list of what you typed at a shell prompt, and the ^ and ! operators let you recall and modify part or all of previous command lines. Knowing your history (operators, that is) is still handy. Let's look at three ways to use shell history.

Replacing and removing text with caret

Did you type too many (or too few) of a character, or type the wrong character? The ^ (caret) operator lets you remove characters from the previous command line — and, optionally, replace them. The syntax is ^OOO^RRR, where OOO is the original text and RRR is the (optional) replacement.

For example, you type lgo instead of log. Change go to og:

$ grep -i warning lgo*
grep: lgo*: No such file or directory
$ ^go^og
grep -i warning log*

The shell echoes the edited commmand line, then runs it. (Interactive editing may seem easier in cases like these, but carets can take fewer keystrokes — and also show your edit.)

Or you type warninnng instead of warning. You terminate the running command, then (at the next shell prompt) remove nn:

$ grep -i warninnng log*
...you press ctrl-c...
$ ^nn
grep -i warning log*

Replacing multiple occurrences

Here's a little of what the ! (exclamation point) operator can do. You can make multiple similar edits to the previous command line with !!:gs/OOO/RRR/, where OOO is the original text, RRR is the replacement, and gs means "edit all occurrences."

For example, you need to see all lines in HTML files containing an opening or closing <tt> tag. Then you'd like to repeat the command to see all <code> tags. Watch as the shell replaces both occurrences of tt:

$ grep -i -e '<tt>' -e '</tt>' *.html
...grep output...
$ !!:gs/tt/code/
$ grep -i -e '<code>' -e '</code>' *.html
...grep output...

This technique saves more time when there are more than two occurrences.

Grabbing previous arguments

When you run a series of commands on a file, the !$ operator is an easy way to get the last argument (typically a filename) from the previous command line.

For instance, you have a set of files you need to search with grep — and to edit if they have the text you're looking for. See below. The first grep finds nothing in log_2010-05-13, so you search the next file (changing the 13 to 14 with the caret operator we saw earlier). That file has a match, so you edit it; the !$ operator pulls in the filename:

$ grep -i error log_2010-05-13
$ ^3^4
grep -i error log_2010-05-14
09:44:07: ERROR: input voltage 9.7
$ vi !$
vi log_2010-05-14

To get the last argument from an earlier command line, add the first argument (or enough of it to be unique) after the ! and folliw it with a colon (:) separator, then the $. Below, after finding a match, you send an email message. Then you edit the file, grabbing the filename from the grep command line (the most recent command line starting with gr):

$ grep -i error log_2010-05-14
09:44:07: ERROR: input voltage 9.7
$ mail manager
Subject: Located the error
James, I found it - May 14 at 9:44 AM.
.
$ vi !gr:$
vi log_2010-05-14

Once your fingers learn this operator, it can save time and mistakes typing complex filenames.

Build strings with curly braces

The little-known curly-brace operators {C,D} work like wildcards, but they can build any type of strings — not just filenames. Since they don't work by matching existing filenames, you can use them to create new files. For instance, to create 15 new files named 2010-05-10 through 2010-05-24, type:

$ vi 2010-05-{10..24}

(Tip: within vi, the operator :n opens the next file.) There are more examples in the article Filenames by Design, Part Two.

Loops aren't only for scripting

Loops repeat one or more commands, changing some value(s) before each repetition. You've probably used loops in shell scripts, but they're also handy at the command line. Say you'd like to automate the grep-and-vi steps in the previous section. For each file in a list, search the file and edit it only if a match was found. The for loop below steps through the log files from May 13-27, searching each file and running vi if grep returns a zero (success) status. (We're using a combination of the range wildcards [A-B] and curly-brace operators to get all 15 filename arguments log_2010-05-13 through log_2010-05-27.) The option -l makes grep suppress output and return only the exit status:

$ for file in log_2010-05-{1[3-9],2[0-7]}
> do
>   if grep -li error "$file"
>   then vi "$file"
>   fi
> done

The shell's secondary prompt > appears until you've entered the whole loop. Here's a shorter version using the shell's && "run on success" operator:

$ for f in log_2010-05-{1[3-9],2[0-7]}
> do grep -li error "$f" && vi "$f"
> done

(How long would that take to do with a graphical application instead of a shell?)

Packaging commands in a function

Got a series of commands that you want to run again later? Package them in a shell function. You can either type the function on the command line or store it in a shell setup file (like .bashrc in your home directory). Let's package the previous loop into a function named finderror:

$ finderror () {
> for f
> do grep -li error "$f" && vi "$f"
> done
> }
$ finderror log_2010-04-*
...search and edit all 2010-04 files...
$ finderror log_2010-05-* today
...do all 2010-05 files and "today" file...

Starting the loop with for variable gets the loop arguments from the command line as the function is run.

Remembering text

One of the simplest tips comes from programming: storing text you need to re-use in a shell variable. It saves typing and can avoid mistakes. In general, quote the value when you set the variable (to preserve spaces), but don't quote when you use the variable (so the shell will break the list at spaces).

For example, let's save a long pathname as $log and three filenames as $files. Later you can use the saved names as many times as needed -- for instance, to open a GUI editor:

$ log="$HOME/data/logs/2010-05-14"
$ files="file23 file34 file45"
...
$ grep ERROR $log
$ grep WARNING $log
$ xeditor $files

If you need to store multiple separate arguments and preserve special characters too, use an array instead. Unless you're an experienced shell programmer, though, the syntax is trickier. One tip: expand the value with "${arr[@]}", where arr is the array name.

Find out more

We've moved through these examples pretty quickly. A good shell book, or your shell's manual page, can fill in details — and show more that you can do on the command line. Basically anything you can use in a shell script can also be typed on the command line.

Jerry Peek is a freelance writer and instructor who has used Unix and Linux for more than 25 years. He's happy to hear from readers; see http://www.jpeek.com/contact.html.

[Read previous article]
[Read Jerry’s other Linux Magazine articles]