Making it Easier

The UNIX Shells

A fundamental concept in the UNIX world is that of shells. A shell is a program that forms an interface between you and the raw operating system. Whenever you enter commands into a UNIX system, you talk to a shell, which takes your input, processes it in various ways, makes the required operating system calls and displays any results. Having this separation allows many useful features to be introduced without changing the underlying operating system. All shells provide certain functionality, much of which will be described in this section, but some shells offer advanced compabilities which will discussed in a later section dedicated to that shell.

The main shells are:

The first two shells listed above are fairly basic, and some versions do not provide some of the facilities dicussed here, such as tab-completion. Therefore it is recommended you use one of the other four, if possible. You can find out which shell you are using by typing echo $SHELL. You can use a different shell by typing its name, or you can change shells completely using the chsh command, if this available on your system (run the command and you'll be prompted for the full pathname of the shell you wish to use - use which followed by the name of the shell to tell you this).


Filenames can be long - up to 256 characters. They are also case sensitive, so for example, file, FiLe, and FILE are all different files. You can used mixed case and other characters like "_" and "." to give your files meaningful names. Don't be put off from using long filenames because you don't want to have to keep typing them in - there is no excuse with tab-completion!

To experiment with this feature, create some files by typing touch psychohistorians foundationsEdge foundationAndEmpire. Don't worry, you'll only have to type that lot in once.

Now let's say you want to rename the first file to seldon. To do this you would use the command mv psychohistorians seldon. But wait! Type mv psy (without pressing Return) and then press the TAB key. Providing your shell supports it, the rest of the filename will be completed for you, as if by magic! What is happening here is your shell is looking for all files that match the prefix you have entered. Assuming there is only one matching file, it will fill in the rest of the name for you. You can then enter the rest of the command.

Next you want to delete the second file. Type rm fou and press TAB. The shell will fill in as such of the name as it can - foundation in this case (assuming there are no other similar files in the directory.) It will then beep to let you know there is more than one completion. Now type s and press TAB again. This prefix should now be unique, so the full filename will be filled in. When there is more than one completion, some shells can display the available choices (for example, press TAB again with the bash shell).

This time, typing mv fou and pressing TAB should cause the full filename of the third file to be filled in, as the other has been removed.

Filename Substitution

This facility, also known as globbing, allows you to select files using wildcards. This form of pattern matching can be used to refer to a collection of files in one go, again saving considerable time and effort.

To experiment with this feature, repeat the touch command from above, either by entering it again, or by pressing the cursor up key repeatedly to select it from your command history, if your shell supports this. Also do touch found.c mule.p.

Patterns are specified using wildcard metacharacters, which are simply characters which are treated specially by the shell. If you wish to use one of these characters as itself, and not its special meaning, precede it by a backslash (\) character; for example, touch the\*star would create a file called the*star. The meaning of the wildcards is given in the table and their use is demonstrated by the following examples.

*Matches any string
?Matches any single character
[ ] Matches any of the characters
between the brackets. A dash
is used to indicate a range
of characters

List all files beginning with fou:
ls fou*

List all files ending with .c:
ls *.c

List all files ending with a dot and a single character:
ls *.?

List all files beginning with f or m:
ls [fm]*

List all files beginning with a letter in the range a..m:
ls [a-m]*

The shell expands the pattern into a sorted list of filenames and passes this to the command involved, exactly as if you had entered each of the names separately. Now use filename substitution and/or tab-completion to remove all the files created here.

Pipes and Redirection

Pipes allow you to send the output of one command to the input of another, using the pipe metacharacter "|". For example ls | wc -l will count the files in the current directory, and ls -l | more will give a detailed file list, pausing between each page if the list is too long to fit on the screen at once.

You can join any number of commands together in this way, to form a pipeline. These are fundamental to UNIX, and can solve complex problems using only the basic building blocks of UNIX commands. For example ls -l | cut -c33-42 | sort -nr | head -1 finds the size of the largest file in the current directory. It works by listing the files, extracting the size column, sorting this in descending numerical order, and taking only the first line.

Redirection allows commands to get their input from a file instead of the keyboard, and send their output to a file instead of the screen. The two metacharacters "<" and ">" are used to do this. For example, ls > contents will store the directory listing in a file called contents and will not display anything to the screen. Verify this by running the command and then using cat or more. Note that the contents file is created in the process, but if it already exists it will be overwritten, so be careful. Input redirection is similar; wc -l < contents will cause the wc command to read from the file.

There are two additional forms of redirection: ls >> contents will send the output to the named file, but it will append this output to the current contents of the file, if any. The input version of this operation creates what are known as here documents. Enter cat <<END. Your prompt will probably change, to indicate that the command is waiting for some input. Type a few lines of text, and then type END exactly, on a line of its own. The command should then complete by repeating what you entered. The <<HERE part of the command tells your shell to accept what is typed as input to the command, until the word END is read on a line of its own - any word can be used instead. The command then behaves as if what you entered was stored in a file and redirected into the command. This facility has numerous uses, such as quickly creating a small file; cat <<HERE >textfile will cause what you type to be stored in the named file.

Input, Output, and Error Streams

The pipe and redirection characters discussed above manipulate streams of characters. There are three standard streams: input, output, and error. Normal communication to and from commands takes place via the input and output streams, but there is a third channel for reserved for errors. For example, wc -l contents > out will count the lines in the file, assuming it exists, and store the results in a file called out. However, if you remove the contents file and try again, you will get an error message: wc: contents: No such file or directory, and the out file will be empty. The error message is not send to the file as it is using a separate stream, namely the error stream.

It is, however, possible to redirect the error stream in a similar way to the standard output stream. Typing wc -l contents 2> out will send any errors to the file, with any standard output going to the screen. Multiple redirection is possible as in wc -l contents >out 2>errors, which sends the two output streams to different files. Note that spaces after the redirection characters are optional.

< Previous: Finding Out ^ Next: Process Control >

Matt Chapman
Last modified: Sun Aug 10 15:16:05 BST 1997