Pipes, scripting on UNIX

When using Unix via command-line, note that there are very few extra messages (e.g confirmations, explanatory messages.)

Example
The result from the LS command is of the form:

```
fred.txt
names.txt
```

rather than:

User xyz123. Files are:
fred.txt
names.txt

Why?
Extra text will 'confuse' programs which receive the piped data.
(i.e what is data and what is commentary?). The tools are cohesive (single-minded)
see 'The Art of Unix Programming' by Eric Raymond -
(http://library.n0i.net/linux-unix/art-unix-programming/ ) chapter 1... silence

Pipe Example

Produce software which creates a concordance of a text file.*

- A concordance is a list of each word, sorted, once only.

E.g the concordance of * above is:


a
concordance
creates
file
of
...

Solution - what tool to choose?

Java? maybe - suitable classes
perl, python - maybe
C - no. No good string-handling
C++ maybe

UNIX tool solution

Concept

put each word on a line by itself
sort the lines
replace duplicate lines by a single copy

From experience, we choose the commands:
tr, sort, uniq

the tr (transliterate) command replaces one char by another throughout its input. Here, we use it to replace every space by a newline.
```
   tr " " "\012"  < myfile > newfile
       ^    ^       (quoted space, and octal code for newline char)
```
sort - arranges the lines of its input data into ascending order.
uniq - spots a group of similar lines and replaces them by one copy

we have a pipeline of 3 programs:


    +------+     +--------+      +-------+
data|  tr  | -->-|  sort  |-->---| uniq  |-->-- output...
 in +------+     +--------+      +-------+

tr " " "\012" < infile.txt  | sort | uniq   (unix command)

Creating your own commands

To avoid typing repetitive commands, we can create a text file containing the commands, then execute it by typing its name.

Example
We might create a concordance command by:

create a file named conc, containing the single line:


  tr " " "\012" < infile.txt | sort | uniq > newfile

we tell unix that it is command, by setting its properties (mode) to 'executable':
```
chmod  +x  conc
```
we can now run it, by typing: conc

Shell Scripting

The shell contains a powerful programming language for connecting commands together. it has:

pipes, IO redirection (as you have seen)
parameter passing to commands
control structures - while,for, if

Example

The above conc command did not allow use to choose our file names. We would prefer to say which files we wish to use:


conc essay myConcordance

Here is how: we amend the line i the file to read:

  tr " " "\012"  < $1 | sort | uniq > $2

$1 and 2 are substituted by command-line parameters 1 and 2 (numbered left-to right)

$1 replaced by: essay
$2 by: myConcordance

To make a more robust command, we might add an IF to check that $1 exists, and display a message if it is not there.

Summary

Scripting is powerful and important on Unix/ linux
Many tools are text-oriented (and not for beginners)
The shell has a built-in scripting language, but nowadays, we might choose to do scripting by Perl, Python, Tcl/tk etc...