Basic

Redirects and pipes

Programs read text and write text, and the shell moves that text around. > sends output to a file, < reads from one, and | feeds one program's output into another's input. This is where a command stops being one step and becomes a pipeline you can read left to right.

What you will see

2 - and each stage alone was checkable, which is the habit.

About 1 min of reading

Try It

Build a pipeline one stage at a time.

1

Stage one

printf 'ok\nERROR one\nok\nERROR two\n' > log.txt && cat log.txt

Before you run it, say out loud what it will print. Then open the answer.

What it prints
Four lines, two of them errors.
2

Add the filter

grep ERROR log.txt

Before you run it, say out loud what it will print. Then open the answer.

What it prints
The two error lines only.
3

Add the count

grep ERROR log.txt | wc -l

Before you run it, say out loud what it will print. Then open the answer.

What it prints
2 - and each stage alone was checkable, which is the habit.

Data Flow

Output streams move between programs.
1 Program writes to standard output
↓
2 The shell sends it to a file or a pipe
↓
3 The next program reads it as input
↓
4 The last stage's output reaches the screen
Why this works
Test a pipeline one stage at a time

Run the first stage, then add the next, then the next. Each stage's output is visible, so a wrong answer is found at the stage that introduced it rather than guessed at the end.

Errors go on a different stream

2> sends errors to a file and 2>&1 folds them into the normal output. A pipeline that produces nothing may have failed loudly on the error stream, which is why capturing both is worth the extra characters.

Examples

The arrow moves text

ls > list.txt writes the listing into a file instead of the screen. The program does not know about the file; the shell arranged the connection. That separation is why the same program works in a pipeline and on the terminal.

Pipes read left to right

cat data.txt | grep ERROR | wc -l counts the error lines. Read it as a sentence: take the data, keep the errors, count them. Each stage is a normal command and can be run alone to check it.

> overwrites, >> appends

A single arrow replaces the file, including truncating a file that had content. Two arrows add to the end. The difference is one character and it decides whether a log survives a run.

Words this lesson uses

Each one is defined here on this site. Open the entry for the full definition and the other words it points at.

← Reading files from the terminal
Previous Lesson
Reading files from the terminal
Next Lesson
Wildcards
Wildcards →