Master

A batch over many files as a queue with a point

A batch that runs out of time is a batch that must be resumable. The queue is the cut and the point is the record, and the record is why a rerun is not a redo.

What you will see

2. A skipped count is what makes a rerun a claim.

About 1 min of reading

Try It

Write a queue and a point

Write the queue

printf 'f1\nf2\nf3\nf4\nf5\n' > /tmp/q.txt; wc -l < /tmp/q.txt

Before you run it, say out loud what it will print. Then open the answer.

What it prints

5. Five names is the cut.

Write the point

printf 'done f1\ndone f2\ndone f3\n' > /tmp/d.txt; tail -1 /tmp/d.txt

Before you run it, say out loud what it will print. Then open the answer.

What it prints

done f3. The point is written, which is what a rerun reads.

Count the skipped

printf 'total 5 done 3\n' | awk '{print $4-$2}' | awk '{print "skipped", $1}' | awk '{print $2}'

Before you run it, say out loud what it will print. Then open the answer.

What it prints

2. A skipped count is what makes a rerun a claim.

Data Flow

Many files to a queue and a point

1 The names are written as a queue

↓

2 Each file is run and reported

↓

3 The finished file is written as the point

↓

4 A rerun reads the point and starts after it

Why this works

Why the queue is a file and not a loop

A loop is a place that closes. A file is a list that survives, and surviving is what a rerun needs.

The one count that must be in the report

The skipped count. A skipped count is what makes a rerun a claim rather than a guess.

Examples

The queue is written before it runs

The names are listed first. A listed queue is a queue that a rerun is compared against.

The point is written per file

The file that finished is written. A written file is a file a rerun does not redo.

A refusal keeps the point

The run stops and the point stays. Staying is what makes a rerun continue rather than restart.

Words this lesson uses

Each one is defined here on this site. Open the entry for the full definition and the other words it points at.

← A rename tool as a plan that is checked before it runs

Previous Lesson

A rename tool as a plan that is checked before it runs

Next Lesson

Shipping a tool with every limit in one file

Shipping a tool with every limit in one file →