Advanced

Parallel and matrix builds

You can run many jobs at once. A matrix runs the same job across many setups. This cuts test time a lot.

What you will see

Two parallel jobs: test + lint Wall-clock time = max(test 40s, lint 30s) = 40s Matrix with extra axis: os x python-version Total jobs: 4 Job: ubuntu-latest on python-3.10 -> PASS Job: ubuntu-latest on python-3.11 -> PASS

The full result is shown at the end of the exercise below.

About 1 min of reading

Try It

A matrix turns one job into many jobs that run at the same time across combinations of values, like several Python versions or operating systems. The wall-clock time is only the slowest job, not the sum.

I define a matrix over three Python versions, fan out one job per version, and print each passing job to show three tests running in parallel.

print("Parallel and matrix builds") py_versions = ["3.10", "3.11", "3.12"] print("Matrix python-version: %s" % ", ".join(py_versions)) for v in py_versions: print("Job: run tests on python-%s -> PASS" % v)

Before you run it, say out loud what it will print. Then open the answer.

What it prints

Parallel and matrix builds Matrix python-version: 3.10, 3.11, 3.12 Job: run tests on python-3.10 -> PASS Job: run tests on python-3.11 -> PASS Job: run tests on python-3.12 -> PASS

One file becomes N jobs. If any version fails, the whole matrix is red. Every combination must pass, not just the first one.

I run two different kinds of jobs at once and show that the wall-clock time equals the slowest one, then expand the matrix with an extra axis to four jobs.

print("Two parallel jobs: test + lint") print("Wall-clock time = max(test 40s, lint 30s) = 40s") oses = ["ubuntu-latest", "windows-latest"] py_versions = ["3.10", "3.11"] rows = [(o, v) for o in oses for v in py_versions] print() print("Matrix with extra axis: os x python-version") print("Total jobs: %d" % len(rows)) for o, v in rows: print("Job: %s on python-%s -> PASS" % (o, v))

Before you run it, say out loud what it will print. Then open the answer.

What it prints

Adding an axis multiplies the job count, not just the coverage. A 2x2 matrix is four jobs; a 3x3 matrix is nine. Keep the matrix small enough to stay cheap and fast.

Put it in order

Before you write anything, put these lines back in the right order. Click a line to move it up one step. This support step is easier than writing code from scratch, and it is here to make the last example stick.

Data Flow

Define a matrix of values, run each in its own job in parallel, and collect the results.

1 List matrix values

↓

2 Launch parallel jobs

↓

3 Wait for all

↓

4 Combine results

Why this works

Matrix strategy

A matrix runs one job many times with different values. Test on many versions, many databases, or many platforms at once.

Split big jobs

If a job takes too long, split it into smaller parallel jobs. They run together and finish faster.

Examples

A build matrix

jobs:
  test:
    strategy:
      matrix:
        python-version: ['3.10', '3.11', '3.12']
    steps:
      - uses: actions/setup-python@v5
        with:
          python-version: ${{ matrix.python-version }}

This runs your tests on three Python versions at the same time. One file, three jobs running in parallel.

Parallel jobs

jobs:
  test:
    runs-on: ubuntu-latest
    steps: [uses: actions/checkout@v4, run: pytest]
  lint:
    runs-on: ubuntu-latest
    steps: [uses: actions/checkout@v4, run: npm run lint]

Two jobs run at the same time. One runs the tests, the other runs the linter. Both must pass, but they finish in the time of the slower one, not both.

Matrix with an extra axis

strategy:
  matrix:
    os: [ubuntu-latest, windows-latest]
    python-version: ['3.10', '3.11']

This doubles the rows: two operating systems times two Python versions equals four jobs in parallel. You now know the code works on every combination.

Words this lesson uses

Each one is defined here on this site. Open the entry for the full definition and the other words it points at.

← Speeding up pipelines

Previous Lesson

Speeding up pipelines

Next Lesson

Security in the pipeline

Security in the pipeline →