Brat, a parallel TAP testing harness for the POSIX shell
quality 7/10 · good
0 net
Tags
sstephenson/brat: Brutal Runner for Automated Tests, a parallel TAP testing harness for the POSIX shell - Codeberg.org sstephenson / brat Watch 1 Star 24 Fork You've already forked brat 1 Code Issues Pull requests Releases 1 Wiki Activity Actions Brutal Runner for Automated Tests, a parallel TAP testing harness for the POSIX shell posix-shell tap unit-testing unix 95 commits 1 branch 2 tags 396 KiB Shell 77.5% Awk 22.5% main Find a file HTTPS Download ZIP Download TAR.GZ Download BUNDLE Open with VS Code Open with VSCodium Open with Intellij IDEA Sam Stephenson 94e95bcb9e All checks were successful Brat CI / macOS (push) Successful in 6s Details Brat CI / Linux (alpine) (push) Successful in 8s Details Brat CI / Linux (debian) (push) Successful in 22s Details Brat CI / Linux (fedora) (push) Successful in 28s Details Brat CI / FreeBSD (push) Successful in 1m1s Details Fix per-user installation instructions 2026-02-20 12:47:31 -06:00 .forgejo /workflows Move CI runner into a top-level script/test 2026-02-18 13:57:51 -06:00 bin brut: Pass original arguments to _unhandled.sh 2026-02-06 13:21:22 -06:00 lib /brat Brat 0.9.0 2026-02-19 12:48:02 -06:00 libexec Escape # and \ in test names, per the TAP spec 2026-02-13 18:38:36 -06:00 script Test suite defaults BRAT_JOBS to the number of CPUs 2026-02-18 15:50:04 -06:00 test Extract script/loc 2026-02-18 13:59:25 -06:00 .gitattributes Mark wcwidth.awk as vendored 2026-02-01 22:13:52 -06:00 CHANGELOG.md Brat 0.9.0 2026-02-19 12:48:02 -06:00 LICENSE.md Initial commit 2026-01-22 11:17:18 -06:00 README.md Fix per-user installation instructions 2026-02-20 12:47:31 -06:00 README.md Brat Brat is the Brutal Runner for Automated Tests , a parallel TAP testing harness for the POSIX shell. Brutal as in architecture. Brat is true to the “materials” it is built with: shell, awk, and the Unix pipeline. It reveals its internal plumbing in the same way a brutalist building might expose its ductwork. Some will find it ugly; others (maybe you?) will appreciate its didactic honesty. POSIX as in zero dependencies. Brat targets the POSIX.1-2024 specification. Practically speaking, it is designed to run on the minimum common subset of contemporary Unix OSes, with no other dependencies or specific implementation requirements. We test Brat under continuous integration against a variety of platforms. Intentionally small. Brat is designed to be embedded directly into your project. It has no build step and nothing to configure. At just under a thousand lines of shell and awk, you can read and understand the codebase in an afternoon. Jump to: Installation | Writing Tests | Running Tests | Implementation Notes | Contributing | License Overview With Brat, you write tests for Unix programs using a special shell syntax: # test/backup.brat setup () { cd " $DIR /.." } @test "prints usage when run without arguments" { run bin/backup.sh [ $status -eq 1 ] match " $stderr " 'usage:' } @test "errors when source directory does not exist" { run bin/backup.sh /nonexistent " $TEST_TMP .tar.gz" [ $status -eq 1 ] match " $stderr " 'not found' } @test "creates backup archive" { run bin/backup.sh " $DIR /fixtures/testdata" " $TEST_TMP .tar.gz" [ $status -eq 0 ] tar tf " $TEST_TMP .tar.gz" } A preprocessor transforms these test cases into shell functions which run with set -eu (exit on error, error when referencing undefined variables). In this way, every line of a test case acts as an assertion. When you run your tests, Brat displays the results in a streaming TAP format: $ brat test/*.brat TAP version 14 1..3 ok 1 - backup.brat:7: prints usage when run without arguments ok 2 - backup.brat:13: errors when source directory does not exist ok 3 - backup.brat:19: creates backup archive # ✓ 3 tests (3 passed, 0 failed, 0 skipped) If any line of a test fails, Brat shows the shell’s xtrace ( set -x ) output up to that point, along with anything written to stdout or stderr: $ brat test/backup.brat:19 TAP version 14 1..1 not ok 1 - backup.brat:19: creates backup archive # + setup # + cd $DIR/.. # + run bin/backup.sh $DIR/fixtures/testdata $TEST_TMP.tar.gz # + '[' 0 -eq 0 ']' # + tar tf $TEST_TMP.tar.gz # tar: Error opening archive: Unrecognized archive format # (test failed with status 1) # ✘ 1 test (0 passed, 1 failed, 0 skipped) Brat formats its TAP stream with color when connected to a terminal. In particular, failing tests are highlighted in red. Parallel Execution Tests run sequentially by default, but Brat has built-in support for parallel test execution. Use -j or set $BRAT_JOBS to run tests in parallel. For example, to run up to 8 tests concurrently: $ brat -j 8 test/*.brat Brat spawns each test in a background process, streaming results as they complete, potentially out of order. The pretty formatter buffers and sorts them live for display. Comparison with Bats Brat is a spiritual successor to Bats , the Bash Automated Testing System. If you’ve used Bats, Brat will feel familiar, but more spartan. Bats Brat Shell Requires Bash Works with any POSIX shell Parallel execution Requires GNU parallel Built-in support, using a FIFO Output TAP or a proprietary pretty format TAP always; pretty format is highlighted and sorted TAP Output capture $output , $lines[] (in-memory strings) $stdout , $stderr (file paths) Built-in helpers Rich standard library and ecosystem Minimal Lifecycle hooks Per-test and per-module setup and teardown Per-test setup and teardown only One important difference is that Brat’s run helper captures output to separate files and exposes their paths to you, avoiding the runtime overhead of reading large outputs into strings and arrays. Portability Brat is written entirely in POSIX shell and awk, targeting the POSIX.1-2024 standard with no other dependencies. It is architecture-independent and does not require a C compiler. We test Brat, using Brat , with continuous integration on the following platforms: sh awk Alpine Linux busybox ash busybox awk Debian Linux dash mawk Fedora Linux Bash gawk FreeBSD FreeBSD ash nawk macOS Bash (3.2) nawk Installation Brat has no build step and no dependencies to install. Installing Brat Globally Download and extract the latest release archive and symlink bin/brat into your PATH. For example, to install Brat in /usr/local : # curl -sL https://codeberg.org/sstephenson/brat/archive/latest.tar.gz | tar -C /usr/local -xf - # ln -s /usr/local/brat/bin/brat /usr/local/bin/brat Or if you prefer a per-user installation (assuming $HOME/.local/bin is in your PATH): $ curl -sL https://codeberg.org/sstephenson/brat/archive/latest.tar.gz | tar -C ~/.local -xf - $ ln -s ~/.local/brat/bin/brat ~/.local/bin/brat Embedding Brat in Your Project Clone Brat into your project and run it directly: $ git clone https://codeberg.org/sstephenson/brat.git vendor/brat $ vendor/brat/bin/brat test/*.brat Writing Tests Test files use the .brat extension by convention. Each file is a shell script containing one or more test definitions: @test "description of what this tests" { # Commands here run with errexit enabled; any # command that exits nonzero fails the test [ 1 -eq 1 ] } It’s a good idea to add a standard #!/bin/sh shebang to the top of each test file so that your editor or code forge applies proper syntax highlighting. Note, however, that Brat test files cannot be executed directly by the shell. About the Test Environment Brat automatically sets the following variables before each test run: $FILE — the path to the test file $DIR — the directory containing the test file $TEST_TMP — a unique temporary path prefix for the current test Use the $DIR variable to source test helper scripts or load fixture data relative to the location of the test file. You can use the $TEST_TMP variable as a prefix for temporary files or directories you create during a test. Filenames matching $TEST_TMP.* are automatically deleted after each test run. Running Commands with the run Helper Use run to execute a command and capture its output and status code: @test "captures exit status and output" { run ls /nonexistent [ $status -eq 1 ] match " $stderr " 'No such file' } After run , three variables are available: $status — the command’s exit code $stdout — the path to a file containing standard output $stderr — the path to a file containing standard error Matching Output with the match Helper Use match to assert that a file contains a string or pattern: match " $stdout " 'hello world' # exact substring match " $stdout " '/^hello .+$/' # ERE pattern If the second argument begins and ends with a / , the match helper treats it as an extended regular expression (ERE) pattern . Otherwise, it is treated as an exact substring to match. Comparing Files with the compare Helper Use compare to assert that two files have identical contents: run my_formatter " $TMPFILE " [ -s " $TMPFILE " ] } teardown runs even when a test fails, so it’s safe to use for cleanup. Top-Level Code Code outside of @test , @skip , and @todo blocks runs twice: once when Brat scans the file to discover tests and their names, and again before each test runs. # Runs during both planning and test execution cd " $DIR /.." setup () { # Only called during test execution } @test "example" { # ... } Keep this in mind if your top-level code has side effects. In practice, most test files only define functions (like setup ) at the top level, which is harmless during planning. Running Tests To… Run… Run all tests in a directory brat test/*.brat Run a specific test file brat test/backup.brat Run a specific test by line number brat test/backup.brat:19 Run tests in parallel (8 concurrent jobs) brat -j 8 test/*.brat Filter tests by exact name match brat -n "creates backup archive" test/*.brat Filter tests by extended regular expression (ERE) brat -n "/backup/" test/*.brat Exclude tests by exact name or ERE brat -e "/usage/" test/*.brat Working with Subcommands Brat exposes the subcommands that make up its internal pipeline. When you run brat test/*.brat , Brat orchestrates the following: For each test file, test-plan extracts test metadata (file, line, kind, name) into a tab-delimited plan. plan-build aggregates these plans, sorts the tests by filename and line number, and applies any -n / -e filters. plan-run executes tests from the plan, invoking test-run in parallel and printing the results as TAP. You can work with this plumbing directly: Subcommand Description brat plan-run The main entry point: build a plan, run tests, format output brat plan-build Build a test plan from files, applying -n / -e filters brat test-plan Extract test metadata from a single file brat test-run Run a single test by file and line number The subcommands are composable. For example, you can build a plan once and pipe it to brat plan-run - : $ brat plan-build test/*.brat | grep backup | sort -r >plan.txt $ brat plan-run -