borg benchmark crud

borg [common options] benchmark crud [options] PATH

positional arguments
	PATH	path where to create benchmark input data
options
	--json-lines	Format output as JSON Lines.
Common options

Description

This command benchmarks borg CRUD (create, read, update, delete) operations.

It creates input data below the given PATH and backs up this data into the repository given via -r/--repo (or the BORG_REPO environment variable). The repository must already exist (it could be a fresh empty repo or an existing repo, the command will create / read / update / delete some archives named borg-benchmark-crud* there.

Make sure you have free space there; you will need about 1 GB each (+ overhead).

If your repository is encrypted and borg needs a passphrase to unlock the key, use:

BORG_PASSPHRASE=mysecret borg -r REPO benchmark crud PATH

Measurements are done with different input file sizes and counts. The file contents are very artificial (either all zero or all random), thus the measurement results do not necessarily reflect performance with real data. Also, due to the kind of content used, no compression is used in these benchmarks.

C- == borg create (1st archive creation, no compression, do not use files cache)

C-Z- == all-zero files. full dedup, this is primarily measuring reader/chunker/hasher. C-R- == random files. no dedup, measuring throughput through all processing stages.

R- == borg extract (extract archive, dry-run, do everything, but do not write files to disk)

R-Z- == all zero files. Measuring heavily duplicated files. R-R- == random files. No duplication here, measuring throughput through all processing stages, except writing to disk.

U- == borg create (2nd archive creation of unchanged input files, measure files cache speed)

The throughput value is kind of virtual here, it does not actually read the file. U-Z- == needs to check the 2 all-zero chunks’ existence in the repo. U-R- == needs to check existence of a lot of different chunks in the repo.

D- == borg delete archive (soft-delete the last remaining archive, measure deletion)

Deletion only removes the archive from the archives list, it does not free any repository space - that is what borg compact does (not measured here). D-Z- == all-zero files archive. D-R- == random files archive.

Please note that there might be quite some variance in these measurements. Try multiple measurements and having a otherwise idle machine (and network, if you use it).

borg benchmark cpu

borg [common options] benchmark cpu [options]

options
	--json	format output as JSON
	--chunking	benchmark the chunkers
	--hashing	benchmark the hashes / MACs
	--encrypting	benchmark the encryption modes
	--compressing	benchmark the compressors
	--data PATH	benchmark the compressors with the data from this file or directory (default: synthetic data)
	--msgpacking	benchmark msgpack item packing
Common options

Description

This command benchmarks miscellaneous CPU-bound Borg operations.

It creates input data in memory, runs the operation and then displays throughput. To reduce outside influence on the timings, please make sure to run this with:

• an otherwise as idle as possible machine

• enough free memory so there will be no slow down due to paging activity

By default all benchmarks run. Give one or more of --chunking, --hashing, --encrypting, --compressing, --msgpacking to run only those.

Some algorithms use multiple threads only above a size threshold, so the hashes and the compressors are measured at more than one buffer size: the hashes at 64MiB (roughly pack-sized) and 2MiB (a typical borg chunk), both above blake3’s threshold, the compressors at 2MiB and 128kiB, which is below zstd’s. Within a section every row processes the same total number of bytes - 1 GiB, or 10 MiB for the compressors (the complete given data when using --data) - so the throughput column is comparable between rows.

By default, the compressors work on synthetic text-like data that compresses about 4x at zstd,3. Random data would be the worst possible input: no codec can compress it, so all of them would take their incompressible fast path and the levels would barely differ.

Synthetic data can still behave differently from your real data, so the compression benchmarks can instead run on data you provide with --data PATH: PATH is a file or a directory (all files below it are read and concatenated, up to 1 GiB). The complete data is compressed: it is cut into consecutive buffers of the measured sizes (a partial buffer at the end is skipped), so ratios and throughput reflect all of it, and the runtime scales with its size. Public benchmark corpora, e.g. the Silesia corpus or the Canterbury corpus, make good reproducible inputs that resemble real-world data (download and unpack them first, then point --data at the result).

The compression rows also show the achieved compression ratio (uncompressed size / compressed size, higher is better).