Skip to content

Testing & benchmarking

Verify archives without writing to disk, and measure performance reproducibly.

Last updated

arca test

Reads every entry and checks its CRC (or its HMAC, when the entry is encrypted) without writing anything to disk.

Terminal
arca test arca.ziparca test secret.zip -p "a password"
Output
174 entries verified, no errors (0.006 s)

Corrupt entries are reported on standard error, and the command exits with status 1 and a count of the entries that failed.

Verifying artifacts in CI

YAML
# with arca on the PATH; see Installation- name: Package  run: arca create dist.zip build/ -l best- name: Verify  run: arca test dist.zip

arca bench

Measures the design’s performance requirements R1 and R2 against a real archive and reports PASS or FAIL.

Terminal
arca bench big.zip

It prints the R2 figure for the archive you give it, and the hyperfine command for R1:

Output
Performance requirements (design document, section 05)  R2  list without extracting      6000 entries in a 828.8 KB archive      1.1 ms   target < 200 ms   PASS  R1  cold start: open and list a one-entry archive, with hyperfine      hyperfine -N --warmup 20 'arca list tiny.zip'

The published figures for R1, R2 and R3, and the script that reproduces them, are on the benchmarks page.

Reproducible benchmarks

  • Report the best of several runs, and delete the output directory before every pass.
  • Small corpora sit in the operating system’s write cache and make every tool look faster. Use data that doesn’t fit.
  • When a run writes gigabytes, alternate the arms (A B B A) and measure the disk with a plain sequential write before and after.
  • Compare CPU time with wall time. If they match, the program is its own bottleneck; if wall time is longer, the disk is.