misa77: decode-first LZ that outruns LZ4

Most compression choices still start from the same three-way fight: ratio, encode speed, decode speed. For a large class of developer workloads the last of those is the only one that actually matters once the bits leave the writer. Think game assets, firmware blobs, static dataset shards, or any pipeline that compresses once and then serves or loads the same payload thousands of times. LZ4 has owned that niche for years because it is simple, battle-tested, and extremely fast on the decode side. A new LZ-family codec called misa77 is trying to push the same frontier further: higher decompression throughput than LZ4 while also beating its ratios, at the explicit cost of slow compression and a still-young codebase.

The claim is concrete enough to be interesting. On the author's single-threaded, pinned-core measurements (Intel Core i7-14650HX, Turbo disabled), misa77 0.2.0 level 0 reaches 5219 MB/s decompress on silesia.tar at a 42.64 ratio while LZ4 1.10.0 sits at 2505 MB/s and a worse 47.59 ratio. Level 1 trades a little speed for denser output (4274 MB/s, 39.65). The same pattern holds on enwik8. Across the Silesia corpus the author reports level 0 beating LZ4 decode on every file; higher effort still wins on 11 of 12 (the exception is essentially incompressible x-ray data). Cross-platform numbers for the earlier 0.1.0 release already claimed 1.5–3× LZ4 on both x86-64 and ARM64. Memory is fixed: ≤5 MB at compress time, zero extra at decompress. There is no entropy stage, so nobody is comparing it to zstd or LZMA; the intended peer group is the fast pure-LZ family (LZ4, LZ4HC, LZSSE, Lizard, Snappy, zxc).

The write-once niche, refined

This is not a general-purpose replacement for LZ4. The README is explicit: slow compression is the deliberate tradeoff that buys the decode numbers and the constant memory. Encode rates hover around 40–55 MB/s on the same machine where plain LZ4 does 270–370 MB/s. That is closer to LZ4HC's high-effort settings than to the fast path most people reach for. In return you get two properties that matter for WORM data. First, higher-effort compression often improves decompress throughput rather than hurting it, because the resulting stream is friendlier to modern CPU microarchitectures. Second, highly compressible inputs tend to decompress even faster, which is the opposite of the usual "harder match = more work" intuition. Experimental modes in the tree push this idea further by spending more encode time to produce streams that decode especially cleanly.

The practical consequence is that the sweet spot is any pipeline where you can afford a slow, offline (or background) encode step and then want the absolute highest single-threaded decode rates you can get without dragging in an entropy coder or a large working set. Constant 0 MB decompress memory also makes the codec attractive for constrained environments or for cases where you simply do not want another dictionary allocation on every load.

Where it sits among the fast LZ codecs

LZ4 still wins on raw encode speed and ecosystem maturity. LZ4HC already demonstrated that you can slow the encoder to improve ratio without slowing the decoder; misa77 takes that idea further and also reclaims ratio ground that pure LZ4 usually cedes. On the author's Silesia numbers it sits ahead of both LZ4 and the various LZSSE/Lizard/zxc configurations on the decode-speed-versus-ratio Pareto front for the compressible files. That is the interesting part of the result: it is not merely "faster than LZ4 at worse ratio." It is frequently faster and denser.

Background context helps set expectations. The fast pure-LZ family has been relatively stable for a decade. LZ4, Snappy, and LZO occupy the "encode and decode both near or above memory bandwidth" corner. High-compression variants (LZ4HC, some LZSSE modes) move along the ratio axis while keeping decode cheap. Codecs that add entropy coding (zstd, even at level 1) jump to better ratios but pay a clear decode tax; on the same Silesia run zstd -1 is already down near 900 MB/s. misa77 is deliberately staying in the pure-LZ lane and trying to move the decode ceiling while still improving ratio over vanilla LZ4. Whether that ceiling holds up under independent multi-machine, multi-compiler testing is still open; the published numbers come from the author's own lzbench fork on one Intel part plus an earlier cross-platform set that the README itself flags as potentially outdated.

What a developer actually does with it

Adoption path is currently a C++20 library plus a POSIX CLI. You need CMake ≥ 3.20 and a little-endian 64-bit target. On x86-64 the build selects AVX2/SSE2 at runtime. Integration looks like any other small C++ compression library: include the headers, call the compress/decompress entry points, pick level 0 or 1. There is no multi-threaded API yet, no streaming framing beyond what you build yourself, and no language bindings outside C++. That is fine for an 0.2.0 project with a few dozen stars; it is not yet a drop-in for a production service that already wires LZ4 or zstd through a stable C ABI.

The real evaluation work is therefore workload-specific:

• Measure your data, not Silesia. The author notes performance is "spiky" and depends on data shape. Highly compressible text and structured binaries are the happy path; already-compressed or high-entropy material collapses toward memcpy speeds for every pure-LZ codec, including this one.
• Time the full pipeline. If you re-encode frequently, the ~50 MB/s compress rate will dominate. If you encode once (asset cook, nightly dataset build, firmware packaging) and then decode on many cores or many machines, the 2× decode win compounds.
• Check the constant-memory claim in your environment. Zero extra decompress RAM is nice for embedded or for processes that already pin a lot of working set.
• Keep a fallback. The project is young, the test surface is small, and there is no long public track record of fuzzing or production crash reports.

A sensible first experiment is exactly the author's harness: build the public lzbench fork, drop your representative blobs next to silesia/enwik8, and compare misa77 levels 0/1 against LZ4, LZ4HC -12, and whatever LZSSE or Lizard variant you already trust. If decode is the bottleneck and the ratio also improves, the slow encode is often acceptable.

Maturity and the usual caveats

Two things keep this from being an automatic "replace LZ4" recommendation. First, independent corroboration is still thin; the detailed 0.2.0 numbers are single-machine and author-supplied. Second, the encode speed and lack of ecosystem (bindings, framing, multi-thread, long-term API stability) mean it is not yet competing for the same "just call the library" slot that LZ4 occupies. The experimental high-effort modes are explicitly marked experimental. Runtime CPU dispatch exists, but the supported platform list is still the classic little-endian 64-bit set.

None of that makes the result uninteresting. The pure-LZ decode frontier has been relatively quiet; a codec that simultaneously improves ratio and single-threaded throughput while keeping decompress memory at zero is a useful data point. For any team whose working set is large, static, and decode-heavy, the numbers are already worth a weekend of measurement. If the claims hold under broader testing and the project continues past 0.2, misa77 becomes a credible specialized tool rather than a curiosity. Until then it is a sharp, narrowly targeted experiment that knows exactly which tradeoff it is making.