TACO

Hard shell. No floats. Same taco on every machine.

Taco is a lossless compressor that uses a language model (GPT-2). The model predicts the next piece of the data, and a range coder stores only what the model did not predict. The engine of Taco, HardShell, runs the model with integer arithmetic only. Thus every CPU and GPU calculates the same predictions, bit for bit. A file that one machine compresses ("folds") decompresses ("unfolds") on every other machine.

taco fold notes.txt          # makes notes.txt.taco
taco unfold notes.txt.taco   # restores notes.txt, byte for byte

Real ./try output of the 0.1.0 release on macOS (CPU), Linux (CUDA) and Windows (CUDA). Long lines are shortened; the full output is in docs/img/try-output/.

Warning

Alpha: a proof of concept (version 0.1.0, 2026-10-19). Taco is an experiment. It is slow, and it is not a replacement for zstd or xz. Keep a second copy of data that you fold. Read Limits before you use it.

A note from me

Hi, I made Taco. I work in ML and backend. I don't know Rust, CUDA or Vulkan.

LLM compressors get great ratios, but they have one ugly problem: the model runs on floats, and floats round a little differently on different CPUs and GPUs. Compress on one machine, decompress on another, and one tiny difference can break the whole file. So moving a compressed file to different hardware is risky.

My idea was simple: take floats out of the model completely. Convert GPT-2 to integers once, run every step in integer math, and every machine has to get the same answer. The design is mine. Claude (with Claude Code) wrote almost all of the code, the tests and the docs, because I can't write the GPU stuff myself. That's also why the docs below read like Claude. So don't trust the code because I say so, trust the tests: conformance hashes, the cross-machine check, fuzzing, and every fast path checked against the slow reference.

It's slow and it's an alpha. The one thing I want you to try: run ./try from the release, and check that you get the same hash as everyone else. If your machine gives a different hash, that's a bug, and I really want to know about it. Open an issue.

Contents

• Proof: the same bytes on every machine
• Try it
• What Taco is, and what it is not
• Results
• What is guaranteed
• Hardware
• Limits
• How it works
• Build from source
• Usage
• Roadmap
• Development
• Licence

Proof: the same bytes on every machine

This is the point of Taco. Speed and ratio come second.

The release program of version 0.1.0 folded the demo sample (demo/samples/sample.txt, 3,672 bytes, --profile text) and unfolded it again on 7 backends: 3 operating systems, 2 CPU architectures and 4 kinds of processors. Each fold ran on the backend in the row, and each unfold restored the sample byte for byte (measured, 2026-10-19):

Thus a file that any of these machines folds unfolds on all of the others. To add a row, run ./try on your machine (Try it) and compare the hash. More checks of the same property, with more inputs, are in Same bits on different machines.

Try it

There are two ways. Both give the same .taco bytes.

A. Download the program (nothing to build)

1. Open the Releases page and download taco-demo-0.1.0.zip (about 120 MB). It has the programs for macOS (Apple Silicon), Linux (x86-64) and Windows (x86-64), the model file gpt2.hsm, and a one-command demo. If you want only the program for one system, download taco-0.1.0-<system>.tar.gz (or .zip for Windows); it does not include the model file.
2. Unpack the zip and open a terminal in the unpacked folder (taco-demo-0.1.0).
3. Run ./try (macOS, Linux) or try.cmd (Windows).

The demo folds and unfolds a short English text (3,672 bytes), checks it, and prints the SHA-256 of the .taco file. It takes about 10 s to 1 minute. The end of the output:

PASS: the file came back byte for byte.
Original: 3672 bytes. Folded: 648 bytes.
SHA-256 of the .taco file: 2fab1ef7a1c70cd4e39ddfd88a8d66389ae07a403111db131326ae425460652e

This hash is the same on every machine and every backend. Run the demo on a second computer and compare. To fold your own files, use ./taco fold notes.txt and ./taco unfold notes.txt.taco in the same folder (Windows: taco.cmd). Details and troubleshooting: demo/README.md.

B. Build the same program yourself

You need: rustup (it installs Rust 1.99.0 from rust-toolchain.toml), uv for the Python tools, a C compiler (for zstd), about 1.5 GB of disk and 8 GB of memory. A GPU is optional.

git clone https://github.com/tac0mania/taco.git && cd taco
uv run --project tools python -m baseline.fetch     # GPT-2 weights; the tool checks each SHA-256
uv run --project tools python -m convert.gpt2       # makes tools/data/out/gpt2.hsm (about 1 minute)
shasum -a 256 tools/data/out/gpt2.hsm               # must be aaa03eb96e9ad795...b51001ba5c2
cargo build --release --locked -p taco-cli
export TACO_MODEL="$PWD/tools/data/out/gpt2.hsm"
./target/release/taco fold demo/samples/sample.txt --profile text -o /tmp/sample.taco
shasum -a 256 /tmp/sample.taco                      # must be the hash in section A
./target/release/taco unfold /tmp/sample.taco -o /tmp/sample.txt
cmp demo/samples/sample.txt /tmp/sample.txt && echo "restored byte for byte"

• The converter calculates with exact fractions, so your gpt2.hsm is the same file, byte for byte, as the one in the release. If its hash is different, do not use it.
• Your .taco file must have the hash of section A. Then your build and the release program are compatible: each one unfolds the files of the other.
• scripts/release/build.sh <target> makes the release archive itself. It builds with fixed paths and times, so two builds of one commit on one machine gave the same binary, byte for byte (measured on Linux). BUILDINFO in each archive records the commit and the SHA-256 of the program.

Caution

Taco is slow. With a GPU, it folds text at about 85 to 129 KB/s and unfolds at about 26 to 31 KB/s. With the CPU only, it folds at about 0.3 to 2 KB/s and unfolds at about 0.1 to 0.5 KB/s (measured). Start with a file of a few KB.

What Taco is, and what it is not

Taco is a compressor whose output does not depend on the machine. Its parts are:

• an open specification of integer-only GPT-2 inference (docs/adr/0004-hardshell-numerics.md);
• an exact converter: it reads each float32 weight as mantissa x 2^exponent and calculates with exact fractions, so every machine makes the same model file (ADR 0014);
• implementations for the CPU (NEON, AVX2), NVIDIA GPUs (CUDA) and other GPUs (wgpu: Vulkan, Metal, DX12), which tests show give the same bits;
• conformance vectors that connect the model to the bytes of real .taco files.

Taco is not the first or the best compressor with a language model:

• ts_zip (Fabrice Bellard) makes smaller files and is faster. Its page also states that its result does not depend on the GPU or the CPU.
• The ideas under Taco have earlier work: language models with entropy coders (LLMZip), integer networks for compression that decode the same on every platform (Ballé et al., 2019), and integer-only Transformers (I-BERT).

What Taco adds is the combination: an inspectable integer specification, a repeatable model conversion, several independent implementations, and tests that show they agree.

Results

All numbers are measured unless they are labelled "published" or "est.". Details are in docs/research/. The sizes do not include the model file (127 MB), which both machines must have.

Size

Against another compressor with a language model, on the whole alice29.txt (152,089 bytes):

ts_zip is 44% smaller. It uses a larger and newer model (RWKV, 169M parameters; GPT-2 small has 124M), and Taco's profile text restarts the model context at each 16 KiB block.

Logs (profile logs: GPT-2, a match model and a mixer)

The 16 Loghub log files (4,458,875 bytes in total), the defaults of the profile logs (blocks of 64 KiB, context overlap 512), whole files for the other compressors (docs/research/2026-10-19-overlap-cli.md, docs/research/2026-10-19-logs-profile.md):

Speed of the profile logs with the GPU mixer (RTX 3060 at 95 W, CUDA, commit 1493af3):

What can make logs smaller (not in this version): the skip rule, 4.8 to 7.3% smaller (est., simulation, ADR 0017); and a model fine-tuned on the logs of known systems, 53.2% smaller on those systems but 4.6% larger on other systems (measured, frozen; Experiments).

Integer model against the float model

The +0.2% is a total. Single files change from −9.0% to +6.6%. The conversion saturates large activations, which changes the predictions; against a float model with the same saturation, the integer model is 2.3% larger (docs/research/2026-10-18-integer-gpt2.md §6).

Speed (RTX 3060 Laptop GPU, 95 W)

For comparison, zstd -3 compresses at hundreds of MB/s and xz -9e at about 0.7 MB/s. ts_zip states "up to 1 MB/s" on an RTX 4090 (published, a different GPU).

Large files (streaming): a memory test, not a speed test

This test checks that the memory does not grow with the file (ADR 0018). It used 200,000,000 bytes of random words and the small test model, on the devbox CPU.

Both unfolds restored the input byte for byte. Its times say nothing about Taco's speed: the test model has random weights, so every block was stored with zstd or raw, and unfold ran no model step. The real speed is in the table above. The largest files folded with GPT-2 so far are 2.4 to 2.7 MB (measured).

Same bits on different machines

What is guaranteed

• No floating-point arithmetic in fold or unfold. The engine, the coder, the mixer and the GPU kernels use integers only. scripts/check-no-floats.sh checks this on each push.

• The same result on every machine, when each operation is fully specified. Integers alone are not sufficient. Taco makes these conditions rules (CLAUDE.md §1):

  • every sum has an accumulator that is wide enough, with the bound written in the code, or it wraps on overflow in a defined way;
  • every rounding and every clamp has one written rule (ADR 0004);
  • signed division uses floor-division helpers.

  Then a compiler, a CPU type or a GPU driver has no freedom to change a result. Tests check these conditions. Tests cannot prove that the code obeys them everywhere.

• The model file is the same on every machine. The Mac and Linux make the same gpt2.hsm, byte for byte. Its SHA-256 is aaa03eb96e9ad7957432168c8ac6b4a99ed55d57e637e4786a664b51001ba5c2.

• A wrong result is never silent. Each .taco file stores the SHA-256 of the original data, and taco unfold checks it.

Important

The checksum finds an error. It cannot repair one. If taco unfold fails, try again with --backend cpu or on a different machine. If the error happened during taco fold (a bug or a hardware fault on that machine), no decoder can restore the file. Keep the original until you have unfolded the .taco file one time.

These problems can still occur, as in every program: a bug in our code, a bug in a GPU driver (Taco runs a self-test before it uses a GPU, and uses the next backend if the test fails), or a hardware fault. In each case the SHA-256 check finds the error.

Hardware

Not tested yet: Intel GPUs; AMD GPUs on Linux; AMD discrete GPUs; macOS on Intel; big-endian and 32-bit CPUs; phones and browsers.

GPU notes:

• AMD and Intel GPUs need no special code. They use wgpu and the normal GPU driver.
• The CUDA backend needs compute capability 8.0 or more (RTX 30 series, A100 and later). Older NVIDIA GPUs use wgpu or the CPU. The bits are the same; only the speed changes.
• On Windows, DX12 with the old FXC shader compiler cannot compile 7 of the 14 kernels. The self-test then fails, and Taco uses the next backend. Vulkan, Metal and DX12 with DXC work.

Limits

How it works

A symbol that the model expects with probability p costs about −log₂(p) bits. For each position, the model gives a probability to each of the 50,257 GPT-2 tokens. The range coder divides a number range in proportion to these probabilities. The decoder runs the same model, divides the range in the same way, and finds the token.

The decoder must calculate exactly the same probabilities as the encoder. If one frequency is different by one unit, the decoder can select a wrong token, and all output after it is wrong. Floating-point results can change with the order of additions, the chip, the math library and the batch size. Specified integer results do not change. Thus HardShell uses i8 weights, i32 and i64 sums, one rounding rule, and tables for GELU and exp that are stored in the model file.

flowchart LR
  A[input bytes] --> B[blocks]
  B --> C[tokenizer<br/>byte-level BPE]
  C --> D[HardShell<br/>integer GPT-2]
  D --> E[frequency table]
  E --> F[range coder]
  F --> G[.taco file<br/>model hash + SHA-256]
  G -. unfold: same model,<br/>same table .-> H[original bytes<br/>checksum verified]

• Blocks are independent, so they run in parallel, and damage stays in one block.
• A block that does not become smaller is stored zstd or raw. The worst-case growth is the 104-byte header and 5 bytes for each block.
• The profile logs adds a match model and a mixer for repeated lines. The profile auto (the default) selects the smaller result for each block.
• The file format is in ADR 0006 and ADR 0011. The model file format is in ADR 0004.

The full explanation of every part, with the measurements and the reasons for each decision, is in [docs/core-architecture.md