Paritok

Paritok

A non-destructive compression gateway for coding agents — cut your input-token bill without changing your agent.

Paritok sits between your agent and the LLM as a drop-in proxy. On every request it strips the tool-schema bloat, compresses tool results and file reads, and summarizes stale history — then forwards upstream, billed on the compressed tokens. Nothing is ever permanently discarded: the agent pulls back any exact original on demand. Works with Claude Code, Cursor, Codex, OpenHands, and any agent that honors BASE_URL — you don't change a line of your agent.

Powered by the first open-source 4B compression model trained specifically for coding agents (45K real agent trajectories). Cut input token bills from ~25% on turn one to past 85% in long, context-saturated sessions, and fit ~3× more turns in the same context window.

What it does · The three levers · Compounding savings · Cost · Pricing · Quick Start · The engine (model) · Team

📢 News

• 2026-08-17   v1.3.8 — a visual /stats dashboard (live token savings + per-request original→compressed before/after, in place of the raw JSON) and the new Paritok VS Code extension.
• 2026-07-31   v1.3.0 — stability release: edit-recovery, read_original API rename (from expand_context).
• 2026-07-19   v1.2.0 ships the embedding-based tool filter — the biggest single-turn lever (~29K → ~8K on a typical Claude Code turn), unlocking prompt-cache-friendly tool selection with gateway_search_tools recall.
• 2026-07-15   Paritok gateway v1.0.0 open-sourced — the proxy/middleware that turns the 4B model into a drop-in Claude Code / Cursor / Codex compression layer.
• 2026-07-14   Paritok-4B-v1 released on Hugging Face Hub with full SWE-bench Lite end-to-end evaluation.
• 2026-06-25   Finished training. 45K teacher-distilled samples on the Qwen3-4B backbone.

🚪 What Paritok does

Paritok runs as a middle layer between your agent and the LLM API — your agent points at Paritok instead of Anthropic/OpenAI, and everything else stays the same.

Your Agent (Claude Code / Cursor / Codex)
  → builds request (tool schemas + history + tool results / file reads)
     ★ Paritok gateway rewrites the request here ★
  → forwarded to Anthropic / OpenAI  (billed on the compressed tokens)
  ← response flows back unchanged; compressed refs expand on demand

The token bill for a coding agent is dominated by input you re-send every turn: dozens of tool schemas, an ever-growing message history, and big file-read / tool-output blocks. Paritok attacks all three — and because it's a non-destructive gateway, anything it compresses or filters is still recoverable on demand. It's lossy on the wire, fully recoverable when it counts.

🎚️ The three levers

Paritok saves tokens through three independent mechanisms. They stack, and they hit different parts of the bill:

1. Tool-schema filter — the biggest single-turn win

Coding agents expose dozens of tools — often 70+ once you add MCP servers — in full JSON schema on every request. Most are irrelevant to the task at hand. Paritok filters them semantically (tool_discovery.strategy: embedding), keeping only the handful relevant to the user's intent in full schema and stubbing the rest.

• This is the largest single-turn lever. On a typical Claude Code turn the tool block alone is ~29K tokens; filtered it drops to ~8K — a saving no amount of file compression matches on a single turn.
• Prompt-cache friendly. The selection is frozen per conversation, so the tools[] block stays byte-stable turn-to-turn and never invalidates the LLM's KV cache.
• Never destructive. Anything filtered is recoverable — the model calls gateway_search_tools and gets the full schema back.
• Runs a small open embedding model (BAAI/bge-small-en-v1.5, MIT, ~130MB) entirely locally on CPU — no API, no per-token fee.
• An agent's core execution tool (shell / exec / apply_patch) is never stubbed, so agents like Codex that expose only a handful of tools always keep the one they can't work without.

2. Content compression — file reads, tool output, history

Each tool_result, file read, and (once the window fills) stale history turn is compressed by the 4B model down to ~26% of its original size, tagged [REF:id]. This is where the trained model earns its keep: it knows a function signature from a debug line, so it protects identifiers, paths, and error strings while dropping the noise.

• Single-turn this is the smaller lever (a few % — most of a turn's cost is the fixed prefix). Its power shows up across a session — see below.
• Every compressed segment is recoverable: the agent calls read_original / expand_context to pull back the exact untouched bytes, locally and instantly.

3. History summarization — keep long sessions under the window

Turns beyond the recent window are summarized once the context fills up, so a long session stays inside the model's context window instead of overflowing (or forcing an aggressive client-side compaction that drops detail).

📈 Savings compound over a session

This is the part a single-turn benchmark hides. In a real multi-turn session the two levers grow at different rates, and together they compound.

We ran the same read-only "find the bug" task for 5 consecutive turns in one Claude Code session (Sonnet, GPU model). Important: in this A/B the tool filter was left on for both sides, so the delta below isolates content compression alone — it does not include the tool-schema saving.

Content compression only (tool filter on both sides):

But that's only one of the three levers. The real "do I use Paritok or not?" comparison must add the tool filter back — and without Paritok the agent sends the entire ~29K tool-schema block every turn, not the filtered ~8K. Folding that ~21K/turn back onto the no-Paritok side:

Full stack (filter + compression vs. no Paritok at all):

So the 4.6% / 22.2% above is the floor (content only). Against a real no-Paritok baseline it's ~25% on turn 1, past ~39% by turn 5 — because the tool filter saves a fixed ~21K every turn on top of the compounding content compression.

Why it grows: every file you read stays in history and is re-sent (cache-read) every subsequent turn — so content compression keeps paying off turn after turn, while the tool filter adds a fixed cut on top.

• Content compression → quadratic. Cumulative saving ≈ 3,350 × N² — each turn's compressed reads keep paying off on every later turn.
• Tool filter → linear. Cumulative saving ≈ 21,000 × N — a fixed block saved every turn.
• Crossover ≈ turn 6: early on the tool filter dominates; past ~turn 6 content compression overtakes it and the gap widens.

Plugging these formulas into a range of N (baseline ~96,500 tokens/turn), capped at a ~200K context budget (typical Sonnet-tier configuration; larger-context models like Opus 1M push the flatten point out proportionally):

_{Capped at a ~200K context budget. The quadratic only holds while history is still growing. Once the accumulated context fills the configured budget (around turn ~8–12 here on Sonnet-tier ~200K), client-side compaction holds it flat, per-turn content saving stops growing (freezes at ~48K/turn), and % saved plateaus toward the ~72% default ceiling instead of diverging. Turns 1–5 match the measured tables above; later rows are in-window projections.}

Ceilings shift with deployment shape:

_{The projection above uses the default ceiling. MCP-heavy setups earn a bigger per-turn tool-filter cut; context-saturated sessions get an even larger effective saving because they're now compared against a compacted, information-lossy baseline.}

Honest cap: the quadratic doesn't run forever — whatever session context budget you configure bounds it. In practice a ~200K budget (typical for Sonnet-tier deployments) makes the curve flatten around turn ~12–20 as the window fills; larger-context models like Opus 1M push the flatten point out proportionally. But that ceiling is itself a feature: because each turn's prefix is smaller, the agent fits more turns before hitting the budget — Paritok effectively buys back context length.

Same budget, more room to think. Because each turn's prefix is smaller, the agent fits far more turns in the same window before compaction kicks in.

That's roughly 3× the runway on any budget, giving the agent more turns to think and finish the task before hitting the wall.

One line: use more, save more. Compression frees up the window and lets the agent go deeper and longer in the same session. Strongest on long, multi-turn, read-heavy work (auditing, Q&A over a big codebase, long debugging sessions).

💰 Cost Impact

The 74% figure is Paritok's content compression rate — file reads, tool output, and history shrink to ~26% of their size. End-to-end savings compound with session length.

Per-session cost at Claude Sonnet input pricing ($3 / M input tokens):

Annual savings by team size (120 turns/dev/day, ~54% average end-to-end):

_{Default ~40-tool config on Claude Sonnet. MCP-heavy setups (70+ tools) push savings to ~78%; context-saturated sessions past 85%. Use the interactive calculator at paritok.com for your specific team size, LLM, and workload — it plugs in Claude Opus ($5/M), GPT-5 ($10/M), Haiku ($1/M), Gemini, or any provider you use.}

Install and running in minutes on your own hardware. No vendor lock-in.

💵 Pricing

Self-host (Apache 2.0): Free forever. The gateway, the 4B model, and every training script ship open. Run on your own hardware. No telemetry, no key, no fees.

Hosted GPU (paritok.com): $0.30 per 1M tokens processed. No GPU required. Free through Oct. 1 as we launch, with full pricing kicking in October 1st.

🚀 Quick Start

Fastest path (self-host, no clone needed)

Everything ships in the PyPI package — you do not need to git clone the repo. In a fresh environment (with Ollama installed):

pip install "paritok[proxy]"     # the gateway + CLI
paritok up                       # pulls the model if missing, then starts the proxy

paritok up checks Ollama, ollama pulls paritok/paritok-4b-v1 and tags it as the local paritok-4b-v1 if it isn't already there (~2.5GB, first run only), then serves on port 8080.

Leave that terminal running. The proxy is a foreground server — every agent request flows through it, so it must stay up for the whole session. Open a separate terminal for the next step.

In the shell that launches your agent:

export ANTHROPIC_BASE_URL=http://127.0.0.1:8080   # then start Claude Code / Cursor / …

No config file is needed — paritok up and paritok proxy run on built-in defaults. Run paritok init to drop a starter paritok.yaml only if you want to tweak settings.

q4 vs full precision. paritok up uses the q4 model (:latest, ~2.5GB) by default. For full precision, run paritok up --registry-model paritok/paritok-4b-v1:f16 (~8GB). If you already ollama pulled either variant, up auto-detects which one you have — no re-download.

1. Install

pip install "paritok[proxy]"
# or, from a clone of this repo:
pip install -e ".[proxy]"

Enable the tool-schema filter (recommended — it's the biggest single-turn lever):

pip install "paritok[toolselect]"   # adds sentence-transformers (CPU-only)

First request warms up the embedding model (~10–15s — downloads bge-small once, then caches locally). Every request after is instant (~15 ms).

2. Pick a backend — self-host or the GPU server

One boolean in paritok.yaml decides where compression runs:

use_gpu_server: false   # ← the only switch that matters

Option A — self-host (false, default). Run the open 4B model on your own machine. No key, nothing leaves your box. Simplest is Ollama:

ollama pull paritok/paritok-4b-v1                # one-time, ~2.5GB
ollama cp   paritok/paritok-4b-v1 paritok-4b-v1  # tag it as the runtime name

Default paritok.yaml already points local_model at Ollama — nothing else to set.

pip install vllm
vllm serve Qwen/Qwen3-4B-Instruct-2507 \
  --enable-lora \
  --lora-modules paritok-4b-v1=paritok/paritok-4b-v1 \
  --max-lora-rank 32 \
  --port 8000

Option B — Paritok GPU server (true). No GPU required. Create an API key at paritok.com → dashboard → API keys, then:

use_gpu_server: true
gpu_server:
  api_key: "pk_live_..."   # or: export PARITOK_API_KEY=pk_live_...

3. Start the proxy

paritok proxy --port 8080 --config-file paritok.yaml

Keep this terminal open — the proxy must stay running for the whole session; run your agent from a separate terminal. On startup it checks the backend and warns (never aborts) if it can't reach one.

4. Point your agent at it

Set the base URL in the shell that launches your agent, then start the agent:

# macOS / Linux
export ANTHROPIC_BASE_URL=http://127.0.0.1:8080   # Claude Code
export OPENAI_BASE_URL=http://127.0.0.1:8080      # Cursor / OpenAI-SDK agents

# Windows PowerShell
$env:ANTHROPIC_BASE_URL = "http://127.0.0.1:8080"
$env:OPENAI_BASE_URL    = "http://127.0.0.1:8080"

Keep your real provider API key set as usual (ANTHROPIC_API_KEY / OPENAI_API_KEY) — the proxy only rewrites the request body and forwards your headers upstream. Compressed prompts go upstream; original responses come back unchanged.

Codex is the exception — it does not read OPENAI_BASE_URL. The Codex CLI takes its endpoint from ~/.codex/config.toml. See Codex setup.

Any OpenAI-compatible upstream (Groq, Gemini, OpenRouter, …)

The gateway is independent of the upstream LLM — point the OpenAI path at any OpenAI-Chat-Completions-compatible endpoint with --openai-url:

paritok proxy --openai-url https://api.groq.com/openai        # Groq
paritok proxy --openai-url https://openrouter.ai/api/v1       # OpenRouter
paritok proxy --openai-url https://api.orcarouter.ai/v1       # OrcaRouter

Then point your OpenAI-SDK agent at the proxy (OPENAI_BASE_URL=http://127.0.0.1:8080), set OPENAI_API_KEY to the provider's key, and use that provider's model names.

Codex CLI

Codex ignores OPENAI_BASE_URL — it only reads its endpoint from ~/.codex/config.toml, so Paritok writes that file for you:

codex:
  enabled: true            # paritok writes ~/.codex/config.toml on `paritok up`
  model: gpt-5