--- title: Why Your Coding Agents Burn Tokens in the Dark description: >- We handed our terminals over to autonomous agents, only to discover our token bills were compounding in silence. The architecture behind Thermal, a local-first FinOps dashboard built in Go. summary: >- We handed our terminals over to autonomous coding agents, only to discover our token bills were compounding in silence. Here is how I built Thermal to track 14 agent harnesses with local SQLite mmap, sub-10ms delta caching, and a Bubble Tea TUI. date: 2026-09-26T00:00:00.000Z heroImage: src: /_astro/hero.CpY8-lvh.webp width: 1024 height: 1024 format: webp tags: - thermal - finops - ai-agents - golang - tui - local-first - sqlite categories: - Project Pilot - The Core Dump published: true featured: true draft: false author: Jad href: /posts/why-your-coding-agents-burn-tokens-in-the-dark/ slug: why-your-coding-agents-burn-tokens-in-the-dark --- A few months ago, around two in the morning, I was sitting in front of my terminal watching three separate tmux panes hum with activity. In one window, Devin was refactoring an AST parser. In another, OpenCode was executing a test-driven loop against a flaky API endpoint. In a third, an experimental agent was indexing symbol references across a dozen repositories. From the outside, it looked like the ultimate dream of modern software development. The screen was a cascading waterfall of syntax highlights, bash commands, unified diff blocks, and passing test assertions. The manual friction of writing boilerplate had vanished into thin air. Then the monthly API invoices arrived. The numbers were not ruinous, but they were deeply unsettling. Not because of the absolute dollar figure, but because I had no honest way of answering the most fundamental engineering question: *what did each of those dollars actually purchase?* When you work alongside human engineers, you have an intuitive grasp of where time and energy go. You know whether an architectural refactor took three hours or three days. But with autonomous coding agents, token consumption happens entirely in the dark. As I learned firsthand when [evaluating coding agents on real tasks](/posts/power-of-jules/), fluency is not the same as discipline. An agent gets trapped in an internal linting dispute, consumes six hundred thousand tokens quietly re-reading the entire project schema on every single turn, and reports back with a single-line patch. You pay for forty-seven model invocations, and you have no idea which prompt triggered the avalanche. We had traded developer ergonomics for financial blindness. And the prescribed solutions (piping private codebases and session transcripts into yet another closed, cloud-hosted SaaS monitoring platform) felt like an insult to developer privacy and software craftsmanship. So I decided to build [Thermal](/project/thermal/). --- ## The Compounding Geometry of Agent Spend Most developers still calculate AI costs using a simple calculator mental model: you send a prompt of five hundred tokens, get back a thousand tokens, and pay pennies. That model dies the moment you deploy autonomous agents. In an agentic loop, token expenditure does not scale linearly with output code. It scales exponentially with *execution uncertainty*. As I explored when breaking down Princeton's research on [how agent skills stabilize execution](/posts/demystifying-agent-skills/), an agent does not merely answer questions; it observes, reflects, probes tool outputs, and adjusts its internal plan. ```mermaid flowchart TD subgraph StepLoop["Compounding Token Footprint"] A["User Task: 'Fix flaky test'"] --> B["Step 1: Read Workspace Tree (45k tokens)"] B --> C["Step 2: Run Test Runner (12k tokens output)"] C --> D["Step 3: Reread File + Test Traces (65k tokens)"] D --> E["Step 4: Attempt Speculative Edit (15k tokens)"] E --> F["Step 5: Rerun Tests & Self-Correct (78k tokens)"] end F --> G["Total Billed: ~215,000 Tokens for a 2-Line Fix"] ``` Consider what happens inside a single agent session: 1. **Context Accumulation:** On step one, the system prompt, tool schemas, and repository summary consume thirty thousand tokens. 2. **Execution Traces:** By step five, the agent has executed three failing shell commands. The terminal stdout and stderr now live inside the message history. 3. **Repeated State Inspection:** To decide what to do on step six, the model must re-read the entire accumulated dialogue from scratch. 4. **Subagent Spawning:** When the primary agent spawns a subagent, a fresh context window opens, often inheriting another slice of the codebase. 5. **Hidden Reasoning Tokens:** Frontier reasoning models (such as o3-mini or DeepSeek-R1) generate thousands of hidden thinking tokens per turn to deliberate before emitting a single line of actionable code. A two-line patch in a test runner can easily burn three hundred thousand tokens across eight round-trips. If that agent runs on a frontier model, an afternoon of aggressive tinkering can quietly cost twenty or thirty dollars. Multiply that across five tmux windows, seven days a week, and three different agent CLIs, and you are no longer doing software development. You are running an unmonitored compute furnace. --- ## Why SaaS Dashboards Miss the Point The standard Silicon Valley answer to this problem is predictable: sign up for a monitoring service, install their npm package, configure an HTTP reverse proxy (`HTTP_PROXY`), and stream every prompt, file path, and agent transcript to their cloud. I refuse to do that. My local code is my own. The architecture decisions, private client keys, internal database schemas, and scratch notes that pass through my agent sessions have no business living on a third-party server. Furthermore, when you work far from central US data centers, adding an external telemetry proxy injects a punishing latency tax into every single turn, a reality I wrote about in [Latency Is Political When You Live Far from the Center](/posts/latency-is-political-when-you-live-far-from-the-center/). It violates the offline-first resilience of Unix tools, breaks when tools enforce custom certificate verification, and creates yet another recurring subscription. When I started digging into how agent harnesses work on disk, I noticed something fascinating: *the telemetry was already there.* Every modern coding agent leaves a rich local trail on your filesystem: - **Claude Code** streams JSONL event logs under `~/.claude/projects/`. - **Codex** maintains `state_5.sqlite` alongside rollout JSONL logs. - **OpenCode / MiMoCode** records session graphs in `~/.opencode/` SQLite databases. - **Grok** logs completed turns in `updates.jsonl`. - **Devin, Gemini CLI / Antigravity, Nous Hermes, and DeepSeek** all maintain local SQLite databases or structured JSON files. The agents were not hiding their costs. They were diligently saving them to disk. We were simply ignoring the files. We did not need another cloud service. We needed a fast, local-first lens. --- ## Building Thermal: Architecture of a Zero-Telemetry FinOps Engine When I sat down to design Thermal, I set three non-negotiable principles for myself: 1. **Zero Cloud Telemetry & Zero Daemons:** Thermal is 100% offline. It never makes an outbound phone-home ping, requires no cloud accounts, and installs no persistent background processes eating battery and RAM. 2. **Sub-10ms Cached Invocations:** In order to fit naturally into interactive developer workflows (even rendering inside a shell prompt like Starship or `.zshrc`), invoking `thermal` must return in single-digit milliseconds. 3. **Strictly Non-Invasive & Read-Only:** Thermal never injects wrappers or modifies user databases. It reads what exists without interfering with active tools. To achieve this, I wrote Thermal in **Go** as a zero-CGO, statically linked binary. ```mermaid flowchart LR subgraph Harnesses["14 Local Agent Harnesses"] H1["Claude Code (~/.claude)"] H2["OpenCode / MiMoCode"] H3["Codex (state_5.sqlite)"] H4["Devin, Grok, Agy, Hermes..."] end subgraph ThermalCore["Thermal Go Core"] D["Parallel Harness Scanner"] --> M["Read-Only SQLite mmap & JSONL Engine"] M --> C["Delta Cache (~/.cache/thermal/)"] C --> P["Disjoint Token Normalization & Pricing"] P --> S["Unified FinOps Aggregator"] end subgraph Presentation["Terminal Output"] S --> TUI["Interactive Bubble Tea TUI"] S --> CLI["Terminal Heatmap & Scriptable JSON"] end Harnesses --> D ``` ### 1. Pure-Go SQLite and Memory-Mapped Zero-Copy I/O Many agent harnesses run on SQLite. If an agent is actively coding while Thermal executes, opening the database with a standard read-write connection causes immediate lock contention (`database is locked`). Thermal opens all SQLite databases in pure read-only URI mode and immediately activates memory mapping via `PRAGMA mmap_size`: ```go // Pure-Go SQLite (modernc.org/sqlite): CGO-free, read-only, zero lock contention db, err := sql.Open("sqlite", "file:"+dbPath+"?mode=ro") if err != nil { return err } defer db.Close() // Memory-map 256MB of page cache for instant scanning without heap allocation if _, err := db.Exec("PRAGMA mmap_size=268435456"); err != nil { return err } ``` Because Thermal uses [`modernc.org/sqlite`](https://pkg.go.dev/modernc.org/sqlite) compiled with `CGO_ENABLED=0`, the binary remains fully statically linked. It compiles across Linux, macOS, and Windows without requiring GCC toolchains or dynamic C libraries. ### 2. The Sub-10ms Delta Snapshot Cache Scanning gigabytes of raw transcripts on every shell invocation is an anti-pattern. To guarantee sub-10ms response times, Thermal employs an incremental disk-backed delta snapshot cache in `~/.cache/thermal//`. On startup, Thermal verifies the target file's modification timestamp (`mod_time`), byte size (`size`), or maximum transaction ID. If the file has not changed, Thermal serves the normalized daily aggregation instantly from cache. If new turns have occurred, it reads only the unseen byte delta, appends the new rows, and updates the snapshot. Cold scans of thousands of sessions take hundreds of milliseconds; warm invocations take **4 to 7 milliseconds**. ### 3. The 32 MiB JSONL Line Ceiling Go's standard library `bufio.Scanner` defaults to a maximum token buffer of 64KB. When reading real agent transcripts from Claude Code or Codex rollout logs, single lines containing whole-file reads or huge git diffs easily reach 2MB to 10MB. A naive scanner crashes immediately with `bufio.Scanner: token too long`. Thermal implements a specialized streaming scanner constructor (`newJSONLScanner`) that configures an explicit **32 MiB buffer ceiling** with bounded worker pools capped at the machine's CPU cores, guaranteeing that large agent payloads never abort scans mid-stream. ### 4. Disjoint Token Arithmetic: Defeating Double-Counting A major pitfall in building multi-agent telemetry is that AI providers disagree on token categorization: - **Codex** and **Grok** nest `Reasoning` tokens inside `Output`, and `CacheRead` tokens inside `Input`. - **ZCode** nests `CacheRead` inside `Input`. - **OpenCode** and **Claude** separate them. If you write a naive `jq` or Python script, you end up double-counting reasoning and cache reads, producing wildly inflated token numbers. Thermal enforces **Disjoint Token Arithmetic** directly inside its loaders: nested tokens are subtracted at ingestion so that: $$\text{Total Tokens} = \text{Input} + \text{Output} + \text{Reasoning} + \text{CacheRead} + \text{CacheWrite}$$ Renderers, financial aggregators, and pricing engines never have to perform defensive arithmetic adjustments. ### 5. Repository Attribution via Symlink-Aware Git Walking Developers rarely run agents from repository roots. You run an agent from `src/components/`, `pkg/api/`, or a symlinked workspace. If a tool tracks paths naively, your project list fragments into dozens of isolated subfolders. Thermal routes every working directory through `ProjectKey`: it resolves filesystem symlinks first (preventing macOS `/var` vs `/private/var` duplication), then walks upwards through parent directories until it finds a `.git` root. All deep subdirectories fold cleanly into their canonical repository name without inventing synthetic hashes. ### 6. FinOps Pricing via the models.dev Catalog Tokens are an engineering metric; dollars are an operational reality. Some tools (like OpenCode or Grok) record their exact expenditure directly to disk. Others (like Claude Code or Devin) record raw token counts without dollar figures. Thermal bridges this gap by prioritizing recorded costs when present, and falling back to an offline-cached pricing catalog from [`models.dev`](https://models.dev) (`~/.cache/thermal/models-dev-catalog.json`). It calculates granular rates across input, output, cache-read, and cache-write tokens, stripping runtime variant suffixes (`-high`, `-mini`) to match base tiers while strictly avoiding fuzzy regex matching. Crucially, when tokens cannot be attributed to a recognized model, Thermal prices them at `$0.00` and discloses them honestly in the footer as unattributed tokens, rather than assuming unpriced compute is free. --- ## The Terminal TUI Experience I spend my workday in `tmux` and `alacritty`. I did not want to leave the terminal to inspect a browser tab or log into a billing portal. Using Charm's **Bubble Tea** and **Lip Gloss**, Thermal categorizes agents into two distinct archetypes: **Token Warriors** (harnesses that report full token telemetry) and **Activity Hunters** (harnesses that measure turns, prompts, and tool steps). ```text THERMAL: Don't break the streak. ╭─ Token Warriors ────────────────────────────────── tokens & spend ─╮ │ # Tool Strk Best Days Tokens Cost │ │ ───────────────────────────────────────────────────────────────── │ │ 1. Devin 2d 33d 45d 19.2B tok ~$20134.07 │ │ 2. Codex 2d 7d 48d 1.0B tok ~$527.11 │ │ 3. OpenCode 1d 11d 30d 10.6B tok $62.91 │ │ 4. MiMoCode 1d 7d 16d 1.2B tok $56.14 │ │ 5. Claude 1d 1d 3d 55.5K tok ~$0.16 │ ╰────────────────────────────────────────────────────────────────────╯ ╭─ Activity Hunters ─────────────────── messages & steps ─╮ │ # Tool Strk Best Days Activity │ │ ─────────────────────────────────────────────────────── │ │ 1. Agy 46d 46d 56d 153.5K step │ │ 2. command-code 3d 9d 48d 10.9K msg │ ╰─────────────────────────────────────────────────────────╯ >> Agy is on fire with a 46-day streak! ``` Running `thermal ` renders a full 52-week ANSI contribution heatmap of daily activity, prompt cache hit ratios, and streak momentum. Subcommands provide immediate period rollups (`daily`, `weekly`, `monthly`), git-level attribution (`projects`), and model distribution audits (`models`). --- ## What 14 Agent Harnesses Taught Me Supporting 14 different tools (Devin, OpenCode, MiMoCode, Codex, Claude Code, Antigravity, command-code, codewhale, Droid, DeepSeek, Grok CLI, Muse, Nous Hermes, and ZCode) uncovered three inescapable truths about modern AI engineering: ### 1. Nobody Uses Just One Tool Developers pick Claude Code for architectural refactoring, OpenCode for deterministic multi-step pipelines, and fast CLI harnesses for quick script generation. When your tools are fragmented, your mental model of expenditure is nonexistent. You assume you spent five dollars, but across four tools, you quietly spent fifty. ### 2. Prompt Cache Hit Rates Are the Ultimate FinOps Lever When an agent harness properly structures its context to take advantage of prompt caching, input token costs drop by up to 90%. Watching your effective cache read rate climb from 25% to 85% in Thermal makes the economic difference tangible: it is the difference between a $0.03 turn and a $0.30 turn on the same model. ### 3. Visibility Directly Restores Intentionality The moment you can see your daily token burn rate rendered as an honest terminal heatmap, your habits immediately improve. You stop letting agents loop aimlessly on ambiguous instructions. You write tighter specifications, provide targeted test constraints, and develop an instinct for when to delegate and when to intervene. --- ## Software Sovereignty in the Age of Agents There is a broader philosophical principle at stake here. In the 1990s and 2000s, Unix taught us that you own your machine. You inspect the running processes with `ps`, monitor memory with `top`, read the system logs in `/var/log`, and maintain total sovereignty over what executes. The early AI era attempted to erode that discipline. As I argued in [the open-source AI rebellion](/posts/open-source-ai-rebellion/), we are encouraged to treat foundational models as opaque cloud magic, to stream our intellectual property into proprietary vendor proxies, and to accept whatever invoice arrives at the end of the month without question. I built Thermal because I believe we can embrace the extraordinary power of autonomous coding agents without surrendering our autonomy as engineers. I keep thinking back to that night at two in the morning, watching the terminal panes scroll in the quiet of my apartment. There is something exhilarating about watching code emerge from an agent's reasoning loop. But engineering has never been about surrendering judgment to the machine; it is about understanding the cost, the structure, and the consequences of what we put into the world. When you can see the numbers clearly, the spell breaks, and the agency returns to your hands. And in an era where software wants to think for you, keeping your eyes wide open might be the most rebellious thing you can do. How do you keep track of what your autonomous agents are actually doing? Are you measuring your token burn locally, or are you still waiting for the monthly invoice to surprise you? --- *Thermal is open source and available as a standalone Go binary. If you want to inspect the architecture, run it against your own local agent history, or read the source, you can find it at [jadmadi.net/project/thermal/](/project/thermal/).*