Freight/Signal Subscribe
Issue 2 · AI in Freight ·

What's actually under the hood of an AI dispatch agent

An AI dispatch agent is an LLM in a tool-use loop — the model orchestrates, the tools compute. The anatomy, the real cost, what breaks, and the 5 questions to ask before you sign.

The companion piece to Freight/Signal Issue #2. Issue #2 ships short — ≤1,000 words in the email. The Deep Dive’s long form lives here.

Every freight software booth this year sells the same promise: an AI dispatcher that books loads, negotiates rates, and runs your desk while you sleep. The pitch lands because the agents are real — C.H. Robinson now says it runs more than 30 AI agents that have executed over 3 million tasks. This is not vaporware. But the gap between deployed and autonomous is where operators get sold, and the only way to see it clearly is to open the thing up.

So here’s the teardown.

The anatomy: it’s a loop, not a brain

An AI dispatch agent is an LLM in a tool-use loop. Strip away the branding and the flow is always the same four steps:

  1. A dispatcher asks something in plain English — “What’s my best route LA to Dallas, and where do I fuel?”
  2. The model reads the request and decides which tools to call: a routing tool, a fuel-price lookup, a rate lookup.
  3. Those tools run and return hard data.
  4. The model reads the results and writes the answer in plain English. If it needs more, it loops — but a well-built agent caps the loop (typically three rounds) so it can’t spin.

The thing to internalize: the model orchestrates and writes. The tools compute. The miles come from a routing API. The fuel price comes from a database query. The load margin is rate − fuel − tolls — third-grade arithmetic, not machine intelligence. The LLM’s actual job is translation: turning a messy human question into the right sequence of tool calls, then turning the tool output back into a sentence a busy dispatcher can act on.

This isn’t a knock. Anthropic’s own engineering guidance describes agents the same way — “LLMs using tools based on environmental feedback in a loop.” Collapsing five lookups into one plain-English question is a genuine efficiency win. But “translates and orchestrates” is a very different product than “dispatches autonomously,” and the price tag rarely reflects which one you’re buying.

The AI is the cheapest part of the AI dispatcher. You’re paying for the data and the markup.

The real cost: the model is cents; the data is the bill

Here’s the part that reframes every per-truck quote you’ll see.

The model layer is nearly free. Current API pricing puts the practical “agent brain” tiers at Claude Haiku 4.5 ($1 per million input tokens, $5 output), Gemini 2.5 Flash ($0.30 / $2.50), or GPT-5.4-nano ($0.20 / $1.25). A full multi-tool dispatch conversation might burn 15–30k tokens. Turn on prompt caching for the static system instructions and tool definitions — a 90% discount on the cached portion — and a conversation costs single-digit cents. Even on a flagship model, you’re talking dimes.

The routing layer is cheap and public. Google’s Maps Routes API is about $5 per 1,000 route calculations (and drops sharply at volume); there’s a free, self-hostable open-source alternative if you don’t need tolls.

The freight data layer is where the money goes. Live rate data — DAT RateView, Truckstop — is gated and contract-priced, with no public rate card. That’s the expensive, defensible input. So when a vendor charges you $X per truck per month, the cost stack underneath is roughly: a few cents of model, a few cents of routing, a real line item for rate data, and the rest is margin.

That’s not a scandal — vendors deserve margin. But it means the right question on a sales call isn’t “how good is your AI.” It’s “what’s the split between model cost, data cost, and markup,” because the answer tells you what you’re actually renting.

What breaks: six failure modes from building one

The demo always works. Production is where you learn what the agent does when reality doesn’t cooperate.

  1. Hallucinated estimates. This is the big one. The moment you let the model answer “how many miles” from its own head instead of forcing a routing-tool call, it invents a plausible, confident, wrong number. The single most important instruction in a production agent is a hard rule: never estimate miles — call the tool. Every other safeguard is downstream of that one.
  2. Eloquent garbage. A model narrates bad tool output exactly as smoothly as good output. If your rate feed returns a stale or null value, the agent will confidently quote it. Garbage in, eloquent garbage out — and the fluency makes it harder to catch, not easier.
  3. Concurrency bugs. When the model asks for several tools at once and you run them in parallel for speed, they can’t share one database connection — they corrupt each other’s state. Each parallel call needs its own session. It’s a subtle bug that costs real debugging time to track down.
  4. Stale caches. You cache prices to keep cost and latency down; cache too long and you serve yesterday’s diesel number as today’s. The time-to-live is a real tradeoff with no clean answer.
  5. Latency stacking. Two model passes plus several tool calls add up to seconds, and dispatchers feel every one. Speed is a feature you have to engineer, not a default.
  6. Cost blowout from naive looping. Let the model loop freely over everything and token use multiplies fast — industry figures put a single-agent loop at roughly 4× the tokens of a plain call, and multi-agent setups far higher. The fix is the discipline of pushing computation into deterministic tools and keeping the model’s loop short.

None of these are reasons not to build or buy one. They’re the reasons the honest version costs real engineering, and the reason “autonomous” is a much bigger claim than it sounds.

Build vs. buy

If you’re a small fleet or brokerage doing the math: the raw ingredients are cheap and mostly public — an LLM API key, a routing API, your own data. A capable engineer can wire a useful internal agent in weeks, and run it for tens of dollars a month plus your rate-data subscription. What you’re really buying from a vendor is the integration work, the failure-mode handling above, the gated data relationships, and someone to call when it breaks at 2 a.m.

That’s a legitimate thing to pay for. Just price it against the alternative honestly, and don’t pay autonomous-dispatcher prices for a lookup-collapsing copilot.

The five questions to ask any “AI dispatcher” vendor

Take these to your next demo. The answers separate real from slideware:

  1. When it quotes miles or rates, is it calling an API or letting the model estimate? If they can’t answer cleanly, walk.
  2. What’s the data source for rates, and how fresh — live API or a cache? What’s the TTL?
  3. What happens when a tool fails or returns nothing — does it say “I don’t know,” or make something up?
  4. Show me a wrong answer it gave, and what you changed. No failure stories means it isn’t in real production.
  5. What’s the all-in cost per truck, and what’s the split — model tokens vs. data feeds vs. markup?

A vendor building the real thing will have crisp answers to all five. A vendor selling a slide will get vague around question three.

The verdict: an AI dispatch agent is a real efficiency layer — it saves your dispatchers lookups and turns five tools into one conversation. It is not a replacement for their judgment, and the tools that claim otherwise are selling you the loop with the cap taken off. Buy the copilot. Interrogate the autopilot.

— Aman Singh · Editor · Freight/Signal · aman@freightsignal.ai · 2026-05-26

Issue #2 of Freight/Signal · back to all issues · subscribe to the Tuesday newsletter

Sources