A long result does not cost you once, it costs you forever

Text that comes back inline stays in the conversation and is re-sent on every turn after it, including the turns about something else.

The problem

A tool returns forty thousand characters and your agent reads them. The call took two seconds and it looked free. It was not: those characters are now part of the conversation, they go back up on every following turn, and they will still be there when the topic has moved on.

That is the difference between a cost and a tax. The answer about the document is fine. The answer three turns later, about something unrelated, is worse, because the model is reasoning with a contract it no longer needs sitting between it and the question. An eighty-page document returned inline is roughly twenty thousand tokens of permanent furniture.

There is no error for this and no line in a bill. The failure mode is a conversation that gets vaguer as it goes on, which everybody blames on the model.

Why it fails locally

Any tool that returns text returns all of the text, and it is not being stupid: it has no idea which part you wanted. Neither do you yet, which is the whole reason you called it. So the tool hands over everything and the decision about what matters happens after the context has already been spent.

The workarounds each break something. Truncating answers questions about the beginning and lies about the rest. Chunking and summarising costs a model call per chunk and leaves you a lossy copy you cannot search: the clause you needed was in the sentence that got compressed. Writing the text to a file and reading it back moves the problem into a second tool that has the same flaw.

The detail almost nobody handles is that a tool response has two channels. The text block is what the model reads; the structured fields are what the client code reads. They arrive together, in the same response, and anything present in both is paid for twice. A response that helpfully repeats the section titles in the structured output and again in the prose has just doubled the part that was supposed to be the cheap summary.

The code

Above an explicit, configurable threshold the response is a preview, a map and a handle. text is simply absent, truncated says so, and tokens_if_inlined tells you what you did not spend. Every result call after that is free.

read(key="<key>")
  -> handle, engine, cost_eur, structure,
     chars: 152340, tokens_if_inlined: 38085,
     truncated: true          # no "text" field at all

result(handle="<handle>", outline=true)
  -> outline, structure_source   # titles and pages, no body

result(handle="<handle>", find="termination")
  -> matches, match_count, matches_omitted

result(handle="<handle>", pages="12-14")
  -> text, chars_total, next_offset

What it costs

The saving is measured, not claimed: on a forty-page document the response came back at 459 tokens instead of 6,355. That is the same work, the same read at 0.007 EUR per page, with fourteen times less of your context spent on it.

Querying the result costs nothing, however many times you do it: upload, result, cancel, account are free calls. Outline first, search second, then pull the window you need. The work was paid for when it ran, and paging through it is not more work.

The threshold is a setting, not a hunch, and the split is deliberate on both channels: section titles and excerpts go in the text block because the model reads them, counts and offsets go in the structured fields because your code reads them, and neither is repeated in the other.

What this does not do

result will not hand you the entire long result back one window at a time as a matter of routine. If you genuinely need all of it in context, nothing here saves you.
The full result is kept for a limited time. Once it expires the handle stops answering and the file has to be processed again, at the normal price.
pages works on documents and time works on transcripts. Asking for the wrong one is an error, not an empty answer.
tokens_if_inlined is an estimate of what inlining would have cost your context. It is a measure of the result, not a billing figure: you are charged per page or per minute, never per token.

Without markup

This page in markdown: /problems/keep-long-results-out-of-your-context.md. All 20 of them in one file: /llms-full.txt.