Finding a run

Search the payload, not the label

Most tracing tools index names and attributes. mcpd indexes the prompt and the completion as well, because what a person complains about is something the model said, and the words they quote back to you are the only handle you have. The q parameter is that index.

What q covers, exactly

The index holds three fields per span: its name, its input and its output. Input and output are JSON, so before indexing mcpd walks the payload and keeps only its strings and numbers, discarding the braces, the quotes and the keys. Indexing the raw JSON would make every punctuation mark a token and bury the words you are actually looking for.

Six consequences are worth knowing before you rely on it:

• A run matches when any single span in it matches. You get whole runs back; open one to see which step it was.
• Words are combined with AND. q=refund delayed finds spans containing both, anywhere in the indexed text, not next to each other.
• Matching is whole-word and case-insensitive, and diacritics are folded, so resume finds résumé. There is no stemming: refund does not find refunds.
• The first 64 KiB of each of input and output goes into the index. Longer payloads are stored and displayed whole; only the tail of a very large prompt is unsearchable.
• Redaction runs before storage, so anything the redactor removed was never indexed either. See Redaction and retention.
• A span written more than once, as a long-running span is when it ends, replaces its index row, so search always reflects the current version.

A stray quote is a search, not an error

The index underneath has its own query language: a double quote opens a phrase, and NEAR, OR and * are operators. Handing it your text unchanged would mean a customer message containing a quotation mark comes back as a syntax error instead of a result, and a search box that can throw is a search box people stop using.

So your text never reaches that parser. mcpd splits what you typed on whitespace, strips any double quote out of each word, wraps each word as a literal term and joins them with AND. q=say "no" searches for the words say and no. q=NEAR searches for the word NEAR. Nothing you can type is a syntax error, and no operators are available to you. That is the trade, made in that direction on purpose.

A q of nothing but whitespace is dropped rather than matching nothing, and an empty q= is treated as absent. Every parameter behaves that way, so a form that submits its empty filter boxes does not accidentally filter on the empty string.

Narrowing a run list

GET /v1/api/traces takes the following. Anything you leave out is not applied.

name, the correlation ids and environment are equality tests, which is what makes them cheap. If you want a partial match on a name, use q: span names are in the text index too.

What a list row costs

There is no second, lighter run list, because the first one is already light. The stored trace projection carries no input, no output and no attributes at all: names, counts, timings, cost, tags and correlation, and nothing a model said. Opening a run is what fetches the payload.

Measured on one realistic run, an eight-step support agent that retrieves three policy documents, calls two tools and makes two model calls with real prompts and completions: the list row is 577 bytes and the whole run is 10,717 bytes, a 94.6% reduction. That is this daemon, on that run, over the real HTTP surface. Your runs will differ, and the direction is what matters: the saving grows with the size of the prompts, because the part that grows is the part the list does not carry.

The figure is produced by a test rather than typed here, so it cannot go stale in one place without failing in the other. Run cargo test -p mcpd-server --test payload_size -- --nocapture and it prints both byte counts. A field added to the projection that carries payload fails that test.

Anything else is refused, by name

That table is the whole vocabulary. A parameter that is not in it is refused with a 400 naming it, rather than accepted and dropped. This changed: mcpd used to ignore a parameter it did not know, so ?statuss=error came back 200 with every run in the project and nothing about the response said the filter had not applied. A plausible wrong answer is worse than an error, because there is nothing to notice.

The trade this makes is worth naming. A client that sends a filter a newer daemon understands now gets a 400 from an older one instead of a wider result set. That is the better of the two failures: the 400 names the parameter and stops.

Some refusals say more than "unknown". mcpd records service_name, service_version, resource_id, run_id, request_id, experiment_id and release on every trace, but the run list cannot filter on them yet. Of the seven, only service_name is a discovery facet, so discovery lists its values and none of the others'. Those seven are refused with a message that says so, because service_name=checkout-api is not a typo and telling somebody it is an unknown parameter sends them hunting a spelling mistake that is not there.

The rule is not the run list's alone. GET /v1/api/spans, /v1/api/logs, /v1/api/scores, /v1/api/metrics and /v1/api/feedback each enforce their own vocabulary the same way. Two are worth knowing before you meet them: the score list takes no order, because it is newest first and its cursor compares one way only, and the feedback list requires a trace_id, because mcpd cannot yet list a whole project's feedback and an empty list would read as though there were none.

Filling a filter box

A filter is only usable if you know what to type into it, and the values are not a fixed list: they are whatever the services reporting to a project happen to send. GET /v1/api/discovery answers that. Ask for one dimension at a time with facet, and it returns the values that dimension has taken in the project, ordered by value, each with when a signal last carried it.

prefix narrows the list for typeahead, and it matches literally and case sensitively, because these values are what a filter will be compared against and folding here would offer one that then matches nothing. limit caps the answer at 1000. Values are ordered by value and not by recency, so a list somebody is reading does not reshuffle under them between polls.

One limit is worth knowing before you build against it. metric_label_key lists every label key seen anywhere in the project, not the keys of one named metric: the distinct-value list is keyed by the label, not by the metric that carried it.

Refreshing a list without running it again

GET /v1/api/deltas returns what has arrived since a cursor, oldest first, for traces, spans or logs. Hold the cursor from the last poll, send it back, and apply the rows to the list you already hold.

A poll is not a page, and the two cursors are deliberately not interchangeable. Paging walks a fixed result set backwards in the sort you chose; polling walks forwards in arrival order and never ends. The watermark is a row's arrival time and its id together, because a batch writes every row in it with one timestamp and a watermark that could only say "after this nanosecond" would either replay the whole batch or skip the rest of it. An empty poll returns the cursor unchanged: moving it forward would drop a row that lands a moment later with an earlier arrival time.

Scores and feedback cannot be polled. They are stamped with the timestamp the caller sent and carry no server-set arrival time, so a watermark over them would either skip a late row or repeat every row for ever. signal=scores is refused rather than answered with an empty page.

What this deployment can do

mcpd has one storage trait and three backends behind it, and they are not equal. On ClickHouse a delete is accepted and applied after the call returns, so the counts in its answer say what matched rather than what has left the disk. On SQLite and Postgres the rows are gone when the call returns. GET /v1/api/capabilities is where that is written down: which backend is behind the daemon, what q matches, whether case is folded outside ASCII, whether a delete is on disk when the call returns, and whether discovery and delta polling answer at all.

Read it once at connect time and offer only what the deployment answers. An endpoint whose capability is false refuses with 501 and a body whose capability field names the same word the capability set publishes, so the two can never disagree and a client that met the refusal can look it up rather than retrying something that will never change.

All three backends that ship today answer both discovery and delta polling, so the 501 below is the shape a backend added later would refuse with, rather than a response you will see from SQLite, Postgres or ClickHouse. The capability set is what tells you which it is, and it is worth reading even now: the three differ on whether a delete is on disk when the call returns, and on how retention is enforced. All three fold case outside ASCII.

# Every environment this project has reported, for a filter box.
curl -s --get http://localhost:4317/v1/api/discovery \
  --data-urlencode 'project=default' \
  --data-urlencode 'facet=environment'

# Typeahead: the service names starting with "check".
curl -s --get http://localhost:4317/v1/api/discovery \
  --data-urlencode 'facet=service_name' \
  --data-urlencode 'prefix=check'

# The values one metric label takes.
curl -s --get http://localhost:4317/v1/api/discovery \
  --data-urlencode 'facet=metric_label_value' \
  --data-urlencode 'key=model'

{
  "facet": "environment",
  "values": [
    { "value": "production", "last_seen": "2026-09-03T08:04:18Z" },
    { "value": "staging", "last_seen": "2026-09-02T17:41:02Z" }
  ]
}

{
  "error": "not_implemented",
  "capability": "discovery",
  "message": "this deployment stores signals in <backend>, which does not support discovery: ... GET /v1/api/capabilities lists what it does support."
}

Windows

from and to accept an RFC 3339 timestamp, or a relative shorthand: a minus sign, a number and one of s, m, h or d. -15m, -1h and -7d all work. The shorthand exists because typing an ISO timestamp by hand to look at the last hour is friction nobody should pay.

from is inclusive and to is exclusive, and both compare against the time the run started. A long run that began before your window and was still going inside it is not returned. If you are hunting something that was running at 09:00, widen from backwards rather than narrowing it.

A value that is neither a timestamp nor a duration is refused by name rather than ignored. Every parameter on every list endpoint fails this way, because a filter that is silently dropped returns a plausible wrong answer, and a plausible wrong answer is worse than an error.

{
  "error": "bad_parameter",
  "field": "from",
  "message": "\"yesterday\" is not an RFC 3339 timestamp or a relative duration like -1h"
}

The query that finds bad answers

has_score=answer_relevance&score_below=0.5 returns every run that scorer put under half. This is the one to reach for when the complaint is about quality rather than a failure, because a bad answer is usually a run that succeeded.

has_score on its own returns every run that scorer has looked at. score_below on its own does nothing at all: a threshold with no scorer attached has no meaning, and mcpd would have to guess which scorer you meant. It does not guess. Scores and feedback covers where those values come from.

Searching steps instead of runs

GET /v1/api/spans searches individual steps across every run at once. Use it for questions about a step rather than a session: which tool call is slow, which calls to one model errored, how long a retrieval takes.

It takes q, kind, status, name, model, trace_id, from, to, min_duration_ms, sort, order, limit and cursor, with the same meanings and the same defaults. There is no tags, session_id or user_id filter here, because those belong to the run, and all three are refused by name rather than accepted and dropped.

sort takes fewer values here than on the run list: start_time, duration or cost, and nothing else. A span carries no span count and no score rollup, so span_count and score have nothing to sort on and are refused by name rather than ignored.

On a run, model asks whether any span used it. On a span, model is that span's own model. The same word, two questions.

# Failed runs from the last day that mention both words, anywhere in a
# span name, a prompt or a completion.
curl -s --get http://localhost:4317/v1/api/traces \
  --data-urlencode 'q=Leeds depot' \
  --data-urlencode 'status=error' \
  --data-urlencode 'from=-24h'

# Every step that called one model and failed, across every run at once.
curl -s --get http://localhost:4317/v1/api/spans \
  --data-urlencode 'model=gpt-4o-mini' \
  --data-urlencode 'status=error' \
  --data-urlencode 'min_duration_ms=2000'

import { Mcpd } from '@mcpd/sdk'

const mcpd = new Mcpd()

const bad = await mcpd.listTraces({
  q: 'Leeds depot',
  status: 'error',
  from: '-24h',
  limit: 20,
})

for (const trace of bad.traces) {
  console.log(trace.traceId, trace.name, trace.status)
}

from mcpd import Mcpd

mcpd = Mcpd()

bad = mcpd.list_traces(q="Leeds depot", status="error", from_="-24h", limit=20)

for trace in bad.traces:
    print(trace.trace_id, trace.name, trace.status.value)

Paging, and why the cursor is opaque

A list returns at most limit rows and a page object. page.next_cursor is present when there is more, and page.has_more says the same thing as a boolean.

Paging is keyset, not offset. The cursor names the last row you were given, and the next page starts strictly after it. Offset paging would drift: traces arrive continuously, so by the time you asked for rows 50 to 99 the window has shifted under you and you read one run twice while another never appears at all. In an investigation, the run you miss is the one you were looking for.

The cursor is a hex-encoded pair of the sort value and the row id, and you should treat it as meaningless. It is opaque so that the encoding can change without breaking every caller, and because the value inside it is a different quantity for each sort: nanoseconds for start_time and duration, a scaled integer for cost, a plain count for span_count, the run's lowest score for score.

That has one rule attached. Send a cursor back with the same filters, the same sort and the same order that produced it. mcpd does not check that you did, and a cursor replayed against a different sort compares against the wrong column and skips rows. A cursor that has been edited or truncated is refused with invalid_query and the message malformed cursor.

There is no total. page.total is reserved for a backend that can count matches cheaply, and the built-in SQLite store does not fill it in, so loop on has_more rather than counting towards a number.

# The first page hands back a cursor in page.next_cursor.
curl -s --get http://localhost:4317/v1/api/traces \
  --data-urlencode 'q=refund' \
  --data-urlencode 'sort=cost' \
  --data-urlencode 'limit=100'

# The next page: the same filters, the same sort, plus the cursor.
curl -s --get http://localhost:4317/v1/api/traces \
  --data-urlencode 'q=refund' \
  --data-urlencode 'sort=cost' \
  --data-urlencode 'limit=100' \
  --data-urlencode 'cursor=MzEzNzQyfDBhZjM...'

let cursor: string | undefined

do {
  const page = await mcpd.listTraces({
    q: 'refund',
    sort: 'cost',
    order: 'desc',
    limit: 100,
    cursor,
  })

  for (const trace of page.traces) {
    console.log(trace.traceId, trace.cost?.totalCost ?? 'not priced')
  }

  // Loop on hasMore. There is no total to count towards.
  cursor = page.page.hasMore ? page.page.nextCursor : undefined
} while (cursor)

cursor = None

while True:
    page = mcpd.list_traces(
        q="refund", sort="cost", order="desc", limit=100, cursor=cursor
    )

    for trace in page.traces:
        cost = trace.cost.total_cost if trace.cost else "not priced"
        print(trace.trace_id, cost)

    # Loop on has_more. There is no total to count towards.
    if not page.page.has_more:
        break
    cursor = page.page.next_cursor

Once you have the run

A trace id from any of these queries goes to GET /v1/api/traces/{trace_id}, which returns the run and every span in it, already ordered so a waterfall renders with no second request. Add include=logs,scores,feedback to get everything attached to it in the same response.

From there, Cost accounting explains the figures on each step, and the HTTP API reference lists every endpoint and field as the specification declares them.