Models and limits
Which models evaluate, draft and rewrite, what an evaluation records about the model that made it, what an evaluation measures, latency budgets, the two caches, data handling, and a dated table of known limits with what to do instead.
This page collects the facts about how the engine behaves that are otherwise spread across the reference pages. Last reviewed 2026-11-09.
What an evaluation measures
An evaluation measures a message against a fixed set of criteria written for outreach email. With evidence: "workspace", four of those criteria measure it against your customers' own conversations: Amdahl's model of what wins with your buyers. Without your data, those four drop out of the score rather than counting as zero.
An evaluation does not predict replies. One evaluation is one reading, so read each criterion and its reason, not the score alone.
Instrument versions
Every evaluation records the evaluation model that produced it, in grade.judge: the model, a judge_version, and role: "final". A prompt run records its evaluation models in judges. The version moves whenever the evaluation work changes.
- Two evaluation-model versions are two instruments. Do not average scores or count passes across a change of
judge_version. - You cannot request an older evaluation model. Every new evaluation uses the current one.
- Eval scores do not compare with evaluations. On 2026-10-22, evaluation moved into the optimize call, and the separate evals product ended. A stored eval run that the MCP
get_eval_runtool reads back was scored by a different evaluation model, and its scores do not compare with an evaluation frommessages.optimize.
The REST API itself is versioned separately, in the URL. See Versioning.
Models
You never choose a model on the optimize call or on Search. The engine uses Anthropic Claude models, and an evaluation names its evaluation model in grade.judge.model.
| Where | Model | Notes |
|---|---|---|
| Research | The research agent's own model | One fixed tier. You cannot choose it. |
| In-loop scoring model of the optimize call (scores every version it tries) | claude-haiku-4-5 | Picks the version to return. On a prompt run it also produces the train scores, which are never lift. |
Evaluation model (mode: "grade", and every email a prompt run evaluates) | Claude Sonnet 5.5, thinking off | The final evaluation model, never the model that picks a rewrite. grade.judge.model names it. |
| Re-check of the version the optimize call returns | claude-sonnet-5-5 | A different model from the in-loop scoring model. Without your data, it answers yes/no questions about the version it returns. When the run read your conversations, it produces lift, which names it in lift.grader. |
| Search, filter and lexical lanes | None | Compiled predicates and full-text matching. |
| Search, fuzzy lane | A model writes the SQL | The SQL it wrote comes back in compiled.sql. |
| Search, semantic lane | An embedding of your query | Ranked against customer-side utterances. |
Search, answer: true | A fast Claude model, named in the model field of answer | Writes 2 or 3 sentences from the returned rows. Each sentence cites a row, and a sentence that does not pass the check is dropped. |
Sampling. The evaluation models run at the model's default temperature, because the Claude 5-series models reject the parameter. The practical consequence is run-to-run variance, covered under Known limits. An evaluation carries the measured spread in disclosures.repeat_spread.
Customizing. Today you cannot fine-tune a model, choose one, or write your own criteria. You shape an evaluation with what you send: context (facts, rules, constraints), the recipient fields where your workspace has them, and evidence: "workspace".
Latency and budgets
| Call | Budget |
|---|---|
POST /messages/optimize | Answers on the same call, usually in 1 to 2 minutes. In test runs on 19 October 2026 the median was about 75 seconds, and 9 runs in 10 finished within about 130. A hard deadline of 300 seconds ends a run with ok: false and optimizer_timeout, so give your client 310. evidence: "workspace" is slower. |
POST /search/query, synchronous | Sized to finish under a 15 second ceiling. A partial answer with typed fields beats a timeout. |
POST /search/query, async: true (fuzzy lane) | Up to about 180 seconds. You get a job_id to poll. |
POST /search/query, answer: true | Adds at most 6 seconds after the rows are in. On a timeout the rows return, and answer has status: "failed". |
POST /messages/optimize, mode: "grade" | Answers on the same call. It makes one evaluation and writes nothing, so it finishes sooner than a rewrite. |
POST /messages/optimize_prompt | Returns 202 with a handle at once. A run takes minutes. Poll GET /messages/optimize_prompt/{id}. |
Prompt run reads with wait_ms | Block for up to 30 seconds, then return the run as it stands. |
Research and message batch reads with wait_ms | Block for up to 45 seconds, then return the job as it stands. Over MCP, get_research and get_message_batch wait up to 25 seconds when no wait_ms is sent. |
| Research turns | The research agent's own turn budget. A job that runs out of turns finishes with a partial answer. See Partial answers. |
Rate limits are on Rate limits: per credential (per IP for a request with no credential) and per minute, 600 MCP protocol calls, 300 reads, 30 expensive calls (optimize, evaluate, batch submit, prompt run, research start, connect) and 60 of everything else, inside a ceiling of 300 per minute per client IP (1,200 for protocol-only requests), and a warehouse-read budget per workspace underneath it (60 reads a minute with a burst of 20, and 4 at once).
Two caches
Both can hand you back an earlier answer. That is usually what you want, and it is the wrong thing when you are measuring a change.
| Cache | What it serves | How to bypass |
|---|---|---|
| Search result cache | An identical warehouse query within the last five minutes, at no cost to your read budget. An implementation detail, not a contract. | Change the query. |
| Optimize run reuse | The same request (same sender, same draft, same options) while it is still running, or its stored result after it finished. A prompt run started with the same idempotency_key within 24 hours returns the first run, with replayed: true. | Change the request, or send a new idempotency_key. |
Data handling
What the code guarantees today:
- Search and evaluations never change your data. An evaluation writes no message and sends nothing.
- Amdahl adds leads to a sequencer campaign only when you ask, from the console. Send to adds a batch's leads, with their messages, to a campaign you pick in Instantly, Smartlead, lemlist or HeyReach (in beta), with the key you connected and after you confirm. It never starts a campaign and sends no email. It keeps which leads were added, skipped or failed, and never the message text.
- The optimize call hands back text and sends nothing. It keeps a record of each run with your workspace: the request you sent, the versions it tried and how each scored, and the per-sender record of how each sender writes. Facts you send in
contextare the exception: they are used for that request only, never stored as evidence and never written to logs as text. - Reads are scoped server-side. Every warehouse read carries your workspace filter and any data-scope rule your admin has set; your credential's scopes cap what it may call.
- Quotes come from your processed customer data. A quote an evaluation reads is a stored utterance, looked up by id. Speaker names are not included.
- Model calls go to Anthropic. The context and drafts you submit and the quotes retrieved for them are sent to Claude models to write and evaluate.
- No search query leaves for the web. Research reads only your workspace's data and fetches nothing from outside it. The opt-in outside search that Chat had retired with Chat on 2026-11-04.
- Runs are stored. Optimize runs and prompt runs are kept with your workspace. Eval runs made before evaluation moved into the optimize call on 2026-10-22 are kept too, and the MCP
get_eval_runtool reads one back.
For contractual terms (retention periods, subprocessors, regions), ask your Amdahl contact; this page does not state them.
Known limits
Last reviewed 2026-11-09.
| Limit | What happens | Do this instead |
|---|---|---|
| No shared demo workspace | A workspace with nothing synced returns empty searches, and an evaluation's four evidence criteria abstain (corpus_empty). | Start with outreach, which needs no data. Run GET /search/overview before you read Search results or an evaluation, and connect a CRM or call recorder. |
| Without your data, the optimize call does not evaluate claims | With no evidence, a message is evaluated on how it reads and how clearly it asks. A wrong fact you send is still wrong in the message. | Send evidence: "workspace" once your conversations sync, and put what you know in facts. |
| The optimize call is only as specific as its context | A run on one thin fact and no draft returns a generic message. | Send who the recipient is, a reason to write now, what you offer and the ask. See Context. |
| Without your data, the optimize call prefers a question to a meeting ask | A message usually closes on a one-line question about whether the problem is real for the reader, not a request for a call. | Put the ask you need in rules, inside context. A version that breaks a rule is never returned. |
| 1,000 optimize results a month | API and connector calls are capped at 1,000 a month per workspace by default, counted when a result comes back: single calls, evaluations, batch items and finished prompt runs alike. Console calls are not counted. The separate evals product had its own cap, max_evals_calls_month. Evals no longer run, so that cap limits nothing. | Ask Amdahl to raise the cap before a launch that needs more. |
| 500 drafts in one optimize batch | A batch takes 1 to 500 drafts and a 1 MB body, and a workspace may have 5,000 batch items queued. A batch has no promised finish time: it runs behind single calls, and its status gives an estimate. | Split a larger campaign into several batches (see Batches). |
| Three optimize calls at once, twelve a minute, per workspace | A call past either limit waits up to 15 seconds, then returns 429 rate_limited with Retry-After, and nothing is rewritten. The limits count every key, connector session and the console together. | For a long list, send a batch: it is not held to these limits. For a few drafts, send at most three at once and wait Retry-After on a 429 (see Running several at once). |
| One evaluation is one reading | Evaluating identical text twice can differ. In Amdahl's measurement, Sonnet 5.5 with thinking off moved its score past 0.2 on about 15% of texts on a repeat. disclosures.repeat_spread carries the figures. | Evaluate each version several times and compare medians. |
| A prompt's train scores are not lift | On an improve run, train holds the in-loop scoring model's scores on the recipients the prompt was tuned on. | Read lift, which is measured on held-out recipients only and is null unless lift_reportable is true. |
| The criteria are written for outreach email | A prompt that names one recipient, holds one fixed email or asks for copy that is not an email is refused with prompt_is_one_off. | Send that one email through messages.optimize. |
| No criteria of your own | Workspace-authored evals ended on 2026-10-22, when evaluation moved into the optimize call. Nothing replaces them. | Put the requirements you need in rules and the constraints, inside context. |
department is not filterable | No surface has the column. | Filter on role_level. |
| The semantic mirror ignores excluded interactions | A semantic match can come from an interaction your workspace excluded. The filter and fuzzy lanes respect exclusions. | Confirm a semantic match on the filter lane before acting on it when exclusions matter. |
| The lexical lane reads a 400-character preview | A term past the first 400 characters of a long utterance is not findable by exact match. | Use the fuzzy or filter lane with contains for long text. |
| A quoted phrase changes the search lane | Under mode: "auto", any query with a quoted phrase runs on the lexical lane. REST cannot force mode: "lexical"; MCP can. | Read mode_ran. Drop the quotes, or set mode, to reach another lane. |
| The fuzzy lane reports throttling as a failed answer | Running out of warehouse reads returns 200 with detail.internal.status: "failed", not 429. | Treat failed during a burst as a throttle and back off. |
| Unknown top-level Search keys are dropped | A typo such as "filter" for "filters" is ignored, and the call runs unfiltered. | Check compiled and corpus.filters_applied. |
| No server-side idempotency | Retrying a timed-out POST /research while the first job still runs can start a second job. There is no Idempotency-Key header on that call. | Deduplicate on the client. See Idempotency. |
| No scenario projection endpoint | There is no simulate or what-if operation. | Measure the baseline with Search's group_by and metrics, and reason over it in your own agent. |