Docs

Evaluate a cold email

One cold email and the prompt behind it, evaluated by Amdahl and read end to end: the five criteria and their reasons, what the score is and is not, a fix and a second evaluation, and the prompt evaluated by the emails it writes.

Level: first run. Calls: messages.optimize with mode: "grade", then messages.optimize_prompt. You need: a Customer agent key or the Amdahl MCP connector, the editor role, and a workspace with synced conversations.

This walkthrough is for GTM engineers and AI GTM teams whose agents write outbound. It evaluates one email, reads every part of the evaluation, and then evaluates the prompt that wrote it. The email, the company and the reasons are an illustrative example in the shape of most first-touch outbound, not data from a real workspace. Your own evaluation reads your own customers' conversations.

The email and the prompt

Here is the cold email:

text
Subject: Quick question

Hi Dana - saw Northwind is scaling fast. We help engineering teams
ship 10x faster with best-in-class rollout tooling. Worth a quick
15 minutes next week?

And the prompt behind it:

text
Write a short cold email to a VP of Engineering about our rollout tooling.
Keep it under 100 words and friendly.

Evaluate the email

Send the email in message, with mode: "grade" and evidence: "workspace". Without evidence, only cta_clarity is evaluated: the other four criteria read your customers' conversations.

bash
curl -s -X POST https://app.amdahl.ai/api/platform/v1/messages/optimize \
  -H "X-API-Key: $AMDAHL_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "mode": "grade",
    "channel": "email",
    "evidence": "workspace",
    "message": "Subject: Quick question\n\nHi Dana - saw Northwind is scaling fast. We help engineering teams ship 10x faster with best-in-class rollout tooling. Worth a quick 15 minutes next week?"
  }'

Over MCP, call optimize_message with the same fields. The evaluation comes back on the same call, and Amdahl writes nothing.

What comes back

This response is trimmed to the evaluation fields. The full response also carries summary, run_id, unsupported_claims and notes, as a write does.

json
{
  "data": {
    "ok": true,
    "outcome": "graded",
    "rewrite": null,
    "optimize_id": "opt_4e18b2",
    "lift": null,
    "evidence": { "requested": "workspace", "used": true, "reason": null },
    "grade": {
      "score": 1.8,
      "passed": 1,
      "total": 5,
      "bar": 4.2,
      "passes_bar": false,
      "judge": { "model": "claude-sonnet-5-5", "judge_version": "v7", "role": "final" },
      "lines": [
        { "id": "relevant_positioning", "pass": false, "applicable": true, "reason": "Nothing here is about Northwind. 'Scaling fast' applies to every company in the segment, and the rest describes the product." },
        { "id": "grounding", "pass": false, "applicable": true, "reason": "No claim matches what these buyers said. Their conversations are about release approvals, not speed." },
        { "id": "verified_specifics", "pass": false, "applicable": true, "reason": "'10x faster' is stated in no conversation and no fact." },
        { "id": "differentiation", "pass": false, "applicable": true, "reason": "'Best-in-class rollout tooling' names a category, not a position." },
        { "id": "cta_clarity", "pass": true, "applicable": true, "reason": "One clear, small ask." }
      ],
      "not_applicable_reason": null
    },
    "disclosures": {
      "single_run": true,
      "not_reply_prediction": true,
      "repeat_spread": { "model": "claude-sonnet-5-5", "moved_past": 0.2, "share": 0.15, "measured": "..." }
    }
  }
}

An evaluation has no message key and no unchanged key. The draft is yours, and nothing was written. Every field is on Evaluate a message.

1. Read evidence first

evidence.used: true means the four evidence criteria read your conversations. When it is false, reason says why (for example corpus_empty), and those four criteria abstain. Say a draft was evaluated against your customers only when used is true.

2. Read the criteria, with the reason for each

CriterionVerdictReason
relevant_positioningfailNothing is about Northwind. "Scaling fast" fits every company in the segment.
groundingfailNo claim matches what these buyers said. They talk about release approvals, not speed.
verified_specificsfail"10x faster" is stated nowhere, and a buyer will ask you to defend it.
differentiationfail"Best-in-class rollout tooling" is a category, not a position.
cta_claritypassThe ask is clear and small. It is the strongest part of the email.

3. Read the fraction, then the score

Headline: 1 of 5 criteria passed. Each criterion is a pass or a fail, so the score is not a holistic rating and not an average of five opinions. It is the pass fraction placed on a 1 to 5 axis:

code
score = 1 + 4 × (passed / total)

With five evaluated criteria the score takes exactly six values: 1, 1.8, 2.6, 3.4, 4.2 and 5. A score of 1.8 is exactly one criterion passed. The response sends passed and total beside score for that reason: "1 of 5 criteria" says what was measured, and "1.8" alone reads like a rating on a scale that does not exist. Quote the fraction.

The bar is 4.2, four criteria of five, and passes_bar: false says this draft does not meet it. A criterion that abstains (pass: null, applicable: false) is left out of both passed and total. It is never counted as a fail. For what the score measures, see What an evaluation measures.

4. Read the disclosures

  • single_run: true: this is one reading. In Amdahl's measurement, the evaluation model moved its score by more than 0.2 on about 15% of texts when it evaluated them again (repeat_spread). A draft near the bar can pass on one evaluation and fail on the next.
  • not_reply_prediction: true: the evaluation measures the message against your evidence and these criteria; it does not predict replies.

Fix the email and evaluate it again

The reasons point at the fix. The buyers talk about approvals, so lead with approvals. Cut "10x faster", because nothing states it. Keep the ask, which passed. If you know something first-hand that no conversation holds, such as a customer result you may cite, send it in facts, inside context: a fact backs only the claim it states.

text
Subject: The four sign-offs

Hi Dana - the engineering leaders we talk to say shipping isn't the
bottleneck, approvals are: four sign-offs, and someone is always out.

We built the approval path for that. One customer went from four
serial approvals to two parallel ones.

Want me to send the two-page write-up of how they did it?

Evaluate the edited email with the same request, "message" changed, and with the customer result in facts, inside context, if it is true. Evaluate it soon after the first evaluation, so both read the same state of your workspace. An evaluation reads your conversations at the time of the call, and no call can pin the quotes an earlier evaluation read.

Read the new fraction criterion by criterion. Note which criteria changed their verdict, not only the new score. Because one evaluation is one reading, evaluate each version several times before you say that one of them is better.

Attribute at the scope you have. Northwind has never spoken to you, so the evidence behind the edit is what buyers like Dana say. "The engineering leaders we talk to say…" is backed by that. "Your engineers said…" is not: it claims a conversation that never happened, and it fails grounding. Send recipient fields, including recipient, too, and the evaluation uses what your own data holds about that account.

Evaluate the prompt

The edit fixed one email. The prompt that wrote the first one will write the same email for the next account. To evaluate the prompt, send it to messages.optimize_prompt (MCP optimize_prompt). It writes one email per sample recipient from the prompt and evaluates each one with the same evaluation model.

bash
curl -s -X POST https://app.amdahl.ai/api/platform/v1/messages/optimize_prompt \
  -H "X-API-Key: $AMDAHL_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "prompt": "Write a short cold email to a VP of Engineering about our rollout tooling. Keep it under 100 words and friendly.",
    "mode": "improve",
    "idempotency_key": "rollout-prompt-v1"
  }'

With no recipients, the run samples up to six of your workspace's own accounts whose customers spoke on calls or meetings, and their company names appear on the emails. In improve mode the first half are train recipients, which the run revises the prompt against, and the rest are held out. To compare runs later, send the same recipients each time, so the sample stays fixed.

The call returns 202 at once with a run_id. A run takes minutes. Read it with wait_ms, which makes the server hold the read for up to 30 seconds:

bash
curl -s "https://app.amdahl.ai/api/platform/v1/messages/optimize_prompt/$RUN_ID?wait_ms=30000" \
  -H "X-API-Key: $AMDAHL_KEY"

Repeat the read until status is complete or failed. Over MCP, call get_prompt_run with run_id and wait_ms.

A finished run, trimmed:

json
{
  "data": {
    "run_id": "opt_9b1d40",
    "status": "complete",
    "prompt_mode": "improve",
    "outcome": "improved",
    "prompt": "Write a first-touch email to a VP of Engineering ...",
    "grade": {
      "original": { "mean": 1.8, "n": 3, "emails_passing_bar": 0, "bar": 4.2, "lines": { "grounding": { "passed": 0, "total": 3, "rate": 0 } } },
      "revised": { "mean": 3.4, "n": 3, "emails_passing_bar": 1, "bar": 4.2, "lines": { "grounding": { "passed": 2, "total": 3, "rate": 0.67 } } }
    },
    "lift": 1.6,
    "lift_reportable": true,
    "lift_not_reportable_reason": null,
    "disclosures": { "single_run": true, "not_reply_prediction": true, "train_scores_are_not_lift": true, "repeat_spread": { "...": "..." } }
  }
}

Read it in this order:

  1. outcome. improved returns the revised prompt. unchanged means no revision beat the original on the held-out recipients, so keep the original.
  2. grade.original and grade.revised. These are over the held-out emails only: the recipients the revision did not learn from. Each lines entry is one criterion's pass rate across those emails.
  3. lift. The held-out mean of the revised prompt minus that of the original. It is a number only when lift_reportable is true. Fewer than 3 held-out emails on a side gives null with held_out_under_3. The train scores come from the model that picks the revision, not from the evaluation model, and are never lift.
  4. emails[] (trimmed above). Read a few of the emails each prompt wrote, with their evaluations, before you adopt the revision.

The revised prompt is the thing to keep: a template with slots, not a rewrite of one email. Paste it for the next account and read its first emails yourself. To evaluate a prompt without a rewrite, send "mode": "grade": every email is evaluated, and the prompt's evaluation is their mean. See Improve a prompt.

Record what happened

When the sender sends a message Amdahl wrote, record it with record_message_feedback (REST POST /messages/optimize/runs/{id}/feedback). evidence is required: did_it_myself, reported_to_me or inferred. Send the text actually sent in sent_text. See Record feedback on a run.

Who uses this, and how

Reps: before you send a sequence

Evaluate step 1 with evidence: "workspace". You get the criteria that fail and the reason for each. Fix the draft, then evaluate the prompt you write the steps from, so the next account starts from a better template.

Managers: audit the sequence, not the rep

Evaluate every step of a live sequence. The pattern across the evaluations is the coaching: if grounding fails on every step, the problem is the template, not the person who sends it.

Enablement: keep the template honest

Run the team's sequence prompt through optimize_prompt on a cadence, with the same six recipients each time. As your conversations change, the held-out evaluation changes with them.

GTM engineers and Ops: optimize before anything ships

One call returns the evaluation. Hold copy where passes_bar is false, and pass through where it is null (nothing was evaluated). Log passed and total, not only the score, so you can see whether outbound quality moves.

A sequence review, end to end

Evaluate every step

One optimize_message call per step, each with mode: "grade", evidence: "workspace" and the step's copy as message. They are independent, but Amdahl limits how many run at once for a workspace, so send a few at a time and wait out a 429 for its Retry-After.

Read the criteria, not the headline

A headline of 2.6 tells you little on its own. grounding failing on all five steps tells you the sequence was written from the category, not from customers.

Evaluate the prompt

Send the shared sequence prompt to optimize_prompt. If the outcome is improved with a reportable lift, put the revised prompt in the sequence brief, so the next person to write a step starts from it.

Fix the failed criteria

Each failed criterion has a reason. Work through the list, step by step.

Evaluate again and compare the fractions

Evaluate each edited step soon after the first evaluation, and compare passed and total criterion by criterion. A criterion that still fails after a real fix is often telling you something about your evidence: the conversations may not back the point you want to make.

Next