Postmark
Transactional email with high deliverability.
Overall score
59.0/100
postmark-rest-v1
Measured Oct 5, 2026
Status
Ranked
Complete benchmark. `overall` is a weighted average of the applicable criteria, comparable to other surfaces in the same category and the same rubric version.
Coverage
98% of the rubric
Surfaces
evaluated.
We score a surface, not a product. One product often exposes several, and they are not worth the same.
postmark-rest-v1restprimary
98% coverage
59.0/100
Criterion
by criterion.
Measured on postmark-rest-v1, rubric v0.2.0. A criterion with no score is not counted as zero: it is excluded and the weights are renormalised.
Offload Valuejudged · weight 15 · Oct 5, 2026
91/100
Postmark performs a genuine, irreversible external side effect that an agent cannot replicate through inference: actually delivering email to recipient inboxes, plus SMTP infrastructure, DKIM/SPF/Return-Path verification, bounce classification and reactivation, suppression lists, spam-complaint and open/click tracking, and inbound email parsing with webhooks. These are deliverability and compliance functions (GDPR data-removal requests, sender signature verification) that no amount of reasoning tokens can substitute for. The documented surface is heavily side-effect oriented rather than storage; the only deduction is that the API reference page itself is largely error codes and auth plumbing, and templating/stats are lighter-value conveniences.
- https://postmarkapp.com/developer/api/overview
Often when implementing your client library or when integrating an existing library into your application you may want to send test emails that don’t actually get delivered to the recipient.
- https://postmarkapp.com/developer/api/overview
Sending email with API - Send a single email - Send batch emails Sending email with SMTP IP Allowlisting Processing email - Configure an inbound server - Inbound domain forwarding - Parse an email
- https://postmarkapp.com/developer/api/overview
Sending email with API - Send a single email - Send batch emails Sending email with SMTP IP Allowlisting Processing email
Interaction Costmeasured · weight 15 · Oct 5, 2026
65/100
Median over 2 successful run(s): 7874.5 tokens, 3 calls. Ratio to the best in "transactional-email" (3 surfaces measured, v0.2 scale). Cold, without documentation: 1/1 succeeded.
Error Recoverabilityprobed · weight 15 · Oct 5, 2026
39/100
unknown_endpoint → HTTP 422, clarity 30/100 (field named: no, allowed values: no, RFC 9457: no) · wrong_type → HTTP 200, clarity 55/100 (field named: yes, allowed values: no, RFC 9457: no) · hallucinated_enum → HTTP 200, clarity 30/100 (field named: no, allowed values: no, RFC 9457: no) · unknown_field → HTTP 200, clarity 30/100 (field named: no, allowed values: no, RFC 9457: no) · missing_required_param → HTTP 422, clarity 50/100 (field named: no, allowed values: yes, RFC 9457: no) · idempotency_replay → HTTP 200, clarity 45/100 (field named: no, allowed values: yes, RFC 9457: no)
- https://api.postmarkapp.com/messages/outbound?count=1&offset=0
HTTP 401: {"ErrorCode":10,"Message":"Request does not contain a valid Server token."}
- https://api.postmarkapp.com/messages/outbound/nonexistent_resource_xyz?count=1&offset=0
HTTP 422: {"ErrorCode":701,"Message":"This message was not found."}
- https://api.postmarkapp.com/messages/outbound?count=1&offset=0&count=not_a_number
HTTP 200: {"TotalCount":13,"Messages":[{"MessageID":"77b942c2-e270-4979-8cb2-ee3ec144a751","MessageStream":"outbound","To":[{"Email":"hello@nicolascharron.com","Name":""}],"Cc":[],"Bcc":[],"Recipients":["hello@
- https://api.postmarkapp.com/messages/outbound?count=1&offset=0&status=nonexistent_status
HTTP 200: {"TotalCount":13,"Messages":[{"MessageID":"77b942c2-e270-4979-8cb2-ee3ec144a751","MessageStream":"outbound","To":[{"Email":"hello@nicolascharron.com","Name":""}],"Cc":[],"Bcc":[],"Recipients":["hello@
Doc Legibilitymeasured · weight 12 · Oct 5, 2026
50/100
text/markup ratio 19.7% (weight 30) · renders without JavaScript: yes (weight 25) · machine-readable spec: none found (weight 25) · llms.txt: https://postmarkapp.com/llms.txt (weight 10) · 1 runnable example(s) (weight 10). Weighted average over 100 points of available sub-criteria.
Auth Frictionmeasured · weight 10 · Oct 5, 2026
90/100
Dedicated X-Postmark-Server-Token header, verified by a successful call. Slightly below Bearer: the header name has to be read in the documentation.
- https://api.postmarkapp.com/messages/outbound?count=1&offset=0
HTTP 401: {"ErrorCode":10,"Message":"Request does not contain a valid Server token."}
- https://api.postmarkapp.com/messages/outbound/nonexistent_resource_xyz?count=1&offset=0
HTTP 422: {"ErrorCode":701,"Message":"This message was not found."}
- https://api.postmarkapp.com/messages/outbound?count=1&offset=0&count=not_a_number
HTTP 200: {"TotalCount":13,"Messages":[{"MessageID":"77b942c2-e270-4979-8cb2-ee3ec144a751","MessageStream":"outbound","To":[{"Email":"hello@nicolascharron.com","Name":""}],"Cc":[],"Bcc":[],"Recipients":["hello@
- https://api.postmarkapp.com/messages/outbound?count=1&offset=0&status=nonexistent_status
HTTP 200: {"TotalCount":13,"Messages":[{"MessageID":"77b942c2-e270-4979-8cb2-ee3ec144a751","MessageStream":"outbound","To":[{"Email":"hello@nicolascharron.com","Name":""}],"Cc":[],"Bcc":[],"Recipients":["hello@
Safety & Reversibilityprobed · weight 10 · Oct 5, 2026
42/100
Two identical POSTs with the same Idempotency-Key header created TWO distinct resources (48edbd49-9abe-42c0-ba32-6e51a447664b then 7af6333e-fd02-4e78-8f7f-455660d6e673). Every network timeout becomes a duplicate. (weight 35) · Isolated test environment available. (weight 25). Weighted average over 60 points of measured sub-criteria.
- https://api.postmarkapp.com/messages/outbound?count=1&offset=0
HTTP 401: {"ErrorCode":10,"Message":"Request does not contain a valid Server token."}
- https://api.postmarkapp.com/messages/outbound/nonexistent_resource_xyz?count=1&offset=0
HTTP 422: {"ErrorCode":701,"Message":"This message was not found."}
- https://api.postmarkapp.com/messages/outbound?count=1&offset=0&count=not_a_number
HTTP 200: {"TotalCount":13,"Messages":[{"MessageID":"77b942c2-e270-4979-8cb2-ee3ec144a751","MessageStream":"outbound","To":[{"Email":"hello@nicolascharron.com","Name":""}],"Cc":[],"Bcc":[],"Recipients":["hello@
- https://api.postmarkapp.com/messages/outbound?count=1&offset=0&status=nonexistent_status
HTTP 200: {"TotalCount":13,"Messages":[{"MessageID":"77b942c2-e270-4979-8cb2-ee3ec144a751","MessageStream":"outbound","To":[{"Email":"hello@nicolascharron.com","Name":""}],"Cc":[],"Bcc":[],"Recipients":["hello@
Payload Efficiencyprobed · weight 8 · Oct 5, 2026
5/100
No discoverable field selection (tested fields, select, _fields, expand, properties): the agent swallows the whole resource. (weight 35) · No cursor pagination marker in the response. (weight 20) · Nested envelope or non-list shape. (weight 10) · No ETag. (Compression is not measurable: the runtime decompresses it and strips the header.) (weight 10). Weighted average over 75 points of measured sub-criteria.
- https://api.postmarkapp.com/messages/outbound?count=1&offset=0
HTTP 401: {"ErrorCode":10,"Message":"Request does not contain a valid Server token."}
- https://api.postmarkapp.com/messages/outbound/nonexistent_resource_xyz?count=1&offset=0
HTTP 422: {"ErrorCode":701,"Message":"This message was not found."}
- https://api.postmarkapp.com/messages/outbound?count=1&offset=0&count=not_a_number
HTTP 200: {"TotalCount":13,"Messages":[{"MessageID":"77b942c2-e270-4979-8cb2-ee3ec144a751","MessageStream":"outbound","To":[{"Email":"hello@nicolascharron.com","Name":""}],"Cc":[],"Bcc":[],"Recipients":["hello@
- https://api.postmarkapp.com/messages/outbound?count=1&offset=0&status=nonexistent_status
HTTP 200: {"TotalCount":13,"Messages":[{"MessageID":"77b942c2-e270-4979-8cb2-ee3ec144a751","MessageStream":"outbound","To":[{"Email":"hello@nicolascharron.com","Name":""}],"Cc":[],"Bcc":[],"Recipients":["hello@
Task Atomicitymeasured · weight 6 · Oct 5, 2026
33/100
Median over 2 successful run(s): 7874.5 tokens, 3 calls. Ratio to the best in "transactional-email" (3 surfaces measured, v0.2 scale). Cold, without documentation: 1/1 succeeded.
Prior Knowledge Coveragemeasured · weight 5 · Oct 5, 2026
100/100
Median over 2 successful run(s): 7874.5 tokens, 3 calls. Ratio to the best in "transactional-email" (3 surfaces measured, v0.2 scale). Cold, without documentation: 1/1 succeeded.
Async Compatibilityprobed · weight 2 · Oct 5, 2026
100/100
POST /email returns an identifier, and GET /messages/outbound/{id}/details reads it back (HTTP 200): the result can be retrieved by polling, with no server to host. Long-polling / SSE not probed, excluded from the weighting.
- https://api.postmarkapp.com/messages/outbound?count=1&offset=0
HTTP 401: {"ErrorCode":10,"Message":"Request does not contain a valid Server token."}
- https://api.postmarkapp.com/messages/outbound/nonexistent_resource_xyz?count=1&offset=0
HTTP 422: {"ErrorCode":701,"Message":"This message was not found."}
- https://api.postmarkapp.com/messages/outbound?count=1&offset=0&count=not_a_number
HTTP 200: {"TotalCount":13,"Messages":[{"MessageID":"77b942c2-e270-4979-8cb2-ee3ec144a751","MessageStream":"outbound","To":[{"Email":"hello@nicolascharron.com","Name":""}],"Cc":[],"Bcc":[],"Recipients":["hello@
- https://api.postmarkapp.com/messages/outbound?count=1&offset=0&status=nonexistent_status
HTTP 200: {"TotalCount":13,"Messages":[{"MessageID":"77b942c2-e270-4979-8cb2-ee3ec144a751","MessageStream":"outbound","To":[{"Email":"hello@nicolascharron.com","Name":""}],"Cc":[],"Bcc":[],"Recipients":["hello@
Tool-Native (MCP)binary · weight 2
—/100
Does the vendor publish an official, maintained MCP server?
Runs, verified
never on the agent’s word.
Every row is a real attempt at the category’s canonical task. Success is verified by reading the created resource back.
| Task | Tokens | Calls | Errors | Docs | Result |
|---|---|---|---|---|---|
| send-transactional-email | 6,036 | 1 | 0 | without | verified |
| send-transactional-email | 8,012 | 3 | 1 | with | verified |
| send-transactional-email | 7,737 | 3 | 1 | with | verified |
The same entry,
without the markup.
You publish this API and dispute a score? The methodology is public and every score above points to the evidence that produced it.