Braintree
PayPal's payment gateway, with a GraphQL API.
Overall score
58.7/100
braintree-graphql
Measured Oct 5, 2026
Status
Ranked
Complete benchmark. `overall` is a weighted average of the applicable criteria, comparable to other surfaces in the same category and the same rubric version.
Coverage
88% of the rubric
Surfaces
evaluated.
We score a surface, not a product. One product often exposes several, and they are not worth the same.
braintree-graphqlgraphqlprimary
88% coverage
58.7/100
Criterion
by criterion.
Measured on braintree-graphql, rubric v0.2.0. A criterion with no score is not counted as zero: it is excluded and the weights are renormalised.
Offload Valuejudged · weight 15 · Oct 5, 2026
94/100
Braintree's GraphQL API performs real, irreversible financial side effects: charging payment methods, moving funds, vaulting/tokenizing payment instruments with PCI-scope compliance handling. No amount of inference can simulate charging a card or settling money; the agent cannot approximate this at all. The documentation is thin but clearly describes transaction creation and payment method vaulting, not storage of arbitrary data.
- https://developer.paypal.com/braintree/graphql/
Create a transaction Charge a payment method to create a transaction See the guide Vault a payment method Store a payment method for future use
- https://developer.paypal.com/braintree/graphql/
Create a transaction Charge a payment method to create a transaction See the guide Vault a payment method Store a payment method for future use
- https://developer.paypal.com/braintree/graphql/
Create a transaction Charge a payment method to create a transaction See the guide Vault a payment method Store a payment method for future use
Interaction Costmeasured · weight 15 · Oct 5, 2026
38/100
Median over 2 successful run(s): 31748 tokens, 7 calls. Ratio to the best in "payments" (3 surfaces measured, v0.2 scale). Cold, without documentation: 1/1 succeeded.
Error Recoverabilityprobed · weight 15 · Oct 5, 2026
61/100
unknown_endpoint → HTTP 404, clarity 30/100 (field named: no, allowed values: no, RFC 9457: no) · wrong_type → HTTP 200, clarity 75/100 (field named: yes, allowed values: yes, RFC 9457: no) · hallucinated_enum → HTTP 200, clarity 75/100 (field named: yes, allowed values: yes, RFC 9457: no) · unknown_field → HTTP 200, clarity 75/100 (field named: yes, allowed values: yes, RFC 9457: no) · missing_required_param → HTTP 200, clarity 50/100 (field named: no, allowed values: yes, RFC 9457: no) · idempotency_replay → HTTP 200, clarity 25/100 (field named: no, allowed values: no, RFC 9457: no)
- https://payments.sandbox.braintree-api.com/graphql
HTTP 200: {"extensions":{"requestId":"140e4dfc-61e2-4aeb-b9ef-99e5da33edc4"},"data":null,"errors":[{"message":"Authentication credentials are missing. Authorization header is required and must contain a value."
- https://payments.sandbox.braintree-api.com/nonexistent_resource_xyz
HTTP 404: {"meta":{"braintree_request_id":"6ed27948-31cb-4e16-ac34-f0f8e6bd29a1"},"error":{"user_message":"There was a problem interpreting this request because the resource does not exist.","developer_message"
- https://payments.sandbox.braintree-api.com/graphql
HTTP 200: {"errors":[{"message":"Validation error (WrongType@[search/transactions]) : argument 'first' with value 'StringValue{value='ten'}' is not a valid 'Int' - Expected an AST type of 'IntValue' but it was
- https://payments.sandbox.braintree-api.com/graphql
HTTP 200: {"errors":[{"message":"Validation error (WrongType@[search/transactions]) : argument 'input.status.in[0]' with value 'EnumValue{name='NOT_A_STATUS'}' is not a valid 'PaymentStatus' - Literal value not
Doc Legibilitymeasured · weight 12 · Oct 5, 2026
17/100
text/markup ratio 2.6% (weight 30) · rendering without JavaScript undetermined (excluded from the calculation) (weight 25) · machine-readable spec: none found (weight 25) · llms.txt: https://developer.paypal.com/llms.txt (weight 10) · 0 runnable example(s) (weight 10). Weighted average over 75 points of available sub-criteria.
Auth Frictionmeasured · weight 10 · Oct 5, 2026
85/100
HTTP Basic, verified by a successful call. Encoding required but standard.
- https://payments.sandbox.braintree-api.com/graphql
HTTP 200: {"extensions":{"requestId":"140e4dfc-61e2-4aeb-b9ef-99e5da33edc4"},"data":null,"errors":[{"message":"Authentication credentials are missing. Authorization header is required and must contain a value."
- https://payments.sandbox.braintree-api.com/nonexistent_resource_xyz
HTTP 404: {"meta":{"braintree_request_id":"6ed27948-31cb-4e16-ac34-f0f8e6bd29a1"},"error":{"user_message":"There was a problem interpreting this request because the resource does not exist.","developer_message"
- https://payments.sandbox.braintree-api.com/graphql
HTTP 200: {"errors":[{"message":"Validation error (WrongType@[search/transactions]) : argument 'first' with value 'StringValue{value='ten'}' is not a valid 'Int' - Expected an AST type of 'IntValue' but it was
- https://payments.sandbox.braintree-api.com/graphql
HTTP 200: {"errors":[{"message":"Validation error (WrongType@[search/transactions]) : argument 'input.status.in[0]' with value 'EnumValue{name='NOT_A_STATUS'}' is not a valid 'PaymentStatus' - Literal value not
Safety & Reversibilityprobed · weight 10 · Oct 5, 2026
42/100
Two identical POSTs with the same Idempotency-Key header created TWO distinct resources (Y3VzdG9tZXJfNDM2MDc2Njc3NTE then Y3VzdG9tZXJfODI0OTc1OTE1MTY). Every network timeout becomes a duplicate. (weight 35) · Isolated test environment available. (weight 25). Weighted average over 60 points of measured sub-criteria.
- https://payments.sandbox.braintree-api.com/graphql
HTTP 200: {"extensions":{"requestId":"140e4dfc-61e2-4aeb-b9ef-99e5da33edc4"},"data":null,"errors":[{"message":"Authentication credentials are missing. Authorization header is required and must contain a value."
- https://payments.sandbox.braintree-api.com/nonexistent_resource_xyz
HTTP 404: {"meta":{"braintree_request_id":"6ed27948-31cb-4e16-ac34-f0f8e6bd29a1"},"error":{"user_message":"There was a problem interpreting this request because the resource does not exist.","developer_message"
- https://payments.sandbox.braintree-api.com/graphql
HTTP 200: {"errors":[{"message":"Validation error (WrongType@[search/transactions]) : argument 'first' with value 'StringValue{value='ten'}' is not a valid 'Int' - Expected an AST type of 'IntValue' but it was
- https://payments.sandbox.braintree-api.com/graphql
HTTP 200: {"errors":[{"message":"Validation error (WrongType@[search/transactions]) : argument 'input.status.in[0]' with value 'EnumValue{name='NOT_A_STATUS'}' is not a valid 'PaymentStatus' - Literal value not
Payload Efficiencyprobed · weight 8
—/100
Can I ask for only the fields I need, or do I have to swallow 200 attributes to extract one?
Task Atomicitymeasured · weight 6 · Oct 5, 2026
50/100
Median over 2 successful run(s): 31748 tokens, 7 calls. Ratio to the best in "payments" (3 surfaces measured, v0.2 scale). Cold, without documentation: 1/1 succeeded.
Prior Knowledge Coveragemeasured · weight 5 · Oct 5, 2026
100/100
Median over 2 successful run(s): 31748 tokens, 7 calls. Ratio to the best in "payments" (3 surfaces measured, v0.2 scale). Cold, without documentation: 1/1 succeeded.
Async Compatibilityprobed · weight 2
—/100
If the operation is asynchronous, can I get the result without hosting a web server?
Tool-Native (MCP)binary · weight 2
—/100
Does the vendor publish an official, maintained MCP server?
Runs, verified
never on the agent’s word.
Every row is a real attempt at the category’s canonical task. Success is verified by reading the created resource back.
| Task | Tokens | Calls | Errors | Docs | Result |
|---|---|---|---|---|---|
| charge-a-customer | 43,062 | 10 | 0 | without | verified |
| charge-a-customer | 47,251 | 8 | 0 | with | verified |
| charge-a-customer | 16,245 | 6 | 1 | with | verified |
The same entry,
without the markup.
You publish this API and dispute a score? The methodology is public and every score above points to the evidence that produced it.