Skip to content
Holding·last review3 May 2026

The four credible 2026 agent-evaluation platforms (DeepEval, Braintrust, LangSmith, Patronus AI) do not compete on capability rank; each fits a distinct deployment shape (engineering-led eval-as-code; SaaS-first eval-as-product; LangChain-stack-native bundled with observability; research-grade hallucination + simulation), and picking by capability matrix produces the wrong procurement outcome for most enterprises. The structurally load-bearing eval-vs-observability split (companion piece AM-123) compounds this: 'is the agent right' and 'what did the agent do' are different procurement decisions answered by different platforms.

Procurement-first deep-dive on the 2026 agent-evaluation tooling category. Verified primary sources: LangChain State of Agent Engineering 2025 (n=1,340, surveyed 18 Nov-2 Dec 2025) cross-validated by McKinsey State of AI Nov 2025 (n=1,993 across 105 nations); DeepEval v3.9.9 release notes (1 Dec 2025, 15.1k stars); Braintrust public pricing (Starter $0/Pro $249/Enterprise on-prem); LangSmith pricing (Developer $0/Plus $39/Enterprise hybrid+self-host) plus published HIPAA/SOC2 Type 2/GDPR posture; Patronus AI homepage (frontier-lab repositioning). Editorial finding: brief's 64% Anaconda+Forrester eval-blocker stat doesn't resolve to a verifiable primary source; substituted LangChain n=1,340 32% quality-as-blocker figure cross-validated by McKinsey. Patronus AI repositioned in 2026 from hallucination specialist (brief framing) to 'frontier lab developing simulation research and infrastructure'; piece surfaces this as a tracked vendor pivot rather than smoothing it. 60-day review cadence because vendor pricing and product positioning churn quarterly in this category.

Published
3 May 2026
Last reviewed
3 May 2026
Next review
+16d· 2 Jul 2026
Embed this claimiframe + oEmbed
HTML iframe
Paste-the-URL (Substack, Medium, Notion, WordPress)

The card auto-updates when the claim's status, last-reviewed date, or correction log changes. Embedders never need to refresh — the card is rendered live from the canonical record.

Watch this claim

Email-me when AM-122's status, next review date, or correction log changes. One email per change. No newsletter subscription, no other mail.

The claim: The four credible 2026 agent-evaluation platforms (DeepEval, Braintrust, LangSmith, Patronus AI) do not compete on capability rank; each fits a distinct deployment shape (engineering-led eval-as-code; SaaS-first eval-as-product; LangChain-stack-native bundled with observability; research-grade hallucination + simulation), and picking by capability matrix produces the wrong procurement outcome for most enterprises. The structurally load-bearing eval-vs-observability split (companion piece AM-123) compounds this: 'is the agent right' and 'what did the agent do' are different procurement decisions answered by different platforms.

About this register

The Reporting register tracks claims published from articles addressed to senior enterprise IT leaders — CIOs, IT directors, heads of platform. Claims are reviewed on a 30–90 day cadence; each review either reaffirms the claim, marks one substantive part as Partial, or marks it Not holding once the underlying evidence has been overtaken.

Recent corrections in Reporting

  • AM-132 · Partial · 10 Jun 2026

    One of four legs unanchored on re-review. The claim text attributes '12% of deployments clearing 300%+ ROI with 88% at or below break-even at 12-18 months' to the Stanford DEL 2026 Enterprise AI Playbook. Full-text verification on 10 Jun 2026 found no such figure in that source: the playbook (Pereira, Graylin, Brynjolfsson, Apr 2026) studies 51 successful deployments by design and contains no ROI distribution, no 300%-plus cohort, and no break-even measurement point (full finding at AM-029, correction of 10 Jun 2026). The only verified figure carrying the same 12/88 numerals is IDC research with Lenovo (via CIO.com, Mar 2025): roughly 88% of AI proof-of-concepts never reach production and roughly 12% graduate — a pilot-to-production graduation metric, not an ROI distribution. The Gartner 28%, McKinsey 23%/17%, and MIT NANDA 95% legs verify; they support a small high-performing tail and a large struggling body, but none documents the two-peak bimodal shape the claim asserts. Status Up -> Partial.

  • AM-129 · Partial · 10 Jun 2026

    One of three read-against anchors unanchored on re-review. The claim text cites 'Stanford Digital Economy Lab Enterprise AI Playbook (12/88 bimodal ROI distribution at 12-18 months)' and frames the realistic ROI band around 'the highest-discipline 12% cohort'. Full-text verification on 10 Jun 2026 found the playbook contains no 12/88 distribution, no bimodal ROI shape, and no 12-18-month ROI measurement point (full finding at AM-029, correction of 10 Jun 2026). The claim's core negative finding — no mid-market enterprise has produced a documented +240% ROI in 90 days under audited conditions — is unaffected; the McKinsey State of AI 2025 and MIT NANDA legs verify and continue to support it. The '12% cohort' framing has no verifiable referent. The only verified figure carrying the 12/88 numerals is IDC's pilot-graduation finding (roughly 88% of AI proof-of-concepts never reach production; via CIO.com, Mar 2025), a different metric. Status Up -> Partial.

  • AM-201 · Partial · 10 Jun 2026

    One of four named datasets unanchored on review. The claim text names 'Stanford DEL's 12% clearing 300%+ ROI vs 88% at or below break-even' as one of four independent datasets. Full-text verification on 10 Jun 2026 found the Stanford DEL Enterprise AI Playbook contains no such distribution — it studies 51 successful deployments by design and carries no ROI-realisation failure data (full finding at AM-029, correction of 10 Jun 2026). The McKinsey (23% scaling, 17% EBIT-attribution), Gartner (28% fully paying off), and MIT NANDA (95% no measurable P&L impact) datasets verify; the claim's spine stands on three datasets rather than four. The only verified figure carrying the 12/88 numerals is IDC's pilot-graduation finding (roughly 88% of AI proof-of-concepts never reach production; via CIO.com, Mar 2025), a different metric from an ROI distribution. Status Up -> Partial.

Reviews coming up in Reporting

  • AM-063 · Holding · next +11d (27 Jun 2026)

    AI agents executing financial transactions need a four-control bundle (action-approval gates by blast radius, kill-swit…

  • AM-061 · Holding · next +11d (27 Jun 2026)

    Production agentic-AI costs at scale routinely run multiples of POC projections, and a layered optimisation programme c…

  • AM-003 · Partial · next +11d (27 Jun 2026)

    GPT-5 Pro's tiered-subscription model forces enterprises to classify problems by computational difficulty — $200/month…