Golden Data

Issue #5 · June 7, 2026

The press release became a split test.

Paul Brown Writes Golden Data from Ogden, Utah 6 min read

Every Sunday I look at the press release a vendor would not have written this way a year ago. This fortnight the entire B2B SaaS cohort wrote the same one. A controlled-experiment lift number, attached to the AI feature, published in the earnings letter.

Pinterest Performance+ posted a 24% conversion lift and an 80% A/B win rate on its Q1 print. Atlassian disclosed Rovo customers growing ARR at twice the rate of non-Rovo customers. Reddit Max reported a 17% drop in cost-per-acquisition and 27% more conversions in split tests. Anthropic shipped Opus 4.8 on May 28 as a benchmark delta against Opus 4.7. CrowdStrike printed $256M of net new ARR on June 3 and framed it as the AI-tagged number.

Same disclosure shape, different vendors, same fortnight. The press release became the place where the control group lives. The buyer’s procurement team got a comparable bar the AI feature on your roadmap now has to fill.

What’s actually shipping this week

1. Pinterest published Performance+ as a 24% conversion lift. Q1 2026, reported May 4. Revenue $1.008B, up 18% year over year. Monthly active users 631 million. Performance+ now carries roughly 30% of lower-funnel revenue, with a retrained ROAS core model lifting returns 11% in experimentation. The disclosure is the news: a 24% conversion-lift figure and an 80% A/B win rate land in the shareholder letter, not a marketing blog. Ad platforms now print their own controlled-experiment numbers in the same place they print revenue.

2. Atlassian reported Rovo customers growing ARR at 2x non-Rovo. Q3 FY26, April 30. Total revenue $1.8B, up 32%. Cloud revenue accelerated to 29% growth. Rovo monthly credit usage up more than 20% month over month. The published metric — Rovo customers expanding at twice the rate of non-Rovo — is the structural move. Atlassian stopped reporting AI ARR. They started reporting an AI-attributed seat-expansion multiplier. The CFO put the lift number where the seat count used to be.

3. Reddit Max campaigns reported 17% lower CPA and 27% more conversions. Launched at CES 2026, broader beta rolling through Q2. Brooks Running’s Ghost 17 campaign discloses a 37% drop in cost-per-click and 27% more clicks across 21 days, with no manual changes. Community Intelligence estimates the value of each impression and reallocates budget without a media buyer. Self-serve ad platforms moved from feature pitches to controlled-experiment case studies as the marketing material.

4. Anthropic shipped Opus 4.8 as a benchmark delta sheet. May 28. Available across Claude Pro, Max, Team, and Enterprise, plus Amazon Bedrock, Google Vertex AI, and GitHub Copilot. Pricing held at $5 per million input tokens and $25 per million output. Dynamic Workflows fans a single session into hundreds of parallel subagents across codebase-scale migrations. The release notes lead with capability deltas against Opus 4.7. The foundation-model vendor publishes in benchmark language because that is the language enterprise procurement now reads.

5. CrowdStrike framed $256M of net new ARR as the AI-tagged number. Q1 FY27, June 3. $5.25B ending ARR — the fastest cybersecurity vendor to that bar. Net new ARR up 32% year over year. Charlotte AI AgentWorks Ecosystem launched March 25 at RSA with Anthropic, OpenAI, AWS, and NVIDIA as model partners. Falcon AIDR went GA. The deck framed the net new ARR as AI-eligible seat expansion. The security vendor crossed the install-base vendors and named the same metric.

What I’d ship in your app this week

The five prints set the shape. Two simple tools your team can ship in two weeks so the AI feature on your roadmap can answer the same procurement question.

Feature one: the holdout-cohort A/B harness, wired into the AI-feature rollout. Pinterest’s 24% lift and Reddit’s 17% CPA drop are publishable because the split test was instrumented from day one. Bake the same shape into your launch. At flag-up, hold 15% of eligible seats in the pre-AI workflow as a control. Roll up the primary metric weekly on each cohort. The CSM team has the per-tenant lift before the customer asks.

  • Shape. Feature flag at the seat level. Treatment seats use the AI surface. Control seats use the legacy workflow. Same product, two surfaces.
  • Data shape. One row per (tenant, seat, week) with cohort flag, primary metric, and three covariates. Lives next to the existing analytics warehouse table.
  • Latency budget. None on the feature. Daily roll-up under five minutes. CSM-facing report lands Monday 6 a.m.
  • Cost ceiling. Storage only — under $40 per month for a 50,000-seat product. Zero new model spend.
  • Eval. 30 control-vs-treatment scenarios reviewed monthly with a senior analyst on "would this lift survive a procurement-team challenge."
  • Instrumentation. Alert when the lift narrows into the noise band. That is the day the feature stopped working and the day procurement starts asking.
  • Two weeks in. Your CSM team quotes the per-tenant lift in three customer reviews. If they don't, the metric was the wrong one — pick one the customer's CFO already tracks.

Feature two: the per-tenant ROI receipt, on the QBR slide. Once the cohort table exists, the artifact the CSM team needs is the per-customer version. Acme’s deal velocity went up 18% on the seats using your AI feature versus the seats that didn’t. One PDF per tenant per quarter. The number is on the QBR slide before the customer asks whether the AI feature is paying for itself.

  • Shape. Aggregation query on the cohort table. One slide per tenant per quarter. No LLM call.
  • Data shape. Per (tenant, cohort, quarter) lift on the primary metric, with a confidence interval.
  • Latency budget. 90 seconds to render. The CSM kicks it off during pre-QBR prep.
  • Cost ceiling. Under two cents per receipt. The constraint is the metric definition, not the compute.
  • Eval. 40 senior-CSM-reviewed receipts per quarter, scored on "is this the number the customer will quote in renewal."
  • Instrumentation. Track whether the customer cites the lift number in renewal call transcripts. That is the conversion event you actually wanted.
  • Two weeks in. Three customers have the per-tenant lift number in their own board deck. If they don't, the AI feature isn't moving a number their board cares about — fix the metric, not the receipt.

Both ship in two weeks with the team you have. Both pull a controlled lift number out of the AI feature and put it in front of the people who decide renewal. The press releases this fortnight set the shape. The vendors that aren’t running controlled experiments will be the ones explaining the AI feature on the next renewal call with vibes.

Sources

Send me an email and we will talk. If something here landed close to what you're working on, the door is open. No calendar funnel, no pitch deck — I read every note that comes in.

Doing the work rather than deciding what to build? Crafting is the column for that chair.

Email me Read more Signal

← All Signal issues · Drafted with Claude · Edited by Paul Brown