Which problems matter most, and how will we know we solved them?

01 Principles

1. 01

   Decisions, not dashboards

   Every screen starts from one decision-maker and one decision. If nobody would decide differently after seeing it, it doesn’t ship.

2. 02

   Show the edges of the model

   Each recommendation links to a model card covering what the model sees, how well it has worked, and what it cannot see. Expert users trust tools that are honest about their blind spots.

3. 03

   Match the decision’s clock

   A catcher has the second after the pitch, with no help from the dugout, and a GM has six weeks. Output takes the form the moment allows: a rule, a one-page card, a ranked list, or a scenario model.

4. 04

   Receipts on every number

   Source, sample size, and “as of” date travel with the number. Small samples are labeled, never hidden.

5. 05

   Measure decision quality, not clicks

   Usage is a leading indicator. The outcome is better decisions: runs, wins, and surplus dollars, judged against a counterfactual.

02 The decision map

Baseball decisions run on very different clocks. The clock decides what the product looks like. A catcher needs a rule. A GM needs a scenario model. Each column is a cadence. Dots show stakes.

03 Personas

Six people with different clocks, incentives, and tolerance for uncertainty. For each: the decisions they own, my hypothesis about their real constraint, the public evidence behind it, and how I would test it.

04 Prioritization

Score = Impact × Confidence × Urgency ÷ Effort, each on a 1–5 scale. The score isn’t the decision. Dependencies and deadlines move things: the Roster Builder scores high but waits for its inputs, and the Pitch Lab ships early because the Opportunity Finder reuses its model.

05 How we’d know it worked

Usage is a leading indicator. Decision quality is the outcome. Every module gets one outcome metric it can be held to, plus a guardrail that catches the obvious way to game it.

06 First 90 days

07 Decision log

Trade-offs made while building Keystone, with the alternatives we rejected, newest first. Decisions without a record get re-litigated. An entry a later decision replaced is marked and links to it.

1. 2026-10-22

   What changed after the October 2026 review

   An outside review read every page the way a front office would, and three rounds of fixes followed. One source for each fact: organization membership, the 40-man and injured lists come from one module; the Winter Plan’s moves, roles and dates from one decisions feed that every page reads; and one price of money ($9.0M a win, today’s rules and the owners’ proposal, with the proposed floor shown at both ends of its shadow price). Honest projections: the win engine hands out exactly one season of team playing time, the browser and the pipeline are checked to agree, and team projections carry an 80% range. An honest record: every Track Record verdict now follows a written rule and shows its noise. The opportunity-flag test was re-run on Stuff grades that never saw the season they forecast, the rookie, sequencing and health rows were re-graded, year-to-year repeat rates stopped counting as forward tests, the dead-weight test’s flags were re-picked with Stuff that never saw 2026 before it counted as one, and forward-tested passes fell to 13 of 32. The Opportunity Finder now ranks by the wins a player adds to the Winter Plan’s roster; skill signals explain a flag but don’t set the rank. A traded player with a major-league record is valued as a blend of FanGraphs’ prospect table and our surplus engine, weighted by how much of his projection is his own record; a prospect with no major-league time still uses the table as published. The Season Review stopped leading with one record: BaseRuns and Pythagorean predict equally well, and records like 2026’s have meant about 83–86 wins the next season. Each fix became a build check: 472 of 472 guardrails pass.

   Rejected: Fix each finding in place with no shared source; publish the review and leave the pages as they were

2. 2026-10-22

   Pitch order retired as a coaching target; matchups get the platoon split; running calls rest on three seasons

   A placebo test, swapping the real previous pitch for a random one of the same type from another at-bat, reproduced the whole tunnel effect, so the Sequencing page no longer makes pair calls: what goes with whiffs and chases is how closely a pitch tracks the pitcher’s other pitches, a pitch-design input. The Matchup Engine had shrunk each hitter’s platoon split toward zero and got same-side matchups backwards; shrinking it toward the league’s split fixed the direction and beat the pitch-type layer in the forward test, and the bullpen grid now shows the Winter Plan’s 2027 relievers and each arm’s fit beyond his usual edge. The Running Game’s calls had come from one season of Savant’s partial counts and reversed on three seasons of official records; success is now regressed with a prior measured from how it repeats, both calls are forward-tested (runners regressed just under break-even ran above it the next season, so a below-break-even flag now needs a shortfall of two standard deviations, and nobody has one; fast runners who rarely run but whose own record was under break-even succeeded at about break-even the next season, so a green light now needs his own record to clear it as well, and those runners are toss-ups with no run value), and the page adds the one-run break-even, bunt success needed, and the 2027 roster on the bases.

   Rejected: Keep pair calls with a caveat; an additive platoon term fit on outcomes; keep single-season Savant counts with a wider interval

3. 2026-10-22

   Split rival pitch-type splits into the club’s level and each pitch against that level, and test both a season ahead

   A staff’s split against the league mostly said how good the whole staff was, and a lineup’s pitch-type pattern barely repeated from one season to the next (r .20, then .01). Rival Intel now leads with each club’s level and its 2027 projection, shows the pitch to sit on against the staff’s own average with its usage and its weak year-ahead persistence, fades cells thrown mostly by players who are leaving, and drops the matcher that could never fire.

   Rejected: Keep one layer against the league; rebuild the splits on each club’s 2027 roster (needs depth charts for every rival)

4. 2026-10-22

   Park fit leads with the effect against an average park, with noise-shrunk factors and a spray-aware home-run factor

   Every WAR on the site is already park-adjusted, so the effect against a hitter’s 2026 park answered the wrong question, and it used one club’s park for hitters who had played in several. The page now adds park fit to the Winter Plan’s branches, both sides of the ball, and tests it on hitters who changed parks.

   Rejected: Against his current park only; a spray-aware model without anchoring to Savant’s index; shrinking every batted-ball bin straight toward the league (it erased most of PNC’s own level, so the prior now shrinks bins toward the park’s own pull-direction regions)

5. 2026-10-22

   Hitting Lab diagnoses carry their uncertainty, their cause, and the matchup

   Tags were shown as certain when some were coin flips, the contact tag sent coaches to bat-to-ball work for hitters whose problem was damage, and platoon-neutral values were read as “handles lefties”. Each estimate now has an SD and a chance of being past the line, tags name their cause, and the outside list adds the league’s value for each batting side back. Decision value scores each choice against the alternative, so it counts every run about twice; every runs figure now uses a factor measured each build (actual run value regressed on both values, about half a run per unit), which roughly halved the decision runs and reordered the outside list toward contact and handedness.

   Rejected: Hide the vs-hand view; rank targets by skill alone; a fixed factor of one half (the measured one is close and is checked every build)

6. 2026-10-21

   Judge gloves on three seasons, and compare positions fairly

   One raw season of fielding runs is a weak guide to the next. Weighting 2024–26 as 5/4/3 and regressing about a full season toward average cuts the error by about a third against that, but the Track Record later showed most of the gain is the regression: one season regressed the same way does nearly as well for fielders and slightly better for catchers. The Defense page ranks upgrades on that, adds the standard positional adjustment so a shortstop and a first baseman compare, and credits each run to the position where it was earned, which surfaced a shortstop leak that filing everything under a player’s main position had hidden. The depth chart now counts arm, double-play and first-base receiving runs, not range alone.

   Rejected: Single-season fielding runs with a caveat; OAA only; a club-built range model

7. 2026-10-21

   Pitching tools held to one standard: forward-tested, side-aware, and gated like the plans

   Superseded by Pitch order retired as a coaching target; matchups get the platoon split; running calls rest on three seasons

   The Pitch Lab advised mix changes against every hitter when the replacement only worked against one side, suggested new pitches to arms with six already (or ones they’d thrown for years), and claimed Stuff “predicts best” from a test that let 2025 grades peek at 2026. Now mix changes are by batter side and keep fastballs at 35%+, new pitches need 15+ comparable pitchers, a short arsenal and a 5-point edge over the pitch they’d replace, and a time-ordered test shows Stuff predicts next season about as well as results do, Location predicts walks but not runs, and Pitching adds nothing going forward. The tunnel effect shrank to a third once location and movement were controlled (a later placebo test erased the rest), and a season-end velocity drop showed no sign of predicting winter injuries, on too few injuries (2) to tell.

   Rejected: Keep cross-fit validation with a caveat; retrain the production models; drop pair-level sequencing

8. 2026-10-21

   Strategy tools sized and shrunk: matchups, ABS, splits and the running game

   Superseded by Pitch order retired as a coaching target; matchups get the platoon split; running calls rest on three seasons

   The Matchup Engine overstated edges about two-fold; with shrinkage measured from the data and calibrated on 2024 → 2025, pitch type adds almost nothing beyond overall quality on 2026 (r .080 vs .079), so it’s labeled a tiebreaker. Batter challenge success doesn’t repeat (r −.05), so there’s no batter green list; catchers are regressed and judged on the same break-even test as the count grid. Hitting Lab diagnoses use regressed platoon splits instead of raw 100-PA ones. Every running call carries its value in runs (about 0.6 a season in all: a small lever), and the Park page flags pull-power right-handers PNC will cost.

   Rejected: Raw splits with sample warnings; keep the batter green list; hand-set shrinkage

9. 2026-10-21

   Call-up and prospect ratings scored without leakage, and prospects measured against their own league

   The rookie test behind rating thin-record hitters by AAA contact quality had scored a model on the players it was fit on. Held out properly, no rating beats “an average rookie”: contact quality and prospect grades both rank rookies only a little (r = .14 and .11). Held-out fits put 55–65% on AAA contact quality, so the depth chart moved from 75/25 to 60/40. Translated AA and High-A lines now count for about 38% of their distance from average, what carried over on the blind test; Perf+ compares each line with its own league first (the PCL and Cal League inflate offense); draft values subtract what a club pays in the control years. The Prospect Pipeline shows each prospect’s trade value and the plan’s Rule 5 and trade calls.

   Rejected: Keep the 75/25 weight; add age to the readiness model (no held-out gain); value picks gross of salary

10. 2026-10-21

    Arbitration builds on the whole career; tender on the role; the plan buys wins, not control

    Separate per-year arbitration fits made star paths jump in the third year (Skenes $10.4M → $21.9M); one pooled fit with a career-WAR term cut held-out error from $1.11M to $0.99M, and the Skenes call moved from “go year to year” to “open talks at five years.” Tender value is now role value plus the upside of later arbitration years, so a third catcher who opens in Triple-A is worth his call-up time: Henry Davis is shopped, then non-tendered, where the brief had said “tender everyone.” And a hard preference for multi-year control had let a +0.6-win player beat a +3.2-win bridge after a small rating change; the plan now buys the most 2027 wins, with control only breaking ties within 0.3 wins, because the prospect price already pays for future years.

    Rejected: Per-year arbitration fits; full-season tender value; always prefer control

11. 2026-10-21

    Split the projected free-agent pool by what each club spends, not evenly

    The Outlook used to hand every club the same share of the wins still unsigned, so a $250M payroll and a $70M payroll gained the same. Payroll predicts free-agent spending (tested on the 2025–26 winter), so each club now gets half its payroll-predicted share and half an even share. Every club’s odds and availability labels shift a little; Pittsburgh’s own signings stay in the Winter Plan.

    Rejected: Even split; last winter’s actual spending (one winter is too noisy to copy directly)

12. 2026-10-21

    Show every player’s 2027 status on his page, from the same sources as the Winter Plan

    A player page said nothing about where he fits next year, so it could quietly disagree with the plan. Each page now shows his projected role on his club’s depth chart, Pittsburgh’s plan for him if it acquires him, his free-agent price, his latest injury, and how available he is. A guardrail checks that every 26-man player shows the same role and WAR as the plan.

    Rejected: Link to the plan without a status line; a status line for Pittsburgh players only

13. 2026-10-21

    Shrink rival pitch-type splits toward zero by their measured reliability

    Split-half reliability is low (lineups 0.36 a season, staffs 0.64), so most raw splits are noise. Each split is now shrunk by its own sample size and marked clear only when it beats 1.5 standard errors. Most “weaknesses” fell inside the error, and the page now says so rather than naming an exploit.

    Rejected: Raw splits with a minimum sample; hide the splits entirely

14. 2026-10-21

    Label every Track Record test by design, with intervals, and count only forward tests on /start

    Superseded by What changed after the October 2026 review

    “Tested out of sample” mixed time-ordered tests with held-out fits, benchmarks and in-sample checks. Each claim now carries its design; persistence correlations carry 95% intervals; the dead-weight test needs its gap to clear two standard errors; Location’s run forecast and the season-end velocity warning are now graded on time-ordered tests (both failed). /start counts only forward tests.

    Rejected: One blended pass rate; drop the in-sample checks

15. 2026-10-21

    Guard the summary pages against drifting from their sources

    The Command Center once showed Washington’s free-agent list after the 30-club export overwrote Pittsburgh’s summary, and summary pages can quote an older plan or Outlook. Six guardrails now check that the Command Center, /start, the Outlook and player pages match the plan, the Outlook and the Track Record.

    Rejected: Manual review before each deploy

16. 2026-10-21

    Reorganize the site around how a front office decides: a Run prevention section

    Defense had grown from one page into four tools (team defense, player gloves, catching, bullpen), and it sat under “Game strategy” with lineups and challenges. GMs think in decision areas: what the roster costs, how we prevent runs, how players develop, how we play games. Six groups of four to seven, each answering one kind of question.

    Rejected: Keep one long Defense page; group by data source

17. 2026-10-21

    Build our own expected-outs model for the team map, but grade individuals with Savant OAA

    The model separates easy plays from hard ones well (Brier 0.097–0.106 by season vs. 0.21 with no model) and maps where a team leaks outs by zone, which OAA doesn’t. As an individual grade it agrees with OAA in the outfield (r ≈ 0.7) but not at shortstop (r ≈ 0.08–0.36), because a zone can’t know where the fielder started. So: our model for “where,” Savant’s for “who.”

    Rejected: Savant OAA only (no zone map); our model for everything

18. 2026-10-21

    Trades cost prospects, everywhere, and the builder can search for the best roster

    A trade target’s surplus value is a price, and somebody has to pay it. Every trade in the Roster Builder now names the package its club would accept: the fewest of our prospects whose value covers his surplus plus a 10% premium (at least half his 2027 on-field value for an overpaid regular), with the top prospects protected by default. A prospect on the 40-man leaves it, so the projection pays for him too. On top of that, a Maximize-wins search finds the best set of additions within a budget, free agents only or trades too, and shows the prospect capital it spends; players can be promoted to or moved off the 26-man to see what a depth-chart choice is worth. The search is greedy and says so: it doesn’t model other clubs bidding or players’ willingness to sign.

    Rejected: Surplus value with no package; a full trade-market simulation; exhaustive search (too slow for the browser at 150+ candidates)

19. 2026-10-21

    Development plans held to what a coach would sign off on

    An outside review found plans asking for things no development staff would: pitch-shape goals that were really the Stuff model grading seasons with different folds, plans for players leaving as free agents, pitch goals for pitchers still on the injured list, chase targets 15 points away, and goals that contradicted the Hitting Lab. Now one model grades every season, changes are tested game by game, injured players get medical clearance first, targets are the smaller of the median, his own best and a big-but-real one-year gain, and pitch-mix changes are by batter side. Plans got shorter; each goal on them is one we’d defend.

    Rejected: Keep percentile targets with caveats; drop development plans; leave goal-setting entirely to coaches

20. 2026-10-21

    Measure adoption, not just accuracy

    A recommendation nobody acts on is worth nothing, and the realism misses showed the risk: advice a coach would reject. Every development goal now carries an ask level (low: a game-plan change; medium: practice and targets; high: mechanics or a new pitch), a coach version in plain language with how we’ll know it worked, and Accept / Modify / Not now buttons with a reason, on coach cards and Decision Briefs. The Adoption page shows how much the plans ask: only a sliver of goals are low-ask, so the staff’s judgment matters most where the models push hardest. Every miss we found also became a build-time guardrail, so the fix for one player is checked for every player.

    Rejected: Accuracy metrics only; a survey after the season; a full feedback backend before validating the workflow

21. 2026-10-21

    Pitching plans aim only at what a pitcher has already shown

    The Stuff Landscape was turning into advice like “add 3 inches of ride,” and a reviewer pointed out what any pitching coach would: pitchers are already throwing with as much movement and velocity as they can, and nobody can promise more. Plans now target velocity a pitcher has thrown before (his best 2024–25 season, net of the normal change for his age) or a pitch shape he has thrown before, and only when the old version graded better by more than an unchanged pitch’s grade drifts between seasons (95% of the time, 11 Stuff points). Arm-angle changes are named as a possible cause, not a prescription. The landscape stays on player pages as a map. Keller’s brief changed with it: his ride held at 15 inches every year, so the case for keeping him is timing and pitch mix, not a fix. The new headline finding came out of the same check: Skenes’s fastball is 2 mph below 2024.

    Rejected: Keep landscape targets with a caveat; drop shape goals entirely; league-percentile velocity targets

22. 2026-10-21

    Ask whether a club would actually trade him, and compute what scouts used to be asked

    Surplus value says what a player is worth, not whether he is for sale: Munetaka Murakami (just signed) and Maikel Garcia (just extended) were showing up as cheap targets. Each trade target now gets a plain-rules availability read with the reason shown (just signed or extended, young controllable core, a contender’s regular → unlikely; a seller’s veteran or short-control player → likely). Unlikely players are hidden by default and kept out of the Winter Plan. That changed the plan’s third-base move: Garcia was off the table, the next option (Jordan Westburg) had missed 2026 on the injured list, so the rule now also requires a healthy 2026 and accepts a one-year bridge when no controllable option fits. The plan now acquires Isaac Paredes from Houston for 2027. Free-agent prices carry the range comparable deals landed in and a flag when October or an option changes the market. The generic “what scouts must confirm” lists became computed checks (where he chases and what fixing it is worth, whether a split held up, BABIP and strand-rate luck); scouts keep the questions numbers can’t answer.

    Rejected: A model of 30 front offices’ trade behavior (no public training data); manual tags; leaving it to the reader

23. 2026-10-21

    Any club in the analysis modules; Pittsburgh stays the focus

    The data was already league-wide (rosters, contracts, Statcast, prospect grades for all 30 clubs); the code assumed Pittsburgh. Thirteen modules now follow a club picked in the sidebar, each measured against that club’s roster as it stands through the same depth-chart engine as the Winter Plan. The Winter Plan, the briefs and the development tools stay Pittsburgh’s: they encode one front office’s decisions, and a generic version would read like a template. Pittsburgh’s pages are unchanged; other clubs’ files are built in the same pipeline run.

    Rejected: Pittsburgh only; a separate site per club; generic auto-written briefs for every club

24. 2026-10-21

    Pair our walk-away with what comparable players signed for

    The extension calculator only knew our side: the highest AAV that beats going year to year. A model fit on 84 extensions signed 2021–26 adds the other side (miss $3.7M a year vs. $8.8M for $9M per WAR): control years sell for under half of what free-agent years do. Put together, most of Pittsburgh’s young players are worth more year to year at comparable prices; Oneil Cruz (buy one free-agent year) and Carmen Mlodzinski are the deal zones, and Skenes sits near break-even. A log-WAR curve fit a hair better but under-priced stars, so the price rises linearly with projected WAR.

    Rejected: Our walk-away only; hand-picked comps; MLBTR’s extension tracker as a list

25. 2026-10-21

    Rate thin-record hitters from AAA contact quality, not just their prospect grade

    Superseded by Call-up and prospect ratings scored without leakage, and prospects measured against their own league

    The depth chart rated players with little MLB record by their Future Value grade. Tested on 108 hitters who went from AAA in 2025 to the majors in 2026, that rule had no signal and rated them about a win per 600 PA too low; AAA expected wOBA did better. A 75/25 blend had the lowest error on that test, so the engine used it. We also published the failures: a translated AAA line barely beats guessing league average, and nothing we have predicts pitcher readiness.

    Rejected: Keep the grade rule; translated lines (harsh and noisy); scouting grades only

26. 2026-10-21

    Price free agents from last winter’s real deals, one price site-wide

    The site priced free agents at $9M per projected WAR with years by age bracket. Fit on last winter’s 143 deals (each player described by a projection made before he signed), a small model cuts the miss in annual value and misses contract length by less (the Track Record has the numbers; on length, “always one year” is exact more often, since most deals are one year). It also showed what the rule missed: relievers cost the most per win, and a big walk year raises the price beyond the projection. The Opportunity Finder, Roster Builder, Free-Agent Board and the qualifying-offer brief now all use this one price. A quadratic fit scored slightly better but bent the wrong way above last winter’s range (an ace priced below a 3-WAR starter), so we took the shape that extrapolates sanely.

    Rejected: Keep $9M/WAR; copy published crowdsource or MLBTR estimates (not reproducible, not available for every player)

27. 2026-10-21

    Lead the season review with BaseRuns, not contact quality

    Superseded by What changed after the October 2026 review

    The obvious “deserved” record values every batted ball at its expected wOBA. Tested on 150 team-seasons, it predicted next season no better than plain BaseRuns, because a team’s results-vs.-contact gap partly repeats: park, team speed and defense live in it. So the page led with BaseRuns and showed contact quality as a check.

    Rejected: xwOBA-based “deserved wins” as the headline; actual record with a luck footnote

28. 2026-10-21

    Call positioning a small lever, and say so

    Pittsburgh shades pull hitters less than the league (27% vs. 32% against heavy pullers), and shading converts slightly more ground balls. But matching the league is worth about a third of a run a season. The third-base zone cost 15 runs. Leading with positioning would point the staff at the wrong problem.

    Rejected: A positioning module with batter-by-batter cards as a headline feature

29. 2026-10-21

    One roster for every recommendation: the Winter Plan

    A reviewer caught two briefs that each claimed second base: the qualifying-offer brief kept Lowe there while the third-base brief moved Gonzales there. An audit found four more collisions: protecting three Rule 5 prospects took the 40-man to 41; the bullpen brief named a reliever who had left and skipped our second-best starter; each brief spent the same budget alone; and the Roster Builder started from 22 players, not the 40-man. Now every brief reads its lineup, rotation, 40-man math and payroll from one plan, outcomes we don’t control are branches, and the build fails if a brief names a player the plan doesn’t have.

    Rejected: Fix the two briefs by hand; a review checklist; one giant brief

30. 2026-10-21

    Fielding priors follow the defensive spectrum, and moving a regular costs 3 runs

    The first depth chart moved Cruz to shortstop and the franchise shortstop to third, and called Horwitz an average second baseman, because thin data was regressed toward an average fielder at the new spot. A second baseman shouldn’t be presumed an average shortstop: he starts 7 runs below one (the 5-run gap in positional value plus 2 for an unfamiliar spot). With that prior and a cost for unfamiliar moves, the plan stopped proposing moves no club would make.

    Rejected: Regress to average at every position; forbid position changes entirely

31. 2026-10-21

    A linear role model for playing time, used everywhere

    It matched a gradient-boosted model on 2022–26 (hitter WAR error 1.312 vs 1.319) and is simple enough to run in the browser, so the Winter Plan and the Roster Builder agree to the decimal. The boosted model was slightly better for team wins (r 0.666 vs 0.653); one model the GM can read beat a marginal gain we would have to explain away.

    Rejected: Gradient boosting (two models); Marcel’s playing-time formula

32. 2026-10-21

    Re-value pre-2026 catcher framing at 57%

    Under the ABS challenge system the spread of framing among regulars fell to 57% of 2024–25, and four-fifths of that came from umpires calling a tighter zone, not overturned calls. Projections that kept counting 2024–25 framing at full value overrated framers (Patrick Bailey went from +32 runs to +5). Re-valued, the 2026 catcher forecasts improve over framing as published. Dropping old framing entirely scored a hair better on the same in-sample test (0.014 WAR on 98 catchers), but 2026 measured framing at 57% of its old spread, not zero, so 57% stays; 2027 is the first blind test.

    Rejected: Keep FanGraphs WAR as published; drop framing from projections entirely

33. 2026-10-20

    Don’t blend Statcast skills into the projections (yet)

    We downloaded 2021–23 tracking so the blend could be fit on earlier seasons and scored on 2024, 2025 and 2026. It had to beat the plain projection overall and in most seasons. It didn’t: hitters 1.342 → 1.343 WAR RMSE, pitchers 0.922 → 0.924. A quarter to two-fifths of the error is playing time, and most of the rest is one-season noise. Two data fixes came first: shrinkage pooled across seasons (per-season estimates swung 17×), and pitchers who batted before the 2022 universal DH removed from the hitter pool. Neither fix changed the answer.

    Rejected: Ship the blend because it sounds better; keep tuning features until it wins (overfitting the three test seasons)

34. 2026-10-20

    Credit every club, Pittsburgh included, with call-ups and depth in the Outlook

    Today’s rosters fall well over 100 wins short of the 2,430 the league will play (the Outlook shows the current figure). Unsigned free agents explain most of it; the rest is call-ups, rookies and depth every club gets. We had been giving all of it to the other 29 clubs, which understated Pittsburgh by about a win. Free agents still go only to the others, because Pittsburgh’s signings are modeled in the Roster Builder.

    Rejected: Spread the whole gap across the other 29 clubs (the old way); spread it across all 30 (double-counts Pittsburgh’s signings)

35. 2026-10-20

    Keep three seasons of history in the projections

    We forecast 13 seasons blind (2012–19, 2022–26) with one to five seasons of history. Going from one season to three cuts hitter error from 1.26 to 1.21 WAR; a fourth or fifth season moves it by less than 0.005. More history adds data to maintain, not accuracy.

    Rejected: Five or more seasons; last season only

36. 2026-10-20

    Measure hitter aging from the data; keep the textbook table for pitchers

    A curve fit only on seasons before each forecast beat the table in 13 of 13 seasons for hitters: they decline faster after 30 than the table assumed. For pitchers it won 4 of 13, so the table stays. One curve for everyone would have been simpler and slightly worse.

    Rejected: One measured curve for both; keep the table for both

37. 2026-10-20

    Publish the misses: a Track Record page with every claim, its test, and its verdict

    At launch, fifteen claims held, nine partly held and four failed. Showing the failures (Location doesn’t add to Stuff for predicting runs; a starter’s own third-time penalty isn’t a skill; bat-speed drops don’t predict injuries; Statcast skills didn’t improve WAR projections) is what makes the passes believable, and each failure changed the product.

    Rejected: Report only validated results

38. 2026-10-20

    No estimated data: if public data can’t support an analysis, it goes on the “blocked by data” list

    Biomechanics, medical, low-minors tracking, amateur data, international bonuses and club finances would each make the product better, and each would be a guess here. A club has them; the product shows where they’d plug in.

    Rejected: Fill gaps with assumptions; omit the gaps silently

39. 2026-10-20

    Calibrate before displaying: matchup edges are halved, lineup runs scaled, win curve built from measured uncertainty

    Superseded by Strategy tools sized and shrunk: matchups, ABS, splits and the running game

    Raw matchup edges were twice the size of what happened; the run model under-scores without steals and wild pitches; team projections miss by ~9 wins. Each number on screen carries the correction the backtest found.

    Rejected: Show raw model output

40. 2026-10-20

    Development plans come only from model outputs, with percentile-based targets

    Superseded by Development plans held to what a coach would sign off on

    A plan a coach can’t trace to evidence won’t get used. Every goal links to the module that produced it, and goals that wouldn’t improve on today are dropped automatically.

    Rejected: Free-text coaching notes

41. 2026-10-20

    Trade values use FanGraphs’ published prospect table as published

    Superseded by What changed after the October 2026 review

    Without historical prospect grades we can’t fit our own FV-to-dollars curve. Citing the best public one (and checking it reproduces their Pittsburgh farm total) beats inventing one.

    Rejected: Our own guess at prospect values

42. 2026-10-20

    A 90-second “Start here” path: six findings, each a deep link that lands on the exact panel

    A busy evaluator decides in the first minute whether to keep reading. Leading with findings (not features) and letting each one open its evidence respects that time and shows the work is checkable. The tour rides on normal pages with a small bar, so nothing is a separate demo.

    Rejected: A modal walkthrough with spotlight overlays; a video

43. 2026-10-20

    Hooks follow the batting order, not the pitch count; individual times-through penalties are shrunk to almost nothing

    Adjusted for pitcher and hitter quality, the penalty is a step at each turn of the order, and the step showed up in all three seasons. A starter’s own third-time penalty barely repeats year to year (r ≈ 0.07), so a pitcher-specific hook would chase noise.

    Rejected: Pitch-count limits; raw TTO splits per pitcher

44. 2026-10-20

    Comps carry a price tag; zone maps show difference from league, not raw values

    A comp is most useful when it answers “who else plays like this, and what does he cost?”, so each comp shows 2027 status and flags free agents. Raw zone run values mostly show where pitches are balls; subtracting the league cell for the same batter side isolates the hitter’s skill.

    Rejected: Comps by results (WAR/wOBA); raw run-value heatmaps

45. 2026-10-20

    Decision Briefs: every deadline gets a one-page memo with a single owner, a confidence level, and what would change our mind

    Exploration tools don’t make decisions; memos do. Generating every figure from the pipeline keeps the memos honest as data changes. Writing them surfaced two calls the dashboards had hidden: Gonzales was about average at second (0, +2 and +3 OAA there in 2023–25, then −11 at third in 2026) before he moved to third, and Lowe’s elite 2026 glove followed a −14 OAA season.

    Rejected: Static write-ups; slide decks rebuilt by hand each week

46. 2026-10-20

    Added Defense & Run Prevention after the v1 scope

    Run prevention is the GM’s stated first priority, and fielding was the largest gap Keystone didn’t cover (−20.5 runs, 25th). Multi-year OAA by position separates “wrong player” from “wrong position.”

    Rejected: Fold defense into Opportunity Finder as one more column

47. 2026-10-20

    Every metric explains itself: plain definition, a data-backed scale, and the hovered player’s own number

    Decision-makers shouldn’t need a glossary open. Scales come from the 2026 data (90th/50th/10th percentile), not memory, so they stay true as the data updates. One shared glossary keeps all ~135 definitions consistent.

    Rejected: Definition-only tooltips; a separate glossary page

48. 2026-10-20

    Opportunity score = skill − acquisition cost, and an overpay counts as a cost

    Superseded by What changed after the October 2026 review

    The first ranking surfaced superstars on long contracts. “Undervalued” has to mean skill per dollar, and absorbing negative surplus is a real cost.

    Rejected: Rank on skill signals alone

49. 2026-10-20

    Roster Builder counts only the best 13 hitters and 13 pitchers

    Superseded by A linear role model for playing time, used everywhere

    Summing every added player’s WAR overstates wins. Displacement is how a roster actually works.

    Rejected: Sum all projected WAR

50. 2026-10-20

    Prospect Perf+ standardized within each league, with age-for-level alongside

    Raw minor-league lines aren’t comparable across levels or leagues. Below Triple-A only Florida State League parks are publicly tracked, so the product labels box-score signals as such.

    Rejected: Show raw slash lines

51. 2026-10-20

    Validate before we polish: year-over-year and backtests run as soon as 2025 landed

    Superseded by Pitching tools held to one standard: forward-tested, side-aware, and gated like the plans

    Expert users ask “how do you know?” first. The first year-over-year test said last season’s Keystone Stuff predicts this season’s results better than last season’s results do. That test let each season’s grades see the season they were scored on; the time-ordered test that replaced it shows an edge inside the noise.

    Rejected: Ship features first, validate later

52. 2026-10-20

    Frame the platform as a decision product, with Product Thinking as its own section

    The role is product management. The posting stresses discovery, prioritization, and making model limits clear.

    Rejected: A pure analytics showcase

53. 2026-10-20

    Neutral brand (“Keystone”), no club marks

    Avoids trademark issues and keeps the platform org-agnostic. Any club could be the focus org.

    Rejected: A club-inspired theme

54. 2026-10-20

    Build our own models instead of republishing public pitch-quality metrics

    Licensing, plus transparency: we can only write an honest model card for a model we built.

    Rejected: Embed FanGraphs Stuff+ directly

55. 2026-10-20

    Prerendered static site with per-page JSON; < 100 KB JS on first load

    “Zero lag” is a product requirement. Expert users abandon slow tools.

    Rejected: A client-side SPA querying APIs live

56. 2026-10-20

    ABS Challenge Coach pulled into v1

    Highest impact-to-effort ratio on the board. An MLB-worst rate is measurable and fixable before spring.

    Rejected: Rival Intel

57. 2026-10-20

    Hitting Lab moved from Phase 2 into v1

    Stakeholder priority. Production vs. LHP is a stated org need and also feeds acquisition fit.

    Rejected: Ship it after Prospect Pipeline

58. 2026-10-20

    Rule 5 decisions handled as a lightweight queue in the Command Center

    The Nov 17 deadline arrives before the full Prospect Pipeline would.

    Rejected: Wait for Phase 2

59. 2026-10-20

    Biomechanics and DSL tracking deferred; designed as plug-in slots

    There is no public biomechanics data at any level, and no public pitch tracking in the DSL (in the minors only Triple-A and Florida State League parks are tracked). Faking coverage would erode trust.

    Rejected: Infer mechanics from release-point proxies

60. 2026-10-20

    Pitch-type colors chosen by a colorblind-safety validator

    Savant’s familiar fastball-red / curve-blue / changeup-green anchors are kept, but red next to orange fails color-vision checks. Reordered until it passes.

    Rejected: Copy Savant colors exactly

08 Roadmap

Blocked by data

Wanted, and deliberately not faked. Each needs data a club has and the public doesn’t. Details.

• Medical · True injury-risk models
• Biomechanics · Whether a pitch design is physically safe for a given arm
• Workload off the field · Acute-to-chronic workload and recovery models
• Workload history · A test of whether an innings jump comes before arm injuries (today there is one measured jump and no season after it), and workload lines for pitchers who came up through the minors before 2025
• High-A and Double-A tracking · Stuff, Location and swing metrics for prospects in the two-level gap between Bradenton and Indianapolis, where push-or-hold calls get made
• Minor-league fielding · Glove ratings for prospects and minor-league depth (a utility infielder at third or short)
• Options and service time · Which demotions would expose a player to waivers
• Minor-league contracts · Who becomes a minor-league free agent in November (estimated today), and who must be re-signed to keep depth
• Rule 5 selections · A pick-risk model fit to real selections
• Other clubs’ draft-pick status · What a qualifying-offer signing costs another club
• Outside free-agent price forecasts · An outside benchmark for every price on the Free-Agent Board before the market sets them, starting with the prices the biggest calls rest on (Brandon Lowe’s, behind the qualifying-offer decision, and the floor list’s picks)
• Prior prospect grades for graduated players · Shopping lists that never release a graduated top prospect of any origin to make 40-man room (they protect first-round picks still under club control)
• Updated prospect grades · Grades that reflect the 2026 season (today’s are from before it), and graded values for this year’s draftees: until then each 2026 draftee carries a provisional value, Keystone’s draft curve at his pick scaled to FanGraphs’ table on the 2025 class, with a bootstrap range
• MLB ids for 13 board prospects · Their values in the trade pools and packages (Pittsburgh’s Neomar Urbina, a 35+ reliever in the DSL, is $1.5M on FanGraphs’ table) and their grades on /prospects and their player pages
• Amateur data · Valuing individual draft prospects, not just draft slots
• International amateurs · International signing strategy and pool allocation for the DSL pipeline
• Intended location · Separating a bad plan from a missed spot in the command model
• Club finances · The win curve in dollars: exactly what the 85th win is worth to Pittsburgh
• Makeup and fit · Every player decision
• Trade market · Which Trade Builder deals are actually available, and what a club would really ask for a target
• Challenge confidence · Whether challenging more would pay: the value of the overturnable calls clubs never challenged