Table of Contents generated with DocToc
- Unreleased
- Added — league-wide NCAA RAPM solver (
mbb_ncaa_rapm_league, #389) - Deprecated — the three NBA
*_v3loaders now read the production releases - Docs — NBA shifted loaders: the
seasonCOLUMN is the END year - New —
sportsdataverse.wexp: win-expectancy bake-off harness (NFL + CFB) - New —
load_nfl_ratings_weekly: per-week as-of NFL ratings vintages - New —
sportsdataverse.scrape.espn: shared ESPN-rawarchive engine
- Added — league-wide NCAA RAPM solver (
- 0.0.75 Release: August 2, 2026
- Fix —
scrape.ncaaCLIs pointed at the wrong repo root (silent no-op) scrape.ncaa.parse— the parse stage is now re-runnable (--season,--force)- New —
sportsdataverse.scrape.ncaa: shared stats.ncaa.org hoops sweep engine sportsdataverse.scrape.stats— league-parameterized capture layer (Phase 2)- Docs — offline full-text search on the documentation site
- CI — docs site builds on GitHub Actions and publishes to
gh-pages - New —
sportsdataverse.scrape.stats: shared stats.nba.com / stats.wnba.com sweep engine (Phase 1) - CFB — model suite retrained on the corrected corpus (BREAKING model change)
- CFB — model cards ship beside every artifact
- Fix —
- 0.0.74 Release: August 2, 2026
- CFB — the ridge opponent adjustment was a no-op (BREAKING rating change)
- CFB — ESPN's
-1end-of-play yardline sentinel corrupted 2016 week 2 - CFB —
fill_null(0.0)is a silent no-op on booleans, pinningrushing_power_rateat 1.0 - Also in this release
- CFB —
opportunity_runcorrected, un-degeneratingopp_highlight_yards(BREAKING) - CFB / PHF — 1,308 loader return-table columns described
- CFB —
team_idcanonicalized toInt64at the loader boundary - Docs — loader returns tables now carry column descriptions
- CFB —
adv_*pos_teamnow holds the team NAME, id moves topos_team_id(BREAKING data change) - CFB —
adv_*declared schemas re-derived from the shipped data
- 0.0.73 Release: August 1, 2026
- CFB — pre-2014
{type}_player_idjoin recovered (2004 +36pp, 2005–2013 +2–8pp) - CFB —
adj_off/def/netrescaled to the Radjust_epanetted statistic (BREAKING scale change) - CFB —
adj_st_eparescaled to true EPA units (BREAKING scale change) - CFB —
cfb_ratingsgameonpaper-parity filters (default ON) - CFB — loaders for 3 published-but-unreachable dataset releases
- CFB — 5 summaries loaders unblocked for 2004–2013
- NBA — human-readable player and team columns for model outputs
- NBA — read the committed raw store over URL + season-level captures
- Fixes
- CFB — pre-2014
- 0.0.72 Release: July 22, 2026
- BREAKING CHANGES
- CFB — loaders for 6 published-but-unreachable dataset releases
- CFB —
load_cfb_ratingsdataset loader - NBA / WNBA — CTG play context (T3.6): possession/shot/lineup/player tables + start-type oracle
- Fixes
- Dependencies
- Release utilities —
sportsdataverse.release(sportsdataversedata R-package port) - PWHL — coordinate-based xG (T5.3b): xg_method default flips quality → coords
- PWHL — per-strength xG calibration + geometry hardening (T5 follow-up)
- CFB — advanced-efficiency spine (opponent-adjusted efficiency/explosiveness/havoc → field position → adjusted tempo)
- NFL — NGS over-expected tracking spine (YAC-OE → RYOE → separation-OE → man/zone rates)
- NFL — scheme & special teams spine (play-call model → game script → kicker/punter value → line grades)
- NFL — projection & draft spine (player projections → usage shares → availability → draft model)
- NFL — ratings & market spine (power ratings → win prob → spread/total → player props)
- CFB — recruiting & roster-projection spine (talent composite → returning production → wins projection → transfer impact → draft projection)
- NBA / WNBA / G-League — shot-value spine (xPoints → context make-prob → talent → selection → zone maps)
- MBB / WBB — shot-quality spine (xPoints → shot selection → shooter talent)
- MBB / WBB — player-value & projection spine (box-BPM → archetypes → recruiting → transfer → draft)
- Recruiting — ESPN NCAA recruiting family + On3 rankings
- MBB / WBB — prediction & tournament stack (ratings → pregame → in-game WP → résumé → bracketology → Monte Carlo)
- NBA — external concurrent validity + walk-forward retrodiction (WP3)
- NBA — RAPM variants (WP2)
- NBA — through-date ratings panel, WAR, and single-game BPM (WP4)
- NBA — v3-to-v2 play-by-play adapter (
nba_v3_to_v2_pbp) - NBA / WNBA — stats.nba.com / stats.wnba.com flat-API family (
nba_stats/wnba_stats) - NBA — possession event-detail columns, per-shooter shooting frame,
game_date - NBA — faithful possession boundaries (pbpstats parity)
- NBA — quarter-box on-court lineup seeding +
lineup_source="quarter_box" - CFB — 0.36-live pbp reconciliation + game-ending WP-perspective fix
- CFB — advanced box score: player-name cleanup +
cpaggregation - NFL / CFB — season standings + simulation engines (nflseedR / cfbseedR ports)
- MBB / WBB — bigballR + wbigballR port:
ncaa_mbb_*/ncaa_wbb_*stats.ncaa.org family - MBB / WBB — college computational-core port (lineup stats → ratings/luck → RAPM → positions → NCAA stint pipeline)
- MBB / WBB / NBA — NCAA LineupStatSet producer + hoopR release-parity producers
- CFB / MBB / WBB / Baseball — stats.ncaa.org parser expansion (football pbp graduation, box tabs, college baseball + softball pbp)
- NBA / MBB / WBB — dataset loaders for the published model releases
- NBA / WNBA — read-through raw JSON store for stats.nba.com per-game payloads
- NBA / WNBA / G-League — tracking-value spine (T3.2): six over-expected models on the
playerdashpt*surface - NBA — model-zoo v1: validation harness + SPM / BPM 2.0 / Bayesian adj-RAPM
- NBA — Kalman + aging-curve player projection + forecast validator
- NHL / PWHL — microstat & EDGE value spine (T5.2) + first-of-its-kind PWHL prediction (T5.3)
- PWHL — shift-derived
strength_state+ shot-level coordinate xG (+ two loaders) - MLB — model spines: game state (T6.4), pitching evaluation (T6.1), fielding/catching/baserunning (T6.3)
- Recruiting / NFL — PFF Premium Stats stem + On3 RDB retarget + 247Sports expansion
- Validation harness —
constant_columncheck, R-lint UTF-8 fix, cron run-tracker ingest
- 0.0.71 Release: June 24, 2026
- 0.0.70 Release: June 24, 2026
- 0.0.69 Release: June 23, 2026
- 0.0.68 Release: June 23, 2026
- CFB — completion-probability (
cp/cpoe) + expected-pass (xpass/pass_oe) surface - CFB — spread-free (naive) win-probability surface (
wp_*_naive) - CFB — QBR model retrained on the full 2004–2025 history
- CFB — fourth-down decision surface (
get_4th_down_probs, cfb4th port) - CFB — two-point-conversion decision surface (
get_2pt_probs, cfb4th port) - CFB — rule-era QBR / FG / fourth-down models +
spread_timesign fix - CFB — pre-2014 play-text player-name extraction
- NFL — expected pass (
xpass/pass_oe) + nfl4th fourth-down decision surface - NFL — self-trained XGBoost field-goal model in the fourth-down surface
- NFL —
load_nfl_espn_qbr(ESPN QBR loader, nflreadpy parity) - NFL — bundled self-derived xpass model (offline, no first-use download)
- CFB — completion-probability (
- 0.0.67 Release: June 17, 2026
- 0.0.66 Release: June 17, 2026
- 0.0.65 Release: June 17, 2026
- 0.0.64 Release: June 17, 2026
- 0.0.63 Release: June 16, 2026
- 0.0.62 Release: June 16, 2026
- 0.0.61 Release: June 16, 2026
- 0.0.60 Release: June 15, 2026
- 0.0.59 Release: June 13, 2026
- CFB — cross-source crosswalk loaders (
load_cfb_*_crosswalk) - ESPN — NCAA men's & women's college hockey (
espn_mch_*,espn_wch_*) - ESPN — NCAA college baseball + softball (
espn_college_baseball_*,espn_college_softball_*) - ESPN — UFL, XFL, and CFL (
espn_ufl_*,espn_xfl_*,espn_cfl_*) - ESPN — soccer/cricket param families + soccer headline aliases (
espn_soccer_*(league=),espn_cricket_*(league=),espn_epl_*,espn_ucl_*,espn_mls_*, ...)
- CFB — cross-source crosswalk loaders (
- 0.0.58 Release: June 12, 2026
- Loaders — NHL core + new NBA/MBB datasets aligned to
sportsdataverse-datareleases - Robustness & infrastructure — typing, CI gates, HTTP, deprecation policy
- The Odds API wrappers (
sportsdataverse.odds,toa_*) - Yahoo Sports college football wrappers (
yahoo_cfb_*) - NFL —
api.nfl.comwrappers cut over to generated; "NFL.com API" docs grouping - NFL — automatic
api.nfl.comtoken caching +NFL_ACCESS_TOKENoverride - Documentation —
api.nfl.comOpenAPI spec - Bug fixes
- Internal — Fox data key single-sourced
- Loaders — NHL core + new NBA/MBB datasets aligned to
- 0.0.57 Release: June 10, 2026
- 0.0.56 Release: June 9, 2026
- HockeyTech — live multi-league scraper (PWHL + AHL/OHL/WHL/QMJHL) + on-ice/Corsi/TOI analytics
- NFL — Next Gen Stats (
nfl_ngs_*) + api.nfl.com football/v2 (nfl_*) modules - NFL — restored the api.nfl.com game schedule + play-by-play wrappers
- ESPN — remove always-erroring endpoint variants + NFL R-parity
- Documentation — per-league Python ↔ R parity tables
- Documentation — example notebooks repaired, expanded, and rendered on-site
- NHL / PWHL — loader naming-parity aliases + games-manifest loaders (fastRhockey parity)
- Documentation — NFL return-table descriptions mined from nflverse
- Documentation — class methods rendered on autodoc pages (CFB / NFL)
- Documentation — accuracy-audit fixes
- 0.0.55 Release: June 8, 2026
- 0.0.54 Release: June 8, 2026
- 0.0.53 Release: June 8, 2026
- ESPN — declarative codegen + factory retirement
- NHL native — codegen cutover + clean names (api-web; in progress)
- Dataset loaders — release manifest + drift audit
- Generated documentation — reference pages + drift gate
- CFB — advanced box score expansion (
create_box_score) - CFB — box-score attribution correctness + ESPN-sourced totals (
create_box_score) - CFB — play-type reclassification: interception-return-fumble guard (
__add_new_play_types) - CFB — blocked-kick turnover flags + ESPN native-flag tripwires
- CFB — pre-2014 era support (
CFBPlayProcess) - Removed — NCAA bracketology
- 0.0.52 Release: June 3, 2026
- 0.0.51 Release: May 30, 2026
- User-facing quality-of-life additions
- New: MLB module (greenfield)
- New: NHL —
api-web.nhle.commigration + EDGE / Stats REST / Records - New: ESPN cross-league port
- New: NCAA bracketology
- New:
_common_espn_parsers.py(polars / pandas parser layer) - New:
return_parsed=Truedispatch shim - New:
nhl_edge_parsers.py - New: Site v2 summary dispatcher (20 sub-parsers)
- New: 100% ENDPOINT_PARSERS coverage (121/121)
- New: weekly cron live-test drift detector
- New: MLB Stats API parser layer
- New: NHL Stats REST + Records parser layers
- New: NHL api-web parser layer
- Bug fixes
- New: NFL drive-plays parser (true PBP parity)
- Test infrastructure
- Documentation
- 0.0.50 Release: May 7, 2026
- Packaging modernization
- Conda installability
- Linting & pre-commit modernization
- Documentation toolchain
- Runnable docstring examples (~190 functions)
- Example notebooks
- Contributor docs and templates
- NFL — nflreadpy parity
- NFL — caching and configuration
- NFL — static datasets
- NFL — pickcenter / odds modern path
- NFL —
load_nfl_scheduleparquet port - WBB / WNBA — new ESPN scrape modules
- CFB —
cfb_play_participantsand__add_player_colscollapse - CFB — pandas → polars 1.x bug-fix reconciliation (
0.36-live→main) - Infrastructure and tooling
- Bug fixes
- Deprecations
- 0.0.40 Release: December 6, 2025
- 0.0.38-39 Release: August 28, 2023
- 0.0.36-37 Release: July 9, 2023
- 0.0.34-35 Release: May 7-9, 2023
- 0.0.18 Release: July 25, 2022
- 0.0.17 Release: July 9, 2022
- 0.0.15 Release: May 8, 2022
- 0.0.14 Release: March 16, 2022
- 0.0.12 Release: February 24, 2022
- 0.0.5 Release: October 20, 2021
Unreleased
Added — league-wide NCAA RAPM solver (mbb_ncaa_rapm_league, #389)
The league-wide half of the NCAA RAPM program ("Path B"): one joint
offense/defense ridge per (league, season) putting every Division-I player on
a common scale — complementing the published ncaa_{lg}_rapm_within_team
datasets, which estimate a DIFFERENT quantity (value relative to teammates)
and must never be cross-joined with these.
aggregate_stintscollapses id-resolved possessions (the #382mbb_ncaa_rapm_inputadapter output) into matchup stints; a possession-weighted stint ridge is mathematically identical to the per-possession ridge at ~1/3 the rows. Possessions with any unresolved on-floor slot are dropped, never imputed.solve_rapm_leagueruns the sparse joint solve (per-100 scale, positivedrapm= good defense, ±1 home-offense column).DEFAULT_RIDGE_LAMBDA = 1000was fitted by game-grouped 5-fold CV on the real 2024 corpora — both leagues minimize there independently. Non-converged solves raise instead of returning a partial iterate.team_aggregateproduces the model-implied team ratings used by the external oracle gate. Validated on the full corpus: Torvik AdjEM Spearman ≥ 0.9434 (MBB 2011–2026, median 0.9653) and ≥ 0.9723 (WBB 2022–2026; 0.9039 in the COVID 2021 season).
The module is league-blind (frames in, frames out) — WBB passes its own
frames; there is deliberately no wbb_ twin. Companion sportsdataverse.mbb
adapter fixes from the same program: cross-season person_id resolution
(#382) and the canonical display_name_to_roster_key (#388).
Deprecated — the three NBA *_v3 loaders now read the production releases
load_nba_stats_pbp_v3, load_nba_stats_possessions_v3 and
load_nba_stats_lineups_v3 read one-asset, season-2025-only release tags
(nba_stats_pbpv3, nba_stats_possessions_v3, nba_stats_lineups_v3) that are
being retired. Those assets were built before the 2025-26 Finals ended and are
missing six games (0042500317, 0042500401–0405).
The Program V pipeline now admits every game type (preseason 001, regular
002, All-Star 003, playoffs 004, play-in 005, NBA Cup 006) across all
30 seasons, so the production tags carry a strict superset — verified live
for the overlapping season: 707,440 rows / 1,400 games vs 704,314 / 1,394, with
zero games lost, zero games short a row, and no column dropped (production adds
season).
All three keep working and are now DeprecationWarning shims (removal in
0.1.0) forwarding to their production successors:
| Deprecated | Use instead | Release now read |
|---|---|---|
load_nba_stats_pbp_v3 | load_nba_stats_pbp | nba_stats_pbp |
load_nba_stats_possessions_v3 | load_nba_stats_possessions | nba_stats_possessions |
load_nba_stats_lineups_v3 | load_nba_stats_game_lineups | nba_stats_game_lineups |
The seasons argument is unchanged. It was the season's START year before
and still is: load_nba_stats_pbp_v3(seasons=2025) meant 2025-26 when it read
play_by_play_v3_2025.parquet and still means 2025-26 now that it reads
nba_play_by_play_2026.parquet. The retired assets were START-year keyed and the
production assets are END-year keyed; that translation lives in the loader's
{season + 1} asset-path template, so the shims forward seasons untouched.
Docs — NBA shifted loaders: the season COLUMN is the END year
The four NBA loaders whose asset path carries {season + 1}
(load_nba_stats_schedules, load_nba_stats_pbp, load_nba_stats_possessions,
load_nba_stats_game_lineups) return frames stamped with the asset's year,
so the season column does not equal the seasons argument:
load_nba_stats_schedules(seasons=2024) returns rows reading season == 2025.
Unshifted NBA siblings (team_boxscores, officials, rosters) stamp 2024
for that same real season.
This was undocumented and is load-bearing — a partitioner keying off the column writes the 2024-25 season over the 2025-26 partition. The divergence is now documented in all four docstrings (and the three shims that inherit it), with a codegen test that fails if a future shifted loader ships silent. The column is documented, not restamped: it is the published asset's own identity, and rewriting it would make the frame disagree with the file it came from.
New — sportsdataverse.wexp: win-expectancy bake-off harness (NFL + CFB)
A typed, mypy-ratcheted harness for walk-forward pre-game win-expectancy modeling across NFL (1999+) and CFB (2004+), built on a structural leakage-proofing philosophy: features flow only through a vintage-keyed store, engines receive outcome-stripped slates, and every fitted build parameter is stamped on its table and refused on mismatch.
wexp.store.VintageStore— vintage-keyed feature store with EXCLUSIVEas_of_weeksemantics (week_semantics="through"shifts inclusive CFB assets at ingest); leak-freejoin_asofwith dtype guards on entity, week, AND season keys.wexp.backtest.run_backtest— the walk-forward driver: engines see only completed prior games plus an outcome-stripped slate; CFB postseason week-reset normalized; predictions validated; per-season + pooled result rows keyed byvariant_hash, plus like-for-likeweek_slice="lined"rows wherever market coverage is partial.wexp.oracle_market— vig-removed market oracles: NFL close from nflverse schedules; CFB close/open consensus from thecfb_line_oddsarchive (loaded viaload_cfb_betting_linesby default) with a LEFT-join contract so every schedule game stays model-scorable.- Engines (
wexp.engines/wexp.elo): margin-Elo (carryover, per-era HFA, continuity priors from talent + returning production), per-week opponent-adjusted ridge vintages (raw / capped / close-game-filtered responses), a Glickman–Stern walk-forward Kalman filter (with continuity-prior composition), net-rating vintage adapters for the publishednfl_ratings_weekly/cfb_ratings_weeklydatasets, and drive-EP response extraction (cfb_drive_deltas). wexp.postgame.postgame_we— post-game deserved-win probability (G3 analytic normal + G2 drive bootstrap, fixed-seed reproducible).wexp.features— walk-forward team features, all keyed(season, as_of_week, team_id)under the same EXCLUSIVE-vintage contract as the engines:sos_sor_vintages(strength of schedule faced and remaining, plus strength of record as wins above what a league-average team would take from the same slate at the same sites),carry_forward_weights(how much of last season's metrics to still believe given returning production and QB / head-coach continuity, ramped to zero once the current season carries its own evidence), andcfb_scoring_opportunities(per team-game scoring opportunities — drives reaching inside the opponent 40 — split intoopp_ratefor creation andpoints_per_oppfor finishing; a team with no opportunity gets a null finishing value, never a 0).wexp.variants— the axis config system: 1,860 valid variants across cores/response/adjustment/priors/wp-map/HFA axes, stable-hashed;wexp.engines.build_predictordispatches implemented cells and raises on unbuilt ones (never a silent fallback).- Committed tune-window leaderboards + variant registry under
results/wexp/— every committed row is tune-window (season <= 2021); no 2022-2025 holdout result is written to the leaderboard.
New — load_nfl_ratings_weekly: per-week as-of NFL ratings vintages
Loads the new nfl_ratings_weekly release on sportsdataverse-data
(per-season assets, 1999-2025 at publish; the producer cron refreshes the current season in-season): one row
per (season, as_of_week, team_id) with STRICTLY EXCLUSIVE semantics —
a row at as_of_week = W was fit only on games before week W's first
kickoff, safe to join onto week-W games with no leakage. SDV-native (no
nflreadpy equivalent).
New — sportsdataverse.scrape.espn: shared ESPN -raw archive engine
The fourth and last duplicated scrape stack in the ecosystem. hoopR-nba-raw,
hoopR-mbb-raw, wehoop-wnba-raw and wehoop-wbb-raw run the same numbered
stage sequence against the same tree shape — but only wehoop-wbb-raw had
grown a shared package (wbb_raw_scrape, 451 LOC), a test suite, and the write
guard. The other three inline those concerns in every numbered script: 22
copy-pasted str2bool definitions between them, and no write guard at all.
That package now lives here, parameterized on LeagueConfig:
scrape/espn/persist.py— the write guard. The raw tree is the scrape checkpoint, so a persisted provider error body is permanent: it yields an empty dataset for that key on every rebuild, forever, with nothing failing. Refusing the write is the whole fix — a refused key simply looks un-scraped, so the next run retries it and the archive self-heals.scrape/espn/cli.py— thestr2bool/season_argscontract.type=boolis a trap: bash passes the string"false", andbool("false")isTrue.rescrape_default=is exposed so the three repos that shippeddefault=Truecan migrate without changing cron behavior, then flip it deliberately.scrape/espn/ids.py— one id canonicalizer; a lossy cast raises rather than silently producing an id that joins to the wrong row.scrape/espn/schedule.py/master.py/paths.py— capture flags, URL columns, the season→master union and coverage index.scrape/espn/league_config.py—NBA/MBB/WNBA/WBBidentity, including which per-game families each league actually publishes (ESPN serves an officials feed for the two women's leagues only).
league is a required keyword on every public entry point. A defaulted
league is how a well-formed capture ends up written under the wrong league's
tree — wrong data, no error — which is exactly the bug the NCAA extraction
found in ncaa-mbb-hoops-raw's capture CLI. An AST-based test enforces that no
module outside league_config.py names a league in executable code.
86 offline tests, ported from wehoop-wbb-raw's suite so the WBB assertions
stand as the parity oracle for the lift, and extended with the league
parameterization.
0.0.75 Release: August 2, 2026
Fix — scrape.ncaa CLIs pointed at the wrong repo root (silent no-op)
The lifted modules defaulted --root to Path(__file__).resolve().parents[1].
That meant "the -raw repo root" while they lived in <repo>/python/, and
silently meant sportsdataverse/scrape/ once they moved into sdv-py — so
every stage CLI in both NCAA repos scanned an empty tree and reported
success having done nothing (bundles=0, EXIT=0). No launcher passes
--root, so all of them were affected.
The engine cannot infer a caller's repo root, so it no longer tries: _main
takes default_root from the shim (each repo's own REPO_ROOT), and the
library-level fallbacks raise a message naming the fix instead of inventing a
path. The parse stage now also exits non-zero when a run matches no bundles —
the silent-success that let this hide in the first place.
scrape.ncaa.parse — the parse stage is now re-runnable (--season, --force)
The stage skipped any contest whose JSON already existed, and globbed every
season. Together that made a reprocess impossible: a parser fix or a later
identity backfill could never reach already-parsed games. That is not
hypothetical — the MBB tree carries null player_id / clean_name /
ncaa_team_id for 2024-2026 because those seasons were parsed before the
reference backfill taught the pipeline to fill them, and re-running the stage
was a no-op on exactly those files.
--season (repeatable) scopes the sweep; --force re-parses existing output.
Default behavior is unchanged — omit both and the stage still skips existing
files across every season, so resumability is intact.
New — sportsdataverse.scrape.ncaa: shared stats.ncaa.org hoops sweep engine
ncaa-mbb-hoops-raw and ncaa-wbb-hoops-raw were the same pipeline twice:
27 shared files, ~4,000 LOC of production code, and only ~90 lines of real
difference between them — the rest maintained by hand-porting each fix from one
repo to the other. That stack now has one home:
scrape/ncaa/discover.py— team/season crosswalk → contest ids → the seasonschedule_master.scrape/ncaa/capture.py— the 3-page bundle per contest: sharded, disk-is-checkpoint, ban-aware.scrape/ncaa/parse.py— bundle → pbp / shots / lineups frames.scrape/ncaa/rosters.py,datasets.py,identity.py,espn_game_xwalk.py,bundle.py,canary.py.scrape/ncaa/league_config.py—MBB/WBBidentity.
league is a required keyword on every public entry point, deliberately.
The capture stack is league-agnostic (stats.ncaa.org serves one contest-id
namespace, and sdv-py's NCAA parsers already select the period model and
three-point arc by league), so a league is only ever a token threaded through
calls — and a shared engine that defaults one is how a women's run silently
reads men's data. That was not hypothetical: the men's repo's capture CLI
hardcoded both the schedule-master path and the capture league to "mbb" and
had no --league flag at all, so it could not be pointed at the other league.
A source-level test guards the literal from creeping back, since the failure it
causes is silent — a well-formed capture written to the wrong league's tree.
Not re-exported at the top-level sportsdataverse namespace: this is
producer-pipeline tooling, not the tidy-data API. The two -raw repos keep a
thin league-binding shim per module, their launchers (which carry real
per-league pacing), and their test suites — those suites are the engine's
parity harness and stay repo-side.
sportsdataverse.scrape.stats — league-parameterized capture layer (Phase 2)
Phase 1 gave the stats-raw twins one transport/proxy/observability engine; this
moves the capture planning behind a LeagueConfig so the two -raw repos
stop carrying near-duplicate copies of it (the remaining ~700 drift lines):
league_config.py— frozenNBA/WNBAidentity (LeagueID, wrapper module and prefix, store env var and subdir), resolvable byby_league_id().endpoints.py— the signature-derived capture registry. Season-string spelling is now league-keyed: the NBA's two-year span ("2023-24", without which several endpoints silently return zero rows) vs the WNBA's bare calendar year.season_capture.py— atomic, resumable season-level captures with the contentless-payload guard (an unparseable{}is never persisted, because resume ispath.exists()and one empty write is permanent).periods.py— league- and era-aware per-period window math: NBA 12-minute quarters, WNBA two 20-minute halves through 2005 and four 10-minute quarters from 2006. Regulation totals 2400s in both WNBA eras, so only the period boundaries reveal a mix-up; a regression test pins the NBA path tonba_lineups._period_start_range, the function that reads these captures back.refill.py— the empty-{}repair pass, driven by aLeagueConfiginstead of per-repo constants.
Resolving the wrapper module from the config fixed a latent crash: the WNBA
repo's own refill shim imported sportsdataverse.nba.wnba_stats, which does
not exist, so a real (non---check) refill run raised ModuleNotFoundError.
Docs — offline full-text search on the documentation site
The Docusaurus site now ships local full-text search
(@easyops-cn/docusaurus-search-local) — the search index is generated into
the static build itself, so there is no Algolia account, external crawler, or
third-party service involved. With versioning enabled the plugin indexes only
the rolling main version, so the index does not grow as release snapshots
accumulate. Docusaurus ships no search by default; the site had simply never
had one configured.
CI — docs site builds on GitHub Actions and publishes to gh-pages
A new docs-deploy workflow builds the static site on pushes to main that
touch docs/** and force-publishes the output as a single orphan commit on
the gh-pages branch. Hosting can then serve gh-pages as a plain static
deploy (no build step on the hosting provider), which removes the heavyweight
per-push Docusaurus build from Vercel.
New — sportsdataverse.scrape.stats: shared stats.nba.com / stats.wnba.com sweep engine (Phase 1)
The scraping machinery that was copy-pasted between hoopR-nba-stats-raw and
wehoop-wnba-stats-raw (and had drifted to byte-identical-or-nearly — 0–6 diff
lines per file) now has one library home, per the 2026-08-02 pipeline audit's
shared-engine decision:
scrape/stats/proxy.py— round-robin ProxyBonanza pool with quarantine + outcome classification (transport_err/blocked/blankquarantine;server_err/notfoundnever count against a proxy).scrape/stats/session_transport.py— thread-local sticky-sessioncurl_cffitransport (Chrome impersonation; the hosts TLS/JA3-block plainrequestswith a silent hang), in-session retry of the cheapserver_errclass only.scrape/stats/observability.py— sweep bookkeeping (endpoint outcome ledger, degradation windows, progress heartbeat).
Deliberately not re-exported at the top-level sportsdataverse namespace —
this is producer-pipeline tooling, not the tidy-data API. The -raw twins
migrate to these imports next (deleting their local copies); league-specific
endpoint sets / season formats stay repo-side until Phase 2's LeagueConfig.
All three modules are typed (mypy ratchet) and covered by the ported offline
observability suite plus a proxy-classifier truth table (23 tests).
CFB — model suite retrained on the corrected corpus (BREAKING model change)
All seven bundled CFB artifacts — ep_model, wp_spread, wp_naive,
qbr_model, fg_model, xpass_model, two_pt_model — are retrained on a
training frame rebuilt from the repaired cfbfastR-cfb-raw corpus.
The previous frame dated from 2026-06-24, before the ESPN -1
end.yardsToEndzone sentinel fix (0.0.74) and 13 other cfb_pbp.py commits. In
that frame, 46 of 2016's games carried an implausible mean EPA; the rebuilt
frame has 1.
Both gated metrics improve under 22-fold LOSO — the same protocol the recorded baselines were measured with:
| model | metric | before | after |
|---|---|---|---|
ep | ep_cal_mae | 0.0140 | 0.0137 |
wp_spread | logloss | 0.3518 | 0.3486 |
EP's mlogloss (1.2333) and accuracy (0.4997) land on the previously documented
values, which is expected: the frame changed ~0.1% of rows and those metrics
pool over 2.2M. What moved is calibration — precisely the metric a corrupt-EPA
tail distorts. mean_pred_ep 1.6886 against mean_realized 1.6888 confirms it.
Feature contracts are unchanged for all seven models (8/5/7/4/13/12/10 features), so this is a weights-only change: no caller adjustment is needed. Downstream EPA/WPA values WILL move, so anything cached from an earlier release should be rebuilt.
CFB — model cards ship beside every artifact
Each .ubj now has a .card.json recording objective, feature list, training
season span, full hyperparameters, boost rounds and the source frame. The
generated cards previously emitted objective: None, training_seasons: None
and hyperparameters: None, so the only way to answer "what was this model
trained on" was to read the training code.
0.0.74 Release: August 2, 2026
CFB — the ridge opponent adjustment was a no-op (BREAKING rating change)
cfb_adjusted_epa._RIDGE_LAMBDA was 325, carried over from cfbfastR's glmnet
call at cv$lambda[[1]]. Two things were wrong with porting that number:
cv$lambda[[1]] is the largest lambda in glmnet's grid — by construction the
value at which the null model wins and every coefficient is zero — and glmnet's
lambda is not sklearn's alpha, which dropped_level_ridge then multiplies by
n. On a 64k-play season that landed at alpha = 2.1e7.
The result was an adjustment that did not adjust. Fitted team strengths spanned
0.0008 EPA/play across all of FBS, and spearman(raw_off_epa, adj_off_epa) was
0.999982 — not one team's ordering changed. Group of 5 teams kept full credit
for weak schedules while Power conference teams were buried: Toledo 8th, James
Madison 5th, Florida 114th, South Carolina 102nd.
It survived review because raw EPA/play is itself correlated with quality, so rank checks on the output looked reasonable. The tell was the coefficient spread, not the ranking.
The default is now 0.035, tuned across 2021–2025 against ESPN FPI joined on
team_id and cross-checked against SP+, FEI and F+ (which agree with each other
at 0.967–0.990). Against their 2025 consensus:
| SP+ | FEI | F+ | FPI | consensus | mean rank error | |
|---|---|---|---|---|---|---|
| before | 0.884 | 0.774 | 0.821 | 0.781 | 0.794 | 19.9 |
| after | 0.930 | 0.966 | 0.956 | 0.965 | 0.967 | 7.8 |
RatingsConfig.ridge_lambda (previously 0.05) shares the value — the two entry
points had disagreed by 6,500x. 0.035 rather than the nominal 0.02 optimum
because the two are statistically tied on the mean (.9303 vs .9307) while 0.035
has the better worst season and clears the PFF-grade oracle gate, which 0.02
fails. Optimizing one oracle into another's red would have meant lowering a gate.
Also fixes the ridge dropping its reference level from the output.
model.matrix drops it from the design, but its effect is 0 by construction and
lives in the intercept, so it belongs in the returned table — one team per side
was silently absent from every fit, and its opponents lost those games from the
adjusted set. Which team it hit was arbitrary (lexicographic on the string id).
CFB — ESPN's -1 end-of-play yardline sentinel corrupted 2016 week 2
ESPN uses -1 to mean "no end-state yards-to-endzone". The parser guarded only
on end.yardLine is not null before trusting end.yardsToEndzone — and ESPN
populates end.yardLine perfectly well on exactly the plays carrying the
sentinel, so the guard passed and the fallback never fired for the case it exists
to handle.
2016 week 2 shipped that way: 72 of 75 games, with -1 on ~every play (SMU @
Baylor: all 238). EP_end was then scored as if the offense were on its own
1-yard line after every snap, running EPA to about −2.6/play. It surfaced in the
published percentiles as a 1st-percentile early-down EPA of −2.97 for 2016,
roughly six times every neighbouring season, while the median and upper tail
looked normal.
Measured across all 19,749 committed raw payloads, plays carrying a negative
end.yardsToEndzone total 13,632 in 88 games — 8.1% of 2016 and 12 plays
across seven other seasons. Reprocessed from raw, the three worst games move
from −2.292/−1.894/−1.733 mean EPA to −0.158/−0.000/−0.041, while eight control
games in 2019 and 2023 come out byte-identical.
A re-scrape does not fix this: the committed raw is identical to what ESPN serves
today. The recovery is local, from end.yard.
CFB — fill_null(0.0) is a silent no-op on booleans, pinning rushing_power_rate at 1.0
polars leaves Boolean nulls untouched when the fill value is a float — no error,
no warning. Aggregation frames prepared with .fill_null(0.0) therefore kept
their boolean nulls, and .mean() on a flag that is null-where-absent averages
over exactly the True rows and returns 1.0.
rushing_power_rate has shipped as 1.0 for every team in every season. In 2024
power_rush_attempt is null on 159,513 plays and True on 3,437, so the published
rate read 1.0 where the real figure is 3437/63017 = 0.055.
Of 44 boolean .mean() aggregations in the module, exactly one was wrong;
rushing_power_success_rate looked like a second victim at 0.774 but is correct,
since its frame is already filtered to power attempts. The fill is now
centralized in _fill_missing() (booleans → False, numerics → 0.0) across all 10
sites rather than patched at the one caught call site.
Also in this release
CFB — opportunity_run corrected, un-degenerating opp_highlight_yards (BREAKING)
Found while writing column descriptions, not by a failing test — nothing pinned either invariant, which is why both survived.
opportunity_run was rush AND yds_rushed <= 4. The cfbfastR oracle
(espn_cfb_15_team_summaries_creation.R:606) is
((rush == 1) & (yds_rushed >= 4)), and the sibling cfb-data producer agrees. An
"opportunity" is a carry where the blocking did its job — one that reached 4
yards. sdv-py had it as a stuff.
That silently degenerated a second column. opp_highlight_yards gates on
opportunity_run, but highlight_yards only accrues from 4 rushing yards up, so
the two conditions could never co-occur: the column was identically 0 in every
published row, verified across 162,950 plays in the 2024 release. It now ranges
0–16 on the same games.
The regression test then caught a second divergence: these gated on
type.text == "Rush", the literal ESPN play-type string, which excludes
"Rushing Touchdown" and "Fumble Recovery (Own)" rushes — so a 4-yard rushing
touchdown was not counted as an opportunity. The oracle gates on rush, and so
does line_yards two statements below, which meant adj_rush_yardage was left
null on exactly the plays line_yards tried to consume. opportunity_run,
highlight_run and adj_rush_yardage now all gate on rush.
Still divergent, deliberately untouched: the line-yards scale. The R oracle
caps adj_rush_yardage at 10 and splits 0–4 / 5–10 / 11+; sdv-py caps at 8 with a
3 + 0.5(adj-3) ramp and a 5.5 ceiling. Changing that shifts published
line_yards / second_level_yards / open_field_yards, so it needs its own
decision rather than riding along with a bug fix.
Breaking: published espn_cfb_pbp assets carry the old values until
republished; the column descriptions say so explicitly.
CFB / PHF — 1,308 loader return-table columns described
Continues the work started in #312. Loader deferred columns 2,407 → 1,099, residual ratchet unchanged at 0.
| batch | cols | grounding |
|---|---|---|
team_summaries + _weekly | 757 | producer arithmetic (_summarize_team) |
adv_team + _gamelog | 131 | empirical profile of published assets |
adv_situational | 71 | empirical profile |
cfb_pbp | 247 | transcribed from cfb_pbp.py, thresholds quoted |
| PHF (4 loaders) | 114 | provider fields, described at face value |
Descriptions are composed from verified vocabularies rather than hand-written
per column, and the generators are committed (tools/codegen/gen_*_descriptions.py)
so each derivation stays reproducible. They enumerate loader_schemas.yaml, so
re-running is idempotent.
Profiling the ESPN blocks against real data contradicted their naming in ways that would otherwise have shipped as wrong documentation:
- the bare
EPA_explosive*andEPA_success*columns are integer play counts, not EPA totals, despite theEPA_prefix (~40 columns) EPA_overall_offandEPA_overall_offenseare exact duplicatesEPA_explosive_rateis notEPA_explosive / EPA_plays— ESPN divides by a smaller qualifying-play count, so deriving it will not reproduce their value
Anything a generator could not ground is left blank and reported, never invented.
CFB — team_id canonicalized to Int64 at the loader boundary
The same ESPN team id shipped with three different dtypes across the CFB release surface, which makes a cross-dataset join match nothing — silently, with no error and a structurally valid frame:
| dtype | datasets |
|---|---|
Int64 | 13 — all adv_*, drives, game_rosters, linescores, player_box, team_box, adv_team_gamelog |
String | 8 — passing, receiving, rushing, team_summaries(+_weekly), ratings(+_weekly), recruiting_proj |
Int32 | 1 — team_info |
All nine non-Int64 loaders now normalize on read, so
load_cfb_team_summaries(...) joined to load_cfb_adv_team(...) on
team_id/pos_team_id resolves 134/134 teams where it previously matched 0.
- Fixed at the boundary, not by republishing. The published assets are untouched; the loader pins the dtype on read, per the repo's "one dtype per id, cast at the boundary" rule. No breaking change for anyone reading the parquet directly.
- New
id_int64:key inreleases.yamldeclares which id columns a loader canonicalizes; the generated loader emits a_cast_ids_int64call. Declaring it is the whole change — no hand-edits to generated loaders. - Lossless or refused, never silent.
_cast_ids_int64converts only when every non-null value survives a round-trip."007"casts cleanly to7and1.5truncates to1— both change the id, so both leave the column untouched. A "no new nulls" check alone would let both through. Float-origin ids go straight toInt64rather than through a string (which would yield"123.0"). - Covered by
tests/codegen/test_id_casts.py, including a manifest gate that fails if a loader declaresid_int64without emitting the call, or declares a canonicalized column as anything other thanInt64. - The two producer-schema contract tests now apply the declared
id_int64overlay: a returns table documents the loader's output, not the raw asset, so a boundary cast is an expected divergence rather than a docs lie.
Docs — loader returns tables now carry column descriptions
Generated loader returns tables rendered col_name | type only, while endpoint
tables carried a third description column. Loader tables now match, resolving
descriptions from manual_column_descriptions.yaml (keyed by the loader's own
function name) and falling back to the R-package column dict — so shared columns
like game_id / season / week fill in automatically.
This exposed ~5,850 loader columns to the description coverage ratchet, which had
never globbed loader_schemas.yaml — so those blanks were invisible to a gate
whose stated intent is "every return-table column renders a description". The
extractor now sees them (59% already covered) and the remainder is tracked via the
existing _DEFERRED_BUCKETS mechanism, mirroring the nba_stats decision.
The first documented use is load_cfb_adv_defensive_players, whose column
availability is season-dependent and previously undiscoverable without loading
several seasons and diffing them:
| seasons | cols | shape |
|---|---|---|
| 2004 | 8 | fumble recoveries only |
| 2005–2013 | 12 | + sacks, sacks_yards, pass_breakups, forced_fumbles |
| 2014–2025 | 14 | + interceptions, interceptions_yards |
CFB — adv_* pos_team now holds the team NAME, id moves to pos_team_id (BREAKING data change)
ESPN's advBoxScore blocks put a team id in a name-shaped column. Verified
against the published assets rather than assumed: all 235/235 distinct
pos_team values in adv_team 2024 resolve to real teams.
The producer (cfbfastR-cfb-data) now surfaces the id as pos_team_id /
def_pos_team_id and fills the original column with the display name, so each
column means what it is named. All 10 adv_* tags have been rebuilt and
republished for 2004–2025 (220 assets, 0 failures, 100% name resolution on
every season).
| before | after | |
|---|---|---|
pos_team / def_pos_team | Int64 id (48) | String name (Ohio State Buckeyes) |
pos_team_id / def_pos_team_id | — | Int64 id |
- Affected tags (10). Eight carry the offense's team and use
pos_team:adv_team,adv_passing,adv_rushing,adv_receiving,adv_turnover,adv_drives,adv_situational,adv_specialists. Two carry the defense's team and usedef_pos_team:adv_defensive,adv_defensive_players. - The summaries family was NOT affected —
cfb_team_summaries,cfb_passing,cfb_rushing,cfb_receivingalready shippedteam_idfor the id and a readablepos_team, so they were deliberately left alone. - Cross-family joins need a cast. The two families spell this differently:
summaries uses
team_id(String),adv_*usespos_team_id(Int64). Joiningcfb_team_summaries.team_idtoadv_team.pos_team_idwithout casting matches nothing, silently. espn_cfb_adv_team_gamelogwas rebuilt for 2004–2025 off the new shape; it continues to exposeteam_id+ a readableteam/opponent, so its consumers see no change.
CFB — adv_* declared schemas re-derived from the shipped data
The declared returns tables for all 10 adv_* loaders are regenerated from a
diagonal union of real published seasons (2004/2014/2024/2025), so the
documented schema now matches what the loaders actually return. Beyond the
pos_team split this closes pre-existing drift the audit surfaced:
-
load_cfb_adv_passingwas missingxComp,CompPct,xCompPct,CPOE, and declaredrush_epa/pen_epaasNullrather thanFloat64. -
load_cfb_adv_defensive_playerswas missingsacks,sacks_yards,pass_breakups,forced_fumbles,interceptions,interceptions_yards.This block's shape ramps in two steps, so the union is the only honest declaration. Columns absent in a requested season come back null (the loaders concatenate diagonally), so a 2004–2025 pull is uniform in shape but sparse in the early years:
seasons cols shape 2004 8 fumble recoveries only 2005–2013 12 + sacks, sacks_yards, pass_breakups, forced_fumbles2014–2025 14 + interceptions, interceptions_yards
loader_schemas.yaml drives the generated returns tables only — nothing casts
from it — so this drift was invisible to the test suite and surfaced purely as
incorrect published documentation. Declared-vs-shipped now diffs clean for all
10.
0.0.73 Release: August 1, 2026
CFB — pre-2014 {type}_player_id join recovered (2004 +36pp, 2005–2013 +2–8pp)
-
Context. ESPN ships no per-play
participants[]before 2014 — verified live against Core v2/events/{id}/competitions/{id}/plays, which returns a full 150–240-play stream for 2004–2013 with zero participants, and ~90% participant coverage from 2014 on. Pre-2014{type}_player_idtherefore comes entirely fromCFBPlayProcess.__attach_player_idsmatching a regex-extracted name against the game roster. Two defects were capping that join far below its ceiling. -
2004 team-abbreviation suffix. 2004 play text renders a name as
"Player Name (TEAM)". The rusher capture strips the parenthetical; the passer capture keeps it, and_norm_player_namefolds it into the join key ("matt ryan bc"never matches"matt ryan"). Passers are the largest name family, which accounted for the entire 2004 gap. -
Narrative-tail bleed (2005–2013). Pass-direction phrases and ESPN's own missing-space concatenations survived into the capture —
"Russell Wilson deep out","Dominique Davis screen","Raynard Hornetackled by". -
Both are now cleaned at the shared chokepoint in
__attach_player_ids, which already iterates every{type}_player_namecolumn, rather than by patching each extraction regex — so the emitted name column and the join key are fixed together in one place. New module-level patterns_PLAYER_NAME_TEAM_SUFFIXand_PLAYER_NAME_TAIL. -
Two fallback join tiers added below exact match: first-initial + surname, then bare surname. Both are team-scoped and uniqueness-gated — an ambiguous surname stays null rather than guessing.
-
Measured on the committed
cfbfastR-cfb-rawtree (25 games/season, 11 player-column families), share of populated name cells resolving to an id:seasons before after 2004 52.1% 87.8% 2005–2013 82.7–94.7% 89.3–96.6% 2014+ 87.0–98.6% 87.8–99.3% End-to-end on 2004 game
242410259: pass+rush id resolution 46.4% → 100%, with names emitted asBryan Randall/Matt Leinartrather thanBryan Randall (VT)/Matt Leinart (USC). -
Player box score is now a second id source.
game_rostersstays primary, but ESPN 404s its roster resource for a large minority of games — 376 of 898 in 2018, 142 of 706 in 2020 — leaving the join nothing to match against. ESPN's per-player box score covers the same athletes in the same athlete-id namespace and already ships inside the summary payload, so it costs no extra request. Namespace agreement was measured, not assumed: 0 id conflicts on the roster/box overlap in 21 of 22 seasons. Where the two disagree the name becomes ambiguous and resolves to null.seasons gain 2004–2013 +0.0pp (box is a strict subset of the roster, ~27 vs ~60 names) 2014–2017 +2.9 to +5.3pp 2018 +32.4pp (63.7% → 96.2%) 2019–2025 +0.1 to +6.4pp End-to-end on three real 2018 empty-roster games with
game_roster=[]: 97.0% / 97.7% / 98.4% — the last of which the season fill probe previously reported at 0.0%.
CFB — adj_off/def/net rescaled to the R adjust_epa netted statistic (BREAKING scale change)
efficiency_ratings/cfb_ratingsnow publish the gameonpaper-parity NETTED values: each team's raw per-game EPA (all pass/rush plays, garbage time included) minus the opponent's ridge-fitted strength, averaged across games — the Radjust_epa/team_agg.Rstatistic. A top team now nets ~0.30–0.40 EPA/play (2024 max 0.35 vs gameonpaper's 0.366). The previous releases carried the ridge coefficient + intercept (a competitive-play model strength) under the same column names — ~1.8× hotter at the top (max ~0.63) and data-volume unstable. Ranks are nearly unchanged (the Spearman oracle gates all hold); magnitudes shrink.- Prediction constants refit on the new scale (
net_points_scale34.49 → 44.54,margin_sd,total_*;hfa_epaunchanged — it is the ridge's own home coefficient). Refit backtest: Brier 0.1416 (beats ESPN FPI 0.1436), spread MAE 3.23 (was 4.06), total MAE 4.88. - New magnitude oracle gate (
test_adj_net_magnitude_matches_gameonpaper_scale) — the failure mode rank-based gates cannot see. Two floors re-derived for the changed statistic (netting includes garbage time by construction), each documented in its test: SP+ offense Spearman 0.84 → 0.82 (observed 0.849 → 0.836) and season-odds expected-wins rank calibration 0.90 → 0.885 (observed 0.928 → 0.899). All other oracle gates held or improved.
CFB — adj_st_epa rescaled to true EPA units (BREAKING scale change)
- The special-teams composite is now real EPA/play: for each unit (field
goal, punt, kick return), the team's mean EPA/play centered on that unit's
league-wide mean, summed across the three units. The previous releases
shipped a sum of three z-scores under the
_epaname — dimensionless, std ~1.7, range ±5. The centered form also tracks the SP+ special-teams oracle better (Spearman 0.865 vs 0.768), so the oracle floor was RAISED 0.75 → 0.84.
CFB — cfb_ratings gameonpaper-parity filters (default ON)
cfb_ratings()now defaults tofbs_only=Trueanddrop_kneels=True, matching the game-on-papercfb-team-summariespipeline's play substrate.fbs_onlykeeps only games where the schedule'shome_division/away_divisionare both"fbs";drop_kneelsstrips kneel-downs via a pipelinekneel_downflag when present, otherwise the play-text regex (kneel/takes a knee) plus the end-of-half anonymized-TEAM-run clock heuristic (pass plays are never treated as kneels). Both filters skip gracefully when the input frame lacks the needed columns, so slim/canonical frames keep working; passfbs_only=False/drop_kneels=Falsefor the previous unfiltered behavior. Downstream consumers (cfb_resume,cfb_season_odds) inherit the defaults. Ratings computed on real released data will shift accordingly (FCS blowouts and kneels leave the ridge fit).model_ledger.json— new repo-root machine-readable ledger of every trained/fitted model across the ecosystem (73 entries): league, artifact, training script, trained-season window, available-data window, window rationale, and known provenance gaps. Seed registry for per-artifact model cards.
CFB — loaders for 3 published-but-unreachable dataset releases
The CFB producer publishes to sportsdataverse-data, but a release asset is
only reachable once a loader exists on this side — the two repos are
independent, so publishing does not make data queryable. Three tags shipped
without one:
| loader | tag | 2024 shape |
|---|---|---|
load_cfb_adv_team_gamelog | espn_cfb_adv_team_gamelog | 1,892 × 90 |
load_cfb_ratings_weekly | cfb_ratings_weekly | 2,068 × 16 |
load_cfb_team_summaries_weekly | cfb_team_summaries_weekly | 2,119 × 384 |
load_cfb_adv_team_gamelog— opponent-adjusted team efficiency joined to the game contextadv_teamlacks (opponent, home/away, scores, margin, result, date). One row per team-game, 2004–2025.- The two
*_weeklyloaders return long format: one asset per season carrying athrough_weekcolumn that stacks every week's cumulative state, so a consumer filtersthrough_week == Wfor as-of-week-W ratings rather than fetching per-week assets. Only the opponent-adjusted team products ship weekly — the ridge is refit on everything through week W, so that state cannot be reconstructed by summing per-game rows.
CFB — 5 summaries loaders unblocked for 2004–2013
load_cfb_passing, load_cfb_percentiles, load_cfb_receiving,
load_cfb_rushing, and load_cfb_team_summaries raised
SeasonNotFoundError("season cannot be less than 2014") for seasons that are
in fact published. The producer's offline-rebuild backfill now ships 22
seasons (2004–2025) for all five tags; the loader-side min_season guard was
never widened to match, so 10 published seasons per dataset (2004–2013) were
unreachable through the public API.
- Verified against the live releases before changing the guard — all four probe seasons (2004/2010/2013/2014) return HTTP 200 for these five tags.
load_cfb_play_participantsand the two crosswalk loaders keep their 2014 floor — that one is a real data cliff, not a stale guard: ESPN ships no per-playparticipants[]before 2014, and those tags genuinely 404 pre-2014. Widening them would have converted a clear error into silent empty frames.- Declared
seasondtype correctedFloat64→Int64forload_cfb_passing/_receiving/_rushing/_team_summaries. The republished assets shipInt64; the declared schema drives the generated returns tables only, so the drift was invisible to the test suite and surfaced as incorrect published documentation.
NBA — human-readable player and team columns for model outputs
Model outputs keyed on bare numeric ids now also carry resolved player and team names, so a published table is legible without a second join.
NBA — read the committed raw store over URL + season-level captures
The possessions / season-compile path prefers the committed raw JSON store over network reads, making season builds reproducible offline.
Fixes
- NBA — result-set schema inference scans all rows rather than the first; a leading run of nulls previously inferred the wrong dtype for the whole column.
- NBA — three real-data bugs in the raw-store impact path.
- NBA — tunable timeout + retry for the
stats.nba.comruntime. - RDS —
write_rdsstreams to its sink instead of buffering the whole object; the temp-file write path keeps the umask-derived file mode. - NFL — NGS season-aggregate (week 0) upstream removal is handled with a warning + xfail sentinel instead of a hard failure.
- NHL — survive the R-arrow
vctrsextension-metadata panic when reading release parquets. - CFB —
load_cfb_passingdeclared schema tracks the republishedInt32sacked/pass_int. - Docs — Docusaurus builds via rspack (
future.v4), resolving the Vercel heap OOM.
0.0.72 Release: July 22, 2026
BREAKING CHANGES
-
NBA season convention is now END-year across the Python API.
compile_nba_season(2024)andnba_availabilitynow use the season ENDING year (2024 = 2023-24), matchingmost_recent_nba_season()and every ESPN-sourcedload_nba_*dataset. Previously the stats.nba.com compile path used the start year — external callers passing a start year must add 1. Unchanged:year_to_season(still a low-level start-year helper — call it asyear_to_season(end_year - 1)), andnba_box_logs(takes the"2023-24"string, not an integer). -
idin Python-produced WBB play-by-play frames is Int64 (was Float64 in R-produced releases). R/jsonlite has no int64, so the released pbpidloses precision above 2^53 — adjacent ~4e17 play ids round to the same double and collide (verified: multiple plays per game share an id in the released assets; fixture game 401804834 yields 447/447 unique ids where Float64 collided). The stored payload carries a true integer, sohelper_wbb_play_by_playnow emits exact Int64 — a deliberate dtype divergence from the R releases, pinned by the wehoop-wbb-data parity suite'sdtype_upgradesgate (values still compared equal under the oracle's lossy Float64 view).
CFB — loaders for 6 published-but-unreachable dataset releases
cfbfastR-cfb-data publishes 18 dataset tags to sportsdataverse-data; sdv-py
had loaders for 9. These six were built, backfilled, and published for years
with no way to read them from Python:
| loader | tag | seasons | cols |
|---|---|---|---|
load_cfb_passing | espn_cfb_passing | 2014– | 43 |
load_cfb_rushing | espn_cfb_rushing | 2014– | 28 |
load_cfb_receiving | espn_cfb_receiving | 2014– | 32 |
load_cfb_percentiles | espn_cfb_percentiles | 2014– | 27 |
load_cfb_team_summaries | espn_cfb_team_summaries | 2014– | 383 |
load_cfb_model_pbp | espn_cfb_model_pbp | 2004– | 43 |
The first five are the season-level "Binion Box Score" tables (opponent-adjusted
team summaries + passing/rushing/receiving leaderboards + per-metric
percentiles); load_cfb_model_pbp is the EP/WP/QBR-enriched play-by-play built
by that repo's model pipeline. Every returns-schema is introspected from the
real published parquet rather than derived, and a live-gated contract test
asserts the declared schema against the published asset so a producer-side
rename or dtype flip can't leave the docs table lying.
espn_cfb_injuries is intentionally not loadable: its release exists but
ships zero assets (write_dataset skips 0-row frames and ESPN's CFB injuries
feed yields nothing), so a loader would 404 on every season.
CFB — load_cfb_ratings dataset loader
load_cfb_ratings(seasons)— loads the publishedcfb_ratingsrelease (one row per team per season, 2004–): opponent-adjusted offensive / defensive / special-teams EPA, FEI,games,off_pace, denseoff_rank/def_rank/net_rank, andnet_z. The tag is produced bycfbfastR-cfb-data'scfb_model_publish ratingsbuilder running sdv-py's own :func:sportsdataverse.cfb.cfb_ratingsover the releasedespn_cfb_pbpplay-by-play, so the loader's returns-schema is the compute function's output schema — a contract test pins the two together (both column order and dtype) so a producer change can't silently leave the published returns-table lying. Like every release loader it is 404-safe: seasons with no published asset are skipped with a warning rather than raising.
NBA / WNBA — CTG play context (T3.6): possession/shot/lineup/player tables + start-type oracle
nba_play_context/wnba_play_context— Cleaning the Glass recreation on the shipped possession engine. Per-possession context (possession_start_typecoarse family +possession_start_type_detailzone-split + the fivepossession_start_type_ctgbuckets,is_transition/transition_source,seconds_to_first_play,is_garbage_time/garbage_time_basis,is_heave_possession) and per-shot context (ctg_shot_zone,is_putback,is_second_chance_shot,shot_context).wnba_play_contextis a real shim (wnba_engine), byte-identical to the NBA core on WNBA fixtures.lineup_play_context/player_play_context— on/off possessions + points per 5-man unit and per player, sharing one aggregation core; the OFF side is derived by subtraction so the on/off split is exact by construction.starters_on_court_counts— implements CTG's garbage-time "<=2 starters on floor" clause; when starter data is joinedgarbage_time_basisupgrades frommargin_onlytomargin+starters(containment-verified against margin-only).- Faithful pbpstats possession start-type — boundary-only timeout detection
(a port of
possession_has_timeout/previous_possession_has_timeout, incl. the asymmetric FT-sandwich technical carve-out), exact CTG shot-zone boundaries from the legacy coordinates (the v3shot_distancecolumn isInt64, rounded to whole feet), and ateam_id == 0team-rebound discriminator (the v3 feed stuffs the team id intoperson_id). Validated like-for-like against pbpstats-live: 99.50% coarsepossession_start_typeagreement (592/595) across the committed fixtures (test_nba_play_context_oracle.py, gated onSDV_PBPSTATS_ROOT).
Fixes
-
fix(cfb):
cfb_ratings()now works against the releasedespn_cfb_pbpasset it documents itself as loading. The function advertises that it pulls play-by-play viaload_cfb_pbp, but that path had never been exercised — the module was built and gated entirely against a 14-column fixture carrying cfbfastR-canonical names, while the published asset is ESPN-shaped. Every real call (cfb_ratings(2023)) raisedKeyErroronpos_team_id/def_pos_team_id/home/neutral_site, then onplay_type/drive_id. The orchestrator now normalizes the released field names (start.pos_team.id,start.def_pos_team.id,homeTeamId,type.text,drive.id, plusneutral_siteoff the schedule join), aliasing only when the canonical name is absent so callers passing an already-canonical frame are unchanged. The HFA term is now guarded onpos_team/homedtype agreement — it derives frompos_team == home, which across mismatched namespaces silently marked every play a road play rather than failing. Verified on the real 2023 asset (153,625 plays → 227 teams); the oracle gates hold on released data (adj_netvs FPI 0.926, vs SP+ 0.936;adj_offvs SP+ off 0.846;adj_defvs SP+ def 0.793). -
fix(codegen): all codegen/capture writers now emit LF explicitly (
newline="\n"), matchinggenerate.py's convention. On Windows the text-mode default translated\nto CRLF, so every codegen-test run (which re-runs the pff / 247 site-pages generators) left ~65 endpoint + schema YAMLs dirty with line-ending-only churn against the LF-normalized index. -
feat(nfl): PFF Premium auth can now auto-refresh from a saved Playwright
storage_state. PointSDV_PY_PFF_STORAGE_STATEat astorage_stateJSON captured once from a headed login andpff_runtimereplays it headlessly so Clerk re-mints the short-lived__sessioncookie, extracting fresh_premium_key+__sessionand caching them in-process forSDV_PY_PFF_STORAGE_STATE_TTLseconds (default 300 — one browser launch per window, not per request). It slots in as auth tier 3 (explicitcookies=>SDV_PY_PFF_*env cookies > storage_state), so existing callers are unaffected. Needs the new optionalplaywrightextra (pip install sportsdataverse[pff]thenplaywright install chromium); a missing install raises a clearImportError. The browser refresh is injectable (refresher=) so the wiring is fully offline-testable. -
fix(nfl): the DynastyProcess CSV loaders (
load_nfl_ff_playerids,load_nfl_ff_rankings) retry with exponential backoff on transient upstream errors (HTTP 429/5xx) instead of failing on the first hit — the raw-GitHub host rate-limits parallel CI runners. -
fix(codegen): reference-doc Returns prose no longer mangles docstrings whose return description is an inline
col: dtype, ...code span.docstring_parsersplits a Google-styleReturns:body on the first colon, so a colon inside the span was mistaken for the type/description separator — dropping the leading column/key and leaving a stray unbalanced backtick. The renderer now recombines the mis-split fragment (only when the parsed type contains a backtick, so legit Google types likepl.DataFrameare untouched), fixing ~15 rendered Returns across the cfb/mbb/wbb/nba/nfl/mlb/wnba reference pages. -
fix(dl_utils):
download()now retries transient status codes (403/408/429/500/502/503/504) with the sameRetry-After-aware backoff it already used for connection failures — previously a 429/403/5xx came back as a normalResponseand was returned without a retry (the root cause the DynastyProcess loader-level retry worked around). The retryable set is configurable via the newretry_statuses=param; when the budget is spent the last response is returned unchanged (callers still key on.status_code), and non-2xx responses are no longer cached. 403 is retried by default because ESPN's Core v2 API returns it under load —download()is the ESPN/nflverse gateway and does not serve the auth'd endpoints. -
fix(mbb): the NCAA fetch layer proves the bm-verify solve instead of assuming it.
_solve_challengelatched_challenge_solved = Truewithout checking whether the Akamai sensor actually passed, so a failed solve poisoned the session — every in-page fetch thereafter returned an unsolved 15-byteNCAA Statisticsstub (invisible to both the ban check and marker matching; 1,485 of them in one live run) while the layer kept hammering, which is what earned the IP a ban. The fetch is now the proof: an unsolved response forces a real re-solve and retry (solve_attempts, default 2 — deliberately low, rotating to a fresh proxy recovers faster), and exhaustion raises so the existing rotate-on-transport-error path moves to a fresh proxy. The stub response is captured live and pinned as a test fixture. (#266) -
fix(wbb): stop trusting ESPN's
header.competitions.boxscoreAvailableflag — it is false for most pre-2014 WBB games even whenboxscore.teams[].statisticsis fully populated, and the box helpers' faithful port of the R gate bug-matched that wrong oracle (the root cause of the WBB 2006–2013 dataset coverage hole:team_box_2009..2013held 10–280 rows against ~5,400-game seasons). Availability is now derived from the payload itself; genuinely boxless games still return typed-empty. Deliberate divergence from the original R behavior (R fixed the same way in wehoop#64), tested on the real 2012 title-game payload. (#275) -
fix(mbb):
raw_game_efficiencyhonors the empty-in/empty-out contract for missing-season boxscores — a season with no released asset comes back from the loader as a column-less empty frame, which crashed the select withColumnNotFoundErrorinstead of returning the documented typed_EFF_SCHEMAempty. One boundary guard covers all callers (mbb_team_ratingsand thewbb_team_ratingswrapper inherit it). (#280) -
fix(cfb): future-season recruiting projections walk back to the newest available teams crosswalk — the crosswalk asset trails the calendar (capped at 2025 while a 2026 projection is already meaningful after early signing), and the missing season's column-less empty frame crashed
_crosswalk_names_to_espn, killing thecfb_recruiting_proj2016:2026 backfill on season 2026. Team-name → ESPN-id identity barely changes year to year, so the walk-back is safe; all-missing degrades to a typed empty. (#282) -
fix(nba):
compile_nba_seasonthreadsproxy_urlthrough game discovery (_season_game_index/_game_ids_for_season), not just the per-game fetches — on a datacenter host the unproxiednba_stats_leaguegamelogcall returned{}, the empty-in/empty-out contract turned it into an empty index, and the whole season compiled to zero games, best-effort, exit 0 — indistinguishable from success. Verified live from the failing environment (0 rows unproxied → 1,230 rows proxied for 2023).proxy_provideris now called N+1 times for an N-game season (documented). (#283) -
fix(nba):
box_featurescanonicalizes the realleaguegamelogparser'sfg3_mcolumn tofg3mat the boundary (applied only whenfg3_mis present andfg3mabsent) — the snake-caser emitsFG3M→fg3_m, so the first real-data run of the SPM/BPM box-feature surface crashed withColumnNotFoundErrorwhile every synthetic-fixture test passed. The regression test feeds the real column name and fails without the fix. (#158) -
chore(phf): PHF dataset URLs cut over from
raw.githubusercontent.comtree paths to dedicatedsportsdataverse-datarelease tags mirroring thepwhl_*convention (phf_pbp/phf_player_boxscores/phf_team_boxscores/phf_schedules, 29 assets uploaded per-file). PHF is a frozen dataset (league ceased operations June 2023; coverage 2016–2023, with 2017–2019 pbp a permanent upstream gap). (#218) -
calculate_nfl_standingsis deprecated in favor ofnfl_season_standings(the nflseedR-parity engine); the old name emits aDeprecationWarningshim.
Dependencies
- chore(deps): dropped the unused
pyreadrdependency (runtime deps + themodels/allextras and the conda recipe). It had no live imports — its only reference was a long-commented.rdscontracts loader — and its removal also drops thelibbz2/liblzmasystem-header requirement that pyreadr's sdist build imposed on Python 3.9 Linux installs.
Release utilities — sportsdataverse.release (sportsdataversedata R-package port)
- feat(release): new
sportsdataverse.releasemodule — a Python port of thesportsdataversedataR package (v0.0.11) used to publish data assets tosportsdataverse/sportsdataverse-dataGitHub releases. Surface:sportsdataverse_save()(writes rds / csv / csv.gz / parquet — the R default set — withsportsdataverse_type+sportsdataverse_timestampstamped into parquet metadata and as R attributes on the rds frame, plus R-parityseason/weekinteger coercion),sportsdataverse_upload()(addstimestamp.*/package_function.*sidecars; retries the whole upload with the sameSPORTSDATAVERSE.UPLOAD.*env-var backoff config the R package reads), plusgh_cli_release_upload/gh_cli_release_tags/gh_cli_release_assets/gh_cli_rate_limits/gh_cli_available. Backed by golden-fixture parity tests generated by running the real R functions (tests/fixtures/release/). Deliberate divergences (documented in the module docstring): onegh release uploadinvocation per file (the multi-file form silently drops large assets), the R-onlyqsformat raisesValueError, unpaddedsize_string, no.tokenargument (gh CLI auth;GH_TOKENfalls back toGITHUB_PAT). - feat(release): native
.rdswriting without R or any new dependency —sportsdataverse/_rds.pyimplements RDS version-2 (XDR) serialization for data.frames of atomic columns (integer / double / logical / character / Date / POSIXct, with NA fidelity, UTF-8 strings, int64→double promotion beyond int32 range, and R symbol back-references). Output is validated byte-for-byte against R 4.5.3saveRDS()via the committedrds_golden.rdsfixture (serialization header excluded) and via a livereadRDS()oracle at fixture-capture time. Nested/list columns raiseValueError.
PWHL — coordinate-based xG (T5.3b): xg_method default flips quality → coords
- feat(pwhl)!:
pwhl_team_game_xg_rates/pwhl_ratings_from_proxygain anxg_methodparameter ("coords" | "quality") and the DEFAULT is the new coordinate distance/angle logistic xG (fit_pwhl_coord_xg, geometry fromhockeytech._analytics.add_shot_distance_angle) — output values change for existing callers that relied on the implicit categorical shot-quality proxy; passxg_method="quality"to keep the old behavior.LEAGUE_CONSTANTS["pwhl"].margin_sdre-fit 1.21 → 1.19 (coords-paired, 2025-only). Held-out 2026 (n=107): coords Brier 0.2444 vs quality 0.2449 vs naive 0.2500 (within noise; gates stay no-worse-than-naive + calibration).
PWHL — per-strength xG calibration + geometry hardening (T5 follow-up)
- feat(pwhl):
fit_pwhl_coord_xggainscalibrate_strength=True(default) — when the frame carries strength columns it fits a per-EV/PP/SH Platt recalibrator (PwhlCoordXGModel.strength_calibrators) that shrinks residual within-bucket per-strength calibration error at ~zero AUC cost (held-out LOSO SH 10-bin ECE 0.0130 → 0.0091, AUC 0.6962 both). Default xG output shifts slightly for strength-bearing frames; identity (unchanged) on 2-feature or thin frames, or withcalibrate_strength=False. Buckets are shooter-relative (is_pp/is_sh). - fix(pwhl,hockeytech): harden the shot-xG geometry against the dual-frame
coordinate footgun —
fit_pwhl_coord_xg/predictnow raise on a RAW-scale (0–600) enrich frame instead of silently scoring it withgoal_x=89, andhockeytech._analytics.add_shot_distance_angleassertsgoal_xis in a plausible rink range (default is the documented NHL-size-rink constant).
CFB — advanced-efficiency spine (opponent-adjusted efficiency/explosiveness/havoc → field position → adjusted tempo)
- feat(cfb):
cfb_advanced_stats— Connelly-style opponent-adjusted offense/defense efficiency, explosiveness (isoPPP), success rate, EPA/play, and havoc fromload_cfb_pbpwith garbage-time filtering. Validated on 2021 vs CFBD advanced stats + SP+: success-rate Spearman 0.954/0.959 (off/def), opponent-adjusted EPA ranks vs SP+ component ranks 0.866/0.843. - feat(cfb):
cfb_field_position— team drive-start field-position value model on a bundled EP-by-yardline curve (fit_field_position_ep/load_fp_curve); avg-start vs CFBD 2021 Spearman 0.897, MAE 0.67 yards. - feat(cfb):
cfb_adjusted_tempo— opponent-adjusted, situation-neutral pace (raw plays/game vs CFBD 2021 Spearman 0.902; adjusted pace validated as a monotone re-ordering of raw, Spearman 0.948). - feat(cfb):
cfb_advanced_constants— shared success/explosive/garbage-time flags + metric helpers and the iterative opponent-adjustment solver. - Committed 2021 CFBD advanced + SP+ oracle corpus under
tests/fixtures/cfb_advanced/with provenance README (hosted pbp covers 2002-2021; 2022+ is a producer-backfill gap).
NFL — NGS over-expected tracking spine (YAC-OE → RYOE → separation-OE → man/zone rates)
- feat(nfl):
nfl_ngs_yac_oe— receiving YAC-over-expected with empirical-Bayes shrinkage (weekly-σ² estimator); shrunk estimates are more next-season-stable than raw (stability oracle: corr(shrunk₂₀₂₂, raw₂₀₂₃) ≥ corr(raw, raw), n=80). - feat(nfl):
nfl_ngs_ryoe— rushing yards-over-expected per attempt with the same shrinkage machinery. The 2022→2023 RYOE stability gate is a documented strict xfail: the base year-over-year signal on that transition is statistically zero (raw→raw corr 0.045, n=33), so no estimator can beat it — escalation noted in the test. - feat(nfl):
nfl_ngs_separation_oe— expected-separation ridge (cushion + air-yards + alignment one-hots, intercept unpenalized) with a positive cushion coefficient by construction; the separation-OE stability gate is a strict xfail on the underpowered 2022→2023 transition. - feat(nfl):
nfl_ngs_man_zone_rates— team-level man/zone coverage snap rates (one row per season/defteam) from the NGS tracking panel. - feat(nfl):
nfl_ngs_constants— shared empirical-Bayes shrinkage, weekly-σ² identification, expected-separation ridge, and the dtype-guardednext_season_stabilityjoin (asserts amin_noverlap floor so a shrunken fixture cannot let a stability gate pass on a handful of players). - Committed NGS panel fixtures + fitting scripts under
tests/fixtures/anddev/nfl_ngs/.
NFL — scheme & special teams spine (play-call model → game script → kicker/punter value → line grades)
- feat(nfl):
nfl_playcall— run/pass + play-family classifier (bundlednfl_playcall.ubj, trained 2016-2021) that beats the shippedxpass_modelon held-out 2022-23 (log-loss 0.498 vs 0.518, AUC 0.822 vs 0.798) with PROE reconciling to pbppass_oeexactly; team tendency profiles included. - feat(nfl):
nfl_gamescript— expected plays / pace / game-script curves (held-out 2023 expected-plays MAE 1.77). - feat(nfl):
nfl_kicker_rating— environment-adjusted FG make probability + empirical-Bayes FGOE (held-out 2019-23 decile calibration gap 0.038; the systematic bias traced to nfl4th's long-kick decision clamp and corrected with a fitted term). - feat(nfl):
nfl_special_teams— per-unit ST EPA decomposition (sums reconcile to team ST EPA exactly) + punter net-over-expected (YoY stability 0.62/0.55). - feat(nfl):
nfl_line_grades— OL/DL pressure-based grades from pbp + PFR advstats (pbp-vs-PFR pressures Spearman 0.794). - feat(nfl):
nfl_scheme_constants— shared metrics/constants + as-of split. - Committed fixture corpus + fitting scripts under
tests/fixtures/anddev/nfl_scheme/; known upstream issue flagged:load_nfl_pbp_participationcrashes on multi-season loads (cross-season schema drift) — work around per-season withhow="diagonal_relaxed".
NFL — projection & draft spine (player projections → usage shares → availability → draft model)
- feat(nfl):
nfl_projection— next-season player stat projections with integer-age aging curves and fitted per-position damping (holdout-2024 Spearman QB/RB/WR/TE 0.61/0.72/0.66/0.73; beats naive carry-forward for QB/WR/TE, RB shortfall shipped as a documented strict xfail). - feat(nfl):
nfl_usage_projection— team-internal target/carry share projections that sum to 1.0 by construction (share Spearman RB/WR/TE 0.73/0.65/0.74 on the 2024 holdout). - feat(nfl):
nfl_availability— expected games played from historical availability (MAE 3.54, decile calibration gap 0.049). - feat(nfl):
nfl_draft_model— combine + college-production draft-position model (Spearman 0.587 vs realized draft slots, n=1269; hit-rate calibration gap 0.08) withw_avcareer labels (car_avupstream is all-null). - feat(nfl):
nfl_projection_constants— shared metrics, league constants, and the as-of split the backtests enforce; constants fitted on 2022/2023 as-of folds only (single-evaluation 2024 holdout). - Committed fixture corpus under
tests/fixtures/nfl_projection/with provenance README.
NFL — ratings & market spine (power ratings → win prob → spread/total → player props)
- feat(nfl):
nfl_ratings— opponent-adjusted ridge team ratings (offense/defense/net EPA per play + HFA) fromload_nfl_pbp, validated vs ESPN FPI (Spearman 0.890) and raw team EPA (0.965, 32/32 matched). - feat(nfl):
nfl_market— pregame win probability (Brier 0.232, quintile calibration gap 0.036), spread and total projections (MAE 2.96 / 3.24 vs closing lines over 208 as-of games, weeks 5-18), fitted constants (points-per-net 23.36, HFA 3.12, margin sd 13.02) from committed fitting scripts. - feat(nfl):
nfl_player_props— empirical-Bayes player projection distributions for passing/rushing/receiving yards with over-probabilities (MAE 70.5/21.1/21.4 vs realized; p_over calibrated against lagged pseudo-lines — ESPN purges historical propbets, documented in-test). - feat(nfl):
nfl_prediction_constants— shared metrics (Brier, log-loss, Spearman, calibration tables), league constants, and the as-of season/week split helper the backtests enforce. - Committed fixture corpus under
tests/fixtures/nfl_prediction/with provenance README.
CFB — recruiting & roster-projection spine (talent composite → returning production → wins projection → transfer impact → draft projection)
- feat(cfb):
load_recruit_classes— per-recruit signing classes from the 247 RDB feed (signed institution with committed fallback; stars, composite grade, position, player name; idsUtf8). - feat(cfb):
blue_chip_ratio+cfb_roster_talent— Bud Elliott blue-chip ratio over a trailing 4-class window and a 247-style class-recency-weighted team talent composite (optionalcomposite_247snapshot override). Oracle-gated vs the 2023 247 Team Talent snapshot (Spearman 0.896, 196/196 teams name-matched) + a percentile champion blue-chip invariant. - feat(cfb):
cfb_returning_production— Connelly-style returning production (offense = attributed yardage; defense = splash events) from the hosted per-play player-stats parquet + rosters; unit weights fitted on FBS 2018-2023 (offense-only; retention gate Spearman 0.229). - feat(cfb):
cfb_recruiting_projection— on-demand ridge projecting wins / scoring margin from preseason features (talent, blue-chip ratio, returning production, prior wins) with the as-of season boundary enforced internally. Backtest 2019-2023: pooled wins MAE 2.19, beating prior-year (2.46) and league-mean (2.34) baselines. - feat(cfb):
cfb_transfer_moves+cfb_transfer_impact— transfer-portal moves from roster year-over-year diffs (name-matched recruit talent points) and a net-talent win-delta ridge. The predictive gate is a documented strict-xfail: net transfer talent shows no team-level win-delta signal on 2018-2023 data (escalation: position-specific values + PFF NCAA grades). - feat(cfb):
load_draft_outcomes+cfb_draft_projection— NFL draft labels from the nflverse picks dataset (the ESPN season-draft endpoint 404s) and an as-of logistic draft-probability model (stars, talent points, career production, class year) with per-team expected-picks roll-up. Holdout AUC 0.78-0.82 (2022-2024 drafts); team draft-capital Spearman 0.62 observed. - Committed oracle fixtures under
tests/fixtures/cfb_projection/(results, 247 talent + 39.6k recruits 2014-2023, returning production, team map, nflverse draft picks, per-player production, net transfer talent) with provenance README.
NBA / WNBA / G-League — shot-value spine (xPoints → context make-prob → talent → selection → zone maps)
- feat(nba):
score_shot_xpoints+xpoints_baseline— per-shot expected points from theLeagueAverageszone-FG% table thatnba_stats_shotchartdetailreturns for free (no bundled artifact, compute-on-demand). Asserts the three zone join keys share dtype before joining;game_idstaysUtf8(zero-padded),player_id/team_idInt64. - feat(nba):
make_prob_by_context+make_prob_joint— FG% by defender distance and shot clock (aggregateplayerdashptshotsbuckets, the only form the public API exposes) plus an independence-combined joint via odds multipliers. - feat(nba):
shooter_talent— regressed make%-above-expected per shooter (split-half-fitted shrinkagek; NBA/G-League 70.1). - feat(nba):
shot_selection_quality(player expected value per shot vs the league) +zone_value_map(per-player per-zone points / expected points). - feat(nba):
nba_shot_valueorchestrator (fetch → score → all five models,include_context=) +nba_shot_value_lineups(shotchartlineupdetailvariant). One league-agnostic core switched byleague_id("00"NBA,"10"WNBA,"20"G-League); court geometry + shrinkage constants keyed by league. - feat(wnba):
wnba_shot_valueby-reference shim (league_id="10") + the five model functions re-exported; G-League isnba_shot_value(..., league_id="20").
MBB / WBB — shot-quality spine (xPoints → shot selection → shooter talent)
- feat(mbb): canonical shot frame + dual-source adapter (
mbb_shot_data,espn_shots_to_canonical,shot_events_to_frame, geometry classifiers) — normalizes the ESPN shots release (basket-anchoredcoordinate_*_rawhalf-court grid, court scale FITTED from rim-make origins + made-three distances, int32-sentinel rows dropped) and the NCAA HTML shot charts (create_shot_event_dataoutput; source axes swapped to the canonical lateral/up-court orientation) into one schema with pinnedUtf8ids. - feat(mbb):
mbb_shot_quality_model+mbb_shot_quality— compute-on-demand empirical-Bayeszone × typemake-rate/xPoints table (cells shrunk toward the parent-zone mean) and the per-shotxmake/xpointsscorer. No bundled artifact. Oracle-gated offline: temporal train/holdout calibration (Σ xpoints / Σ actual ≈ 1.00, per-zone bands ≤ 0.03) and blended 2P/3P within ±0.02 of observed Barttorvik national aggregates. - feat(mbb):
mbb_shot_selection— per shooter/team expected points per attempt vs a league-average shot mix (selection_value, attempt-weighted zero-sum by construction). - feat(mbb):
mbb_shooter_talent+fit_shrinkage_k— per-shooter make%-over-expected regressed by a split-half-fittedk(mens 233.2, womens 92.4); reliability gated on splits the fit never saw. - feat(wbb): by-reference shims for all of the above
(
wbb_shot_data/wbb_shot_quality*/wbb_shot_selection/wbb_shooter_talent) with era-matched women's gates (season 2026 — thewbb_shotsrelease floor).
MBB / WBB — player-value & projection spine (box-BPM → archetypes → recruiting → transfer → draft)
- feat(mbb):
mbb_box_bpm— team-constrained box Plus/Minus from a game-level minutes-weighted fit (lineup-free APM identification); bundled ridge artifact; oracle-gated vs Barttorvik BPM (Spearman 0.88 mens / 0.91 womens) plus an independent 125-game NCAA stint-RAPM validation at ~95% of the grain's noise ceiling. - feat(mbb):
mbb_archetypes— bundled KMeans role clusters (k=6 mens, k=8 womens incl. women-specific "midrange big" / "slashing guard"), gated on bootstrap ARI ≥ 0.70 + hand-labeled role-certain players. - feat(mbb):
mbb_recruiting_projection— expected freshman box-BPM from pre-arrival composite/rank/height (as-of safe); LOSO-gated ≥ 0.45 per held-out class. - feat(mbb):
mbb_transfer_projection+transfer_cohort— post-transfer box-BPM projection over the boxscore-discontinuity cohort (the roster release under-reports moves ~70×); beats the naive post=pre baseline. - feat(mbb):
mbb_draft_projection— dual-head draft probability (AUC 0.97+) + log-pick projection with tier bucketing; WNBA pick head's data floor documented as an xfail at the unlowered gate. - feat(wbb): by-reference shims + women's artifacts for all five models.
Recruiting — ESPN NCAA recruiting family + On3 rankings
- feat(espn): Core v2
recruitingfamily —espn_{league}_recruiting_years/espn_{league}_recruiting_players/espn_{league}_recruiting_rankingsacross the 7 NCAA-scope leagues (cfb, mbb, wbb, college baseball/softball, m/w college hockey).recruiting/{year}/athletesships inline athlete objects that flatten to a 33-column recruit frame (identity, grades, recruiting class); all three shorts route throughparse_itemsvia thereturn_parsedshim. Live-captured MBB fixtures + offline tests. - feat(cfb): On3 recruiting rankings stem (
on3_player_rankings,on3_industry_player_rankings,on3_team_rankings,on3_industry_team_rankings) over on3.com's Next.js data routes — the only public JSON surface; industry = On3/Rivals/247Sports/ESPN consensus, including NIL valuations and commitment / transfer status.on3_runtime._getauto-discovers the rotating Next.jsbuildIdfrom the rankings page and refreshes it once on the stale-buildId 404 (an unchanged buildId is treated as an authoritative miss). Returns-schemasnative/on3/*with all 224 column descriptions authored; trimmed real-capture fixtures + offline runtime, parser, and wiring tests. - feat(cfb): 247Sports Recruit Database stem (11 wrappers) over
ipa.247sports.com/rdb/v1/—sports247_recruits(individual recruit rankings: 247 + industry-composite ratings/stars/ranks, commit status),sports247_transfers(transfer portal),sports247_coaches,sports247_target_predictions(expert "crystal ball"),sports247_institution_rankings/sports247_teams/sports247_composite_team_ranking_feed/sports247_transfer_portal_team_feed/sports247_transfer_portal_player_feed/sports247_sport_years/sports247_tags_autocomplete. One genericparse_sports247_result_setcovers every payload shape (bare array /{players|results|rankings|list: [...]}envelope / scalar array / single object). The Fastly edge fingerprint-blocks plainrequests, so the runtime uses lazy-optionalcurl_cffiChrome impersonation with an injectable transport (thenba_statspattern) and normalizes slash-less paths (the RDB 301s them). Most routes need anAuthorization: Bearerguest JWT —GET https://247sports.com/mints one with no login (~12 h TTL); the runtime mints/caches/refreshes it automatically (re-mints once on a 401/403). The ~14 remaining routes stay 403 even with the guest token (logged-in/premium) and are not wrapped. Returns-schemasnative/sports247/*with all 211 column descriptions authored; real-capture fixtures + offline parser/runtime/wiring tests.
MBB / WBB — prediction & tournament stack (ratings → pregame → in-game WP → résumé → bracketology → Monte Carlo)
- feat(mbb): opponent-adjusted team ratings (
mbb_team_ratings: AdjO/AdjD/AdjEM/AdjTempo KenPom-style fixed points) oracle-gated vs barttorvik 2024 (Spearman 0.990, MAE 2.37). - feat(mbb): closed-form pregame predictions (
predict_margin/win_prob_from_margin/predict_total+ vectorizedmbb_predict_games) with backtest-fitted constants (em_scale/HFA/sigma/tempo-anchor, joint least squares on a leakage-free as-of-date walk of 2024); gates: Brier beats the ESPN BPI predictor (0.2006 vs 0.2031), spread MAE 1.95 / total MAE 2.90 vs the closing line. - feat(mbb): in-game win probability (
in_game_features+mbb_in_game_win_prob) from a bundled shallow-xgboost artifact trained on 2023 pbp; out-of-sample 2024 decile calibration max gap 0.0298 (gate ≤ 0.03). - feat(mbb): strength of schedule + NET-style Quad 1–4 résumé + Wins Above Bubble
(
mbb_strength_of_schedule); SoS Spearman vs ESPN BPI SOS 0.923. - feat(mbb): bracketology (
mbb_bracketology/project_bracket+ conference auto-bids) — committee-style résumé blend, 68-team field selection, seeds; seed-order Spearman vs the actual 2024 committee 0.938. - feat(mbb): season + bracket Monte Carlo (
mbb_season_sim/mbb_bracket_sim/simulate_game, seeded + deterministic); neutral-site calibration slope 1.03 on 413 neutral games (includes the NCAA tournament). - feat(wbb): full women's parity — six thin shims (
wbb_team_ratings,wbb_predict_games,wbb_in_game_win_prob,wbb_strength_of_schedule,wbb_bracketology,wbb_season_sim/wbb_bracket_sim) over the league-agnostic mbb cores with women's fitted constants + a women's-trained WP artifact; every oracle gate re-run on WBB 2024 at the same thresholds (Torvik 0.995, SoS 0.985, seed-order 0.976, WP deciles 0.0224). - test(mbb,wbb): committed 2024 oracle corpora under
tests/fixtures/{mbb,wbb}_prediction/(results/team-box/barttorvik/ESPN BPI/predictor/odds samples/pbp WP samples/actual NCAA tournament seeds) with provenance READMEs; all gates run offline in CI.
NBA — external concurrent validity + walk-forward retrodiction (WP3)
- feat(nba): model-zoo v2 WP3 — Oracle 5 external concurrent validity
(
external_validity, five published-metric loaders innba_oracle_data.py: Ryan Davis RAPM, Dunks & Threes EPM, LEBRON season/daily, daily plus-minus (DPM), Dunks & Threes ewins) and Oracle 6 walk-forward retrodiction (walk_forward, time-ordered "predict tomorrow" with carry-forward and random-fold baselines) innba_model_validation.py.
NBA — RAPM variants (WP2)
- feat(nba): RAPM variants (
nba_rapm_variants) — luck-adjusted (nba_la_rapm), four-factor (nba_four_factor_rapm), and time-decay (nba_decay_rapm) RAPM, all reusing the plain-RAPM design matrix; concurrent-validity vs the Ryan Davis oracle CSVs gated onSDV_PY_NBA_ORACLE_DIR.
NBA — through-date ratings panel, WAR, and single-game BPM (WP4)
- feat(nba): through-date ratings panel (
nba_ratings_panel+ratings_as_ofprimitive, leakage-free by construction — works with any harness model), WAR layer (nba_war+calibrate_pts_per_win/calibrate_replacement_levelcalibration helpers), andnba_bpm(granularity="game")single-game BPM 2.0.
NBA — v3-to-v2 play-by-play adapter (nba_v3_to_v2_pbp)
New sportsdataverse/nba/nba_v3_v2_adapter.py ports hoopR's .v3_to_v2_format() to Python:
nba_v3_to_v2_pbp(pbp_v3, box_v3, *, return_as_pandas=False) turns a playbyplayv3 payload +
boxscoretraditionalv3 boxscore into the full 61-column v2-schema frame (NBA retired playbyplayv2,
which now returns 0 rows for every season — this restores v2-dataset compatibility from the live v3 feed).
- Recovers the secondary players v3 drops: assist via the
(Name N AST)description parenthetical, block/steal via the standaloneactionType == ""rows (the blocker/stealer ships aspersonId, associated to the shot/turnover at the same period+clock), sub-in viaSUB: X FOR Y, and jump-ball viavs. / Tip to— each resolved through a 4-tier roster name-match (family → name_i → "F. Family" → fuzzy). Validated 1-to-1 against the cdn live feed's structuredassistPersonId/blockPersonId/stealPersonIdfields: 100% agreement on all three committed fixture games. Documented gap: the foul-drawn player is unrecoverable from v3 (fouls carry nullplayer2/player3). - v2 schema faithful to hoopR — event/action-type codes (EVENTMSGTYPE/EVENTMSGACTIONTYPE),
home/neutral/visitor description split by
location, forward-filledscore/score_margin/team_leading, person types, time columns from the ISO clock, string-typed ids (leading zeros preserved), plus the v3 passthrough columns. One deliberate divergence:player2/player3are enriched by id from the extraction rather than hoopR's name re-resolution (which can mismatch on family-name collisions). - pbpstats interop: a
stats_nbafeed shim (resultSetsenvelope) lets the adapted frame drive thepbpstatslibrary's v2 provider. A gated round-trip test feeds our v3-derived output through pbpstats-stats_nbaand matches pbpstats' ownliveprovider on the same games (possession counts within 0–2, period starters exact 8/8). The round-trip surfaced two fixes:PLAYER1_TEAM_IDserializes as null (not 0) on team-rebound rows, and the"Transition Take"foul subtype (EVENTMSGACTIONTYPE 31, added with the 2022-23 transition take foul rule) joined the foul table. Opt in locally withSDV_PBPSTATS_ROOT=<path to a pbpstats checkout>. - Six cdn oracle fixtures committed under
tests/fixtures/nba_engine/{gid}/cdn_{playbyplay,boxscore}.json(provenance documented in the fixtures README); exported assportsdataverse.nba.nba_v3_to_v2_pbp.
NBA / WNBA — stats.nba.com / stats.wnba.com flat-API family (nba_stats / wnba_stats)
Two new codegen-generated flat-API stems wrap the official stats API surface:
nba_stats(sportsdataverse/nba/nba_stats.py) — 112 wrappers targetingstats.nba.com. League routing is a singleleague_idparameter on each endpoint:"00"→ NBA,"20"→ G-League,"15"→ Summer League. Namednba_stats_<slug>(e.g.nba_stats_leaguedashplayerstats,nba_stats_playercareerstats,nba_stats_boxscoreplayertrackv3).wnba_stats(sportsdataverse/wnba/wnba_stats.py) — 95 wrappers targetingstats.wnba.com(WNBALeagueID=10), namedwnba_stats_<slug>. Implemented as a thin shim re-exporting the NBA-stats runtime with the WNBA host.- Codegen surface = capture-confirmed live, non-deprecated endpoints only. The wrapper count is driven by a live capture sweep (committed under
sdv-internal-refs/nba/): endpoints that annba_api/hoopR/wehoopsource marks deprecated (lifecycle::deprecate_*, runtime warnings, or release-note retirements — 26 endpoints) and endpoints with no capture confirming a populated table (untested/barren/deadfor that league) are excluded. The full active/dying/barren/dead matrix lives insdv-internal-refs/nba/ENDPOINT_HEALTH.md. - One generic parser
parse_nba_stats_result_sets(raw, result_set=None, *, return_as_pandas=False)handles the uniform{resultSets: [{name, headers, rowSet}]}envelope. Returns a singlepolars.DataFramewhen aresult_setname is given or the payload has one set; returnsdict[str, DataFrame]for multi-set payloads (e.g.playercareerstats). Empty / malformed payloads return a zero-row frame; columns are snake-cased viadl_utils.underscore. It also handles the two non-uniform shapes in this family: the shot-location endpoints (leaguedash{player,team}shotlocations) whoseresultSetsis a single dict with 2-level grouped headers (flattened to composite columns likeless_than_5_ft_fgm), andscoreboardv3whose data lives underscoreboard.games(one row per game, home/away team objects inlined).parse_wnba_stats_result_setsis a re-export alias. - Browser-TLS runtime:
stats.nba.comTLS/JA3-fingerprint-blocks plainrequests(silent timeout, not an IP block). The runtime_getusescurl_cffiwithimpersonate="chrome".curl_cffiis a lazy optional import shipped under thetestsandallextras — not a hard runtime dep. A clearImportErrorguides users topip install curl_cffi(orpip install sportsdataverse[all]). The HTTP transport is injectable so wrappers and tests can run fully offline. - Wrappers default to
return_parsed=True(tidy polars DataFrame). Passreturn_parsed=Falsefor the rawDictorreturn_as_pandas=Truefor pandas. There is no user-facingheaders=param — the TLS impersonation is handled inside the runtime, not via a user token. - Generated from the enriched canonical catalog (
tools/codegen/gen_nba_stats.py) and registered inFLAT_APISintools/codegen/generate.py. Paramdefault/examplevalues are mined from the hoopR/wehoop roxygen signatures +@examples. Returns-table descriptions are authored for the pilot slugs and back-filled by column name from the SDV R-package docs (_r_col_desc); the remaining un-authorednative/nba_stats+native/wnba_statscolumns are a tracked follow-up exempted from the coverage ratchet viaextract_residual_columns._DEFERRED_BUCKETS(surfaced bydeferred_columns()).
NBA — possession event-detail columns, per-shooter shooting frame, game_date
- feat(nba): possession event-detail columns (
fg2a/fg2m/fg3a/fg3m/fta/ftm/oreb/tov), per-shooterbuild_possession_shootingcompanion frame, andgame_dateoncompile_nba_seasonoutput (possession cachePIPELINE_VERSION1 -> 2).
NBA — faithful possession boundaries (pbpstats parity)
- feat(nba):
_build_possession_groupsrewritten to pbpstatsstats_nbais_possession_ending_eventsemantics (and-1 + FT-trip exceptions, real-rebound and no-turnover filtering, jump-ball logic); technical FTs are inline again with team-filtered event detail (per-possession points identity preserved exactly). - feat(nba): possessions gain
dreb,number_in_period,possession_start_type(coarse vocabulary),count_as_possession; shooting frame gainsteam_id(possession cachePIPELINE_VERSION2 -> 3). - test(nba): pbpstats-live oracle gate — like-for-like possession counts +
boundary-by-boundary diff on the committed cdn fixtures (
SDV_PBPSTATS_ROOT).
NBA — quarter-box on-court lineup seeding + lineup_source="quarter_box"
- feat(nba): exact quarter-box on-court seeding —
players_on_court_from_quarter_boxscoresandlineup_source="quarter_box"onnba_possessions(auto chain: rotation -> quarter_box -> pbp), seeded from per-periodboxscoretraditionalv3range payloads. - fix(nba):
players_on_court_from_quarter_boxscoresgains an optionalraw_boxfull-game-boxscore name-map source (mirrorsplayers_on_court_from_pbp's own signature), closing a mid-period name-resolution gap that regressed one fixture to 0.8817 gamerotation-agreement; withraw_boxthreaded through, quarter_box now matchesplayers_on_court_from_pbpexactly on all 3 fixture games (0.9689 / 0.9686 / 0.9662).
CFB — 0.36-live pbp reconciliation + game-ending WP-perspective fix
Two bodies of CFB play-by-play parity work, each parity-test-first on real
captured fixtures (tests/cfb/test_cfb_036_reconciliation.py, with measured
before/after EPA/WP deltas). The reconciliation ports the valuable CFB pbp
fixes from the pandas 0.36-live branch into the polars main pipeline:
kneel-down flag + scrimmage exclusion; cleaned_text +
yds_rushed/yds_receiving direction-word parsing; box-score volume sort;
kickoff fair-catch → touchback (era-gated ≥2018); errored-punt end-yardline
flip; end_state_missing fill (end.team.id + end.yardsToEndzone
backfill); overtime play-sort by sequenceNumber; and the
penalty-assessed-on-kickoff EP/WP touchback — a refined penalty_flag guard
excludes Timeouts and completes the end-state touchback 0.36-live left
partial (resets down_*_end + pos_score_diff_end) so EPA neutralizes to
0.0 on the affected 2024 fixture plays. Interception return yardage is now
excluded from offensive / drive / total_yards at the aggregation sites
(the shared statYardage column is untouched, preserving the
penalty-residual chain), and statYardage==0 completions are reconstructed
from the yardline delta. Separately, a game ending on a possession-flipping
play whose type is absent from end_change_vec — notably a safety —
reported home_wp_after = 1.0 for the losing team; the game-ending WP
perspective is now correct.
CFB — advanced box score: player-name cleanup + cp aggregation
- feat(cfb): the advanced box score's player-name cleaning regex strips more
invalid cases, and the passing box score now aggregates
cp(the xCompletion model results) alongside the existing passing columns. Verified against the livetest_cfb_adv_box_scoresuite so existing box scores don't break with the new field.
NFL / CFB — season standings + simulation engines (nflseedR / cfbseedR ports)
- feat(nfl):
nfl_season_standings— a port of the nflseedR v2 standings engine with the real NFL division/conference/draft tiebreaker cascades and thetiebreaker_depthladder (RANDOM<PRE-SOV<SOS<POINTS). Namednfl_season_standingsbecausesportsdataverse.nfl.nfl_standingsis already taken by the api.nfl.com codegen wrapper. Parity vs real nflseedR 2.0.2 output for 2023: exact ondiv_rank/conf_rank/draft_rank/exit/games/true_wins/losses/ties/pf/pa/pd, <1e-9 on wins/win_pct/div_pct/conf_pct/sov/sos (golden fixtures undertests/fixtures/seedr/with R-provenance README). - feat(nfl):
nfl_simulations/nfl_compute_results— week-by-week season simulation with a pluggablecompute_resultsgenerator (default = the nflseedR ELO generator: init N(1500,150), +20 home, ×1.2 postseason, marginrnorm(estimate, sd=13)rounded away from zero, K=20 log-MOV), playoff rounds with reseeding +byes_per_conf, and the 5-frame summary dict (standings/games/overall/team_wins/game_summary). - feat(cfb):
cfb_standings+cfb_playoff_seeds+cfb_simulations/cfb_compute_results+cfb_games_from_schedule— the CFB adaptation (sibling of the NFL port), cross-validated against the new cfbseedR R package on a designed toy fixture (all 13 output columns × 9 teams agree ≤1e-6). Binding semantic ruling: the conference-rank tiebreaker cascade AND the reportedsov/sosare conference-REG-scoped (independents 0.0) — a deliberate divergence from nflseedR's overall games-weighted convention, caught and locked in by the Python↔R toy-fixture diff.CONF_CHAMPgames count toward the overall record and decide the champion but not the conference record/rank.cfb_playoff_seedsimplements CFP 12-team straight seeding (2025 rule): 12 best-ranked teams with the 5 highest-ranked conference champions guaranteed, optional committeerankings=frame.
MBB / WBB — bigballR + wbigballR port: ncaa_mbb_* / ncaa_wbb_* stats.ncaa.org family
Ports bigballR (MBB) and wbigballR (WBB) — the community
stats.ncaa.org R scrapers — as a first-class 33-function
ncaa_mbb_* / ncaa_wbb_* surface: one shared polars core in mbb/ +
thin WBB binding shims, built on the existing proxy-bound NcaaFetcher
transport (browser path for bm-verify game pages; no new transport code).
Surface: ncaa_{mbb,wbb}_game_pbp / _play_by_play (row-level pbp with
both-teams on-floor lineups, possession numbering, transition +
garbage-time flags), _box_scores, _team_schedule, _team_roster,
_date_games, _shot_locations, _join_pbp_shots, _lineups,
_player_lineups, _player_combos, _on_off, _player_stats,
_team_stats, _possessions, _team_ids (+ resolve_ncaa_team_id,
refresh_ncaa_team_ids), with bundled season-scoped team-id crosswalks
(2009-10→2025-26 M / →2024-25 W).
Parity is golden-master against the R sources on real fixtures: 27 raw HTML
captures (8 games chosen to stress edges — blowout/garbage-time, close,
1 OT, 2 OT, 2019-era markup) + 32 R-oracle CSVs produced by running
bigballR/wbigballR from source, with SHA/version provenance READMEs. MBB
play-by-play parity is cell-for-cell exact on all 35 columns including row
order; every transform (lineups, player/team stats, possessions, combos,
on/off, box, schedule, roster, scoreboard, shots, pbp↔shots join) is strict
against its oracle on both leagues. R numeric fidelity is ported exactly:
R ≥4.0 round() (ties-to-even with back-converted-double candidates),
long-double sum() via math.fsum, NA-poisoning max()/sum()
semantics, dplyr C-locale group ordering. One deliberate fix over the R
sources (documented in-module + in tests): modern WBB pbp ships one table
per quarter, and wbigballR — an older fork of bigballR — applies MBB halves
math and misreads regulation WBB as 2-OT; the shared core takes a
period_model knob (WBB binds (4, 600, 300)), with time-derived columns
validated by invariants + a 100% chart↔pbp join match.
MBB / WBB — college computational-core port (lineup stats → ratings/luck → RAPM → positions → NCAA stint pipeline)
A nine-phase port of an open-source college-basketball RAPM computational
core (TypeScript on/off analyzer + Scala NCAA pipeline upstream) into
sportsdataverse/mbb/ with wbb_* shims throughout,
jest-/upstream-oracle-validated:
| Phase | Modules | What |
|---|---|---|
| 0–1 | mbb_lineup_stats | LineupUtils.ts port — weighted-sum accumulators, complete_weighted_avg (transition/scramble possession recompute, regressed diffs), calculate_aggregated_lineup_stats, lineup_to_team_report + get_stats_diff per-player on/off partition (replacement on/off ported bug-for-bug, documented) |
| 2 | mbb_ratings, mbb_luck | Dean-Oliver individual ratings (build_o_rtg 93-key / build_d_rtg 47-key diagnostics matching the jest oracle exactly, build_productivity "Adj Rtg+" RAPM prior) + the complete 950-LOC luck engine (Bayesian 3P% shrink with 10-bit bit-packed shot-info deserializer, eFG→PPP→AdjEff propagation, both directions) |
| 3 | mbb_rapm | Ridge RAPM over lineup design matrices — priors/player context (weak/strong/adaptive-correlation modes), sqrt(poss/total) design matrix + unbias row, (XᵀX+λI)⁻¹Xᵀ solve with standard errors, adaptive-λ picker scaled by avgEigenVal |
| 4 | mbb_positions | Box-score → position classifier (17-feature linear discriminant → softmax over PG..C, normal-CDF height reweighting, small-sample shot-quality shrinkage), build_position decision tree (PG/s-PG/CG/WG/WF/S-PF/PF/C), order_lineup greedy slot assignment |
| 5a–5d | mbb_ncaa_{models,events,possessions,data_quality,names,stints,lineup_enrich,stint_validation} | The cbb-explorer NCAA stats.ncaa.org pipeline: 20-dataclass model layer with the possession-count formula, 40 pbp event extractors, the substitution-tracking stint state machine (incl. the hand-curated 144-key duplicate-name / misspelling data-quality tables), the 1,772-LOC enrich_lineup counting-stat tree (18-category event dispatch, scramble 6.5s ORB-follow + transition tagging, assist source/target pairing), and stint validation + self-healing (validate_lineup three sanity checks, clump_bad_lineups, the fixer pipeline) |
| 5e | mbb_ncaa_html, mbb_ncaa_{roster,boxscore,pbp}_parser | JSoup→bs4 semantics helpers (:eq(N), :matches, text-collapse) + the roster/boxscore/pbp HTML parsers, making the NCAA pipeline end-to-end runnable from saved HTML; oracle'd against 5 vendored upstream fixtures + the upstream inline test suites |
| 5f | mbb_ncaa_fetch, strength engine | Cache-first, proxy-bound stats.ncaa.org fetch layer (original sdv-py code — deliberately no direct-fetch mode, the host is IP-ban-happy; fetch-once caching, NcaaFetchConfig singleton with SDV_PY_NCAA_* / SDV_PY_PROXYBONANZA_* env, secret-redacting __repr__, curl_cffi chrome impersonation) + the Phase-4-deferred strength-adjusted stats engine |
MBB / WBB / NBA — NCAA LineupStatSet producer + hoopR release-parity producers
- feat(mbb,wbb):
lineup_stats_bucket/lineup_stats_buckets(mbb_ncaa_lineup_aggregation) — a field-for-field port of the upstreamcommonLineupAggregations.ts(the Elasticsearch lineup aggregation), the missing stage-2 layer that mints the 254-fieldLineupStatSetbucket from the ported raw-count tree. This connects the finished college model tier to real NCAA data: the ported models (mbb_rapm,mbb_lineup_stats,mbb_luck,mbb_ratings) consume an ES-aggregation bucket that until now came only from vendored jest fixtures — now a real HTML game runs parse → enrich →lineup_stats_buckets→ models. Two field families that need external opponent D1 baselines emit a documented fallback rather than a faked reproduction. - feat(nba,mbb): release-parity producers for the hoopR NBA + MBB Python
cutover —
helper_{nba,mbb}_*functions reproducing the hoopR R creation scripts against the released parquet oracles sohoopR-nba-data/hoopR-mbb-datacan compile their datasets in Python (the sister-league counterpart of the merged WNBA producers). NBA is the WNBA variant (zero new parameters); MBB is the WBB variant with one code delta; two genuinely new helpers — officials (projected fromgame_rosters.gameInfo.officials[]) and player_season_stats (flat career payload, Totals/teamSlug preference) — are shared by NBA + MBB. Every dataset was validated full-frame equal against R's on-disk released parquet oracle (NBA across the full 2025 season; MBB per-game on three fixtures + full-season for the delegates). One deliberate, flagged data fix: pbpidis emitted Int64, not R's Float64 — MBB's 18-digit concat overflows R's double (~906k colliding released ids in 2025, 41%); parity is asserted through the oracle's lossy Float64 view (#245 precedent).
CFB / MBB / WBB / Baseball — stats.ncaa.org parser expansion (football pbp graduation, box tabs, college baseball + softball pbp)
The stats.ncaa.org parser surface now spans four sports, mirroring the MBB NCAA split everywhere: the library owns the parser, discovery/capture stay producer concerns.
- feat(cfb):
parse_cfb_ncaa_pbp(cfb/cfb_ncaa_pbp.py) — the college-football play-by-play parser graduated from thencaa-mfb-football-rawproducer, socfb/now holds both providers (ESPN + stats.ncaa.org) likembb/does. One row per play, cfbfastR-style, from thediv.drivesmarkup: drive context, situation (down/distance/yard_line/end_yard_line),play_typeclassification (0 unknowns across all fixtures), players (passer,rusher,receiver,kicker,punter,returner,tackler_1/tackler_2with suffix-safe "Last,First" handling), signedyards_gained, kick/return/punt/FG detail, turnover + penalty flags, and a frame-wide derivedqb_scramble(a rush by a player who also passes in the game — NCAA text doesn't label scrambles). 21 offline tests on 3 real captured games. - feat(cfb):
cfb_ncaa_box.py— parsers for the non-pbp football game-detail tabs:parse_cfb_ncaa_drives(one row per drive with start/endhow/clock/yard_line),parse_cfb_ncaa_team_stats(team box with a genuine per-quarter breakdown),parse_cfb_ncaa_player_stats(dict of one frame per category),parse_cfb_ncaa_officials, andparse_cfb_ncaa_linescore(+game_date/venue/attendance). Validated on a real 2024 capture (California @ Auburn): 26 drives in clean sequence, linescore quarters sum to finals, officials never a nav tab. NCAA's per-quarter team values don't necessarily sum to the total — extracted faithfully, not "corrected". - feat(mbb):
mbb_ncaa_box_tabs.py—parse_ncaa_bb_officials,parse_ncaa_bb_team_stats(one row per stat/period; the period regex handles WBB quarters and MBB halves), andparse_ncaa_bb_linescore, closing the three basketball contest tabs the bigballR-port parsers didn't cover — the contest page is now fully mapped. WBB re-exports them (parse_ncaa_wbb_*). - feat(baseball):
parse_college_baseball_ncaa_pbp+parse_college_softball_ncaa_pbp(a by-reference twin — softball uses the identical page layout + grammar) — a structured NCAA baseball pbp decomposition: inning context and scores, the batter clause (play_type~20 values,hit_trajectory,fielded_position, hit/out/strikeout/ sacrifice/double-play flags,rbi,count_balls/count_strikes,pitch_sequence, error detail) and the runner clauses (runs_scored,scoring_runners,runners_advanced,outs_on_play). On 3 real D1 games: 0 unknown play types across 322 plays andruns_scoredreconciles exactly to the final score in every game.
All of these return tidy polars frames (pandas via return_as_pandas=True)
with empty input → the documented zero-row schema, and are fully typed in
the mypy ratchet.
NBA / MBB / WBB — dataset loaders for the published model releases
- feat(nba):
load_nba_player_impact(seasons)— consumer loader for thenba_player_impactrelease produced byhoopR-nba-stats-data'snba_model_publish(the RAPM / adj-RAPM / SPM / BPM / WAR / Kalman-projection per-player-season impact table). Codegen-generated fromreleases.yaml+loader_schemas.yaml; 404-safe with a 1996 floor (the stats.nba.com lineup/pbp era, erring low so unpublished seasons skip rather than raise); documented 22-column schema pinned by an offline round-trip test. - feat(mbb,wbb):
load_mbb_ratings/load_mbb_player_value/load_wbb_ratings/load_wbb_player_valueover the four published model dataset tags. Floors were probed per tag rather than inherited from the 2002 boxscore loaders: MBB 2006 (2003–05 are genuine archival thinness), WBB ratings 2008 (unlocked by theboxscoreAvailable-flag fix below — better than the originally-published 2014), WBB player value 2014 (wbb_box_bpm's ≥10-games hygiene floor is unreachable on partial archival coverage). Declared returns-schemas are pinned to the producer modules' schema constants by contract tests.
NBA / WNBA — read-through raw JSON store for stats.nba.com per-game payloads
- feat(nba): the possession engine's module-level fetchers (
_fetch_pbp/_fetch_box/_fetch_rotation/_fetch_box_periods) now route through an env-gated read-through raw store: payloads live in a raw-repo checkout (canonicallyhoopR-nba-stats-raw'snba_stats/json), laid out{endpoint}/{season}/{game_id}.json. Hit → served from disk, no network (offline rebuilds; aPIPELINE_VERSIONbump no longer refetches the corpus). Miss → live fetch, then atomic persist (tmp+rename) with corrupt-file refetch; persist failures never fail the pipeline. Read-only mode disables the persist half so compile/build consumers stay pure readers — only the raw repo's own sweep fills the store. Configuration is explicit-first, env-fallback:raw_store_dir=/raw_store_readonly=onnba_possessions,compile_nba_season, and the fetchers;raw_store_diraccepts a single root or a per-endpoint mapping, and per-endpointSDV_PY_NBA_RAW_JSON_DIR_{ENDPOINT}env vars override the genericSDV_PY_NBA_RAW_JSON_DIR. With nothing set, every existing caller is byte-identical to before. - feat(wnba): the WNBA engine's fetchers route through the same
league-agnostic store (the season decode is WNBA-aware —
10-prefixed game ids are single calendar years, no end-year shift) under a separate env namespace (SDV_PY_WNBA_RAW_JSON_DIR+SDV_PY_WNBA_RAW_JSON_READONLY) so a WNBA compile can read awehoop-wnba-stats-rawcheckout offline; a test asserts no bleed from the NBA env var. - fix(nba): the store never persists an empty
{}payload — a present file is a cache hit that never refetches, so a cached{}(a transient scrape failure) poisoned the game permanently, silently degrading possession / lineup building to pbp-inferred lineups (the backfill had left 2,732 empty rotation files, including 311 of 431 in 2023 alone). Falsy payloads stay retryable misses; the shared helper covers WNBA too. - fix(nba):
nba_player_positionsdedups to one row perplayer_id— the stats.nba.complayerindexlists a mid-season-traded player once per team, violating the documented grain and fanning out through thepositionsjoin innba_bpm/nba_spm(the model-publish builder correctly halted on the duplicate). Listed position is a player attribute, sokeep="first"is deterministic and lossless.
NBA / WNBA / G-League — tracking-value spine (T3.2): six over-expected models on the playerdashpt* surface
- feat(nba): six player-tracking "over-expected" value models over the
second-spectrum
playerdashpt*surface, sharing one_over_expectedcentering engine — rebounding over-expected (chance-adjusted), passer value (assists over expected), drive value, catch-&-shoot vs pull-up shot value, touch value (points per touch over expected), and rim protection (points saved). WNBA (league_id="10") ships as by-reference shims with a G-League degradation path. - Oracle gates on real 2023-24 fixtures: each over-expected metric is a
minutes/possession-weighted centering that sums to ~0 by construction
(verified Σ≈2.8e-14, tol 1e-6), plus a rank-sanity gate against an
externally rate-sourced elite allowlist (top-K exact-cover membership;
qualified populations ~300–450 per model). Review-driven hardening added
minimum qualified-N asserts on all six rank gates (so a truncated
re-capture can't pass vacuously) and a position-bucket join match-rate
floor (
matched/height >= 0.9) that catches the dtype-agrees-but-id-spaces-disjoint bug which would silently collapse the by-position baseline into one league-wide bucket. Real-data capture corrected two wrong column names from the design doc (documented in the fixtures README).
NBA — model-zoo v1: validation harness + SPM / BPM 2.0 / Bayesian adj-RAPM
The first generation of the NBA player-value model zoo: a validation harness
wiring four external oracles plus a meta-oracle and a season compiler; a
trained SPM (box-score features regressed onto a RAPM target) with the
RatingsModel harness extension; a faithful BPM 2.0 port (box-score player
value) validated in a three-way head-to-head; and adj-RAPM-with-prior
(Bayesian RAPM) with the calibration oracle activated. The v2 tier-1
estimators (RAPM variants, external concurrent validity, the through-date
ratings panel + WAR) build on this base.
NBA — Kalman + aging-curve player projection + forecast validator
- feat(nba):
nba_darko— a per-player Kalman filter over a multi-season rating panel with an empirical aging curve (AgingCurve/fit_aging_curve, delta method) that forecasts each player's next-season rating with a posterior SD. Noise parameters are MLE fit (q/obs_baseviascipy.optimizewith a moment-basedq-floor); observation noise scales ∝ 1/possessions. Input is a pre-built{player_id, season, rating, weight}panel (assemble it from adj-RAPM/SPM per season) so the projection layer is decoupled from the estimators.nba_player_ages.pysupplies bulk per-seasonAGEvialeaguedashplayerbiostats. - feat(nba):
darko_forecast_accuracy+ForecastResult— the projection is evaluated by forecast accuracy (predict season N+1 from history ≤ N), not the possession harness. The meta-oracle has teeth: on a skill panel the projection beats carry-forward (RMSE 0.82 < 0.94, corr 0.98), and on a pure-noise panel it does not manufacture skill — building that noise test exposed and fixed a real MLE pathology.
NHL / PWHL — microstat & EDGE value spine (T5.2) + first-of-its-kind PWHL prediction (T5.3)
- feat(nhl): five microstat value models over the api-web pbp + EDGE
tracking surface, with PWHL by-reference shims — context-adjusted faceoff
win value (zone × strength, fit from post-faceoff xG), penalty
drawn/taken net value (man-advantage goals per minor), expected
primary/secondary assists (relative-danger xG credit), zone-entry/exit
value (pbp-derived controlled/dump inference), and an EDGE skating
z-composite value (NHL-only; PWHL zero-row) from
nhl_edgeskater detail. All oracle gates pass on the real 2024 corpus (faceoff context-cell calibration 0.021 ≤ 0.03; penalty net conservation ~4e-16; ΣxA≈ΣA unbiasedness 0.006 ≤ 0.05; split-half stabilities 0.21–0.28 vs ≥0.15 floors; EDGE component rank-corr 0.52–0.76 vs ≥0.5), with the rare-event stability gates using an independent games-played denominator to avoid the conditioning-on-the-sum trap. No gate lowered. - feat(nhl): the T5.2 deferrals fleshed out — the zone-entry
controlled/dump heuristic is now event-sequence-aware (the entering team
must win the next possession event within the window, cross-period
sign-flip guarded) with a directional gate (controlled entries precede a
same-team shot 0.98 vs 0.77 for dumps); EDGE skating gains
method="percentile"plus a joint face-validity gate; xAssists ordering was re-measured, confirmed genuinely underpowered (real numbers documented + reproducible via a committed check script), and replaced with a powered population-level gate (secondary-assist goals carry higher mean relative danger). - feat(pwhl):
pwhl_xg_proxy— the T5.3 PWHL prediction, a first-of-its-kind model on real PWHL data (3 live seasons, 2024–2026): a 2-tier empirical xG proxy from PWHL's categoricalshot_qualityfeeding the existing league-agnostic opponent-adjustment + market core. De-leaked after review: tier weights fit on strictly pre-cutoff pbp (per-as-of in the backtest) andmargin_sdfit on train (2024+2025) and evaluated on held-out 2026 only. Honest held-out result (n=107): Brier 0.2449 vs naive 0.2500 — within ~1 SD of noise, so the beats-naive magnitude assertion is deliberately dropped in favor of the held-out calibration gate.
PWHL — shift-derived strength_state + shot-level coordinate xG (+ two loaders)
- feat(pwhl):
add_strength_state(pbp, goalie_ids)(hockeytech/_analytics.py) — derivesskaters_home/skaters_away, home-vs-awaystrength_state(5v5/5v4/6v5), and astrength_state_validsanitize flag from the on-ice ids, unlocking real offline strength context on every PWHL shot for EV/PP/SH strength-split xG. Empty-net is intentionally not derived here — HockeyTech goalie shift-tracking is unreliable (~40% false positives); use the authoritative goal-levelempty_netfield. - fix(hockeytech):
build_on_iceend intervals are now half-open (> end_s), fixing a line-change double-count that produced impossible ~10-v-10 on-ice states — a shared-core fix that also corrects Corsi/TOI for AHL/OHL/WHL/QMJHL. - feat(pwhl):
pwhl_shot_xg()— public shot-level counterpart topwhl_team_game_xg_rates: same pre-shot context derivation over the full pbp,PwhlCoordXGModelscoring, returned as a curated 21-column_SHOT_XG_SCHEMAframe (identity, rink-feet geometry, strength context, outcome,xg), with dtypes cast at the boundary soload_pwhl_pbpoutput and the pwhl-data committed parquet land on one published schema. Takesmodel=so a producer can fit once on pooled seasons and score every season consistently. Real-data smoke: pooled 2024–2026 fit scoring 2025 → 5,671 shots, xG sum 464.8 vs 499 goals. - feat(pwhl):
load_pwhl_shifts()+load_pwhl_xg_pbp()— codegen loaders for thepwhl_shiftsandpwhl_xg_pbprelease tags (the latter's declared returns-schema pinned to_SHOT_XG_SCHEMAby a contract test).
MLB — model spines: game state (T6.4), pitching evaluation (T6.1), fielding/catching/baserunning (T6.3)
- feat(mlb): game-state spine — the MLB substrate + the RE24 /
run_valuedenominator the sibling spines import:mlb_run_expectancy_matrix+run_value(...)(exported atsportsdataverse.mlb), a win-expectancy table + WPA + Tango leverage index (pre-play-aligned), an umpire strike-zone logistic with per-umpire bias (Statcast bridge), team projection (pythagenpat + as-of-date Elo), and prop projection (team-runs log5 + strikeouts Poisson). Ships its ownstatsapi.mlb.comcollector. Gates: RE24 per-state |diff| vs Tango 0.048 ≤ 0.05, WE corr vs statsapi 0.974 ≥ 0.95, exact WPA-sum telescoping, umpire calibration gap 0.075 ≤ 0.08, pythagenpat MAE 0.029, props as-of-date MAE 2.64 runs. As-of leakage enforced (Elo updates only after a team's own game; prop backtest uses strictly-priorcum_sum().shift(1)). - feat(mlb): pitching-evaluation spine — compute-on-demand pitcher
models over Baseball Savant Statcast:
mlb_stuff_plus(xgboost run-value model on pitch physics, plus-scale),mlb_command_plus(Location+/Command+ with a bundled command model; fixed a train/score-inconsistency in the categorical encoding),mlb_pitch_era(parametric xERA + SIERA-like estimator, oracle vs the Savant xERA leaderboard),mlb_pitch_features(physics/location/sequence/TTO/workload substrate),mlb_pitch_sequencing(tunnel geometry + sequence run value),mlb_pitch_fatigue(times-through-order / fatigue penalty),mlb_pitch_classify(per-pitcher GMM reclassification with an agreement gate), andmlb_pitch_injury(leakage-safe injury-risk index). All oracle gates green on real Savant pitch + leaderboard captures; model artifacts bundled. - feat(mlb): fielding / catching / baserunning spine —
mlb_run_values(RE288 count table + event/count-strike run values with an as-of-date split),mlb_catcher_framing(framing runs from a called-strike probability grid),mlb_catcher_defense(blocking runs + throwing / caught-stealing value via a pop-time model),mlb_fielding_oaa(outs-above-average from a catch-probability surface over BIP trajectory features),mlb_baserunning(extra-bases-above-expected + advancement opportunities), andmlb_stolen_base(SB success surface + value; SB/CS attempts derived fromdestext to fill a documented Savant capture gap). All gates green against the committed real-capture Savant corpus; rate-metric oracle joins assert dtype agreement + match-rate floors.
Recruiting / NFL — PFF Premium Stats stem + On3 RDB retarget + 247Sports expansion
- feat(nfl,cfb): PFF Premium Stats 2.0 stem (
premium.pff.com/api/v1, cookie-auth) — a newpff_corestem (46 wrappers: 32 facet reports, player reports, meta) +make_pff_league_moduleshims for nfl / ncaa (cfb) / aaf / ufl. Cookie-supply auth path withSDV_PY_PFF_LIVE-gated live tests;pff_loginships as an experimental stub; the transport is injectable so offline tests run against real logged-in captures. - feat(cfb): On3 RDB retarget — the 4-endpoint
_next/datascrape becomes an 82-endpoint RDB stem; deprecated_next/datashims keep the 4 released names working. - feat(cfb): 247Sports expansion — the guest-usable
positionsRDB route (residential-gated live tests) plus a new auth-freesports247_site_pagesstem (curl_cffi chrome; string-numeric casts at the boundary; nested entities surfaced as integer FKs).
Validation harness — constant_column check, R-lint UTF-8 fix, cron run-tracker ingest
- feat(validation): new
constant_columncheck — flags all-null / all-NaN and zero-variance (single-valued, incl. all-zero) columns as WARNneeds_judgment, minus a per-datasetexpected_constant_columnsallowlist; the class of dead-column bug the existing checks can't catch (extractionis null-based,sweepis release-over-release,numeric_parityneeds an oracle). All 7 previously-unmonitored CFB modeling-suite datasets (cfb_passing,cfb_rushing,cfb_receiving,cfb_percentiles,cfb_team_summaries,cfb_rosters_crosswalk,cfb_rb_eval) are now registered with committed schema snapshots, join keys, and allowlists — and the check immediately surfaced real standing producer findings (e.g. all-zerosacked/pass_int/sack_ydsincfb_passing). - fix(validation): the R leakage lint decoded Rscript output as cp1252 on
Windows and
UnicodeDecodeError'd on UTF-8 R sources — silently dropping those files instead of linting them; nowencoding="utf-8", errors="replace". The workflow's hardcodedLINT_TARGETShad also drifted from the registry (the CFB R lint never ran); a new contract test asserts the workflow'sDATASETS/LINT_TARGETSmatchregistry.pyso future drift fails CI. - ci(validation): the weekly validation cron now ingests one platform run
per validated dataset into the sportsdataverse.org run tracker — one gate
per harness check (pass iff zero ERROR findings; WARN-only findings
surface in
metrics), with links back to the Actions run. A dataset whose release download was unavailable is not ingested (an empty findings file would masquerade as a perfect run), and ingest failures never fail the cron.
0.0.71 Release: June 24, 2026
CFB — opponent-adjusted EPA (cfb_adjusted_epa): season + walk-forward
sportsdataverse.cfb.cfb_adjusted_epa() and cfb_adjusted_epa_by_game() add a reusable ridge / RAPM-style opponent-adjustment primitive — separating a team's per-play EPA from its schedule with a ridge regression on offense/defense team indicators (plus home-field), fit over the competitive (0.1 ≤ wp_before ≤ 0.9) pass and rush plays. The season function returns one row per team (adjusted off / def / net EPA + ranks); the walk-forward function returns one row per team-game and is point-in-time — each week is adjusted using opponent strengths fit only on prior weeks, so the values are leak-free and valid as in-season power-rating or model inputs (week 1 has no prior, so its adjustments are null; not-yet-seen opponents fall back to the league baseline, the intended early-season shrinkage). This is an in-sample per-season estimator lifted out of the cfb-data team_summaries builder — not a bundled .ubj artifact. scikit-learn is now a runtime dependency.
NFL — era-aware decision models + both-path (ESPN + nflverse) model parity
Ships the era-aware NFL model suite and brings both PBP construction paths to model parity. Rule-era one-hots (era0..era4, cuts 2001/2005/2013/2017) are added to the xpass / fourth-down / fg models so the curves are era-aware across all of 1999–2025 (fourth-down 14-feature, fg 7-feature, xpass 19-feature), and the bundled nfl/models/* are refreshed to the 1999–2025 retrain (two-point on 2010–2025).
Both builders now produce the same modeled columns: the ESPN path (NFLPlayProcess) gains qb_epa, wp / vegas_wp (+ def_wp / home_wp / away_wp), and xpass / pass_oe, wired into run_processing_pipeline in nflfastR order; the nflverse path (enrich_nfl_pbp) gains the per-play QBR EPA components. The fourth-down decision surface is default-on in both builders, scored on its play-type subset and merged back by play id, with each model applied on its correct play-type shape (xpass on scrimmage dropbacks, cp/xyac on pass + air-yards, fourth-down on down == 4). xYAC remains the documented null stub on the ESPN path (no air_epa). A latent bug the era refresh introduced is fixed: calculate_xpass now backfills the era0/era1 features _make_cp_mutations did not build. Pairs with the nfl-data 1999–2025 retrain that produced the artifacts.
0.0.70 Release: June 24, 2026
CFB — qbr / fg / wp_spread models refreshed on the consensus-odds full-corpus reprocess
The bundled CFB qbr_model, fg_model, and wp_spread XGBoost artifacts are retrained on the full 2004–2025 play-by-play corpus after it was re-reprocessed with two upgraded modeling inputs: the cfb_line_odds multi-book consensus pregame spread/total (replacing ESPN's single pickcenter as the EPA/WPA odds source) and roster-backed pre-2014 player IDs. Feature contracts are byte-identical to the shipped models (qbr 10-feature incl. era0–3, fg 5-feature, wp_spread 13-feature), so this is a drop-in artifact refresh — no model-application changes.
Leave-one-season-out CV over all 22 seasons confirms the gains: qbr RMSE 17.60 → 17.29 (r² 0.598 → 0.612), fg logloss 0.5265 → 0.5247, and wp_spread baseline logloss 0.3616 → 0.3486 — the win-probability model improves most, since the consensus odds sharpen the spread_time feature directly (the signal the rule-era one-hot dummies previously had to recover). The fourth_down model is intentionally left unchanged: on the refreshed corpus its era variant no longer beats the consensus-odds baseline, so it was not promoted.
0.0.69 Release: June 23, 2026
CFB — roster-backed {type}_player_id + player-name cleanup fixes
CFBPlayProcess now emits a {type}_player_id for every extracted {type}_player_name, resolved team-aware against the game roster: each player type maps to the team that fielded it (offense pos_team / defense def_pos_team / special-teams kicking_team / return_team / recovery), so identical names on opposing rosters don't collide; a globally-unique name is the fallback. Ids resolve for all years — pre-2014 (no structured participants[] array) via the roster, 2014+ from the clean participant names.
- New
CFBPlayProcess(game_roster=, participants=)constructor params let offline rebuilds pass the stored roster + participants — fetch-free, and keeping 2014+ clean names whenjoin_participantsis off.__join_participantsnow accepts a caller-supplied participant frame /{"data": [...]}/ row list instead of always fetching. - Player-name cleanup fixes the roster-match exposed: the receiver state-abbrev strip (
ST/GA/FL/…, with the leading space) is anchored to a trailing standalone token so it can't corrupt real names (it used to eat the " St" inside "Stewart" → "ewart"); a garbage guard nulls obvious play-text artifacts ("bea loss of") before the id-join.
0.0.68 Release: June 23, 2026
CFB — completion-probability (cp/cpoe) + expected-pass (xpass/pass_oe) surface
CFBPlayProcess now emits per-play completion-probability and expected-pass columns, mirroring nflfastR's cp/cpoe and xpass/pass_oe.
- Two new bundled models —
cfb/models/cfb_cp_model.ubj(8-featbinary:logistic:down,distance,yards_to_goal,score_diff,seconds_remaining,is_home,period,passing_down) andcfb/models/xpass_model.ubj(7-featbinary:logistic:down,distance,yards_to_goal,pos_score_diff,TimeSecsRem,era,period). Both are ~400 KB and ship via the existingcfb/models/*package-data glob (no download-on-demand). - New per-play columns —
cp= P(complete) on pass plays withcpoe = 100 * (completion - cp)(percentage-point scale, null on non-pass plays);xpass= P(pass) on scrimmage rush-or-pass plays withpass_oe = 100 * (pass - xpass)(null elsewhere). Added as two pipe steps (__process_cpoe/__process_xpass) after the EPA/WPA steps inrun_processing_pipeline(); each degrades to null columns rather than raising when a source column is absent.
CFB — spread-free (naive) win-probability surface (wp_*_naive)
CFBPlayProcess now emits a second, spread-free win-probability surface alongside the existing spread WP, completing the play-level model handoff begun in 0.0.67 (which retrained EP + spread WP on the full 2004–2025 history).
- New bundled model
cfb/models/wp_naive.ubj— the faithful cfbscrapR "naive" recipe (12-feat =wp_final_namesminusspread_time,binary:logistic, 65 rounds), retrained on the same full-history corpus (2,219,607 plays, 2004–2025) as the spread model. Ships via the existingcfb/models/*package-data glob. - New per-play columns
wp_before_naive/wp_after_naive/wpa_naive(plusdef_/home_/away_analogues), mirroring the spread columns under a_naivesuffix. The naive surface answers "given only game state, who wins?" while the spread surface bakes in the pregame line; the two correlate ~0.90, diverging most early-game where the market prior carries the most information. - Refactor (no behavior change to the spread surface): the win-probability prediction + game-logic derivation in
__process_wpawas factored into shared_wp_predict/_apply_wp_derivationhelpers routed once per model. The spread (un-suffixed) output is byte-identical to the prior release — verified against a captured per-play baseline.
CFB — QBR model retrained on the full 2004–2025 history
The bundled cfb/models/qbr_model.ubj (6-feat XGBoost: qbr_epa / sack_epa / pass_epa / rush_epa / pen_epa / spread) was retrained on the full-history corpus, replacing the legacy 2020-lineage model.
- Decisively better against the ESPN raw-QBR reference. On a 2021–2025 holdout (out-of-sample for the legacy model): RMSE 23.2 → 16.1 (−31%), MAE 18.7 → 12.5, R² 0.29 → 0.66, correlation 0.69 → 0.82. The retrained model's honest leave-one-season-out metrics (RMSE 17.9, R² 0.585) confirm the gains are real generalization, not in-sample fit.
- Drop-in swap — same 6-feature contract, ships via the existing
cfb/models/*package-data glob; no caller changes.
CFB — fourth-down decision surface (get_4th_down_probs, cfb4th port)
A full college-football fourth-down decision surface, a faithful Python port of cfb4th's add_4th_probs(), against this package's bundled EP / WP-spread boosters.
sportsdataverse.cfb.get_4th_down_probs(pbp_df)scores all three options on a frame of fourth-down situations and adds:go_wp/first_down_prob/wp_succeed/wp_fail(go),punt_wp(punt),fg_make_prob/make_fg_wp/miss_fg_wp/fg_wp(field goal), afourth_down_recommendation∈ {go,punt,field_goal} (max-WP choice), per-option*_wp_diff, andgo_boost(cfb4th's headline100·(go_wp − max(fg_wp, punt_wp))).CFBPlayProcess.add_fourth_down_probs()applies the same to a processed game's fourth-down rows afterrun_processing_pipeline().- New models:
fg_model.ubj(CFB-native field-goal make-probability by distance, trained on 42.6k attempts) andpunt_distribution.parquet(punt end-yardline distribution) are bundled undercfb/models/; the 6-feat / 76-classfd_model.ubj(yards-gained, with the ordinal CFB rule-era factor) is download-on-demand (~16 MB, fetched from theespn_cfb_model_artifactsrelease and cached under~/.cache/sportsdataverse/cfb_models/, mirroring the NFL xYAC pattern; override withSDV_PY_CFB_MODEL_DIR). The go path reuses the reviewed cfb-data decision-layer machinery; punt/FG mirror cfb4th's possession-flip + end-game scoring.
CFB — two-point-conversion decision surface (get_2pt_probs, cfb4th port)
The extra-point vs go-for-2 decision, a faithful Python port of cfb4th's get_2pt_wp(), against this package's bundled EP / WP-spread boosters and a new bundled CFB two-point model.
sportsdataverse.cfb.get_2pt_probs(pbp_df)treats each row as "the scoring team just made a touchdown; decide". For each of the three point outcomes (0/1/2) it subtracts the points, flips to the opponent's ensuing kickoff-return drive (1st-&-10 at the 25,yards_to_goal = 75), scores EP → WP, and flips WP back to the scoring team. It addstwo_pt_wp(=prob_2pt·wp(2) + (1−prob_2pt)·wp(0)),xp_wp(=prob_xp·wp(1) + (1−prob_xp)·wp(0)),prob_2pt, atwo_pt_recommendation∈ {go_for_2,kick_xp} (go for 2 ifftwo_pt_wp > xp_wp), andtwo_pt_wp_diff(=two_pt_wp − xp_wp, positive ⇒ go for 2). The ensuing-drive frame reuses the reviewed 4th-down state machinery (_flip_team_state+ EP/WP scorers).CFBPlayProcess.add_2pt_probs()applies the same to a processed game's point-after / two-point-conversion rows (those withpointAfterAttempt.textpresent) afterrun_processing_pipeline(); every other row carries nulls.- New model:
two_pt_model.ubj(abinary:logistic4-feature booster —posteam_spread,posteam_total,pos_score_diff, ordinalera) is bundled undercfb/models/.prob_2ptcomes from this model (cfb4th hardcodes 0.45);prob_xpis the empirical CFB extra-point make rate0.9851(cfb4th derives XP from its FG GAM, but the empirical rate is more accurate for CFB).
CFB — rule-era QBR / FG / fourth-down models + spread_time sign fix
The QBR, field-goal, and fourth-down (yards) models gain one-hot rule-era dummies (era0..era3, cuts 2006/2013/2020) where they improve out-of-fold, and the bundled boosters are swapped to the era-augmented versions.
- QBR —
qbr_varsgainsera0..era3(LOSO RMSE 17.9 → 17.4);__process_qbrinjects the per-game era one-hot before prediction; bundledqbr_model.ubjswapped to the 10-feature era model. - Fourth-down —
fd_model.ubjswitched to the 9-feature one-hot era model (first-down cal-MAE 0.0035 → 0.0027) and bundled in the package (was download-on-demand);fg_model.ubjswapped to the 5-feature era model. - WP-spread — bundled
wp_spread.ubjretrained on the odds-backfilled frame (the ~2,167 missing-spread games now carry real consensus spreads; LOSO logloss 0.362 → 0.352; same 13-feature contract, no inference change). spread_timesign fix —_predict_wpcomputedspread_time = −pos_team_spread·exp(…), inverted vs the trained-on convention (favorites scored as underdogs inget_go_wp/get_fg_wp/get_punt_wp); corrected to+pos_team_spread·exp(−4·elapsed_share).- Decision surfaces on by default —
run_processing_pipeline(fourth_down_probs=True, two_pt_probs=True)now appends the fourth-down and two-point decision columns to a processed game by default.
CFB — pre-2014 play-text player-name extraction
CFBPlayProcess now recovers per-play player names for 2004–2013 games, where ESPN ships no structured per-play participants array (only teamParticipants). Two latent bugs in the play-text regex extraction were fixed: a multi-alternative str.extract group-index bug (the matched branch's name landed in a non-default capture group, returning null for ESPN "rush" / "Punt by" / "on-side" / "returned by" phrasings) and a \d-escaping bug (a literal backslash instead of a digit, which broke field-goal-kicker extraction). Pre-2014 games now populate rusher / passer / receiver / sack / fg-kicker / punter / returner / fumble player names (all null before); 2014+ output is unchanged (the structured-participants overwrite still wins).
NFL — expected pass (xpass / pass_oe) + nfl4th fourth-down decision surface
calculate_xpass adds xpass (P(dropback)) and pass_oe = 100·(pass − xpass) to the enriched NFL PBP, plus a faithful Python port of nfl4th's fourth-down decision surface (nfl/nfl_fourth_down.py) scoring go / field-goal / punt win probability + a recommendation.
calculate_xpass— the self-derived dropback booster;xpass/pass_oemirror nflfastR'sadd_xpass.- nfl4th surface — go / FG / punt WP via the download-on-demand
fd_model/wp_modelartifacts (cached on first use), mirroring nfl4th'sadd_4th_probs.
NFL — self-trained XGBoost field-goal model in the fourth-down surface
get_fg_wp / get_2pt_wp switched from the mgcv-GAM prediction grid to a self-trained binary:logistic XGBoost FG model (fg_model.ubj, features yardline_100 / fg_roof / fg_era) with the unchanged nfl4th long-kick clamps. Oracle parity (2022): fg_wp 0.9995, go_wp 0.9998, punt_wp 0.9996.
NFL — load_nfl_espn_qbr (ESPN QBR loader, nflreadpy parity)
New load_nfl_espn_qbr (also aliased load_espn_qbr) — the last nflreadpy dataset without an sdv-py loader. summary_type= season|week, 2006+ floor, source= dual (nflverse espn_data release or the SDV-native nfl_espn_qbr release, 2006–2025), read-once-then-filter, with a 23-column returns-schema.
NFL — bundled self-derived xpass model (offline, no first-use download)
xpass_model.ubj (the self-derived dropback booster, 1121 trees, 7.4 MB) moves from download-on-demand to bundled under nfl/models/, so calculate_xpass works offline. It is the same model the release ships — xpass output is unchanged; removed from _MODEL_URLS (the bundled path wins in _load_model's resolution order).
0.0.67 Release: June 17, 2026
Documentation — return-table column descriptions filled (~3,061 columns)
Every generated reference page renders a col_name | type | description returns table; ~3,061 of those cells previously rendered blank because the column name had no entry in the R-package-mined dictionary that backfills descriptions at render time (sdv-py-/provider-specific columns: ESPN Site v2, MLB Stats API, NHL api-web / EDGE, nflverse Shield, HockeyTech, etc.). Those cells are now filled.
- New hand-curated source
tools/codegen/manual_column_descriptions.yaml, keyed by the schema'sschema:field (with a_globaltable-agnostic fallback), consumed at render time bygenerate.py:_table_cell_desc. Resolution order: captured-stored value →manual[schema][col]→manual._global[col]→ R-dict mined fill → empty. Descriptions live only here (theschemas/**.yamlare clobbered blank on every capture), so they survive re-capture. - Coverage: NFL (1,158 — nflverse / Next Gen Stats / Pro Football Reference / ESPN), MLB (599 — Stats API + ESPN), NHL (588 — api-web / EDGE / ESPN), CFB (177 — ESPN + cfbfastR), plus the ESPN cross-league game summary (sport-agnostic), NBA/WNBA/MBB/WBB, PWHL + CHL junior hockey (OHL/QMJHL/WHL/AHL), and the shared
standings/leaders/team_roster/news/team_scheduleschemas. - Regression guard:
tools/codegen/extract_residual_columns.pycomputes the render-blank residual;tests/codegen/test_manual_descriptions.pyasserts it stays at 0 (a newly-captured undocumented column fails CI until authored), plus an orphan guard (no stale dict keys) and a filler-lint (rejects terse/generic descriptions). Every bucket was adversarially accuracy-reviewed; corrections included PFRrec_br, MLBbase_on_balls, NHL EDGE goalie goal-differential / pbp assist totals, and the long-formatload_cfb_betting_linescolumns.
Documentation — doctest-prompt cleanup, native returns-tables, new tutorials
- No more raw
>>>doctest prompts. The generated ESPN-wrapper + loader docstring templates emitted>>> callunderExample:(whichsphinx.ext.doctestwould try to verify); both emission sites now produce the napoleonQuick start::literal block, clearing ~3,559 generated hazards. The remaining ~55 hand-written prompts (NFL NGS / parsers, The Odds API,find/discover, etc.) were converted in source. - 78 new native returns-tables. Wired
returns_schemafor NHL api-web (9), stats-rest (10), records (37), EDGE (15), and MLB Stats API (8) endpoints that previously rendered no return table — captured from live fixtures; the 676 new columns are fully described. (24 endpoints were skipped: off-season EDGE top-10 leaderboards, retired record paths, and auth-gated MLB endpoints.) refresh_return_schemasno longer writes 0-column per-league schemas — an emptycolumns: []file shadowed and suppressed the genericschemas/{name}.yamlfallback, leaving some leagues with no table; it now skips them so the generic table renders.- Three new intro tutorials under
examples/notebooks/(rendered todocs/docs/tutorials/): Soccer (espn_soccer_*(league=)+ headline aliases), Cricket (espn_cricket_*+ the 8-section matchcard summary), and Other ESPN leagues (UFL/XFL/CFL, college baseball/softball, NCAA M/W hockey).
NFL — PBP ETL ↔ nflfastR alignment + faithful model artifacts
enrich_nfl_pbp()lead-diff orchestrator computes nflverse-native EP/EPA/WP/WPA/CP/xYAC on a real nflverse PBP frame, aligned to nflfastR; runs on live nflverse data.- Shared derivations
calculate_epa()/calculate_wpa()lifted intosportsdataverse/nfl/ep_wp.py; the NFL EP/WP constants + shared column contract centralized insportsdataverse/nfl/model_vars.py. - Faithful NFL model artifacts replace the byte-identical CFB 8-feature placeholders that previously shipped under
nfl/models/:ep_model.ubj(18 features),wp_spread.ubj(12),wp_naive.ubj(11),cp_model.ubj(18) — resolving the long-standingxgboost num_feature >= num_col (8 vs 18)mismatch that left the NFL model path red. - New test coverage:
tests/nfl/gains enrich, enrich-derive, EPA, WPA, and column-contract suites.
CFB — EP + WP models retrained on the full 2004–2025 history
- Canonical
cfb/models/ep_model.ubjandcfb/models/wp_spread.ubjretrained on the complete cfbfastR-cfb-raw finals — 2,219,607 cleaned/labeled/weighted plays, seasons 2004–2025 — now that the raw backfill is complete. Shipped XGBoost recipes unchanged (EPmulti:softprob7-class/525 rounds; WP-spreadbinary:logistic/760 rounds). - Leave-one-season-out validated (22 folds, out-of-fold): EP mlogloss 1.233 / accuracy 0.500 / EP-value calibration MAE 0.014 pts; WP logloss 0.362 / Brier 0.118 / AUC 0.916 / weighted-cal-error 0.0147. Drop-in safe (feature names/order match
cfb_pbp.ep_final_names/wp_final_names). QBR is intentionally unchanged (LOSO R² 0.585 — remains the Dec-2020 canonical model).
0.0.66 Release: June 17, 2026
CFB — cfb_pbp sparse-game ColumnNotFoundError guard (end.team.id et al.)
Sparse pre-2010 games (e.g. 2005 game 252440154) crashed CFBPlayProcess.run_processing_pipeline() with polars.exceptions.ColumnNotFoundError: unable to find column "end.team.id". The per-play start.*/end.*/period.*/clock.*/type.* columns are produced only by pd.json_normalize flattening the plays array, so when no play in a game carries a given nested object the column is never created — and the downstream with_columns chain dereferences it via pl.col(...) unconditionally, which raises at plan time before the existing fill_null / when-otherwise logic can substitute a value.
Added a column-materialization guard in __helper_cfb_pbp_features (after the early-return length checks, before the main play chain) that diffs the 15 unconditionally-referenced json_normalize-origin columns against the live frame and creates any missing one as a Null literal:
- String-typed source columns (
clock.displayValue,type.text,text,start.downDistanceText) are created aspl.lit(None, dtype=pl.String)because the chain runs.str.*ops on them (an untyped Null column raisesSchemaError). - Numeric/bool columns stay untyped
pl.lit(None)so their explicit downstream.cast(...)owns the final dtype. - The guard is a no-op for healthy games —
with_columnsis skipped when nothing is missing, so output is byte-identical (verified: 5 control games reprocessed to identical 406-column frames and exact play counts). Resolves all 7 known-failing 2005 games.
0.0.65 Release: June 17, 2026
Namespace — minor/alias leagues nested under sport-group packages
- refactor(namespace): nest minor/alias leagues under sport-group packages (
sportsdataverse.soccer.epl,.hockey.ahl,.football.ufl,.baseball.college_baseball); the 8 majors + pwhl/soccer/cricket stay top-level. Legacy names (sportsdataverse.epl,import sportsdataverse.ufl) still resolve with aDeprecationWarning. NOTE:discover.function_count()/list_functions()keep flat-leaf keys (function_count(league="ufl")works);import sportsdataversenow eagerly loads the 12 soccer-alias submodules to support attribute access.
All sports — espn_*_game_rosters vectorized logo extraction
Pre-2010 ESPN team payloads omit the logos key entirely, causing helper_{sport}_team_items to raise polars.exceptions.ColumnNotFoundError: "logos" not found. The row-by-row item-assignment fallback (teams_df[row, "logo_href"] = ...) also triggers TypeError: the truth value of a Series is ambiguous in polars 1.x because the row-index selector internally evaluates Series.__bool__.
Replaced the logos block in all seven espn_*_game_rosters modules (cfb, mbb, nba, nfl, nhl, wbb, wnba) with vectorized with_columns:
if "logos" in teams_df.columns:guard handles pre-2010 payloads where the key is absent.pl.col("logos").list.get(i).struct.field("href").fill_null("")— expression-engine extraction, null-safe, no Python-level row iteration.except Exception:fallback to empty-string literals if the logos payload doesn't match the expectedList(Struct)shape.
MLB — mlb_api_* renamed to mlb_*
The 64 Stats API wrapper functions in mlb_api.py (generated via tools/codegen/endpoints/mlb_api.yaml) and the 15 hand-written functions in mlb_api_extra.py were renamed from mlb_api_{short} to mlb_{short} — parallel to the statcast_* → mlb_statcast_* rename in 0.0.64. The _api_ infix was a disambiguation artifact from when multiple backends shared the module; it is now redundant. No aliases — update call sites accordingly.
0.0.64 Release: June 17, 2026
MLB — comprehensive Baseball Savant / Statcast surface (mlb_statcast_*, 43 endpoints)
Expanded the Baseball Savant integration from a 12-endpoint representative slice to the full ~43-endpoint catalog under the mlb_statcast_<family>_<name> naming (search / leaderboard / gamefeed / player), with every endpoint parsed to a tidy frame by default (return_parsed=False / raw=True for the raw payload).
- 39 codegen-generated wrappers — 37 leaderboards (expected stats, sprint speed, bat tracking, pitch arsenals/movement/tempo, OAA, arm strength, catcher framing/blocking/throwing, baserunning, park factors, …) plus
mlb_statcast_gamefeed(one row per pitch) andmlb_statcast_schedule(one row per game). Savant mixes CSV / JSON / HTML, so the family uses a content-type-aware getter (dictfor JSON,strfor CSV/HTML); the two HTML-embedded leaderboards (fielding-run-value,statcast-park-factors) are parsed from their embeddeddata[]blob. - Hand-written search —
mlb_statcast_search(+_minors,_wbc) auto-chunks the 25,000-row Savant cap and translates friendly filters (season,pitch_type,at_bat_result,batters_lookup, …) to Savant'shf*params.mlb_statcast_playerparses a player page'sserverValssection (defaultstatcast, ~260 metrics) to a tidy frame (section=for others,raw=Truefor HTML). - Returns-schemas (
col_name | type | description) for every frame-returning function, andexamples/notebooks/09_mlb_intro.ipynbmodernized to the new surface. - The pre-0.0.64
statcast_*names were renamed (no aliases) to themlb_statcast_*convention.
Documentation — nfl_api (NFL.com Shield) returns-schema tables
Added live-captured col_name | type | description returns-schemas for all 11 api.nfl.com endpoints (standings, rosters, teams_history, team, weeks, weeks_by_date, combine_profiles, draft_picks, injuries, game_summaries, weekly_game_details), wired via returns_schema: into nfl_api.yaml and rendered into the reference docs — bringing nfl_api to parity with the other six native API families (mlb_api, nhl_*). Docs/codegen-metadata only; no runtime change.
0.0.63 Release: June 16, 2026
All sports — espn_*_game_rosters diagonal per-team concat (fixes silent roster loss)
The per-team roster concat in espn_wbb/wnba/nba/mbb/nfl/cfb_game_rosters used pl.concat(..., how="vertical"), which hard-fails with polars.exceptions.ShapeError when a game's two teams ship different roster columns (e.g. one entry list has jersey, the other didNotPlay). The whole game then errored and was discarded as empty despite having roster data. Switched to how="diagonal" (union + null-fill), matching nhl_game_rosters and the teams/athletes concats in the same modules.
HTTP — download() no longer retries a definitive 404
sportsdataverse.dl_utils.download retried a NoESPNDataError (ESPN 404 / code:404 body) for the full num_retries budget — wasting ~51s of backoff and N requests per genuinely-absent resource, amplifying load against a rate-limited host. A 404 is definitive "no data", so it now fails fast (one attempt) instead of retrying. Connection/timeout/5xx errors still retry as before.
0.0.62 Release: June 16, 2026
All sports — espn_*_game_rosters robust to long-tail ESPN payloads
Applies the two espn_cfb_game_rosters robustness fixes from 0.0.61 to every sibling rosters builder — espn_wbb_game_rosters, espn_wnba_game_rosters, espn_nba_game_rosters, espn_mbb_game_rosters, espn_nhl_game_rosters, and espn_nfl_game_rosters — which were templated from the same source and shared both bugs verbatim:
statistics_hrefstrict-rename of the competitors payload now renames only keys actually present, so older games that omit the team-levelstatistics$refno longer raisepolars.exceptions.ColumnNotFoundError.- Per-team roster 404 is now tolerated: a single team's missing
/roster(NoESPNDataError) no longer fails the whole game; the other team's roster is recovered, andNoESPNDataErroris raised only when every team is empty.
Adds parametrized offline regression tests across all six modules (tests/test_sibling_game_rosters.py).
0.0.61 Release: June 16, 2026
CFB — espn_cfb_game_rosters robust to long-tail ESPN payloads
Surfaced by the 2004–2023 cfbfastR-cfb-raw backfill, two deterministic failures used to empty a game's rosters entirely (then get caught upstream and banked as empty "hollow" extras):
statistics_hrefstrict-rename. Older games (e.g. pre-2021) omit the team-levelstatistics$refin the competitors payload, sostatistics_hrefnever exists and the unconditionalitems.rename({..., "statistics_href": "team_statistics_href"})raisedpolars.exceptions.ColumnNotFoundErrorfor the whole game. The renamed column is unused downstream, so the rename now applies only to keys actually present.- Per-team roster 404. A single team's
/rostersub-endpoint can 404 (NoESPNDataError) — common for older games and FCS opponents — while the other team's roster exists. The per-team loop now skips a 404 team and recovers the other, raisingNoESPNDataErroronly when every team is empty (genuinely no roster data).
Adds offline helper unit tests (tests/cfb/test_cfb_game_rosters.py, no network).
0.0.60 Release: June 15, 2026
NFL — expected points, win probability, completion probability (CP/CPOE), and expected YAC (XYAC) models
sportsdataverse.nfl.ep_wp gains nflfastR-parity modeling functions — calculate_expected_points, calculate_win_probability, calculate_completion_probability (CP + CPOE), and calculate_xyac (four XYAC sub-models: mean/median/SD yardage + completion probability) — fed by ESPN-adapter feature builders and wired into NFLPlayProcess. Ships the bundled XGBoost .ubj model files.
CFB — espn_cfb_schedule guards null-competitor placeholder events
ESPN's 2010 and 2014 college-football scoreboards include placeholder events with null competitions/competitors. espn_cfb_schedule now skips those events instead of raising TypeError: 'NoneType' object is not subscriptable and failing the entire season.
0.0.59 Release: June 13, 2026
CFB — cross-source crosswalk loaders (load_cfb_*_crosswalk)
New 404-safe dataset loaders read pre-built CFB identity crosswalks from the cfb_crosswalk release tag on sportsdataverse-data, so callers can translate ids across providers without re-scraping every source. They cache the output of the live cfb_teams_crosswalk / cfb_schedule_crosswalk / cfb_rosters_crosswalk builders (ESPN × Fox × Yahoo, keyed on an aggressively-normalized team name; see sportsdataverse.cfb.cfb_crosswalk) — a full-season schedule build otherwise fans out hundreds of requests across three providers.
ESPN — NCAA men's & women's college hockey (espn_mch_*, espn_wch_*)
- feat(espn): add NCAA men's & women's college hockey (espn_mch_, espn_wch_)
ESPN — NCAA college baseball + softball (espn_college_baseball_*, espn_college_softball_*)
- feat(espn): add NCAA college baseball + softball (espn_college_baseball_, espn_college_softball_)
ESPN — UFL, XFL, and CFL (espn_ufl_*, espn_xfl_*, espn_cfl_*)
- feat(espn): add UFL, XFL, and CFL (espn_ufl_, espn_xfl_, espn_cfl_*)
ESPN — soccer/cricket param families + soccer headline aliases (espn_soccer_*(league=), espn_cricket_*(league=), espn_epl_*, espn_ucl_*, espn_mls_*, ...)
-
feat(espn): add league-parameterized soccer + cricket families (espn_soccer_(league=), espn_cricket_(league=)) + soccer headline aliases (espn_epl_, espn_ucl_, espn_mls_*, ...)
-
feat(soccer): full-parity soccer parsers — scoreboard→matches, standings→league table (group column), summary→11-section dispatcher (header/lineups/key_events/team_stats/commentary/leaders/standings/head_to_head/last_five/game_info/shootout), teams, roster — routed via per-sport codegen overrides; feat(cricket): cricket parsers — scoreboard, standings, summary→8-section matchcard dispatcher (batting/bowling/partnerships)
-
load_cfb_teams_crosswalk(seasons=)andload_cfb_schedule_crosswalk(seasons=)are per-season (min_season2014) — teams and schedules are genuinely historical, published per year for 2014–2025. -
load_cfb_rosters_crosswalk()is season-less: ESPN's and Fox's team-roster endpoints expose only the current roster, so the artifact is a single snapshot (noseasons=argument) rather than a misleading per-season series.
All accept return_as_pandas=. Artifacts are produced by cfbfastR-cfb-data/scripts/build_cfb_crosswalk.py (the rosters table fans the per-team cfb_rosters_crosswalk out over the current season's ESPN↔Fox team-id pairs and concatenates them with espn_team_id / fox_team_id provenance). The companion on-demand builder cfb_odds_events_crosswalk (The Odds API event-id ↔ ESPN game-id) remains live-only — near-term events aren't worth caching.
0.0.58 Release: June 12, 2026
Loaders — NHL core + new NBA/MBB datasets aligned to sportsdataverse-data releases
The four core NHL loaders (load_nhl_pbp, load_nhl_player_boxscore, load_nhl_team_boxscore, load_nhl_schedule) now read the SDV-native sportsdataverse-data releases (nhl_pbp_full, nhl_player_boxscores, nhl_team_boxscores, nhl_schedules) instead of the legacy R fastRhockey-data branch — gaining the 2010 season (min_season 2011 → 2010). Added loaders for NBA/MBB datasets that were already published but had no loader, bringing them to parity with the WBB/WNBA surface: load_nba_player_season_stats, load_nba_team_season_stats, load_nba_draft, load_nba_rosters, and load_mbb_standings, load_mbb_player_season_stats, load_mbb_team_season_stats, load_mbb_rosters, load_mbb_officials, load_mbb_game_rosters — each with a generated return-schema table. Also fixed the --audit-releases drift check to key on the release tag (it parsed the human-readable title), which had been falsely flagging valid releases as missing.
Robustness & infrastructure — typing, CI gates, HTTP, deprecation policy
A package-wide hardening pass with no change to public data outputs:
- Typing + CI: ships a PEP 561
py.typedmarker; a newquality.ymlCI gate runsruff+ruff format --check+mypyon every PR, with a[tool.mypy] filesratchet (modules join the strict gate as they reach clean typing), and the test workflow now emits coverage. - Errors + logging: a
SportsDataverseErrorbase class (withSeasonNotFoundError/NoESPNDataErrorre-parented under it) and a package logger with aNullHandler; previously-silentexceptpaths now log. - HTTP layer:
dl_utils.download()reuses a module-level pooledrequests.Sessionand backs off honoringRetry-After(numeric and RFC 7231 HTTP-date, clamped non-negative, 120s ceiling) instead of a fixed sleep. - Deprecation policy: a centralized
sportsdataverse._deprecation(warn_deprecated+@deprecated) with a documented removal window; the 11 per-type NFL loader aliases migrated to it. - Codegen determinism: generator output is LF-only on every platform and the ruff format pass is pinned to the project's ruff (no CRLF phantom diffs); idempotency tests lock it in.
- Tests: a VCR-style record/replay harness (committed cassettes, secret-scrubbing) exercises the real
download()→ parser call path offline.
The Odds API wrappers (sportsdataverse.odds, toa_*)
New sportsdataverse.odds module wrapping The Odds API v4 — live + historical sports betting odds, scores, events, markets and participants across a wide range of bookmakers. Mirrors the sister R package oddsapiR's toa_* surface: toa_sports, toa_sports_odds, toa_sports_scores, toa_sports_events, toa_event_odds, toa_event_markets, toa_sports_participants, the three *_history snapshot variants, and toa_usage (cached quota, no network). The odds endpoints return tidy long-format frames (one row per event × bookmaker × market × outcome). Auth resolves from the ODDS_API_KEY env var (same variable as oddsapiR) or an api_key= argument; the call routes through the shared dl_utils.download() gateway. Same return_parsed / return_as_pandas contract (polars by default). Built from the the_odds_api OpenAPI spec.
Yahoo Sports college football wrappers (yahoo_cfb_*)
Read-only Yahoo Sports wrappers for college football over Yahoo's shangrila stats graph (graphite-secure.sports.yahoo.com/v1/query/shangrila) and editorial feed (api-secure.sports.yahoo.com/v1/editorial/s): yahoo_cfb_player_season_stats, yahoo_cfb_team_season_stats, the legacy per-category *_season_stats_legacy variants, yahoo_cfb_scoreboard, and a yahoo_cfb_boxscore scaffold. Same return_parsed / return_as_pandas contract (polars by default).
NFL — api.nfl.com wrappers cut over to generated; "NFL.com API" docs grouping
The hand-written sportsdataverse.nfl.nfl_api wrappers (nfl_standings, nfl_rosters, nfl_injuries, …) are now generated from tools/codegen/endpoints/nfl_api.yaml, like the NHL/MLB native families. The flat-API codegen gained getter_module + auth support so an authenticated family (the NFL.com WEB_DESKTOP bearer token) can be generated; the auth getter lives in nfl_api_runtime.py and the per-endpoint record extraction in nfl_api_parsers.py. As a result the NFL docs index now lists a dedicated "NFL.com API" reference grouping (11 functions) instead of burying those wrappers in "Additional functions". Wrapper signatures gain return_parsed / **kwargs.
NFL — automatic api.nfl.com token caching + NFL_ACCESS_TOKEN override
The api.nfl.com bearer token is now minted once and cached in-process, then auto-renewed just before its JWT exp — so back-to-back nfl_* / nfl_api_* calls reuse a single token instead of POSTing to /identity/v3/token on every call, with no setup and no manual refresh. A new optional NFL_ACCESS_TOKEN env var injects a pre-minted bearer token verbatim (skipping the mint + cache); the existing NFL_CLIENT_KEY / NFL_CLIENT_SECRET credential overrides still apply. nfl_clear_token_cache() forces a fresh mint, and nfl_token_gen(force_refresh=True) re-mints on demand.
Documentation — api.nfl.com OpenAPI spec
Added an OpenAPI 3.1 description of the modern NFL.com "Shield" data API (api.nfl.com: /identity/v3/token device-token auth + /football/v2/* + /experience/*) to the reference repos (sdv-internal-refs/nfl/, sdv-swagger/nfl_api_openapi.yaml).
Bug fixes
load_nfl_players()now reads the nflverse players release (players/players.parquet) on both the polars and pandas paths; the default polars path previously returned the officials dataset by mistake.- The generated
api.nfl.comwrappers route their HTTP call through the sharedsportsdataverse.dl_utils.download()gateway (retries + cache + ESPN-aware error handling) like every other wrapper, instead of callingrequests.get()directly. Boolean query flags and thenfl_weeksseason/season_typepath params are hardened soNonecan no longer leak onto the wire.
Internal — Fox data key single-sourced
sportsdataverse.cfb.cfb_fox_ext.FOX_DATA_KEY is now imported from sportsdataverse._fox_layout.DATA_KEY so the bundled public Fox key and its SDV_PY_FOX_DATA_KEY env override live in exactly one place instead of being duplicated.
0.0.57 Release: June 10, 2026
Fox Sports Bifrost wrappers (CFB, NBA, MBB, NHL, MLB)
Read-only Fox Sports "Bifrost" wrappers (fox_<sport>_*) over api.foxsports.com/bifrost/v1/<sport>/*, complementing the espn_<sport>_* families. The Bifrost API is a layout API (sections → tables → rows → cells) that is uniform across sports; a shared parsing layer (sportsdataverse/_fox_layout.py) backs every league module. Same return_parsed / return_as_pandas contract (polars by default).
CFB (cfb module): fox_cfb_pbp (quarters → drives → plays), fox_cfb_boxscore, fox_cfb_odds, fox_cfb_team_roster, fox_cfb_team_stats, fox_cfb_team_gamelog, fox_cfb_standings, fox_cfb_league_leaders.
NBA / MBB / NHL (nba / mbb / nhl modules): the same eight wrappers per sport (fox_<sport>_pbp, _boxscore, _odds, _team_roster, _team_stats, _team_gamelog, _standings, _league_leaders). Play-by-play is period-based (QUARTER / HALF / PERIOD → plays); boxscore is tidy long per player-stat.
MLB (mlb module): fox_mlb_team_roster, fox_mlb_team_stats, fox_mlb_team_gamelog, fox_mlb_standings, fox_mlb_league_leaders, fox_mlb_odds. Fox does not expose MLB play-by-play or boxscore via event/{id}/data, so those two are intentionally omitted.
Live-tested (gated behind SDV_PY_LIVE_TESTS=1). Reverse-engineering notes + an OpenAPI 3.1 spec live in the sdv-internal-refs repo. Parallels the cfbfastR / hoopR / fastRhockey / baseballr fox_* families.
CFB — Fox as a backup source for the EPA/WPA play processor (fox_cfb_play_process)
Where fox_cfb_pbp returns the raw Fox play rows, fox_cfb_play_process runs Fox data through the same CFBPlayProcess pipeline ESPN games use — producing EPA / WPA / advanced box score — as a backup/alternative when ESPN is unavailable. The new module sportsdataverse.cfb.cfb_pbp_fox adapts a Fox cfb/event/{id}/data payload into the ESPN-summary shape the processor consumes (fox_to_espn_summary), so the 6,000-line pipeline runs unmodified.
fox_cfb_play_process(event_id)— fetch + adapt +run_processing_pipeline(orprocess=Falsefor cleaning-only,raw=Truefor the adapted summary). Returns the processed payload taggedsource="fox".fox_to_espn_summary(fox_data)— the adapter (modalPlay.events[].yardStart→ yards-to-goal, play title → down/distance,events[].text→ ESPNtype.textvocab, team logo → possession).- Validated offline (5 tests) on a captured blowout (FSU 66-10 → FSU +0.50 vs Kent −0.94 EPA/play — game-consistent). High fidelity on the structured/numeric path (down/distance/yards-to-goal/EPA/WPA); text-grammar features (detailed player attribution, penalty yards) degrade vs ESPN. Archive-format Fox games (no
modalPlaygeometry) are detected and rejected. A Fox event id differs from an ESPN game id; backing up a specific ESPN game needs matching by teams + date.
0.0.56 Release: June 9, 2026
HockeyTech — live multi-league scraper (PWHL + AHL/OHL/WHL/QMJHL) + on-ice/Corsi/TOI analytics
A new sportsdataverse.hockeytech core powers live wrappers over the HockeyTech
feeds, alongside the existing offline load_pwhl_* loaders:
- PWHL (
sportsdataverse.pwhl): 20 livepwhl_*()functions at fastRhockey output parity —pwhl_schedule,pwhl_scorebar,pwhl_game_info,pwhl_game_summary,pwhl_pbp,pwhl_player_box,pwhl_teams,pwhl_team_roster,pwhl_standings,pwhl_player_info,pwhl_player_stats,pwhl_player_game_log,pwhl_player_search,pwhl_stats,pwhl_leaders,pwhl_streaks,pwhl_transactions,pwhl_playoff_bracket,pwhl_season_id, andmost_recent_pwhl_season. - AHL / OHL / WHL / QMJHL (
sportsdataverse.{ahl,ohl,whl,qmjhl}): per-league families (schedule, pbp, standings, teams, team_roster, player_stats, leaders, game_summary, season_id,most_recent_<lg>_season) over one shared core. - Analytics across all five leagues:
<lg>_game_shifts,<lg>_player_toi, and<lg>_game_corsi(player-level on-ice Corsi/Fenwick), reconstructed from the shift tables via countdown-clock interval matching.<lg>_pbpis enriched to a superset (coordinate transforms, clock columns, shot distance/angle, scoring chances, on-ice players, game-meta join,blocked_shot/hitevents). - Corsi/Fenwick caveat: the HockeyTech feed has no missed-shot event, so both
metrics are computed from shots-on-goal + blocked + goals and every analytics
output carries
corsi_includes_missed = False. - All returned columns are snake_case; PWHL columns match fastRhockey exactly. A companion fastRhockey (R) release mirrors this surface, verified by a cross-language parity test pinning identical Corsi/TOI numbers.
NFL — Next Gen Stats (nfl_ngs_*) + api.nfl.com football/v2 (nfl_*) modules
- New
sportsdataverse/nfl/nfl_ngs.py— token-free Next Gen Stats wrappers overnextgenstats.nfl.com/api(browser session, no auth). 10 functions / 21 endpoints:nfl_ngs_statboard(passing/receiving/rushing),nfl_ngs_statboard_leaders,nfl_ngs_leaders(speed/distance/time-to-sack + completion/ery/yac expectation, season & week),nfl_ngs_league_schedule[_current],nfl_ngs_league_teams,nfl_ngs_gamecenter_overview,nfl_ngs_microsite_chart[_players],nfl_ngs_play_is_highlight. The/live/*NGS endpoints are anonymous-403 (need elevated auth) and are documented as omitted. - New
sportsdataverse/nfl/nfl_api.py—api.nfl.com/football/v2+/experiencewrappers on the bearer token (reusesnfl_headers_gen). 11 functions:nfl_standings,nfl_rosters,nfl_teams_history,nfl_team,nfl_weeks,nfl_weeks_by_date,nfl_combine_profiles,nfl_draft_picks,nfl_injuries,nfl_game_summaries,nfl_weekly_game_details. - Both return tidy polars DataFrames by default (
return_as_pandassupported) and are documented on the NFL reference pages. Catalogued from a full crawl of the NFL API surface (api.nfl.com + NGS). - Captured autodoc return-column tables for all 23 new NFL functions (live
introspection ->
schemas/autodoc/nfl/*.yaml+autodoc_example_args.yaml), so each renders acol_name | type | descriptiontable on its reference page.
NFL — restored the api.nfl.com game schedule + play-by-play wrappers
nfl_game_schedule/nfl_game_detailswere broken because NFL.com retired the old/v1/rerouteclient-credentials token endpoint (404 ->JSONDecodeError). Rebuiltsportsdataverse/nfl/nfl_games.pyon the modern flow the NFL.com web app (and nflverse'snflapi) now use:nfl_token_gen()mints a bearer token from/identity/v3/token(form-encoded device grant,X-Domain-Id: 100);nfl_game_schedule()reads/football/v2/games/season/{s}/seasonType/{t}/week/{w};nfl_game_details()reads/experience/v1/gamedetails/{id}and unwraps the shielddata.viewer.gameDetailobject (plays, drives, scoring summaries, line scores).- Auth uses the NFL.com public
WEB_DESKTOPweb-client credentials as defaults, overridable viaNFL_CLIENT_KEY/NFL_CLIENT_SECRETenv vars or function args (no personal account; the token carries the anonymousfreeplan). Verified live: 16 games for 2024 REG wk1, 194 plays / 20 drives for the opener. - Added a parsed surface over the raw dicts:
nfl_game_pbp(game_id)returns a tidy polars/pandas DataFrame (one row per play, withgame_id/home_team/visitor_teamcontext), andnfl_week_games(season, season_type, week)returns one row per game. (Named to avoid colliding with thenfl_pbp/nfl_schedulesubmodules.)
ESPN — remove always-erroring endpoint variants + NFL R-parity
- Removed dead ESPN endpoint variants (all leagues). A live health sweep found
these generated wrappers 404 /
NoESPNDataErrorat ESPN for every league and season: the season-lessespn_*_coacheslist (/leagues/{league}/coaches) and the fourespn_*_calendar_{offseason,regular_season,postseason,ondays}sub-paths. They are dropped from the codegen so the package no longer ships endpoints that always raise. The working counterparts remain:espn_*_season_coaches(/seasons/{season}/coaches), the coach-detail endpoints (espn_*_coach, ...), and the baseespn_*_calendar. (~40 dead functions removed across 8 leagues.) - NFL Python ↔ R parity. Added curated
r_parity_aliases.yamlentries mapping the canonicalload_nfl_*loaders to their nflreadr equivalents (e.g.load_nfl_pbp→load_pbp,load_nfl_schedule→load_schedules), so the NFL parity table links both naming styles (nfl rows 26 → 49). Theload_nfl_*/ bareload_*dual-naming itself was verified already consistent (intentional nflreadpy parity; the only unaliasedload_nfl_*are deprecated or sdv-specific).
Documentation — per-league Python ↔ R parity tables
- Each league's
index.mdnow carries a Python ↔ R parity table mapping everysportsdataversefunction to its equivalent in the sister R package (cfbfastR / hoopR / wehoop / baseballr / fastRhockey), linking the Python doc page and the R pkgdown reference. Driven by a newtools/codegen/build_r_exports.pyminer (NAMESPACE → committedr_exports.yaml, so links never 404 and the offline--checkstays deterministic) plus a curatedr_parity_aliases.yamlfor divergent names (e.g.mlb_api_*→ baseballrmlb_*, +36 verified). Coverage: nhl 202, mlb 107, wnba 83, nba/wbb 74, mbb 69, cfb 55, nfl 26, pwhl 15. - Fixed a self-referential codegen bug the parity table exposed:
render_autodoc_pagecomputed "already documented" against a corpus that included the index, so the index's parity table (which names autodoc functions) caused those functions to be dropped fromadditional.mdand their parity links to 404. It now uses the reference-pages corpus only, matching the autodoc-name count used for the index.
Documentation — example notebooks repaired, expanded, and rendered on-site
- Repaired the example notebooks. Live execution (
nbclient) surfaced runtime schema/usage drift that import/compile checks miss: ESPN schedule team columns renamed tohome_display_name/away_display_name;espn_*_pbp()['plays']is a raw list using dot-notation keys (period.number,clock.displayValue,scoringPlay,shootingPlay,coordinate.x/.y) built viapl.DataFrame(...); ESPN scores are strings (cast before arithmetic);espn_cfb_scheduletakesdates=notseason=; ESPN team rosters usefull_name;espn_*_team_statsreturns a dict{Averages, Totals, Misc}; some hardcoded dates had no games. All notebooks now execute clean end-to-end. - Split + expanded the suite to ten notebooks. The combined
wbb_wnbanotebook was split into separate05_wbb_introand08_wnba_intro, both expanded; the NHL notebook gained an ESPN-NHL section alongside the native api-web surface; and two new notebooks were added:09_mlb_intro(MLB Stats API + Statcast + ESPN MLB) and10_pwhl_intro(PWHL loaders). - On-site rendered Tutorials. New
tools/codegen/render_notebooks.pyexecutes each notebook and renders it (with real outputs, as clean monospace tables) to a themed page underdocs/docs/tutorials/, surfaced in a new Tutorials sidebar section. Execution is quarantined to the weeklylive-tests-cronworkflow, which now re-executes + renders and opens a refresh PR (main is branch-protected); the normal offline docs build just consumes the committed pages. Each league index's Examples section now links the on-site tutorial pages instead of GitHub.
NHL / PWHL — loader naming-parity aliases + games-manifest loaders (fastRhockey parity)
- Added 4 NHL short-name aliases in
sportsdataverse/nhl/nhl_loaders.py:load_nhl_team_box→load_nhl_team_boxscore,load_nhl_player_box→load_nhl_player_boxscore,load_nhl_skater_box→load_nhl_skater_boxscores,load_nhl_goalie_box→load_nhl_goalie_boxscores. - Added 5 PWHL short-name aliases in
sportsdataverse/pwhl/pwhl_loaders.py:load_pwhl_team_box→load_pwhl_team_boxscores,load_pwhl_player_box→load_pwhl_player_boxscores,load_pwhl_skater_box→load_pwhl_skater_boxscores,load_pwhl_goalie_box→load_pwhl_goalie_boxscores,load_pwhl_schedule→load_pwhl_schedules. - Added
load_nhl_games()(noseasonsarg) reading the NHL games-in-data-repo manifest parquet from thenhl_schedulesrelease asset (primary URL verified working:sportsdataverse-data/releases/download/nhl_schedules/nhl_games_in_data_repo.parquet). - Added
load_pwhl_games()(noseasonsarg) reading the PWHL games-in-data-repo manifest parquet from thepwhl_schedulesrelease asset (primary URL verified working:sportsdataverse-data/releases/download/pwhl_schedules/pwhl_games_in_data_repo.parquet). - Added
tests/test_loader_parity.pycovering importability,__all__membership, docstring-based forwarding assertions, and live alias shape-parity + manifest tests (gated behindSDV_PY_LIVE_TESTS=1).
Documentation — NFL return-table descriptions mined from nflverse
- Extended
tools/codegen/build_r_col_descriptions.pywith two nflverse source parsers:mine_csv_dictionaries()reads nflreadr's canonicaldata-raw/dictionary_*.csvfield docs (delimiter-sniffing for the semicolon-delimited NGS file, BOM-stripping forroster_status, and case-insensitive Field/Description column resolution across 6 header variants), andmine_item_list()reads nflfastR'sdata-raw/variable_list.txt(\item{Field}{Description}form). Yieldsnflreadr(941 columns) andnflfastR(372 columns) dictionaries inr_column_descriptions.yaml. - Mapped
nfl → nflreadringenerate.py's_LEAGUE_R_PACKAGE; nflfastR's fields still contribute via the_mergedcross-package fallback. - NFL generated reference-page description fill rose from ~36% to ~85%. The
enlarged
_mergedunion (7.3k → 8.1k columns) also backfilled previously-blank shared football/stat columns on the CFB and MLB reference pages (e.g.passing_yards,receptions,kicker_player_name,name_short).
Documentation — class methods rendered on autodoc pages (CFB / NFL)
- Hand-written classes (
CFBPlayProcess,NFLPlayProcess) previously rendered on theadditionalreference pages as a bare constructor signature with no description and an empty parameter table — their public methods, returns, and examples were omitted entirely. The autodoc renderer now treats a class specially:_doc_view()attaches a per-method doc-view list (via_augment_class_view()), and theautodoc_page.md.jinjatemplate renders each public method as a nested#### Class.method(...)entry with its description, parameters, returns, and example. Both classes now document all 7 of their public methods (espn_*_pbp,*_pbp_disk,*_pbp_json,corrupt_pbp_check,create_box_score,run_cleaning_pipeline,run_processing_pipeline). - Constructor parameter descriptions are backfilled from the class's
__init__docstring when the class object itself carries none (CFBPlayProcessdocuments its ctor args on__init__), so the constructor parameter table now renders with descriptions instead of blank cells. - Added a class-level docstring to
CFBPlayProcess(it had none) mirroringNFLPlayProcess, so the class entry leads with an overview + runnable example instead of aNo description available.placeholder.
Documentation — accuracy-audit fixes
- Stable autodoc anchors. Every autodoc function/class heading now carries an
explicit
{#name}id, so it is reliably deep-linkable instead of relying on a signature-derived slug. This fixes a broken cross-link inecosystem.md(espn_nhl_teamsnow resolves to itsadditionalpage entry) and future-proofs any reference to a hand-written wrapper. - Invalid example code.
_clean_example()mis-handled reST literal-block intros that wrap across multiple prose lines (only the line ending in::was recognized), leaking a prose sentence into the rendered```pythonblock as a broken statement. It now absorbs the preceding contiguous intro lines into the step comment. Fixes theNflConfigandespn_wbb_team_statsexamples; all non-REPL doc examples now compile. - Notebook reachability.
ecosystem.mdnow links all seven example notebooks individually (previously only01_quickstartwas linked; the per-sport intros 02–07 were an un-linked "for your league" mention). Each league's generatedindex.mdlanding page also gained an Examples section linking the quickstart plus that sport's intro notebook (render_league_index+ a league→notebook map ingenerate.py); mlb/pwhl show the quickstart until they get a dedicated intro.
0.0.55 Release: June 8, 2026
Documentation — richer per-function reference
- Autodoc "Additional functions" pages now render full Parameters tables (name/type/default/description), Returns, and runnable Example blocks parsed from each function's docstring (previously just a signature + one line).
- Endpoint reference pages gained a Description column on the parameter table; shared query params carry authored descriptions.
- Function Returns are now
col_name | type | descriptiontables: endpoint pages from introspected per-sport schemas, and autodoc DataFrame functions from a newgenerate.py --autodoc-schemaslive-introspection pass (best-effort, with prose fallback where a function can't be introspected offline). - Return-table column descriptions are filled by column name from the sibling
SDV R packages'
@returndocs (cfbfastR / hoopR / wehoop / baseballr), mined totools/codegen/r_column_descriptions.yamland applied at render time (hand-curated descriptions take precedence; unmatched columns stay blank).
Bug fixes
espn_mbb_game_rosters/espn_wbb_game_rosters/espn_nfl_game_rosters: fixed aShapeError(positional column rename broke when ESPN ships extra*_$reffields); columns are now renamed by source key.espn_nhl_schedule: fixed'NoneType' object has no attribute 'get'with default args (a helper was missing itsreturn event).- The
espn_*_game_rostersrename-by-source-key fix is additive: NBA/WNBA roster frames now include ateam_alternate_ids_sdrcolumn when ESPN ships it (the old positional rename would have raised once that field appeared).
0.0.54 Release: June 8, 2026
Per-sport return schemas (correctness)
@return tables are now derived per league by running the real parsers against
captured per-sport fixtures (generate.py --schemas), replacing the previous
sport-agnostic schemas that showed (e.g.) basketball boxscore columns on MLB/NHL
pages. Native API pages (nhl_api_web/nhl_edge/nhl_records/nhl_stats_rest/
mlb_api) gained accurate return schemas. Schemas are now introspected truth,
gated by generate.py --check.
BREAKING — parser-backed wrappers return a DataFrame by default
return_parsed now defaults to True for the parser-backed wrappers; they
return a tidy polars DataFrame instead of the raw Dict. Pass
return_parsed=False to recover the raw Dict; return_as_pandas=True switches
polars→pandas. Wrappers without a registered parser are unchanged (still Dict).
The sportsdataverse.parsed.{league} mirror modules are unaffected.
Docs coverage gate + autodoc
Every user-facing function now reaches the docs. A new generate.py --coverage
audit enumerates in-scope exported functions per league and fails --check if
any is undocumented (allowlist for cross-cutting internals in
tools/codegen/coverage_allowlist.yaml). ~180 hand-written wrappers/loaders/
statcast/utility functions that the endpoint-YAML codegen never documented are
now rendered into per-league "Additional functions" reference pages
(autodoc from live signatures + docstrings).
MLB - full MLB Stats API coverage
The codegen now wraps the full statsapi.mlb.com surface: 38 previously
unwrapped endpoints were added (home run derby, all-star ballots, conferences,
free agents, game pace, jobs/datacasters/official-scorers, team coaches/
personnel, schedule variants, seasons/all, sport, teams history/stats, etc.).
28 are publicly serviceable and ship with captured fixtures + introspected
return schemas; the remaining handful are auth-gated/internal MLBAM feeds
(analytics/guids/color), wrapped with valid example args for if/when access
exists.
Deprecations
sportsdataverse.parsed.{league}is deprecated (since the default modules now return parsed DataFrames by default). Importing aparsed.*module emits aDeprecationWarning; it still works and will be removed in a future release. Migrate tofrom sportsdataverse.{league} import <fn>directly.
0.0.53 Release: June 8, 2026
ESPN — declarative codegen + factory retirement
The runtime "magic" that mass-registered each league's espn_<league>_* ESPN
wrappers at import time (_common_espn.make_league_module / _bind + the
_UNIVERSAL_WRAPPERS / _NCAA_WRAPPERS / _FOOTBALL_WRAPPERS / _MLB_WRAPPERS
tables + ~127 private _site_v2_* / _espn_* / _core_v2_* core functions) has
been replaced by a declarative codegen pipeline (tools/codegen/). Endpoint
metadata lives in tools/codegen/endpoints/*.yaml; generate.py renders concrete,
fully-documented wrapper modules into sportsdataverse/<league>/<league>_espn_ext.py.
- New
espn_nhl_*surface (115 functions). NHL previously had no ESPN cross-league wrappers; it now gets the full Site v2 / Web v3 / Core v2 surface, andfind()works for NHL for free. - Identical behavior, real signatures. Every generated function builds a
byte-identical URL + query string to the function it replaced (verified by a
URL+params parity gate across all scopes), but now exposes concrete parameter
names, type hints, and docstrings instead of an opaque
*args, **kwargsshim. - Names aligned to the R sister packages (universal, token-level convention).
Across all eight leagues the generated
espn_*names follow the cfbfastR/hoopR/wehoop taxonomy (behavior unchanged). The rename is applied at the underscore-token level (not just prefixes), soathlete/eventconvert in every position incl. plurals:athlete -> player(athlete_vs_athlete -> player_vs_player,athletes_index -> players_index,season_athletes -> season_players),event -> game(bareevent -> game,events -> games,event_* -> game_*,season_week_events -> season_week_games). Two combined mappings run first:event_competitor* -> game_team*(a competitor is the game's team) andevent_competition -> game_competition/event_competition_* -> game_*. Compound tokens likeeventlogare preserved (athlete_eventlog -> player_eventlog). cfb additionally getsseason_*cleanups vs cfbfastR (futures/groups/recruits/week_rankings;powerindex -> team_powerindex). Rule engine:generate._convention_rename; cfb-specific exceptions:tools/codegen/espn_rename_map.yaml. - Collision-guarded. Renames that would clash with a hand-written sibling or
another generated name are skipped automatically:
teams_site(raw endpoint, != parsedespn_*_teams) andespn_cfb_season_{team,awards,coaches}(vs the catalog). SAME-endpoint duplicates are dropped: the generated rawespn_{wbb,wnba}_game_officialsis suppressed (viaespn_rename_map.yamldrop:) because the hand-written parsedespn_{wbb,wnba}_game_officials(renamed fromevent_officials, core-api officials with ids) exposes the same endpoint. One->many splits (e.g.summary) remain for curation (seedocs/superpowers/specs/espn-r-naming-worksheet.md). - Versioned collision rule (dynamic, "one stays bare"). When a generated name
would collide with an existing function but they hit different endpoints, both are
kept: ONE keeps the bare name and the larger/newer one is version-qualified. This is
now decided dynamically by the generator (
_league_module_sourcepass 2 +_versioned_on_collision), not hard-coded. The web-common-v3/athletes/{id}/statsendpoint wants the bareplayer_stats; it is version-qualified toespn_*_player_stats_v3only when a hand-written bareplayer_statsalready claims the name — a league without that sibling would get the bare name automatically (no orphaned*_v3). - Cross-league
player_statsparity (core-v2 season) for ALL eight ESPN leagues. Every league now exposes a bareespn_<league>_player_stats(core-v2/seasons/{season}/types/{type}/athletes/{id}/statisticsseason line) returning one wide, self-describing row (athlete identity + season line as{category}_{stat}columns +team_*identity), plus the generatedespn_<league>_player_stats_v3(web-v3 comprehensive) — matching the hoopR/wehoop/cfbfastR convention exactly. nba, mbb, nfl, nhl, mlb, and cfb gain new hand-written wrappers; wnba/wbb were already converted. A single sport-aware core (sportsdataverse._common_espn_player_stats._espn_player_stats) backs all eight (basketball/football/baseball/hockey share the core-v2splits.categories[].stats[]shape and athlete/team$refgraph). Newseason_type("regular"/"postseason") andtotalparams mirror the wehoop signature. BREAKING:espn_wnba_player_stats/espn_wbb_player_statspreviously hit web-v3 and returned adictof category frames; they now return a single core-v2 seasonDataFrame(the web-v3 payload moved to*_player_stats_v3). _get/_csvsingle source. The HTTP + coercion helpers now live insportsdataverse._codegen_runtime(shared by all generated wrappers);_common_espnre-exports them. Note for test authors: mocksportsdataverse._codegen_runtime.download(not_common_espn._get) to intercept the generated wrappers.- Drift guard.
python tools/codegen/generate.py --check(and thesdv-codegenpre-commit hook) fail if the committed wrappers fall out of sync with the endpoint metadata.
BREAKING (internal): sportsdataverse._common_espn no longer exposes the
factory (make_league_module / _bind / the _*_WRAPPERS tables) or the private
_site_v2_* / _core_v2_* core functions. Public espn_<league>_* wrappers are
unchanged in name and behavior.
NHL native — codegen cutover + clean names (api-web; in progress)
The hand-written NHL native modules are being regenerated from endpoint specs
(via tools/codegen/extract_native.py -> flat-API YAML -> generate.py) with
clean, R-aligned names, family by family. First family: nhl_api_web.
- BREAKING renames (
nhl_web_* -> nhl_*where the clean name is free; the qualifier is kept only on collision with a hand-written composite, sonhl_web_pbpandnhl_web_scheduleare unchanged): e.g.nhl_web_boxscore -> nhl_boxscore,nhl_web_standings -> nhl_standings,nhl_web_roster -> nhl_roster,nhl_web_scoreboard -> nhl_scoreboard(26 functions; full map intools/codegen/rename_map.yaml). Behavior (URL + params + parser) is identical -- faithfulness was verified bytest_parity_nativebefore the swap. nhl_scoreboard(the 3-way team/date/now branch) stays hand-written innhl_api_web_extra.py-- the single-URL-builder codegen can't represent it.- Removed the deprecated
sportsdataverse.nhl.nhl_apimodule (targeted the retiredstatsapi.web.nhl.com); usenhl_api_web/nhl_pbpinstead. nhl_edge(family 2) andnhl_stats_rest(family 3) are now generated too. Both keep their meaningful API namespaces (nhl_edge_*,nhl_stats_rest_*) so they are non-breaking codegen-ifications (35 + 21 functions). stats_rest's arbitrary**filterspower feature (cayenneExp/sort/limit/...) is preserved via a newpassthrough_queryengine mode that forwards None-filtered**kwargsas query params;return_parsedis additionally wired where a parser exists.nhl_records(family 4) is generated too -- keptnhl_records_*(distinct records.nhl.com product), non-breaking (50 functions: 44 generated +passthrough_query, 6 value-embedded/scope-conditional ones preserved hand-written innhl_records_extra.py).mlb_api(family 5, final) is generated too -- keptmlb_api_*(the raw MLB Stats API namespace, distinct from the curatedmlb_*composites), non-breaking (41 names: 26 generated +passthrough_queryfor hydrate/fields, 15 conditional-_csv/ multi-param //api/v1.1/-host functions preserved hand-written inmlb_api_extra.py).- All five native families are now codegen-generated. Only
nhl_api_webwas a breaking rename (itswebqualifier was host-noise); the other four kept their meaningful API namespaces. The codegen engine gained flat-API collision resolution (FlatApi.qualifier+resolve_name),passthrough_query, and abuild_flat/--checkdrift gate.test_parity_nativelocked in each family's faithfulness before its swap.
Dataset loaders — release manifest + drift audit
releases.yamlmanifest expanded 24 -> 92 loaders, seeded from the live sportsdataverse-data release list: every release tag shipping season-partitioned*.parquetassets gets a 404-safe loader entry whose URL is derived from the actual asset names (verified to resolve). New coverage: WNBA (espn_wnba_*+wnba_stats_*), PWHL (15 datasets, a new league), NHL (fullnhl_*family incl. EDGE/lite/boxscores), WBB, NBA, MBB.generate.py --audit-releasescompares the manifest against the live release list (gh CLI) and reports tags with no loader (gaps) + orphans -- a CI-oriented drift gate (separate from the offline--check).tests/codegen/fixtures/release_tags.txtsnapshots the live tags for offline coverage tests.- Release tags that don't yet ship parquet (empty / csv-only / season-less -- e.g.
several
espn_cfb_*advanced-box tags,nba_stats_*boxscores) are intentionally absent and surfaced by the audit until parquet lands. @returncolumn tables (Task 4): every non-stub loader's parquet footer is introspected intotools/codegen/schemas/loader_schemas.yaml(92 datasets) and rendered as a|col_name|type|table in the generated loader docstrings (reproducible viagenerate.py --loader-schemas).- All loader modules are now generated (Task 5 complete). The new
sportsdataverse.pwhlleague (15 loaders) plus the six existing leagues (cfb/mbb/nba/nhl/wbb/wnba) are rendered from the manifest into{league}/{league}_loaders.py-- expanding from 4 hand-written loaders per league to the full release-backed set (nhl 24, wnba 25, wbb 11, nba 9, pwhl 15, ...), each with@returncolumn tables. Zero loss (verified before/after): the season-less / helper functions the loop template can't express are preserved hand-written in{league}_loaders_extra.pyresiduals --cfb:load_cfb_betting_lines+get_cfb_teams;nhl:nhl_teams. The codegenbuild/--checkdrift gate covers all generated loader modules (_GENERATED_LOADER_LEAGUES). Verified live:load_pwhl_pbp(2024)-> 10,456 rows,load_nhl_pbp_lite(2010)-> 400,512,load_wnba_shots(2024)-> 45,480.
Generated documentation — reference pages + drift gate
generate.py --docsrenders the full reference docs tree from the same endpoint/loader/parameter metadata that drives the wrappers, directly into the live Docusaurus "Next" surface (docs/docs/{league}/). 64 files: per-leagueindex.md+_category_.json, a per-API reference page for every ESPN API (site/web/core) and native flat family (nhl_api_web/nhl_edge/nhl_stats_rest/nhl_records/mlb_api), areference/loaders.mdper loader league, and a sharedreference/parameters.md. This replaces the legacy Sphinx apidoc dumps — the 7 per-leagueindex.mdSphinx pages plus the hand-authored NHL/MLB conceptual pages were regenerated/removed; package-wide conceptual pages (intro,quality-of-life,architecture/,parsers/) are preserved untouched.- 8-section function block (
templates/_reference_block.jinja): summary, endpoint URL, a concrete valid example URL, an nba_api-style| API Parameter | Python | Pattern | Required | Nullable |table, a@returncolumn table sourced from thereturns_schema(handles bothkind: dataframeand multi-framekind: framespayloads), a runnablepythonexample, and a validated-date line. - Names never drift from code: the per-endpoint name-resolution passes were
extracted into shared
_espn_league_views()/_flat_views()helpers used by both the module renderer and the docs renderer, so a reference page always documents the exact wrapper name that gets emitted (e.g. the collision-qualifiednhl_web_pbpalongside the cleannhl_boxscore). - Drift gate:
--checknow also fails on stale generated docs (and orphans inside the fully-generated league/reference/dirs — conceptual pages outside them are never flagged); the defaultbuildwrites them.tools/codegen/fetch_packages.pysnapshots the SDV package list for an optional packages page (network tool; the gate stays offline by omitting the page when no snapshot is committed). New offline teststests/codegen/test_docs.py+test_doc_parity.pyassert the 8-section contract across every league x API and that the live tree is current. - Docusaurus migration:
docs/sidebars.tsnow drives each league as a clickable category (link → generatedindex) expanding to an autogenerated reference subtree, so new endpoints surface in the nav with no sidebar edit; added a top-level "Parameter reference" entry. The legacy Sphinx pipeline (create_docs.sh+Sphinx-docs/) is deleted, along with its now-unusedsphinx/sphinx-markdown-builder/sphinx-materialdev dependencies (dropped from thedocsextra/group + theallextra;uv.lockre-resolved).yarn buildpasses with a link-clean/docs/next/surface (remaining broken-anchor warnings are confined to the frozen0.0.50version + the CHANGELOG doctoc fragments). - Example notebooks are CI-executed (
nbmake): the example notebooks were audited against the post-rename API — only02_cfb_introbroke (the standaloneespn_cfb_pbp(game_id=...)is gone), and its PBP cells were rewritten to theCFBPlayProcess(gameId=...).espn_cfb_pbp()+.run_processing_pipeline()flow (verified live).nbmakewas added to thetestdependency group, and the weeklylive-tests-cronworkflow now runspytest --nbmake examples/notebooks/as an informational (non-blocking) leg so notebook breakage surfaces as drift. - Cohesive intro docs: a new Ecosystem & philosophy
page ties the docs together — the design philosophy, the full function-naming
paradigm (
espn_<league>_*, native<league>_*,load_<league>_*,parse_*, plus the R-aligned athlete→player / event→game conventions and collision rules), the Python ↔ R sister mapping (hoopR / wehoop / cfbfastR / baseballr / fastRhockey, plus oddsapiR / recruitR / sportyR / sportypy / sportsdataverse.js), and how the package relates to nflverse (the NFL module mirrors nflreadpy) and the wider PySport ecosystem.intro.mdand all seven example notebooks now open with a consistent philosophy/naming blurb and link to it. The page also documents the companion data repositories (sportsdataverse-data releases, cfbfastR-data, fastRhockey-data, nflverse-data) behind theload_*family and links each league's generated Automation status loader table, and includes a 1:1 function map — a table whosesportsdataverse-pyfunctions deep-link to their reference pages and whose R-sister functions link to the matching hoopR/wehoop/cfbfastR/fastRhockey/baseballr pkgdown docs (verified against each package's NAMESPACE). The ESPN cross-league architecture page was realigned from the retiredmake_league_module()runtime factory to the current declarative-codegen reality. - Docs default flipped to the overhauled tree + per-release snapshot policy:
docusaurus.config.tssetslastVersion: 'current'(labelledmain), so the generated reference + conceptual docs are the live DEFAULT at the root/docs/and auto-refresh on every deploy instead of sitting at/docs/next/behind the frozen 0.0.50 Sphinx dumps; the legacy docs stay archived at/docs/0.0.50/. The site builds on Vercel on push tomain(no in-repo deploy workflow — a GitHub Pages action would double-publish). At each release, freeze a per-release archive with the newcd docs && yarn version:docs <x.y.z>helper (keepingcurrent/mainthe default) — so the live docs never drift from the code (codegen--check-gated) yet every release still gets a frozen record. The release step is documented in CLAUDE.md. - Home page refreshed:
docs/src/pages/index.tsxwas rewritten from the stale MBB/CFB/EPA cards to the full current surface — Basketball / Football / Baseball / Hockey (incl. native NHL & MLB APIs, loaders, the tidy-by-default parser layer) — each card naming its R sister, with an "Ecosystem & philosophy" call-to-action. - Declined follow-up: a data-driven SDV navbar dropdown —
projects.jsoncarries no canonical doc URLs, so the curated navbar indocusaurus.config.ts(which has them) stays authoritative.
CFB — advanced box score expansion (create_box_score)
CFBPlayProcess.create_box_score() (and therefore run_processing_pipeline()'s
advBoxScore) now emits two additional per-player sections alongside the existing eight:
defensive_players— per-defender havoc events attributed by player and defensive team:sacks(+sacks_yards),pass_breakups,interceptions(+interceptions_yards),forced_fumbles,fumble_recoveries(+fumble_recoveries_yards). Keyed bydef_pos_team+player_name. Columns present vary per game (only populated stats appear); all values derive from existing enriched play columns (no new tracking data).specialists— per-player kicking/punting/return production keyed bypos_team+player_name:field_goals(+field_goals_yards),punts(+punts_yards),kick_returns(+kick_returns_yards),punt_returns(+punt_returns_yards).
Both are additive and degrade to [] when no events are attributable. The existing
pass/rush/receiver/team/situational/defensive/turnover/drives sections are
unchanged.
CFB — box-score attribution correctness + ESPN-sourced totals (create_box_score)
A correctness pass on team/player attribution in the advanced box score, reconciled against ESPN's official box score for a 5-game fixture set (all turnover totals now match ESPN exactly). All output is additive — existing field names are preserved; previously wrong values are corrected and new fields/sections are added.
- Per-play attribution layer (
__add_attribution_cols): resolves the credited team for every play from the play text + flags, aware thatpos_team/def_pos_teamswap roles by play type (on a kickoffpos_teamis the receiving team; on a punt it is the punting team). Produceskicking_team,return_team,fumbling_team,recovery_team,recovery_team_2,penalized_team, and per-side turnover flags. - Special-teams turnovers are now counted. Previously the turnover box filtered to
scrimmage plays, dropping muffed punts, kickoff-return fumbles, and blocked-kick
recoveries; these are now included. Muffs (
"muffed by …") are detected as fumbles, and overturned plays ("(Original Play: …)"after a reversed review) are stripped before parsing so a reversed fumble is not counted. - Per-side turnover model. A single play can register a turnover for both teams via
is_pos_team_turnover/is_def_pos_team_turnoverand a 2-deep recovery chain — e.g. an interception returned and fumbled back, or a sack-strip where the recovering defense fumbles it back. Turnover margins/luck are keyed by team identity (fixing a prior group-order bug that could swap or sign-flip them). Theturnoverlist is now ordered[home, away]and every row carriesteam_id— consumers should key byteam_idrather than list position (the previous order came from an unordered group-by). - Correct team attribution for fumble recoveries (own recoveries credited to the
recovering team, not always the defense), punt returns (credited to the returning team,
not the punting team), and penalty yards (charged to the penalized team via
penalty_yards, with the legacytotal_pen_yardsretained). - End-of-period play-drop fix. A dedup heuristic was dropping the real play immediately before an "End of period/half/game" marker (which inherits its start state) — losing end-of-half turnovers such as a Hail Mary interception. Guarded so end markers never trigger dedup of the preceding play.
- ESPN-sourced totals. New
espn_teamandespn_playerssections surface ESPN's official box verbatim (turnovers, fumbles lost, interceptions, total/passing/ rushing yards, penalties, first downs, player stat lines) as the authoritative source for countable totals. Theturnoversection sourcesturnovers/Int/fumbles_lostfrom the ESPN box (espn_sourced=True), keeping the play-by-play derivation under*_pbpkeys as the fallback and as a validated cross-check. - Clean player names.
run_processing_pipeline()joins ESPN's per-play participants (espn_cfb_play_participants) to replace regex-extracted names (which carried team prefixes, e.g."BYU Dayan Ghanwoloku") with clean display names, with graceful fallback to the regex names when offline. Setjoin_participants = Falseto skip the fetch (used by offline reprocessing and the offline test suite).
CFB — play-type reclassification: interception-return-fumble guard (__add_new_play_types)
-
Interceptions are no longer mislabeled as fumble recoveries. The "strip-sack → fumble" reclassification rules fire on
fumble_vec & pass & change_of_poss==1. An interception also setschange_of_poss=1, so a pick whose returner subsequently fumbled matched the predicate and was relabeled"Fumble Recovery (Opponent)", erasing the interception (and, because the downstreamintflag is derived fromtype.text, zeroing it for EPA/WPA and the box score). Both pass strip-sack rules now additionally requiretype.textnot be an interception label (int_vec), so these plays keep their interception classification (normalized to"Interception Return"later in the method). Genuine strip-sacks — and the rush strip-sack rule, which cannot match an interception — are unchanged. Verified across a 20-game / 3,439-play before/after diff: exactly one play changed (Fumble Recovery (Opponent)→Interception Return), zero other plays affected. -
Post-attribution play-type refinement (
__refine_play_types_post_attribution). A new pipeline step (after__add_attribution_cols) corrects two labels that need the turnover signal the step-5 reclassifier lacks (it can only seechange_of_poss, which isTrueon every possession flip, not just turnovers):- A sack-fumble the offense recovers itself was relabeled
Fumble Recovery (Opponent)(spuriouschange_of_poss);is_turnover == FalserestoresFumble Recovery (Own). - A punt-return fumble the punting team recovers (
recovery_team == pos_team) becomesPunt Team Fumble Recoveryinstead of stayingPunt Return.
Only the package's own first-pass relabels are undone (guarded on
orig_play_type); the two frozentype.text-derived columns EPA/WPA read (downs_turnover,pos_score_diff_end) are recomputed so EPA stays consistent (e.g. a 4th-down self-recovery short of the sticks is now correctly scored a turnover on downs). ESPN-sourced box turnover totals are unaffected. Verified across a 20-game / 3,439-play before/after diff: exactly two plays changed (both intended relabels), with EPA moving only on those two plays — no collateral drift. - A sack-fumble the offense recovers itself was relabeled
CFB — blocked-kick turnover flags + ESPN native-flag tripwires
- New
is_blocked_punt_turnover/is_blocked_fg_turnoverper-play flags (additive).is_turnovermodels only giveaways (INT + fumbles lost) to match ESPN's official-boxturnoversdefinition (so the*_pbpcross-check stays exact). A blocked kick the defense recovers is a possession loss but not a giveaway — ESPN's official box does not count it (verified) — so each is surfaced as a standalone flag kept out ofis_turnover/is_st_turnover:Trueon aBlocked Punt/Blocked Field GoalTouchdown, or the non-TD variant with a possession change. These are the possession-losing classes ESPN's per-playisTurnoverflag catches that the giveaway-based derivation does not. - Blocked-FG mislabel fix. ESPN sometimes types a blocked field goal returned by the defense
as
Extra Point Missed, routing it through PAT-scoring EPA logic.__add_new_play_typesnow relabels these toBlocked Field Goal[ Touchdown](gated on"blocked"+ an FG/field goaltext token, so a genuine blocked PAT is untouched), which also corrects the EPA. Because the relabel runs before thetype.text-derived flag computation, all downstream flags recompute cleanly (no staleness). - ESPN native
isTurnover/isPenaltyare kept as cross-checks, not sources of truth. They pass through the flattener as columns (populated back to 2018).isTurnoveris coarser (it silently drops ~16% of plain interceptions on sparse-text plays and has no per-side/ST concept);isPenaltyflags only primary-penalty plays. New regression tripwires (test_espn_flag_tripwires.py) assertisTurnover ⇒ is_turnover OR is_blocked_punt_turnover OR is_blocked_fg_turnoverandisPenalty ⇒ penalty_flagon the fixtures — the first would have caught the interception-erasure bug above. Validated across 150 games (24,876 plays): all blocked-punt and blocked-FG possession losses captured with 100% ESPN agreement and zero leakage into the giveaway signals; penalty tripwire 0 violations;isTurnover/is_turnoveragreed 99.6% (residual disagreements are ESPN false positives — self-recovered fumbles — the stricter derivation correctly excludes).
CFB — pre-2014 era support (CFBPlayProcess)
Validated across a 240-game sweep (15 games × 2004-2019): every game that has play-by-play
produces valid EPA/WPA and a full advanced box score in every era (209/209 of the sampled
games-with-plays; games without PBP exit early gracefully). Legacy ESPN labels that only appear in
older seasons are now normalized in __add_new_play_types (each rule is gated on the raw label, so
it is a no-op on modern data):
2pt Conversion— ESPN's pre-2014 successful two-point label — is resolved viascoringPlaytoTwo-Point Conversion Good/Two-Point Conversion Missed, so it routes through the two-point EPA/scoring path instead of being scored as a generic play.- 2004
Unknownrows are relabeled from their text: period/game markers →End Period(so these non-plays are excluded from aggregates instead of producing garbage EPA), and the handful of misclassified kicks →Field Goal Missed/Extra Point Missed/… Good. Kickoff Return (Defense)(pre-2014 onside-kick-recovered) →Kickoff.- Separate extra-point rows are normalized to the no-down sentinel (
down/distance = -1) for the few pre-2005 games that ship a real down on them (2005+ and two-point rows already use it).
Era notes (documented in the architecture reference): pre-2014 player attribution is
text-extraction only (the participants endpoint returns nothing before 2014; __join_participants
already falls back to regex names); ESPN's own win-probability array is empty before ~2016 but
wpa is computed in-house in every era; and PBP coverage is sparse before 2008 (~47% of 2004
games have no PBP), handled by the existing early-exit.
Removed — NCAA bracketology
espn_mbb_bracketology()/espn_wbb_bracketology()removed. The non-leaguesports.core.api.espn.com/v2/tournament/{22,23}/seasons/{y}/bracketologywrappers added in 0.0.51 — along with the_common_ncaa.pymodule and theregister_ncaa_bracketology()registration machinery — have been removed. The endpoint is ephemeral (ESPN only publishes it during the Jan–Mar projection window) and sat outside the per-league URL pattern, so it is no longer carried. The universalespn_mbb_*/espn_wbb_*wrappers are unaffected.
0.0.52 Release: June 3, 2026
CFB — offline reprocess support (CFBPlayProcess)
Three additive, non-breaking changes that let college-football games be rebuilt from
on-disk raw JSON without re-hitting ESPN, in support of the cfbfastR-cfb-raw scraper's
reprocess pipeline:
- Raw summary allowlist now keeps
injuriesandgameNotes.- Before:
CFBPlayProcess(gameId=..., raw=True).espn_cfb_pbp()filtered the ESPN summary to 15 keys and droppedinjuries/gameNoteseven when ESPN returned them. - After: both keys are retained (defaulting to
[]when ESPN omits them). All previously returned keys are unchanged — this is purely additive.
- Before:
- New
CFBPlayProcess.odds_sourceattribute.- Before: there was no way to tell where the resolved spread/total came from.
- After:
proc.odds_sourceis set to one of"summary_pickcenter","core_odds_api","default", or"injected"during odds resolution.
- New
CFBPlayProcess(odds_override=...)constructor argument.- Before: odds resolution always consulted the summary
pickcenterand, for 2024+ games with an emptypickcenter, cascaded to the livesports.core.api.espn.comodds endpoint — falling back to hardcoded defaults(2.5, 55.5, True, False)on failure. An offline rebuild could therefore silently hit the network or inherit wrong spread inputs that corrupt every play's EPA/WPA. - After: passing
odds_override={"gameSpread": ..., "overUnder": ..., "homeFavorite": ..., "gameSpreadAvailable": ...}short-circuits resolution to use exactly those values, setsodds_source="injected", and never touches the network or the defaults. With no override supplied (the default), behavior is unchanged. The override is validated and type-coerced at the constructor (a missing key or non-dict raisesValueErrorinstead of a laterKeyError).
- Before: odds resolution always consulted the summary
odds_sourceis also written into the returned payload (not just the instance attribute), so dict consumers ofrun_processing_pipeline()/run_cleaning_pipeline()retain odds provenance.
0.0.51 Release: May 30, 2026
User-facing quality-of-life additions
Three top-level helpers that significantly reduce friction for new users and notebook-driven exploration.
sportsdataverse.parsed.* — DataFrame-by-default mirror of every
league's wrappers. The standard sportsdataverse.nba.espn_nba_scoreboard()
returns raw Dict; the new sportsdataverse.parsed.nba.espn_nba_scoreboard()
returns a polars DataFrame. Both share the same underlying function
and accept the same return_parsed=False / return_as_pandas=True
overrides, but the default flips per import path. Available for all 8
leagues (parsed.nba, parsed.wnba, parsed.mbb, parsed.wbb,
parsed.cfb, parsed.nfl, parsed.mlb, parsed.nhl). Wrappers
without a registered parser pass through unchanged.
find_team / find_athlete / find_event — name-to-ID
resolvers in sportsdataverse.find (also re-exported at the package
top level). Eliminates the "what's the magic ID for X" friction:
from sportsdataverse import find_team, find_event
find_team("lakers", league="nba")["id"] # '13'
find_event(date="2024-06-17", league="nba", home="Boston") # NBA Finals G5
All three support multi=True for every match, case-insensitive
substring matching against the relevant fields, and an in-process
team-list cache (clearable via clear_team_cache(league=None)).
list_functions / function_count — searchable function index
in sportsdataverse.discover (also re-exported at the package top
level). Replaces dir() + grep:
from sportsdataverse import list_functions, function_count
function_count()
# {'cfb': 149, 'mbb': 146, 'mlb': 196, 'nba': 143, 'nfl': 208,
# 'nhl': 199, 'wbb': 151, 'wnba': 148} — 1,340 callables total
list_functions(search="pbp") # cross-league PBP wrapper inventory
list_functions(league="mlb", parsers_only=True) # just the parsers
list_functions(league="nhl", wrappers_only=True) # everything except parse_*
24 new offline tests in tests/test_qol.py cover all three QoL
additions including the backwards-compatibility invariant (importing
parsed.* must NOT mutate the raw module's default).
New doc page docs/quality-of-life.md with a side-by-side comparison
showing the four-line "before 0.0.51" equivalent vs the two-line
"after" recipe (find_event → parsed.espn_nba_summary). Intro page
Quickstart updated to show the parsed.* import path first.
Tiered TTL response cache — new sportsdataverse.cache module
adds a six-tier HTTP cache layer that sportsdataverse.dl_utils.download
consults before hitting the network. Three modes (off (default),
memory, filesystem) and six TTL tiers picked by URL inspection:
import sportsdataverse as sdv
sdv.set_cache_mode("filesystem") # persists to ~/.cache/sportsdataverse/
# IMMUTABLE (30d): completed-game PBP/boxscore, glossaries, NHL Records
# REFERENCE (7d): venues, franchises, divisions, seasons, draft picks
# SLOW (24h): team rosters, athlete /landing
# MODERATE (1h): default — leaders, season-to-date stats
# FAST (5m): news, injuries
# LIVE (0): /scoreboard/now, /standings/now — never cached
Scoreboard URLs with dates=YYYYMMDD get special handling: past dates
become IMMUTABLE (game results don't change), future dates stay LIVE.
Per-call cache_ttl= kwarg on download() overrides the tier picker,
and $SDV_PY_CACHE_DIR overrides the on-disk location. Invalidation:
sdv.clear_cache(), sdv.clear_cache(pattern="*roster*"),
sdv.clear_cache(url="https://..."). 19 offline tests in
tests/test_cache.py.
404 error messages with actionable next-action hints —
NoESPNDataError messages now include a tailored Suggestion: line
inferred from the URL. A 404 on /teams/9999/roster suggests
find_team(name, league='nfl'); an athlete 404 suggests
find_athlete(name, league='mlb', team=<team>); a summary 404 suggests
find_event(date, league='nba', home=..., away=...). League is
extracted from both ESPN URL shapes (the flat site.api/.../sports/<sport>/<league>/
form and the nested sports.core.api/v2/sports/<sport>/leagues/<league>/
form). 14 offline tests in tests/test_errors_suggest.py.
sdv console script — argparse-based CLI installed via
[project.scripts] in pyproject.toml. Six subcommands wrap the
top-level QoL helpers so users can poke at the package without
spinning up a Python REPL:
sdv find-team lakers --league nba
sdv find-event 2024-06-17 --league nba --home Boston
sdv list-functions --league mlb --search statcast
sdv function-count
sdv cache mode --set filesystem
sdv cache stats
sdv cache clear --pattern "*roster*"
A --json flag on any command emits raw JSON for piping to jq; the
default is a human-readable format. Exit codes: 0=success, 1=no match,
2=runtime error. 19 offline tests in tests/test_cli.py.
A second big release on top of 0.0.50. The headline items:
- New
sportsdataverse.mlbmodule (greenfield) — 175 functions spanning three data surfaces:- 113 ESPN cross-league wrappers + 5 ESPN originals
- 40 official MLB Stats API wrappers (
statsapi.mlb.com) - 17 Baseball Savant / Statcast wrappers including auto-chunked
25,000-row truncation handling on
/statcast_search/csv
- NHL migrated to
api-web.nhle.com/v1/— the deprecatedstatsapi.web.nhl.comhost is gone; replaced with 26 modernnhl_web_*wrappers grounded in the OpenAPI spec atfastRhockey/data-raw/nhl_api_web_openapi.yaml. - Cross-league ESPN port from hoopR / wehoop / cfbfastR — 804 new
wrappers across 8 leagues (NBA, MBB, WNBA, WBB, CFB, NFL, MLB, NHL)
via a single ~80-function core (
_common_espn.py) parameterized on the(sport, league)slug. Each per-league extension module is a 5-line file callingmake_league_module()to mass-register the wrappers with proper__name__/__qualname__/__doc__for IDE discoverability. - 3 new NHL modules for the historical / Statcast surfaces:
nhl_edge— 35 wrappers for the NHL EDGE player-tracking system (api-web.nhle.com/v1/edge/*)nhl_stats_rest— 21 wrappers for the official stats REST API (api.nhle.com/stats/rest/) with verbatim Cayenne filter expression supportnhl_records— 50 wrappers for the records site (records.nhl.com/site/api/) covering awards, coaches, franchises, HOF, draft, all-star, GMs
- NCAA bracketology —
espn_mbb_bracketology()andespn_wbb_bracketology()for the non-leaguesports.core.api.espn.com/v2/tournament/{22,23}/seasons/{y}/bracketologyendpoint (live during the projection window, Jan-Mar). - 20 polars/pandas parsers in
_common_espn_parsers.pycovering the most-used ESPN payload shapes (scoreboard, teams, standings, groups, athlete overview/stats/gamelog/splits, leaders, coaches, draft, event-competitor surface, team schedule/roster, news, injuries, generic Core v2 paginated lists). - 4 NHL EDGE family parsers + 3 sub-frame parsers in
nhl_edge_parsers.py, schema-grounded against live captures from 2026-05-23. return_parsed=Truedispatch shim — every wrapper whose short name has a registered parser (57 keys currently inENDPOINT_PARSERS) gains an optionalreturn_parsed=Truekwarg that routes the raw response through the parser and returns a polars DataFrame (pandas viareturn_as_pandas=True). The raw-Dict path is unchanged — the shim is backwards-compatible and strictly additive.- 80 offline parser tests (NHL EDGE 32 + universal ESPN 16 + the
cross-league shim suite) + 32 live-gated integration tests under
SDV_PY_LIVE_TESTS=1so default test runs never hit live endpoints.
New: MLB module (greenfield)
- New top-level
sportsdataverse.mlbpackage with 8 submodules. mlb_api.py(40 functions) wraps the official MLB Stats API. IDs to know:sportId=1is MLB,leagueId103=AL /104=NL,gameTypeslugsR/F/D/L/W/S/A/E/PO. Player IDs (personId/batter/pitcher) are the same MLBAM id space shared with Baseball Savant.mlb_statcast.py(17 functions) wraps Baseball Savant. The unofficial CSV search at/statcast_search/csvtruncates at exactly 25,000 rows with no pagination;statcast_searchraisesRuntimeErrorwhen the response hits that cap (default,raise_on_truncation=True). Usestatcast_search_chunkedfor multi-week ranges — it auto-chunks the date range and stitches client-side.mlb_espn_ext.pyregisters 113 cross-league ESPN wrappers viamake_league_module(..., include_mlb=True), which adds the MLB-onlyespn_mlb_athlete_hotzonesto the universal surface.
New: NHL — api-web.nhle.com migration + EDGE / Stats REST / Records
-
The deprecated
statsapi.web.nhl.comis gone.nhl_api.pykeeps a small set of backward-compatible aliases that warn and delegate tonhl_api_web. -
nhl_api_web.py(26 functions) covers the modern game-feed API athttps://api-web.nhle.com/v1/. -
nhl_edge.py(35 functions) wraps the NHL EDGE player-tracking surface — skater / goalie / team detail, shot-location, shot-speed, skating distance, zone time, plus 12*_top_10leaderboards.Note: all 12
*_top_10URL paths return 404 as of 2026-05-23 — the OpenAPI spec lists them but they're not live. The wrappers andparse_edge_top10are kept for forward-compatibility. -
nhl_stats_rest.py(21 functions) wraps the official Stats REST API atapi.nhle.com/stats/rest/. Verbatim Cayenne filter expression support viacayenneExp/factCayenneExpkwargs. -
nhl_records.py(50 functions) wraps the records site atrecords.nhl.com/site/api/— awards, coaches, franchises, skaters, goalies, draft, all-star, HOF, GMs, attendance, fastest goals, team records.
New: ESPN cross-league port
_common_espn.pyexposes ~80 core functions parameterized on(sport, league).make_league_module(sport, league, prefix, globals(), include_ncaa=, include_football=, include_mlb=)mass-registers wrappers in the caller's namespace. Each per-league extension file is a 5-line wrapper.- Wrappers use
functools.partialwith explicit__name__/__qualname__/__doc__so they behave like real functions forhelp(), IDE auto-complete, andinspect.signature(). - The
_NCAA_WRAPPERStable addsrankings,season_recruits,season_week_rankingsformbb,wbb,cfb. - The
_FOOTBALL_WRAPPERStable addsseason_qbr,season_qbr_weekfornfl,cfb. - The new
_MLB_WRAPPERStable addsathlete_hotzonesformlb.
New: NCAA bracketology
espn_mbb_bracketology(season, iteration=None)/espn_wbb_bracketology(...)atsports.core.api.espn.com/v2/tournament/{22,23}/seasons/{y}/bracketology.- The endpoint is seasonal — live during the projection window
(roughly January through March each year) and 404s the rest of the
year. Integration tests handle this with
pytest.xfailso off-season CI runs don't fail.
New: _common_espn_parsers.py (polars / pandas parser layer)
- 20 parsers covering the highest-traffic ESPN payload shapes. All parsers are league-agnostic — the same parser handles MLB, NFL, NBA, etc. because ESPN's payload shapes are identical across leagues.
- Every parser returns polars by default;
return_as_pandas=Trueyields pandas. Empty / malformed payloads return zero-row frames rather than raising. - Output columns snake-cased via
sportsdataverse.dl_utils.underscore. ENDPOINT_PARSERSregistry has 57 short-name keys mapped to 20 unique parsers; covers the universal table plus NCAA / football / MLB extras.parser_for(short_name)lookup helper.
New: return_parsed=True dispatch shim
_bind()in_common_espn.pywas extended with an optionalparser=argument. When present, the bound wrapper is a closure that addsreturn_parsed=Falseandreturn_as_pandas=Falsekwargs; whenreturn_parsed=True, the closure dispatches the raw response through the parser and returns a DataFrame.make_league_module()looks up the parser viaparser_for(short)on each wrapper registration. The lookup is lazy-imported so a missing parsers module doesn't break the package.- API contract: every existing caller continues to get raw
Dict— the shim is opt-in via the new kwargs.
New: nhl_edge_parsers.py
- 4 family parsers (
parse_edge_top10,parse_edge_detail,parse_edge_shot_location,parse_edge_zone_time) + generic fallback (parse_edge_payload). - 3 sub-frame parsers (
parse_edge_sog_details,parse_edge_sog_summary,parse_edge_hardest_shots) for unrolling the rich nested lists inside detail payloads thatparse_edge_detaildeliberately stringifies. EDGE_ENDPOINT_PARSERSregisters 33 of the 35 EDGE wrappers (the remaining 2 fall through to the generic parser viaparser_for_edge).EDGE_SUBFRAME_PARSERSmaps each detail wrapper to the tuple of sub-frame parsers that apply.
New: Site v2 summary dispatcher (20 sub-parsers)
The Site v2 summary endpoint
(espn_{league}_summary(event_id=...)) ships ~19-22 top-level sections
per game (~700 KB to 1.8 MB per call). Rather than collapse that into
one parser, the summary surface now has 20 targeted sub-parsers plus a
dispatcher:
parse_summary_boxscore_player— one row per (team × athlete) with the parallelkeys/statsarrays zipped (e.g. NBA produces 27 rows withmin,fg,3pt,ft,reb,ast, columns).parse_summary_boxscore_team— one row per (team × stat) withstat_name,stat_label,stat_display_value.parse_summary_plays— one row per play (~450 rows per NBA game).parse_summary_winprobability— one row per win-prob tick (joinable to plays viaplay_id).parse_summary_leaders— one row per (team × category × leader) from the 3-levelleaders[]nesting.parse_summary_game_info,parse_summary_officials,parse_summary_header,parse_summary_season_series,parse_summary_against_the_spread,parse_summary_standings,parse_summary_broadcasts,parse_summary_format,parse_summary_pickcenter,parse_summary_odds,parse_summary_article,parse_summary_injuries,parse_summary_news— one row per (or one row total for) the corresponding summary section.parse_summary_drives,parse_summary_scoring_plays— NFL / CFB specific (NFL summary shipsdrives.previous[]+scoringPlaysinstead of top-levelplays). Return zero-row frames for non-football leagues.parse_summary(payload, section=None)— dispatcher. Withsection=Nonereturns a dict of all 20 sub-frames keyed by section name; withsection="<name>"returns just that frame. Empty payload returns a dict of 20 zero-row frames.SUMMARY_SECTION_PARSERS— public registry mapping section name to parser.
Cross-league parity tests verify the dispatcher works against captured fixtures for NBA / MLB / NFL / NHL / WNBA — same code path handles every league's summary endpoint.
New: 100% ENDPOINT_PARSERS coverage (121/121)
Every wrapper short name across all 4 wrapper tables
(_UNIVERSAL_WRAPPERS, _NCAA_WRAPPERS, _FOOTBALL_WRAPPERS,
_MLB_WRAPPERS) is now registered in ENDPOINT_PARSERS. Every
factory-bound wrapper plus the hand-bound NCAA bracketology helpers
accepts return_parsed=True and return_as_pandas=True.
Two new generic fall-through parsers cover the long tail:
parse_single_entity— flattens any single-resource Core v2 payload (team, venue, franchise, coach, award, position, season_info, athlete_core, event_competitor, etc.) to a one-row frame.parse_itemswas already generic for{items: [...]}Core v2 lists and Core v2{entries: [...]}(athlete_statisticslog); this release expands its registration to ~30 more list-shape endpoints (calendar variants, event lists, season_powerindex, talentpicks, etc.).
register_ncaa_bracketology was upgraded to wrap the bracketology
helpers in the same return_parsed=True shim used by make_league_module
— previously they were hand-bound without the shim.
Three regression tests lock in the invariant:
test_every_wrapper_short_name_has_a_registered_parsertest_no_stale_entries_in_endpoint_parsers_registrytest_return_parsed_shim_active_on_every_wrapper_across_all_leagues(walks the__all__of every league extension module and verifies 819+ wrappers carry the shim).
New: weekly cron live-test drift detector
.github/workflows/live-tests-cron.yml runs the full live test suite
(tests/test_espn_live.py and any other SDV_PY_LIVE_TESTS=1 gated
tests) every Monday 13:00 UTC and on workflow_dispatch. On failure,
the workflow uses actions/github-script to find or create a tracking
issue labeled live-tests:drift:
- First failure opens a new issue with the last 4 KB of pytest output plus a run URL.
- Subsequent failures comment on the existing open issue instead of duplicating.
- Closing the issue resets state.
Catches upstream API drift (ESPN schema changes, NHL EDGE 404s, MLB Stats API URL moves) on a regular cadence even when the repo is otherwise quiet between releases.
New: MLB Stats API parser layer
sportsdataverse.mlb.mlb_api_parsers turns the 40 raw-Dict
mlb_api_* wrappers into tidy polars / pandas DataFrames. Mirrors
the design of _common_espn_parsers:
- Every parser returns polars by default; pandas via
return_as_pandas=True. - Empty / malformed payloads return zero-row frames.
- Output columns snake-cased via
sportsdataverse.dl_utils.underscore. - Most parsers use
pandas.json_normalizefor one-pass flattening.
Five dedicated parsers handle the high-traffic endpoints with their own unrolling logic:
parse_mlb_api_schedule— walksdates[].games[]and prefixes the schedule date onto each game row (one row per game withteams.home.*/teams.away.*/venue.*/status.*flattened).parse_mlb_api_teams— one row per team fromteams[].parse_mlb_api_team_roster— one row per player fromroster[]withperson,position,statussub-dicts flattened.parse_mlb_api_standings— walksrecords[].teamRecords[], prefixes division identifiers (namespacedstandings_*to avoid column collisions with team-record fields likelastUpdated), and produces one row per (division × team).parse_mlb_api_person_stats— walksstats[].splits[](also handlesmlb_api_team_statswith the same shape), prefixesstats_type/stats_groupfrom the parent block, and flattens the innerstatblock to wide stat columns.
A generic parse_mlb_api_list fallback handles every list-shape
endpoint that doesn't need extra unrolling (venues, sports, leagues,
divisions, seasons, awards, umpires, draft, draft_prospects,
attendance, team_leaders, team_alumni, team_affiliates, stats,
stats_leaders, stats_streaks, people, sport_players).
MLB_API_ENDPOINT_PARSERS registry has 26 entries (7 dedicated + 19
generic). parser_for_mlb_api(fn_name) returns the registered
parser; unknown names fall back to parse_mlb_api_list so the
caller always gets a DataFrame-returning callable.
Test fixtures captured 2026-05-24 from statsapi.mlb.com (8 captures
in tests/fixtures/mlb_api/). 17 offline tests in
tests/test_mlb_api_parsers.py exercise each dedicated parser plus
the generic fallback against the live fixtures.
New: NHL Stats REST + Records parser layers
sportsdataverse.nhl.nhl_stats_rest_parsers.parse_nhl_stats_rest and
sportsdataverse.nhl.nhl_records_parsers.parse_nhl_records turn every
wrapper in their respective surfaces into a tidy polars / pandas
DataFrame.
Both APIs ship the identical {data: [...], total: N} envelope on
every endpoint, so a single parser handles every wrapper:
parse_nhl_stats_restcovers the 21 wrappers insportsdataverse.nhl.nhl_stats_rest(api.nhle.com/stats/rest/en/*).parse_nhl_recordscovers the 50 wrappers insportsdataverse.nhl.nhl_records(records.nhl.com/site/api/*).
The meta Stats REST endpoints (config, componentSeason, ping) ship
non-data-keyed payloads — both parsers return zero-row frames for
those instead of raising.
Registries: NHL_STATS_REST_ENDPOINT_PARSERS has 17 entries (excluding
the meta endpoints). parser_for_nhl_stats_rest and
parser_for_nhl_records always return a callable (fall back to the
generic parser — never return None).
New: NHL api-web parser layer
sportsdataverse.nhl.nhl_api_web_parsers covers the modern game-feed
API at api-web.nhle.com/v1/ — 16 dedicated parsers + 2 dispatchers
covering all 26 nhl_web_* wrappers across game-center, schedule,
score, scoreboard, standings, team, player, leaders, and draft
families.
Game-center parsers:
parse_nhl_web_pbp— one row per play (~330 plays per game) witheventId,typeCode,typeDescKey,periodDescriptor,detailsflattened.parse_nhl_web_boxscore— unrolls the 6-bucketplayerByGameStats: {away,home}Team.{forwards,defense,goalies}structure into one long-form frame, tagging each row withhome_awayandposition_group.parse_nhl_web_landing— single-row game profile with venue, teams, periodDescriptor, gameState, summary stringified.parse_nhl_web_right_rail— dispatcher returning 6 sub-frames:season_series,shots_by_period,team_game_stats,game_info,linescore_by_period,season_series_wins. Withsection="..."returns just one frame.
Schedule / score parsers:
parse_nhl_web_schedule— walksgameWeek[].games[], prefixes the day's date onto each game row.parse_nhl_web_score— flattensgames[]for a single date.parse_nhl_web_scoreboard— walksgamesByDate[].games[], prefixesscoreboard_date(multi-day scoreboard).parse_nhl_web_club_schedule— flattensgames[]withclub_timezone/club_current_season/club_previous_season/club_next_seasoncontext columns from the parent payload.
Standings + team / player parsers:
parse_nhl_web_standings— one row per team (84 stat columns covering full win/loss/OT/SO/ROW/L10/streak/home/away breakdowns).parse_nhl_web_standings_season— one row per season (108 NHL seasons since 1917-18).parse_nhl_web_club_stats— dispatcher returning{skaters, goalies}as separate frames.parse_nhl_web_roster— mergesforwards,defensemen,goaliesinto one long-form frame with aposition_groupcolumn.parse_nhl_web_player_landing— single-row player profile (~130 columns for a player like McDavid with full career totals, features, recent games).parse_nhl_web_player_game_log— one row per game fromgameLog[].
Leaders + draft:
parse_nhl_web_leaders— walks the category-keyed leaders payload ({points: [...], goals: [...]}for skaters;{wins: [...], savePctg: [...]}for goalies), tags each row with the category it came from, concatenates.parse_nhl_web_draft_picks— one row per pick.
Registry: NHL_API_WEB_ENDPOINT_PARSERS has 24 entries covering all
the data endpoints. parser_for_nhl_api_web(fn_name) returns the
registered parser or None for the 2 idiosyncratic endpoints
(playoff_series, player_spotlight, draft_rankings,
draft_rankings_now) whose payloads are too idiosyncratic for a
useful generic fallback — callers null-check.
Test fixtures captured 2026-05-24 (17 captures from
api-web.nhle.com/v1/). 37 offline tests in
tests/test_nhl_api_web_parsers.py verify each parser against the
captured fixtures plus dispatcher contracts, empty payload contract,
pandas opt-in, and registry consistency.
Test fixtures captured 2026-05-24 (8 from api.nhle.com/stats/rest/,
6 from records.nhl.com/site/api/). 21 offline tests in
tests/test_nhl_aux_parsers.py verify parsing across:
- 7 Stats REST data endpoints (season, franchise, country, glossary, skater_summary, goalie_summary, team_summary).
- 6 Records endpoints (franchise, franchise_team_totals, coach, draft, player, attendance).
- Empty-payload contract, pandas opt-in, registry consistency, and the config-as-meta zero-row case.
Bug fixes
parse_team_rosternow handles both ESPN roster shapes. The flat shape (athletes[]= list of athlete dicts; used by NBA / WNBA / MBB / WBB) continues to work unchanged. The newly- handled position-grouped shape (athletes[i] = {position, items}; used by MLB / NFL / NHL / CFB) is auto-detected by inspecting the first element — each player from a group'sitems[]is tagged with aposition_groupcolumn carried over from the parent group. Without the fix, MLB / NFL / NHL / CFB rosters were collapsing to ~5-6 group rows instead of unrolling to the full per-player list (e.g. Alabama CFB went from 6 group rows to 100 player rows).
New: NFL drive-plays parser (true PBP parity)
parse_summary_drive_plays rounds out the football PBP story. NFL
and CFB summary payloads don't ship a top-level plays[] array (the
NBA / MLB / NHL / WNBA convention); they nest plays inside each
drive at drives.previous[i].plays[]. The existing
parse_summary_drives returns one row per drive with the plays
stringified. This new parser unrolls those nested plays into a true
one-row-per-play frame with drive_id + drive_sequence columns
carried over from the parent drive — letting callers join back to
the drives frame for drive-level context.
Verified against Super Bowl LIX: 26 drives + 186 plays unrolled
into a 186-row × 43-column polars frame. Returns zero rows for
NBA / MLB / NHL / WNBA fixtures (those leagues use top-level
plays[], exercised by parse_summary_plays).
SUMMARY_SECTION_PARSERS registry grows from 20 to 21 entries.
The summary dispatcher's output dict now includes the drive_plays
section alongside drives and scoring_plays.
Test infrastructure
- New
tests/test_espn_universal_parsers.py(128 tests, +22 since last roll-up: 8 sparse-section tests coveringbroadcasts(present for MLB / NHL, empty for NBA / NFL / WNBA in past-game captures) and the universally-sparseagainst_the_spread/pickcenter/odds; 3 MBB/WBB/CFB NCAA summary fixture additions to the cross-league parametrized tests, expanding the dispatcher + boxscore_player + plays + drives + officials assertions from 5 leagues to all 8 ESPN leagues),tests/test_mlb_api_parsers.py(17 tests),tests/test_nhl_aux_parsers.py(21 tests),tests/test_nhl_api_web_parsers.py(37 tests), andtests/test_nhl_edge_parsers.py(32 tests) run offline against captured fixtures. - New
tests/test_espn_live.py(56 live tests, +24 since last roll-up: 9 NCAA-side wrapper tests (CFB/MBB/WBB × team_roster/ news/team_schedule), 3 NCAA summary dispatcher tests, 3 MLB Statcast pitch-search tests (small-range happy path / multi-week chunked stitch / raise-on-truncation guard), plus 9 parametrizedreturn_parsed=Trueshim-parity tests confirming the raw-Dict / polars / pandas round-trip is internally consistent for the NCAA surface — same wrapper invocation with vs without the kwarg must produce equivalent data, andreturn_as_pandas=Truerow count must match the polars row count). Gated bySDV_PY_LIVE_TESTS=1for live integration verification. - Captured fixtures live under
tests/fixtures/espn/(43 captures — the original 7 plus summary captures for all 8 ESPN leagues (NBA / MLB / NFL / NHL / WNBA + the new NCAA captures: MBB final Purdue@UConn, WBB final Iowa@SC, CFB national championship OSU@ND) plus the 28-fixture cross-league parity set coveringteam_roster/team_schedule/news/injuriesfor each league),tests/fixtures/mlb_api/(8 captures: schedule, teams, roster, standings, person_stats, venues, sports, divisions),tests/fixtures/nhl_stats_rest/(8 captures: season, franchise, country, glossary, config, skater_summary, goalie_summary, team_summary),tests/fixtures/nhl_records/(6 captures: franchise, franchise_team_totals, coach, draft, player, attendance),tests/fixtures/nhl_api_web/(17 captures: pbp, boxscore, landing, right_rail, schedule, score, scoreboard, standings, standings_season, club_schedule, club_stats, roster, player_landing, player_gamelog, skater_leaders, goalie_leaders, draft_picks), andtests/fixtures/nhl_edge/(7 captures), each with a README documenting provenance. - Parametrized cross-league parity tests in
test_espn_universal_parsers.pyexercise the summary dispatcher against all 5 captured leagues and assert the full 20-section dispatch contract for each (boxscore_player + boxscore_team + plays + winprobability + leaders + 13 metadata sections + 2 football-only).
Documentation
- README.md and docs/docs/intro.md both gain two new sections:
- "Supported leagues and data sources" — a per-league table showing every module + the data surfaces it covers + wrapper counts (NBA=118, WNBA=124, MBB=121, WBB=126, CFB=123, NFL=119, MLB=175, NHL=132, total ~1,030).
- "Polars / pandas parser layer" — quick overview of the
return_parsed=Trueshim for ESPN wrappers + the compose-wrapper-with-parser pattern for the NHL / MLB sibling APIs. Links to the architecture + parsers docs pages.
- New documentation pages:
docs/architecture/espn-cross-league.md— the factory + shim architecture.docs/parsers/index.md— the parser layer +ENDPOINT_PARSERS.docs/mlb/index.md— MLB module overview (ESPN + Stats API + Statcast); brief pointers to the new dedicatedparsersandstatcastpages.docs/mlb/parsers.md— dedicated MLB Stats API parsers page (split out fromindex.md) with the full parser table, registry +parser_for_mlb_api, four chaining examples, and a fixture inventory.docs/mlb/statcast.md— dedicated Baseball Savant / Statcast page (split out fromindex.md) covering the 17statcast_*wrappers, the 25,000-row truncation handling + thestatcast_search_chunkedauto-chunked variant, Statcast coverage windows by metric, MLBAM ID-space chaining with the Stats API, and two end-to-end examples (catcher pop times + World Series pitch-by-pitch). Both new pages are wired into the MLB category indocs/sidebars.ts.docs/parsers/fixtures.md— comprehensive index of all 89 captured live payloads across the 6 fixture directories (espn/,mlb_api/,nhl_api_web/,nhl_edge/,nhl_stats_rest/,nhl_records/). Includes the full endpoint mapping table per directory, the championship-game event IDs used for the cross-league summary captures, and a maintenance section explaining how to refresh a fixture.docs/architecture/building-blocks.md— meta-documentation page enumerating the five low-level patterns reused across every parser module:_bindshim factory,make_league_modulefactory call,_row_per_item/_single_rowjson_normalize helpers, theENDPOINT_PARSERSregistry +parser_for_*lookup, and the dispatcher pattern (used byparse_summary,parse_nhl_web_right_rail,parse_nhl_web_club_stats). Closes with a step-by-step "Adding a new parser" checklist. Sidebar entry added under the Architecture category.
docs/sidebars.tsregrouped by sport family — leagues now cluster by basketball (NBA / WNBA / MBB / WBB) / football (NFL / CFB) / baseball (MLB) / hockey (NHL) instead of alphabetical, surfacing the cross-league helper relationships (e.g. NCAA basketball pair with NBA via the same ESPN factory). Architecture + Parsers categories now default to expanded (collapsed: false) so newcomers see the package-wide overview first.nhl/nhl_loaders.pylint cleanup: 4 sites ofif type(seasons) is int:replaced withisinstance(seasons, int)to clear pre-existingE721ruff warnings (no behaviour change — both forms accept the same input).tests/conftest.pyis now the single source of truth for theSDV_PY_LIVE_TESTS=1gating mechanism.tests/test_espn_live.pywas previously redefiningLIVE+ its ownpytestmark.skipifmarker; it now imports the sharedskip_if_no_livefromconftestand assigns it directly topytestmark. Behaviour is identical (no env var → 56 tests skip; env var set → 56 tests run) but the duplication is gone and the conftest docstring now documents both the per-test decorator and module-level marker patterns for futuretest_*_live.pyfiles.tests/conftest.pyalso gains a sharedload_fixture(category, stem)helper that all 5 parser test modules now use instead of each carrying their own copy of the samejson.loads((FIXTURE_DIR / f"{stem}.json").read_text(...))boilerplate + per-fileFIXTURE_DIRconstant. The helper raisesFileNotFoundErrorwith the expected path baked into the message when a fixture is missing — easier debugging of typo'd stems. Each test file still keeps its thin local_load(stem)alias bound to its category, so call sites (_load("summary_nba")) remain unchanged.test_nhl_aux_parsers.pykeeps its 2-arg_load(directory, stem)signature for its dual-category load pattern but the underlying helper is now shared.pyproject.tomlkeywordsexpanded from 6 to 21 entries reflecting the 0.0.51 surface — full league set (nba, wnba, nfl, college football, ncaa basketball, mlb, nhl), data sources (espn, mlb stats api, statcast, baseball savant, nhl edge, nhl api-web), and concepts (data, epa, statistics, win probability, play-by-play, web scraping, polars, parser). Improves PyPI search discoverability for users searching by individual league or data source.- New
localpre-commit hooksync-docs-changelog(in.pre-commit-config.yaml): when staging changes toCHANGELOG.md, automatically re-copies the file todocs/src/pages/CHANGELOG.md(the docusaurus-rendered copy) and stages the synced file so both copies land in the same commit. Replaces the manualcp CHANGELOG.md docs/src/pages/CHANGELOG.mdstep that contributors used to remember by hand. - Module docstrings on every parser + wrapper module now carry a
Documentation:block linking to the matching docs page sohelp()/pydocusers land on the right reference without hunting. Updated modules:_common_espn.py,_common_espn_parsers.py,nhl/nhl_api_web.py,nhl/nhl_api_web_parsers.py,nhl/nhl_edge.py,nhl/nhl_edge_parsers.py,nhl/nhl_stats_rest.py,nhl/nhl_stats_rest_parsers.py,nhl/nhl_records.py,nhl/nhl_records_parsers.py,mlb/mlb_api.py,mlb/mlb_api_parsers.py,mlb/mlb_statcast.py. nhl/nhl_pbp.py::espn_nhl_pbpdocstring gains a prominent cross-reference + comparison table distinguishing it from the modernnhl_web_pbp/parse_nhl_web_pbpsurface (different ID spaces, different schemas, not interchangeable). A matching:::caution:::admonition added todocs/nhl/api-web.mdso users coming from either direction find the cross-reference.docs/nhl/api-web.mdgains a "Parser deep-dive" section between the registry and the full example: documents theparse_nhl_web_boxscore6-bucket unrolling pattern, both dispatchers (right_rail6-section +club_stats2-section breakdowns with example invocations), the roster merge-with-tag pattern, and the leaders category-keyed payload unrolling.
docs/docs/intro.mdgains a "Quickstart" section directly under the goal paragraph showing three one-liners across NBA / MLB / NHL covering the three primary usage modes (return_parsed shim, Stats API compose-with-parser, NHL EDGE compose-with-parser).CLAUDE.mdgains two new top-level sections ("ESPN Cross-League Architecture (0.0.51+)" and "Parser Layer (0.0.51+)") that document the factory pattern,make_league_module,_bind+shim, ENDPOINT_PARSERS invariant, summary dispatcher contract, cross-league shape divergences captured by tests, the fixture inventory, and the test-file structure. ~210 lines added to keep future AI assistants and contributors aligned on the parser-layer conventions.docs/nhl/api-web.md— the modern game-feed surface (api-web.nhle.com/v1/) with the full endpoint table and a parser layer section covering all 16 dedicated parsers + 2 dispatchers (right_rail,club_stats).docs/nhl/edge.md,edge-parsers.md,stats-rest.md,records.md— the NHL surface (EDGE, Stats REST, Records). Each now includes cross-links to the other three NHL docs pages and a "Parser layer" section.
0.0.50 Release: May 7, 2026
This release is a big one. The headline items:
- A near-drop-in nflreadpy-parity surface inside
sportsdataverse.nfl: six new loaders, two unified per-type loaders, a caching layer, runtime config, three static datasets, 25load_*aliases, and current-season / current-week helpers. - 11 new ESPN scrape modules across
wbbandwnba(team rosters, season player & team stats, standings, draft, event officials), each with full@overloadtyping. - A new
cfb_play_participantsmodule and a corresponding ~340-line collapse insidecfb_pbp.__add_player_cols. - The long-running
0.36-live→mainpolars-1.x reconciliation across all seven*_pbp.pymodules (~165 API translation sites). - Packaging fully modernized to PEP 621
pyproject.toml(no moresetup.py), conda-installable via the newrecipe/meta.yaml. - Lint chain re-baselined on Ruff (replacing black + isort + pycln + flake8) plus a richer pre-commit set.
- Runnable
Example:sections on ~190 public callables and seven new intro / intermediate Jupyter notebooks underexamples/notebooks/. - Sphinx docs build is clean under
sphinx-build -W.
Round bump to 0.0.50 (rather than 0.0.41) to signal scope; we are still alpha.
Packaging modernization
- Migrated all packaging metadata from
setup.pyto PEP 621[project]inpyproject.toml.setup.pyis removed;python -m buildis the only supported build path. - License switched from classifier (
License :: OSI Approved :: MIT License) to SPDX expression (license = "MIT"+license-files = ["LICENSE"]) for Metadata 2.4 compliance. - Python target widened to 3.9–3.14 (3.6/3.7/3.8 dropped). Dependency lower bounds modernized (
polars>=1.0,<2.0,pyarrow>=14.0,numpy>=1.23,pandas>=2.0, etc.). [tool.setuptools.packages.find]excludestests*,Sphinx-docs*,docs*,examples*,archive*,recipe*,dev*from the wheel.[tool.setuptools.package-data]retains thecfb/models/*+nfl/models/*shipping list.MANIFEST.intrimmed to current-relevance patterns..gitignoreextended to ignoredev/,dist_check/, and the Sphinx_build/+_static/artifacts; trackedSphinx-docs/_build/files were untracked.
Conda installability
- New
recipe/meta.yaml:noarch: pythonconda-build recipe that mirrors[project.dependencies]and consumespyproject.tomldirectly. Two source modes documented — localpath: ..for dev, PyPIurl:+sha256:for conda-forge submission. - New
recipe/README.md: walks through the localconda build recipe/workflow and the conda-forgestaged-recipessubmission flow. - New
.github/workflows/conda-build.yml: verifies the recipe on every PR that touchesrecipe/orpyproject.toml, plus on every release. Usesconda-incubator/setup-miniconda@v3+ miniforge / mamba; builds, installs the resulting.conda, smoke-imports all seven sport subpackages, uploads the built package as a workflow artifact.
Linting & pre-commit modernization
- Replaced the legacy black + isort + pycln + flake8 chain with Ruff (lint, import-sort, pyupgrade, format, unused-import removal).
pyproject.toml [tool.ruff]pinsline-length = 120,fix = true,show-fixes = true. The standaloneisorthook is retained ONLY to injectfrom __future__ import annotationsat the top of every Python file via its--add-importflag — Ruff handles all other import concerns. pyproject.toml [tool.ruff.lint]ignoresE712(intentionalpl.col(...) == True/Falsefor polars boolean masks),E501/E402(long-URL docstrings + module-level imports),F601/F841(legacy parser idioms). Per-file ignores cover star-imports + re-exports in__init__.pyfiles (F401/F403).- New pre-commit hooks alongside Ruff:
pre-commit-hooks(trailing-whitespace, check-merge-conflict, check-ast, check-toml/json/xml/yaml, check-symlinks, end-of-file-fixer, requirements-txt-fixer, check-added-large-files, debug-statements). Thecheck-yamlhook excludesrecipe/meta.yamlbecause its Jinja2 templating isn't valid pre-substitution YAML.pygrep-hooks:python-use-type-annotations,python-no-eval,python-no-log-warn,rst-backticks,rst-directive-colons,rst-inline-touching-normal,text-unicode-replacement-char,python-check-mock-methods,python-check-blanket-noqa,python-check-blanket-type-ignore.add-trailing-comma,sync-pre-commit-deps.check-jsonschema --check-github-workflowsvalidates.github/workflows/*.ymlagainst the GitHub Actions schema.actionlintfor workflow expressions / shell.yamlfmt(config in.yamlfmt:line_ending: lf,eof_newline: true).doctocregenerates Markdown TOCs.markdownlint-cli2against.markdownlint-cli2.yaml. The config disables a handful of rules that fight legacy README / CHANGELOG content (MD013 line-length, MD030 list-marker-space, MD045 alt-text, MD051 link-fragments, MD060 table-column-style) and allows<a>,<img>,<br>,<sub>,<sup>inMD033for the README's badge / logo HTML.
Documentation toolchain
- Added
sphinx.ext.napoleontoSphinx-docs/conf.pywith explicit Google-style settings — the newwbb/wnba/nfl/cfbmodules use Google-style docstrings (Args:/Returns:/Raises:) and these were producing 22 docutils warnings on build before napoleon was wired up. - Added a no-op
visit_abbreviationshim to the markdown translator inSphinx-docs/conf.py. Sphinx 9 emitsabbreviationnodes for the keyword-only*separator in rendered function signatures, andsphinx-markdown-builder0.6.10 has no visitor for that node type. The shim emits the inner text and skips the node, so the build is now warning-free undersphinx-build -W. - Module docstrings in
cfb_play_participants.pyandnfl/utils_date.pyhad bullet lists immediately following aCaveats:/NFL season convention:paragraph header. Added the required blank line + asterisk markers so docutils parses them as proper RST bullet lists. Sphinx-docs/sportsdataverse.{cfb,mbb,nba,nfl,nhl,wbb,wnba}.rstregisterautomoduleentries for every new ESPN scrape module shipped this release.Sphinx-docs/setup.rstdeleted (was an auto-generated apidoc page for the now-removedsetup.py).Sphinx-docs/index.rstfixed a single-backtick\toctree`typo so therst-backticks` pre-commit hook passes.
Runnable docstring examples (~190 functions)
- Every public callable across
cfb,nfl,nba,nhl,mbb,wbb,wnba,dl_utils,decorators,errors,nfl/cache,nfl/config,nfl/datasets,nfl/utils_date, and the top-level package now ships a multi-blockExample:section: a quick-start invocation, one or two useful parameter combinations, a one-line pipeline next-step, and aSee Also:block with cross-links to companion R packages (wehoop,hoopR,cfbfastR,baseballr,fastRhockey),nflverse,nflreadpy,nba_api, andnhl-api-pywhere applicable. - Examples use the napoleon literal-block format (heading +
::+ 4-space indented code) so they render as proper code blocks in the markdown docs without triggeringsphinx.ext.doctest. Users can copy-paste any block and run it as-is. - Existing one-line backtick-wrapped examples (the legacy
Example: <inline call>shape) were replaced (not appended) so each function has exactly oneExample:section.
Example notebooks
- Seven new Jupyter notebooks under
examples/notebooks/:01_quickstart.ipynb,02_cfb_intro.ipynb,03_nfl_intro.ipynb,04_nba_intro.ipynb,05_wbb_wnba_intro.ipynb,06_mbb_intro.ipynb,07_nhl_intro.ipynb. Intro / intermediate level — schedule, pbp, team / player / season-stats endpoints, thenfl.update_config/clear_cache/get_current_*runtime surface, and a small pipeline example per sport. Outputs cleared so the user runs them locally; cross-references link to companion R packages and alternative Python libraries. .gitignorekeeps*.ipynbignored at the repo level (so scratch + checkpoint notebooks aren't accidentally tracked) but adds a negative pattern!examples/notebooks/*.ipynbso the curated tutorial notebooks are explicitly tracked.
Contributor docs and templates
- New
CLAUDE.mdand.github/copilot-instructions.mdcapture the project conventions for AI-assisted development: branching, conventional commit messages, polars 1.x rules, HTTP layer, module patterns, NFL nflreadpy-parity surface, CFBcfb_play_participants, test conventions, packaging, Sphinx toolchain, the docstring conventions for new functions, common pitfalls. - New
CONTRIBUTING.md: canonical onboarding doc covering uv workflow, conda fallback, Python target 3.9–3.14, code standards (ruff, mypy), polars 1.x rules, test gating withskip_if_no_live, new-module spec. - New
.github/PULL_REQUEST_TEMPLATE.mdand.github/ISSUE_TEMPLATE/(config.yml,bug_report.yml,feature_request.yml,data_quality.yml). The PR template includes an "I have NOT included AI agents (Claude / Copilot / Cursor / GPT / Gemini) as commit co-authors" checkbox enforcing project policy.
NFL — nflreadpy parity
- Six new loaders:
load_nfl_team_stats,load_nfl_ftn_charting,load_nfl_trades,load_nfl_ff_playerids,load_nfl_ff_rankings,load_nfl_ff_opportunity. - Two new utility helpers in
nfl/utils_date.py:get_current_nfl_season(),get_current_nfl_week(). - Unified
load_nfl_nextgen_stats(stat_type=...)consolidating the per-type variants. The per-type functions are kept as aliases that emitDeprecationWarningand forward to the unified entry point. - Unified
load_nfl_pfr_advstats(stat_type=, summary_level=)consolidating eight per-type / per-summary functions, with the same deprecation alias pattern. - 25 nflreadpy-parity aliases inside
sportsdataverse.nfl(load_pbp↔load_nfl_pbp, etc.). Identity-equivalent — no perf overhead, just a friendlier import surface for nflreadpy users. kind=parameter added toload_nfl_ff_rankingsas the preferred name;type=retained for nflreadpy parity.
NFL — caching and configuration
- New caching layer in
sportsdataverse.nfl.cachewith both memory and filesystem backends and TTL support. clear_cache()for explicit invalidation.- New
NflConfigplusupdate_config()/get_config()/reset_config(), with env-var initialization:SDV_PY_NFL_CACHE,SDV_PY_NFL_CACHE_DIR,SDV_PY_NFL_CACHE_DURATION,SDV_PY_NFL_VERBOSE,SDV_PY_NFL_TIMEOUT,SDV_PY_NFL_USER_AGENT. - All 23 canonical loaders plus the 11 deprecated aliases are decorated with
@cached_loader. return_as_pandas=Trueround-trips correctly through the cache: a single polars frame is stored, and conversion happens on read.
NFL — static datasets
team_abbr_mapping(143 entries, relocations folded into the modern abbreviation:OAK -> LV,SD -> LAC,STL -> LA).team_abbr_mapping_norelocate(143 entries, history preserved).player_name_mapping(136 entries, common-variant → canonical).- All three are eagerly loaded at import time and inline-bundled in the package — no separate JSON files to ship.
NFL — pickcenter / odds modern path
__helper__espn_nfl_odds_information__now hits the modernsports.core.api.espn.com/v2/.../events/{gid}/competitions/{gid}/oddsendpoint when the legacysummary?event=pickcenterarray is empty (true for all 2024+ games).- Cascades to defaults
(2.5, 55.5, True, False)only if both modern and legacy paths fail. - For example, the 2024 CFP semifinal previously returned
(2.5, 55.5, True, False)and now correctly returns(-3.5, 67.5, True, True).
NFL — load_nfl_schedule parquet port
- Switched from the stale
nflverse-pbp/master/schedules/sched_{season}.rds(which was 404'ing on every season) to the modernnflverse-data/releases/download/schedules/games.parquet. One combined file, 1999–2025, 7,276 rows × 46 cols.
WBB / WNBA — new ESPN scrape modules
Eleven new modules across sportsdataverse.wbb and sportsdataverse.wnba, plus their __init__.py re-exports and live-gated smoke tests. The WNBA modules (other than wnba_draft) are thin shims onto a shared _espn_basketball_* helper that lives in the corresponding wbb_*.py file (league slug fixed to "wnba"), keeping the wbb/wnba pair DRY.
wbb_team_roster/wnba_team_roster: per(team_id, season)roster, flattened to one row per athlete. Snake-case columns; stable schema on empty rosters.wbb_player_stats/wnba_player_stats: per(athlete_id, season)stats. Multi-table dict with canonical keysAverages/Totals/Misc(always present, empty-frame fallback) plus anOtherbucket only added when ESPN ships a non-canonical category.wbb_team_stats/wnba_team_stats: per(team_id, season)stats. Same multi-table shape as player stats; ESPN ships these asGeneral/Offensive/Defensivecategories that map onto the canonical Averages / Totals / Misc keys. Endpoint corrected tosite.web.api.espn.com/.../teams/{id}/statistics?season=...(thecommon/v3path the original spec named 404s).wbb_standings/wnba_standings: one-row-per-team season standings. WBB defaults togroup=50(Division I women); WNBA has no group filter.wnba_draft: one-row-per-pick draft history. Modern endpoint atsite.web.api.espn.com/apis/site/v2/sports/basketball/wnba/draft(thesite/v3variant 404s).wbb_event_officials/wnba_event_officials: one-row-per-official game-level officials list.- All eleven ship with full
@overloadtyping (mypy-strict), polars 1.x APIs, andsnake_casecolumns viadl_utils.underscore.
CFB — cfb_play_participants and __add_player_cols collapse
- New
cfb_play_participantsmodule hits the ESPNevents/{gid}/competitions/{gid}/playsparticipants endpoint, with$refresolution (default-on,resolve_missing=True) for athletes missing from the sidecar. cfb_pbp.__add_player_colsshrunk from 471 lines of regex extraction to ~130 lines that delegate to the participants module.- All 19 legacy
_player_namecolumns preserved via an alias mapping. - Hybrid scalar + list-column output:
{type}_player_nameplus{type}_player_names, so multi-entry types like split sacks aren't silently collapsed to a single name. - Targeted regex fallbacks retained as a tertiary safety net for
sack_player_name2,fg_block_player_name,punt_block_player_name, andinterception_player_name— ESPN's sidecar has documented gaps for those.
CFB — pandas → polars 1.x bug-fix reconciliation (0.36-live → main)
- Foundation: new
cleaned_textcolumn normalizes ESPN play descriptions and is the single source of truth for downstream feature extraction. - Behavioral: kneel-down semantics flag plus
scrimmage_playexclusion. - Yardage: structural rewrite of
__add_yardage_cols(~150-linenp.selectchain →pl.when().then()chain), pass-yards regex tightened from(?<=for)to(?<=[\s,]for), full punt rewrite, fair-catch fix. - Helper-features: end-state edge cases, NCG 2025 GW play hardcode,
lead_halfend-of-half fix, OOB punts block, FG classification correction,end.TimeSecsRemshift direction flipped from lag to lead — which is what WPA inputs expected all along. - WPA:
__process_wpaend-of-game branch rewrite plus onside-kick rewrite, pluspenalty_assessed_on_kickoffplumbing across__setup_penalty_data+__process_epa+__process_wpa. - Player names: extraction migrated to
cleaned_texteverywhere.
Infrastructure and tooling
- Polars 1.x migration across
cfb/cfb_pbp.py,nfl/nfl_pbp.py,mbb/mbb_pbp.py,nba/nba_pbp.py,nhl/nhl_pbp.py,wbb/wbb_pbp.py,wnba/wnba_pbp.py. Roughly 165 API translation sites:groupby→group_by,with_row_count→with_row_index,apply→map_elements(with explicitreturn_dtype), struct list-arg → varargs,shift_and_fill→shift,cumsum→cum_sum,str.strip→str.strip_chars,str.n_chars→str.len_chars, outer-join →full+coalesce,write_jsonkwargs. - Polars 1.x
is_insame-datatype deprecation: switched to.implode()for the global-containment idiom. pkg_resources.resource_filename→importlib.resources.files()incfb_pbp.pyandnfl_pbp.pyvia small_cfb_resource_filename/_nfl_resource_filenamehelpers. Setuptools 81+ removedpkg_resources, which made the legacy import emit aUserWarningat module load and (eventually) break entirely.download()retry rewrite: iterative loop instead of recursion, defensiveresponse = Noneinit, re-raises the last captured exception when the retry budget is exhausted.psutilmade optional indecorators.py(lazy import, previously an undeclared transitive dep that broke autodoc).pytest.inifilterwarnings for the transitivesphinxcontrib-jsmathlegacynspkg.pthUserWarningand thepkg_resourcesAPIDeprecationWarningsurfacing from setuptools 81+.- New tests under
tests/wbb/,tests/wnba/,tests/conftest.py(with the@skip_if_no_livedecorator gated bySDV_PY_LIVE_TESTS=1), andtests/README.mdcapturing the test conventions. NFL test files renamed to drop legacy-phase-jargon filenames in favor of descriptive names (test_nfl_loaders_parity_loaders.py,_unified.py,_aliases.py).
Bug fixes
test_havoc_ratecorrected for bothcfbandnfl:def_intfield name fix, bounded<=assertion,def_box.sort()for deterministic group_by emit order, andturnover_boxnow produces a cli warning instead of silently padding an empty dict.yds_puntedduplicate definition removed.drive.idNCG 2025 GW play hardcode.is_in(col)→is_in(col.implode())for global containment, applied acrosscfb_pbpandnfl_pbp.- Pickcenter regression test added for both CFB and NFL: a 2024+ game must NOT silently fall back to the
(2.5, 55.5, True, False)defaults; a pre-2024 game with populated legacypickcentermust continue to use that legacy path. - NFL
__helper_nfl_pbp_featuresdefensive cast for the case where ESPN returnsoverUnderas a Python float (no.astype()); same shape fix as the cfb_pbp version.
Deprecations
- Four NFL loader families now consolidate per-type variants into a single unified function:
load_nfl_nextgen_stats(stat_type=...)andload_nfl_pfr_advstats(stat_type=, summary_level=). The per-type names continue to work but emit aDeprecationWarningpointing at the unified function. No removal yet.
0.0.40 Release: December 6, 2025
- Minor changes to mbb_calendar and wbb_calendar functions to include all games, even when top 25 teams are not competing
0.0.38-39 Release: August 28, 2023
- Minor changes to cfb_pbp functions to improve WP calculation and player parsing.
0.0.36-37 Release: July 9, 2023
- Switched most under the hood dataframe operations to use the python
polarslibrary and many functions now have a parameterreturn_as_pandaswhich defaults toFalsebut can be set toTrueto return a pandas dataframe instead of a polars dataframe. This is a breaking change. - Added
**kwargswhich pass arguments to thedl_utils.download()function, includingheaders,proxy,timeout(default 30s),num_retries(default = 15),logger(default = None) - Function
espn_cfb_game_rosters()added. - Function
espn_nba_game_rosters()added. - Function
espn_nfl_game_rosters()added. - Function
espn_nhl_game_rosters()added. - Function
espn_wbb_game_rosters()added. - Function
espn_wnba_game_rosters()added. - Function
load_cfb_betting_lines()added (only 2006 through 2019).
0.0.34-35 Release: May 7-9, 2023
- Reconfigured some imports
- Improved compliance with pandas upgrades
- Updated loader locations to use sportsdataverse-data releases and nflverse releases
- Flattened the returned results somewhat for "sportsdataverse.cfb.espn_cfb_schedule()" functions, but also now including some nested data frame and list columns
0.0.18 Release: July 25, 2022
- Added ondays parameter to ESPN calendar functions
- Renamed "sportsdataverse.cfb.cfb_teams()" to "sportsdataverse.cfb.espn_cfb_teams()" to avoid an edge case issue when running the function.
0.0.17 Release: July 9, 2022
- Added MLBAM API functionality to the sportsdataverse-py package. For more information on how to use these new functions, refer to the docs.
- Fixed a bug where the "sportsdataverse.nfl.load_nfl_schedule()" function would cause a 404 error when run.
- For functions where multiple files are loaded in, progress bars have been added to indicate how far along the sportsdataverse-py package is in completing its task(s).
- Renamed "sportsdataverse.cfb.cfb_teams()" to "sportsdataverse.cfb.get_cfb_teams()" to avoid an edge case issue when running the function.
0.0.15 Release: May 8, 2022
- Refactor schedule and teams functions for all existing leagues.
- Created more robust home/away mappings to simplify assignment.
0.0.14 Release: March 16, 2022
- Refactor schedule and teams functions for all existing leagues.
- Created more robust home/away mappings to simplify assignment.
0.0.12 Release: February 24, 2022
- Minor refactor to all the pbp functions, attempting to normalize behavior.
- Adding raw parameter to same functions to return object as it comes in without any transformation
- Adding some config file corrections.
0.0.5 Release: October 20, 2021
- f'in round
- findin' out