From f691056492b146d6b2866a07c1fee88b9facd23e Mon Sep 17 00:00:00 2001 From: Alexander Akait <4567934+alexander-akait@users.noreply.github.com> Date: Sat, 29 Aug 2026 15:21:46 +0000 Subject: [PATCH 1/2] ci: deprecate the repository and stop the scheduled measurement MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Measurement moved to webpack/webpack, where benchmarks.yml runs test/benchmarkCases/ through CodSpeed on every pull request and every push to main. The scenarios this repository carried that webpack/webpack lacked — a loader in the pipeline, a cold persistent cache and HMR — were ported there. Nothing fires on its own any more. The random-missing cron ran every 30 minutes for 16k runs and roughly two in five ended in a build failure: the cases pin dependenciesDate for their dependencies but leave dev dependencies floating, so rome resolves a typescript that rejects the case's own tsconfig. Verify-stability went dispatch-only with it, so merging this does not itself start a run. The gh-pages archive stays as it is: ~77k measurements of webpack releases back to 2020 against the npm ecosystem of the time, which CodSpeed cannot reconstruct. README.md records all of it. --- .github/workflows/measure-random-missing.yml | 4 +- .github/workflows/verify-stability.yml | 5 +-- README.md | 46 ++++++++++++++++++++ 3 files changed, 50 insertions(+), 5 deletions(-) create mode 100644 README.md diff --git a/.github/workflows/measure-random-missing.yml b/.github/workflows/measure-random-missing.yml index 3e05a9922c..c4a4d98072 100644 --- a/.github/workflows/measure-random-missing.yml +++ b/.github/workflows/measure-random-missing.yml @@ -1,8 +1,8 @@ name: Measure random missing data point +# Ran every 30 minutes until the fixtures stopped building; see README.md. +# Kept dispatch-only so a data point can still be filled in by hand. on: - schedule: - - cron: "*/30 * * * *" workflow_dispatch: jobs: bench: diff --git a/.github/workflows/verify-stability.yml b/.github/workflows/verify-stability.yml index d7b1e38d9e..f590688a4c 100644 --- a/.github/workflows/verify-stability.yml +++ b/.github/workflows/verify-stability.yml @@ -1,9 +1,8 @@ name: Verify Stability +# Dispatch-only since the repository was deprecated; a push to main would +# otherwise start a benchmark run nobody reads. See README.md. on: - push: - branches: - - main workflow_dispatch: jobs: diff --git a/README.md b/README.md new file mode 100644 index 0000000000..8479bf997f --- /dev/null +++ b/README.md @@ -0,0 +1,46 @@ +# webpack/benchmark + +> [!IMPORTANT] +> **This repository is deprecated.** Performance measurement for webpack now +> lives in [webpack/webpack](https://github.com/webpack/webpack). Nothing here +> runs on a schedule any more, and the results are a frozen archive. + +Long-horizon build measurements for webpack: real applications, installed with +their dependencies as of a given date, built with the webpack of that same date. +The historical results are plotted at . + +## Where measurement happens now + +`.github/workflows/benchmarks.yml` in webpack/webpack runs the cases in +`test/benchmarkCases/` through [CodSpeed](https://codspeed.io/webpack/webpack) +on every pull request and every push to `main`, in both simulation and memory +mode. That is where a regression is caught and where the trend is tracked; this +repository never gated anything. + +The scenarios it carried that webpack/webpack lacked — a loader in the pipeline, +a cold persistent cache, and HMR — have been ported into `test/benchmarkCases/` +there. + +## What is kept, and why + +The `gh-pages` branch holds ~77k measurements of webpack releases going back to +2020, each against the npm ecosystem as it stood at the time. CodSpeed cannot +reconstruct that: it knows only the commits it has measured since it was +enabled, and it does not build real applications. **That archive stays as it +is** — please do not rewrite or delete the branch. + +## Why the schedule was switched off + +A cron measured a random missing data point every 30 minutes, for over 16,000 +runs. Roughly two in five ended in a build failure: the cases pin +`dependenciesDate` for their `dependencies` but leave dev dependencies floating, +so `rome` resolves `typescript` and `ts-loader` to a compiler that rejects the +case's own `tsconfig.json` (`TS5107: Option 'moduleResolution=node10' is +deprecated`). The newest data point ended up older than the historic ones the +cron kept backfilling. + +## Running something by hand + +Every workflow is still available through **Run workflow** in the Actions tab — +`compare.yml` takes two versions or dates. Reviving any schedule means pinning +each case's dev dependencies first. From baed88f52bf6f2fe5000cadb04223f7f0c6e7d0b Mon Sep 17 00:00:00 2001 From: Alexander Akait <4567934+alexander-akait@users.noreply.github.com> Date: Sun, 30 Aug 2026 17:41:46 +0000 Subject: [PATCH 2/2] docs: record exactly which scenarios were ported and which were not MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The earlier wording claimed the three scenarios webpack/webpack lacked had been ported, which understated the gap: eight of the 21 addons were already covered, ten are ported across two pull requests, and five are not portable — swc, thread-loader and preset-env each need a dependency webpack does not have, PnP needs an install layout the harness cannot produce, and no-cache is a no-op there. The fixtures are not ported either, and the README now says why. --- README.md | 26 +++++++++++++++++++++++--- 1 file changed, 23 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index 8479bf997f..07368c2d07 100644 --- a/README.md +++ b/README.md @@ -17,9 +17,29 @@ on every pull request and every push to `main`, in both simulation and memory mode. That is where a regression is caught and where the trend is tracked; this repository never gated anything. -The scenarios it carried that webpack/webpack lacked — a loader in the pipeline, -a cold persistent cache, and HMR — have been ported into `test/benchmarkCases/` -there. +Of the 21 addon scenarios this repository carried, webpack/webpack already +covered eight (both source-map variants, `future-defaults`, `persistent-cache`, +and the CPU and heap profiles through its own runner modes). Ten more were +ported into `test/benchmarkCases/` there: a loader in the pipeline, a cold +persistent cache and HMR in webpack/webpack#21858, then `no-minimize`, +`no-concatenation`, `no-exports-analysis`, the three unsafe-cache variants and a +modern `common-libs` in webpack/webpack#21864. + +Five are deliberately not ported, and stay recorded here rather than being lost +quietly: + +| Scenario | Why | +| --------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------- | +| `swc-env`, `swc-env-minimize`, `swc-minimize` | need `@swc/core` / `swc-loader` / `terser-webpack-plugin`, which webpack does not depend on | +| `thread-babel-env` | needs `thread-loader` | +| babel `preset-env` | needs `@babel/preset-env`; the ported `loader-babel` uses `preset-react` | +| `pnp` | needs a Yarn PnP install layout the in-repo harness cannot produce | +| `no-cache` | inexpressible there — the watch scenario overrides `cache`, and each iteration builds a fresh compiler, so a memory cache never carries over | + +The `cases/` fixtures are not ported. `rome` is an archived project whose case no +longer builds, `atlaskit-editor` pins React 16 with `@atlaskit/editor-core` 120, +and the old `common-libs` pinned Material-UI 4, Vue 2, moment and jQuery; the +replacement uses the modern libraries webpack already depends on. ## What is kept, and why