Repository navigation
Conversation
… workflow lint The PR build checked out with fetch-depth: 0 (every branch, full history, 364 s in run 40742517893) only for the .mvn/maven.config guard. On pull_request the checkout is the merge commit, whose first parent is the base tip, so depth 2 is enough to diff HEAD^1..HEAD. The guard now fails closed if HEAD is not a merge commit. The workflow-lint actionlint job gets the same treatment.
…ip unused uploads Build and save the test image in the background while maven-repo and the docker build context upload, store already-compressed artifacts without re-deflating them, stop uploading build-classes (no workflow downloads it) and skip the docker-build-context upload on pull request builds, which are never deployed.
Pull request and merge-queue runs now save the Maven repository and plugin caches only when they restored nothing. Their entries are scoped to their own ref, so no other run can read them, and at ~690 MB each they kept the 10 GB repository cache full and evicted the main-branch entry that every run reads. On 2026-10-28, builds that restored main's entry scanned projects in 1-2 s and ran Maven in 5-8 min; builds that started cold took 76-127 s to scan and ~18 min in Maven, most of it downloading dependencies.
…fact build The Initial Artifact Build ran nx run-many -t build over 21 projects, but the dotcms-core-web war only takes dotcms-ui, dotcms-block-editor, dotcms-binary-field-builder, dotcms-webcomponents, edit-content-bridge and the standalone sdk-analytics bundle. The SDK libraries, mcp-server, ai-evals, dotcdn and http are now tagged skip:build, so build-test leaves them out; the SDK libraries that dotcms-ui and sdk-analytics depend on still build as their dependencies. They are still compiled before merge: the validate profile, which only the Frontend Unit Tests job uses, gains a build-unshipped execution that builds exactly the skip:build projects. That job runs on every change under core-web.
Every test job rebuilds its own module with -pl (dotcms-integration, dotcms-postman, dotcms-ui-e2e, dotcms-test-karate, the CLI pair), and no step of a PR run reads them from the maven-repo artifact. dotcms-reports is an aggregator nothing in CI invokes. In run 40742517893 these modules took 3.5 min of the reactor; with a warm ~/.m2 they take about 1.5 min. Merge-queue, trunk, nightly and release builds are unchanged: trunk reuses the queue's maven-repo for the CLI native build and the dotcms-cli deploy.
Every build job ran the whole core-web Nx build from scratch. A restored .nx/cache cannot fix that: Nx 21+ indexes it in .nx/workspace-data/<machine-id>-v3.db, so on a new runner the restored artifacts are unrecognized and never used. @nx/shared-fs-cache, Nx's own answer for CI-provider caches, is commercially licensed and deprecated over CVE-2025-36852 (CREEP). Instead, maven-job starts a small local server (nx-cache-server.mjs) that speaks Nx's documented self-hosted remote cache protocol from a directory, and points Nx at it with NX_SELF_HOSTED_REMOTE_CACHE_SERVER. actions/cache carries the directory. Only builds of main save it, keyed on the core-web tree so main stores a new entry only when core-web changed. Pull requests and the merge queue restore the newest one. The saved directory holds only the artifacts that run used or built (about 37 MB), so it does not grow. Release builds do not use it. .github/workflows/tests/nx-cache.test.sh runs the step's real run block and checks the server's protocol behaviour.
# Conflicts: # .github/workflows/cicd_comp_build-phase.yml
check-previous-build searched every workflow for a successful maven-repo artifact at github.sha. A pull_request run's head_sha is the PR head commit, so if main received that exact commit (an admin push bypassing the queue), trunk would reuse a PR build. PR builds now leave the CLI and test modules out of maven-repo, so trunk's CLI native build and dotcms-cli deploy would fail. Restrict the search to cicd_2-merge-queue.yml. That is what trunk reuses in practice (run 40740068900 reused merge-queue run 40730136845). Only trunk and a nightly dispatched with reuse-previous-build set the search; every other caller leaves it off.
The Nx cache reuses a build when its task hash is unchanged, but the targets behind the war read files their hash did not cover, so a change to only those files got the previous output back: - dotcms-ui, dotcms-block-editor, dotcms-binary-field-builder and dotcms-webcomponents import libs/dotcms-scss by path (styles, includePaths, relative @import), and nothing in the project graph depends on it. - dotcms-ui and dotcms-block-editor run Tailwind v4, which builds utility CSS from every text file under core-web, not only their dependency graph. A class written in libs/sdk/vue showed up in dotcms-ui's styles.css. - npm files read by path (primeicons, prismjs, gridstack, tinymce, monaco-editor, flatpickr, regenerator-runtime, daisyui) and the build tool versions are invisible to Nx's external dependency hashing. nx.json gains three named inputs: warBuild (pnpm-lock.yaml, package.json and the Node version), dotcmsScss and tailwindSources (the whole workspace, which Nx and Tailwind both read minus git-ignored files). The six shipped build targets use them as they need. This also fixes local nx build runs, which could serve stale CSS from the local cache for the same reasons. Checked with real builds through the CI cache server, each run starting empty and restoring a cold run's cache: a libs/dotcms-scss edit rebuilt the four apps that read it and restored the rest; an edit to an unrelated libs/sdk/vue file rebuilt only the two Tailwind apps.
Restricting previous-build reuse to the merge queue dropped one case that used to work: a commit that reached main without the queue is built by its trunk push run, and a later trunk dispatch or reuse-enabled nightly on that commit (build-on-missing-artifacts false) now failed instead of reusing it. When the merge-queue search finds nothing, search cicd_3-trunk.yml for the same commit. Trunk checks out github.sha, so that maven-repo is complete and matches the commit. The nightly is not a source: it builds the midnight commit (build-ref) rather than its own head_sha. The PR workflow is never searched.
… server Save gating. The save was gated on github.ref == refs/heads/main, the ref a workflow was dispatched from, not the code it built: a manual deploy of a feature branch or a CLI release of an auxiliary branch, dispatched from main, would have saved its outputs as main's newest entry for every PR to restore. maven-job now saves only when the caller passes save-nx-cache, which the build phase forwards and only trunk (push to main) and the scheduled nightly (which always builds a commit of main) set. Empty saves. A Count Nx Cache Entries step skips the save when the run used and built nothing, for example when Maven restored core-web from its own build cache and Nx never ran. Readiness. The step waited for any listener on the port. The server now writes a ready file once it is listening and exits if it cannot listen, so a taken port means a build without the cache, not Nx talking to whatever holds the port. Server. GET streams through pipeline and cuts the connection on a read error instead of leaving the response hanging; a failed PUT removes its partial file so it is never saved. The Restore Nx Cache comment now says plainly that only release-build: true skips the cache, so images from trunk, the nightly and manual deploys can contain restored outputs. nx-cache.test.sh covers all of it: 27 checks pass, and 12 of them fail against the previous version.
…figuration Frontend Unit Tests passes no -Dprod, so build-unshipped compiled dotcdn and the SDK libraries with their default configuration, not the production one the artifact build used before they were moved out of it. Pass --prod explicitly; locally that took 26 s against 25 s.
…ast when it vanishes The background subshell inherits the step's errexit, so a failing docker build exited it before the status file was written and the wait step polled for 20 minutes. Capture the exit code with || rc=$?, record the pid, and have the wait step fail at once when the process is gone without a status.
|
Claude finished @swicken's task in 4m 43s —— View job Claude Code ReviewI reviewed the reviewable portions of this PR — the New Issues
Looked at and OK
No 🔴/🟠 findings — nothing blocking. The two mediums are non-blocking; the first is the one worth a second look before merge. |
actionlint reported 45 shellcheck findings in cicd_comp_initialize-phase.yml on PR #38329. 44 were already on main; one SC2006 came from the trunk fallback added to Set Outputs. Fixed mechanically: the two backtick substitutions become $(...) (SC2006), and every >> $GITHUB_OUTPUT / >> $GITHUB_STEP_SUMMARY is quoted (SC2086, 40 findings). Both paths are runner-provided with no spaces, so behaviour is unchanged. The three SC2129 findings (group the redirects) are in the pre-existing Change Detection Summary step, where the fix is a restructure of that script. They get a scoped ignore in .github/actionlint.yaml, as the lint workflow's policy prescribes for pre-existing debt.
|
dotbot code review:
Traced reactor exclusions, cache-save gates, background image sync, shallow-checkout guards and Nx inputs; no provable P0/P1 regression introduced by this patch. Tip: comment with "/dotbot address comments" to attempt automated fixes for unresolved review threads. reviewed by dotbot · meta/muse-spark-1.3 · medium |
|
Thanks for the review. On the two mediums:
No code changes from this round. |
|
dotbot code review:
Traced the Nx cache server (port-conflict fail-closed, tmp-write-then-rename, 409-on-exists, empty-cache never saved, save gated on push-to-main/schedule and github.ref), the background docker image build/wait status-file handshake and its upload gating, the Maven cache-save gating on cache-matched-key plus event name, the shallow-checkout HEAD^2 fail-closed diff guards, and the PR reactor exclusion against the restricted previous-build reuse (merge-queue and trunk only, both of which build the full reactor). No provable P0/P1 regression introduced by this patch. Tip: comment with "/dotbot address comments" to attempt automated fixes for unresolved review threads. reviewed by dotbot · ~z-ai/glm-latest · medium |
dotCMS-Machine-User
left a comment
There was a problem hiding this comment.
✅ dotbot review: all reviewer models (meta/muse-spark-1.3, ~z-ai/glm-latest) agree — patch is correct.
approved automatically by dotbot
Fixes #38326
The "Initial Artifact Build" job gates every test job. A cold PR build (run 40742517893) took about 29 minutes in it; a warm one about 13.5. This PR attacks both halves: it keeps the Maven cache warm, and it shortens the warm build.
Changes
Maven repository cache (
maven-job/action.yml)~/.m2caches only when they restored nothing at all. Their entries are scoped to their own ref, so no other run could ever read them. Each one was about 784 MB, and about a dozen filled the 10 GB repository cache and evicted main's entry, the only one every build can use. On 10-27, PR builds were cold from 14:20 to 22:28 UTC because of this.Checkout (
cicd_comp_build-phase.yml,cicd_pr_workflow-lint.yml).mvn/maven.configguard now diffs the PR merge commit against its first parent (HEAD^1..HEAD), which is exactly the PR's change set. It fails closed when HEAD is not a merge commit. The workflow-lint job uses the same approach.Reactor
dotcms-integration,dotcms-postman,dotcms-reports,dotcms-ui-e2e,dotcms-test-karate,dotcms-api-data-modelanddotcms-cli. Each test job builds its own module with-pl. Merge-queue, trunk, nightly and release builds still build the full reactor.check-previous-buildnow reuses only merge-queue builds, falling back to trunk's own builds. A PR build is incomplete now, so it must never be reused for trunk.Uploads and the test image
maven-repoanddocker-build-contextupload withcompression-level: 0, because their contents are already compressed.docker-imageuses level 1: measured at 755 MB, against 726 MB at level 6, and 2.3x faster to zip.build-classesis no longer uploaded by default. Its only reader is the uncalleddeploy-javadocaction.docker-build-context, which only deployments download.Frontend (core-web)
.nx/cachealone doesn't work in Nx 23 (it reports "Unrecognized Cache Artifacts"), so the job serves the restored directory through a small local server,nx-cache-server.mjs. The server implements Nx's documented self-hosted cache API.libs/dotcms-scss, Tailwind v4's workspace-wide class scan, npm files read by path (throughpnpm-lock.yamlandpackage.json), and the Node version. These gaps could already serve stale CSS from a local Nx cache.skip:build, including the SDK framework libraries, dotcdn, mcp-server and ai-evals. They still build, with--prod, in the Frontend Unit Tests job through thevalidateprofile. The WAR file list is unchanged: the same 2,356 entries with the same sizes.Expected savings on a PR build
These are estimates from logs and local measurements. This PR's own CI run is the first real measurement. Its Nx cache will be empty until a trunk push or nightly saves one on main.
Verification
bash .github/workflows/tests/release-build-reuse.test.sh: 179 passed. It covers:bash .github/workflows/tests/nx-cache.test.sh: 27 passed. It covers:What to check on this PR's CI run
Linux-mavencore-v2-…from main and has no "Save Maven Repository Cache" step.