Skip to content

fix(maintenance): configurable peer port and self-signed certs in cluster log download - #38386

Open
oidacra wants to merge 6 commits into
mainfrom
cluster-log-download-verified-https
Open

oidacra wants to merge 6 commits into
mainfrom
cluster-log-download-verified-https

Conversation

@oidacra

@oidacra oidacra commented Oct 30, 2026 •

Copy link
Copy Markdown
Member

Change risk: Medium. Peer requests in the "All Cluster Nodes" download now go over HTTPS to CLUSTER_LOG_DOWNLOAD_PORT — 8443 by default — and accept self-signed peer certificates. Clusters serving HTTPS on another port set CLUSTER_LOG_DOWNLOAD_PORT; clusters whose nodes talk to each other over plain HTTP set CLUSTER_LOG_DOWNLOAD_SCHEME=http.

Where to look

  • ClusterLogCollector: one HttpClient per collection. By default it uses a trust-all SSLContext scoped to that client alone — no JVM-wide jdk.internal.httpclient.* property is set — so peers with self-signed certificates work and the peer token and logs still travel TLS-encrypted. CLUSTER_LOG_DOWNLOAD_VERIFY_PEER_CERTS=true switches to the JVM's default trust store with host name verification.
  • Scheme selection (resolveScheme): HTTPS by default; plain HTTP only when CLUSTER_LOG_DOWNLOAD_SCHEME is http (trimmed, any case). auto, empty or unknown values mean HTTPS, and a failed HTTPS request is no longer retried over HTTP.
  • Port selection (resolvePeerPort): CLUSTER_LOG_DOWNLOAD_PORT when set; otherwise 8443 under HTTPS, or the local server's port under plain HTTP.
  • _errors.txt in the ZIP: each failed peer has a short reason (HTTP <status>, could not connect, TLS handshake failed, connect timed out, timed out, connection failed, request failed). Failures that may come from TLS also name CLUSTER_LOG_DOWNLOAD_PORT and CLUSTER_LOG_DOWNLOAD_SCHEME.
  • QA: Maintenance > Log Files > Download, "This Server" (unchanged) and "All Cluster Nodes" on a multi-node cluster with self-signed certificates (default settings), then with CLUSTER_LOG_DOWNLOAD_PORT and CLUSTER_LOG_DOWNLOAD_SCHEME=http.

Proposed Changes

  • Peer log requests in the cluster log download go over HTTPS by default, on the port given by CLUSTER_LOG_DOWNLOAD_PORT (8443 by default; the local server's port under plain HTTP).
  • Peer certificates are accepted without verification by default, via a trust-all manager scoped to the collector's client; CLUSTER_LOG_DOWNLOAD_VERIFY_PEER_CERTS=true verifies certificates and host names against the JVM trust store. No JVM-wide HTTP client property is set either way.
  • Plain HTTP is used only when CLUSTER_LOG_DOWNLOAD_SCHEME=http, with no automatic retry between schemes.
  • The _errors.txt entry of a failed peer says why it failed and, when the failure may be TLS related, which port and scheme settings control it.
  • MaintenanceResource#downloadClusterLogFile Javadoc updated to the new port, scheme and certificate rules.

Release note

The "All Cluster Nodes" log download now reaches the other nodes over HTTPS by default, on the port given by CLUSTER_LOG_DOWNLOAD_PORT (8443 by default; env DOT_CLUSTER_LOG_DOWNLOAD_PORT). Peer certificates are accepted without verification, so clusters with self-signed certificates work out of the box; set CLUSTER_LOG_DOWNLOAD_VERIFY_PEER_CERTS to true (env DOT_CLUSTER_LOG_DOWNLOAD_VERIFY_PEER_CERTS) to verify peers against the JVM trust store. Clusters whose nodes serve HTTPS on a different port set CLUSTER_LOG_DOWNLOAD_PORT to it; clusters whose nodes talk to each other over plain HTTP must set CLUSTER_LOG_DOWNLOAD_SCHEME=http (for example DOT_CLUSTER_LOG_DOWNLOAD_SCHEME=http). Each unreachable peer is listed in _errors.txt with the reason and the settings that control it. The value auto is no longer special and means HTTPS.

Checklist

  • Tests
    • Unit test ClusterLogCollectorPeerRequestTest (25 cases, local HTTP/HTTPS peers with runtime-generated certificates): scheme selection, peer port defaults and override, certificate acceptance by default, verification with CLUSTER_LOG_DOWNLOAD_VERIFY_PEER_CERTS, per-failure reasons, no JVM-wide properties.
    • Integration test ClusterLogCollectorTest (MainSuite2b): plain-HTTP path unchanged; finally restores the previous Config values.
  • Translations (none: no UI text changed)

Additional Info

No REST contract change: same endpoint, same ZIP layout; only the free-text lines of _errors.txt change. No database, index or UI change.

Fixes dotCMS/private-issues#903

Peer requests made by the cluster log download are verified with the
JVM's default trust store and hostname check, and collecting logs no
longer changes JVM-wide HTTP client properties. CLUSTER_LOG_DOWNLOAD_SCHEME
selects plain HTTP only when set to http; any other value, or none,
uses HTTPS, and a failed peer request is not retried with another
scheme.
The _errors.txt entry of a failed peer now gives the HTTP status the
peer answered with, or says the connection failed. The note naming
CLUSTER_LOG_DOWNLOAD_SCHEME is added only to connection failures over
HTTPS. The integration test restores the configuration values it
changes.
The _errors.txt entry of a peer that could not be reached now says
whether no connection was made, the TLS handshake failed, the request
timed out, or the connection failed otherwise. The note naming
CLUSTER_LOG_DOWNLOAD_SCHEME is left out when no connection was made,
since the scheme makes no difference there.
…load

A peer that does not finish connecting is reported as "connect timed out"
and keeps the CLUSTER_LOG_DOWNLOAD_SCHEME note; a download that times out
after connecting is reported as "timed out" without it.
Adds unit tests for a peer that answers after the request timeout and for
a peer that closes the connection under the http scheme.
@oidacra
oidacra requested a review from a team October 30, 2026 17:15
@github-actions github-actions Bot added the Area : Backend PR changes Java/Maven backend code label Oct 30, 2026
@claude

claude Bot commented Oct 30, 2026 •

Copy link
Copy Markdown
Contributor

Claude finished @oidacra's task in 2m 26s —— View job


PR Review: verify peer certificates in the cluster log download

  • Read ClusterLogCollector.java
  • Read MaintenanceResource.java
  • Read test files
  • Analyze and post findings

I reviewed the diff against the prior state of these files. The change correctly implements what the reviewer (@wezell) asked for: port is configurable and defaults to 8443 (resolvePeerPort, ClusterLogCollector.java:230), and peer certificates are accepted by default while still travelling over TLS.

Correctness notes (verified, no action needed)

  • Hostname verification is actually bypassed now. The old code leaned on the JVM-wide jdk.internal.httpclient.disableHostnameVerification property; the new TRUST_ALL_MANAGER is an X509ExtendedTrustManager whose checkServerTrusted(chain, authType, SSLEngine/Socket) overloads are no-ops (ClusterLogCollector.java:340-380). Since the JDK HttpClient runs its "HTTPS" endpoint-identification check inside those extended methods, a no-op extended manager suppresses both chain and host-name validation for this client only — no JVM-wide state touched. The accompanying comment is accurate and the unit test asserts the property is never set.
  • exceptionally() throwable ordering is right. In fetchLog (ClusterLogCollector.java:273-297) HttpConnectTimeoutException is checked before its supertype HttpTimeoutException, and all four branches (SSLException, connect timeout, read timeout, ConnectException) are IOException subtypes, so they're reached. The possiblyTls flag is set correctly per case (connect timeout → true, read timeout → false, pure ConnectException → false).
  • Scheme/port resolution matches the documented contract — resolveScheme (:214) treats auto/empty/unknown as HTTPS and only http (trimmed, any case) as plain HTTP; resolvePeerPort (:230) honours CLUSTER_LOG_DOWNLOAD_PORT first, else 8443 for HTTPS / local port for HTTP. Removal of the cross-scheme retry is intentional and documented in the release note.
  • Convention checks pass: Config.get*Property (no System.getProperty/getenv), Logger throughout, token is sent as a Bearer header and not logged. The integration test is a modification (no suite re-registration needed) and the new ClusterLogCollectorPeerRequestTest is a Surefire unit test, so no MainSuite registration applies.

Non-blocking observations (pre-existing, not introduced here)

  • 🟡 ClusterLogCollector.java:136 — the HttpClient is created per collection and never close()d. HttpClient is AutoCloseable (Java 21+), so its selector thread/executor are released only when the instance is GC'd. The prior code had the same pattern (and actually built two clients), and this path is admin-triggered and infrequent, so impact is low — but since the PR consolidated to a single client, wrapping the collection in a try-with-resources (or calling close() after join()) would tidy it up. Fix this →

No blocking issues found — the reviewer's two requests (configurable port defaulting to 8443, accept untrusted certs while keeping TLS) are both satisfied, and the implementation is sound.
· cluster-log-download-verified-https

wezell
wezell previously approved these changes Oct 30, 2026

@wezell wezell left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is no going to work. dotCMS secure port is 8443, not 443 and each dotCMS uses a self signed cert that is not going to pass verification.

  1. Make the port configurable and default to 8443
  2. Allow untrusted certs.

This will secure the output. Maybe the endpoint on the receiving server should add

@wezell
wezell dismissed their stale review October 30, 2026 17:32

pressed the wrong button

@wezell wezell left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is no going to work. dotCMS secure port is 8443, not 443 and each dotCMS uses a self signed cert that is not going to pass verification.

Instead, we should

  • Make the port configurable and default to 8443
  • Allow untrusted certs.

This will secure the output.

@oidacra

oidacra commented Oct 30, 2026

Copy link
Copy Markdown
Member Author

Done.

  • Port: added CLUSTER_LOG_DOWNLOAD_PORT (env DOT_CLUSTER_LOG_DOWNLOAD_PORT). Peers are requested on it when set; otherwise on 8443 — dotCMS's default secure port — over HTTPS, or on the local server's own port over plain HTTP.
  • Untrusted certs: peer certificates are accepted by default now, self-signed included, so stock clusters work out of the box. The acceptance is scoped to the collector's own client: no JVM-wide jdk.internal.httpclient.disableHostnameVerification property (the old code's real problem), and no fallback to plain HTTP, so the peer token and the log content still travel TLS-encrypted. CLUSTER_LOG_DOWNLOAD_VERIFY_PEER_CERTS=true restores strict verification against the JVM trust store for clusters that can provision trusted certificates.
…ster log download

Peer requests now go to CLUSTER_LOG_DOWNLOAD_PORT — 8443 by default under
HTTPS, the local server's own port under plain HTTP — and accept the peer
certificate without verification by default, through a trust-all manager
scoped to the collector's own client: no JVM-wide hostname-verification
property, no plaintext fallback. CLUSTER_LOG_DOWNLOAD_VERIFY_PEER_CERTS=true
restores verification against the JVM trust store.
@oidacra oidacra changed the title fix(maintenance): verify peer certificates in the cluster log download Oct 30, 2026
@oidacra
oidacra requested a review from wezell October 30, 2026 20:31

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

AI: Safe To Rollback Area : Backend PR changes Java/Maven backend code

2 participants