Skip to content

Follow the redirects HttpURLConnection declines (307/308, cross-protocol) - #1151

Open
aalhossary wants to merge 1 commit into
biojava:masterfrom
aalhossary:aa/follow-redirects
Open

Follow the redirects HttpURLConnection declines (307/308, cross-protocol)#1151
aalhossary wants to merge 1 commit into
biojava:masterfrom
aalhossary:aa/follow-redirects

Conversation

@aalhossary

Copy link
Copy Markdown
Member

Fixes #1149.

HttpURLConnection follows 301, 302 and 303 within a protocol. It never follows 307 or 308, and never follows a redirect that changes http to https. Both gaps have broken this build:

When Service Redirect
2025-12 CATH 301 http→https — cached the redirect body as classification data (#1138)
2026-08-29 ECOD 308 to a rewritten path — nightly red on all EcodInstallationTest

A browser follows either without comment, so neither service had reason to expect the change would break us.

What this adds

openConnectionFollowingRedirects(URL, timeout) handles what the JDK leaves:

  • 307 and 308, and cross-protocol 301/302/303 — the only ones that reach us, since the JDK already consumed the rest.
  • Relative Location resolved against the request. ECOD's is relative.
  • Five-hop cap, and a loop reported by name rather than chased.
  • https→http is refused. A redirect must not downgrade the transport silently. Anything refused is returned as-is, so checkHttpStatus still decides what a non-2xx means.

Used by downloadFile, downloadFileWithValidation and the URL overload of createValidationFiles.

The decision is split into redirectTargetFor(code, location, url) so the rules are testable without a server.

Note on ECOD specifically

This deliberately does not change DOMAINS_PATH. ECOD's own site and API still publish /ecod/distributions/…; the 308 to /ecod-legacy/ is an internal rewrite they introduced when their new app took over /ecod/. BioJava's URL is already the advertised one — the only defect was not following the redirect. Hardcoding /ecod-legacy/ would pin us to something unadvertised.

Testing

mvn test -pl biojava-core -Dtest=FileDownloadRedirectTest — 13 tests, no network:

  • 7 rule tests, pure, covering each status, relative and absolute locations, the https→http refusal, and unusable locations.
  • 6 end-to-end through a local HttpServer: a 308 and a 307 followed, the validated download storing the real body with a matching .size, a loop, an over-long chain, and a redirect with no Location.

Full biojava-core suite: 535 tests, no failures.

Scope

Other call sites have the same exposure — AllChemCompProvider, SiftsChainToUniprotMapping, Astral, PDBStatus, URLConnectionTools, and UniprotProxySequenceReader, whose hand-rolled version carries the comment "This method should be moved to a utility class in BioJava 5.0". Migrating them is a follow-up rather than part of this, since each needs its own judgement; inventory filed separately.

The JDK follows 301, 302 and 303 within a protocol, but never follows 307
or 308, and never follows a redirect that changes http to https. Both gaps
have broken this build: CATH began answering http with a 301 to https, and
ECOD now answers with a 308 to a rewritten path. A browser follows either
without comment, so neither service had reason to expect it would break us.

openConnectionFollowingRedirects handles what the JDK leaves, resolving
relative locations, capping the chain at five hops and reporting a loop
rather than chasing it. A redirect from https to http is refused: the
transport must never be downgraded silently. Anything refused is returned
as it is, so checkHttpStatus still decides what a non-2xx status means.

The decision is split into redirectTargetFor(code, location, url) so the
rules can be tested without a server; the end-to-end tests use a local
HttpServer rather than a real service.

Fixes biojava#1149
aalhossary added a commit to aalhossary/biojava that referenced this pull request Aug 30, 2026
Covers what is merged since 7.2.6 and what is open and expected to land:
the download and checksum work, the CATH and ECOD fixes, the contact and
ASA performance tweaks, electron density, and the JUnit 5 migration.

Five entries are for pull requests that are still open - biojava#1134, biojava#1147,
biojava#1148, biojava#1150 and biojava#1151 - and should be checked against what actually
merged before the release is tagged.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

FileDownloadUtils detects redirects but does not follow the ones HttpURLConnection won't (307/308, cross-protocol)

1 participant