Add metadata-only S3 source connectivity dry-run
Implemented SeaTunnel's existing source connectivity validation SPI for supported S3A configurations using bounded metadata requests that neither download object contents nor modify storage.
apache/seatunnel · #12242
Deployment preflight feature
SeaTunnel can now detect inaccessible S3 objects, prefixes, buckets, credentials, and endpoints before a supported ingestion job is submitted.
Problem
The S3 source factory did not expose the connectivity dry-run SPI, so `--dry-run connect` skipped it. Endpoint, credential, bucket, and path mistakes therefore could remain undiscovered until after job submission.
Approach
Reuses Hadoop S3A configuration to issue one object metadata request or a single-entry prefix listing, with capped connection and socket timeouts, retries disabled, explicit eligibility checks, and cleanup after success or failure. Missing batch paths and denied requests fail early while accessible empty continuous prefixes remain valid.
Impact and scope
- Moves common S3 access failures from job execution into a pre-submission validation step.
- Keeps the preflight metadata-only and bounded: it does not fetch contents, paginate listings, upload, delete, or initialize a shared filesystem.
- Returns explicit unsupported-scope errors instead of presenting unvalidated configurations as successful.
Validation
- All 68 connector unit tests and 13 factory integration tests passed without failures or skips on Java 8 and Java 11.
- Six pinned-MinIO tests and seven HTTP-wire tests covered objects, prefixes, empty roots, missing buckets, credentials, metadata-only requests, bounded listing, denials, timeouts, and cleanup.
- A SeaTunnel maintainer approved the feature with explicit recognition, the GitHub-verified merge commit credits Goutam Adwant, and GH-12239 closed. These are factory-level tests rather than submitted engine jobs; the historical fork run also contained an unrelated Maven-wrapper HTTP 403 failure.