Apache Software Foundation / SeaTunnel
Merged upstreamCorrectnessMerged Sep 6, 2026

Fix file comparison across stream read boundaries

Corrected strict checksum fallback in SeaTunnel's batch and continuous file update paths by comparing complete byte sequences independently of individual stream read boundaries.

apache/seatunnel · #12080

File synchronization correctness fix

Byte-identical files are no longer recopied or re-enqueued merely because their input streams return different chunk sizes.

Problem

The fallback read source and target streams in lockstep and treated unequal per-call byte counts as unequal content. Because Java streams may legally return different chunk sizes for identical bytes, unchanged files could be classified as changed and copied again.

Approach

Replaces both duplicated hand-written loops with the module's existing `IOUtils.contentEquals` implementation. Complete content and length are compared with bounded buffers while stream ownership and the existing fail-open-to-copy behavior on I/O errors remain unchanged.

Impact and scope

  • Avoids unnecessary copying and split enumeration for unchanged files in strict update synchronization.
  • Corrects both batch and continuous paths so equivalent content receives consistent treatment across execution modes.
  • Uses an established dependency instead of maintaining boundary-sensitive byte-comparison logic.

Validation

  • Added deterministic streams returning identical content in different 3-byte and 5-byte chunks, plus different-content, different-length, empty-stream, and continuous-enumeration cases.
  • All 276 connector-file-base tests passed on Java 8 and Java 11; module compilation, Spotless, packaging, and final PR checks passed.
  • A SeaTunnel maintainer approved the authored change with explicit appreciation; the GitHub-verified merge commit credits Goutam Adwant and closed GH-12078.