Preserve SQS messages on deserialization failure
Corrected SeaTunnel's Amazon SQS source so deserialization failures preserve unreadable messages by default, while an explicit option supports controlled skip-and-continue processing.
apache/seatunnel · #12038
Data-loss prevention fix
Malformed Amazon SQS messages are no longer forwarded as null records and deleted before operators can retry or route them to a dead-letter queue.
Problem
The deserializer swallowed IO failures or returned null, after which the source could forward a null record and delete the original SQS message. This bypassed visibility-timeout retries and dead-letter-queue redrive, creating a silent at-least-once delivery violation.
Approach
Converts thrown and null deserialization failures into a connector exception before collection or deletion. Added `ignore_parse_errors`, defaulting to false, so operators can deliberately skip malformed records; deletion behavior remains explicitly governed by `delete_message` and is documented in English and Chinese.
Impact and scope
- Prevents malformed source messages from being silently lost under the connector's default safety behavior.
- Restores SQS-native retry and dead-letter-queue handling without changing the public deserializer interface.
- Provides an opt-in operational escape hatch for poison messages while making its deletion trade-off explicit.
Validation
- Added deterministic reader and factory regressions for thrown and null failures, mixed batches, collector failures, deletion modes, and factory-created JSON parsing.
- All 10 Amazon SQS module tests passed on Java 8 and Java 11; Spotless and the complete final Apache Build also passed.
- Both authored commits and the GitHub-verified merge credit Goutam Adwant; maintainer approval closed GH-12036 with explicit appreciation for the work.