Support empty Schema Registry Protobuf payloads
Fixed SeaTunnel's Schema Registry-aware Protobuf decoder to accept zero-byte message bodies only after structurally validating the complete Confluent frame header.
apache/seatunnel · #12039
Streaming correctness fix
Valid Confluent-framed empty Protobuf messages now deserialize into default-valued rows instead of failing with an invalid-tag error.
Problem
Empty messages and proto3 messages containing only implicit default values can legitimately have no data bytes. The decoder rejected every payload shorter than two bytes, retried the complete framed record as plain Protobuf, and failed on the framing bytes as an invalid field tag.
Approach
Structurally decodes the Confluent zigzag-varint message-index vector and permits an empty data section only when the magic byte, schema identifier, index vector, and complete header are valid. Plain-Protobuf fallback and legacy non-empty framing remain supported.
Impact and scope
- Restores valid Kafka ingestion for empty and all-default Protobuf records using Schema Registry framing.
- Avoids weakening format detection by requiring a fully valid frame before accepting an empty body.
- Preserves compatibility for plain Protobuf, nested message indexes, and existing non-empty records.
Validation
- Added coverage for optimized and explicit index framing, empty and default-only messages, nested indexes, fallback, and malformed headers.
- All 19 Protobuf module tests passed on Java 8 and Java 11, followed by the complete final Apache Build.
- The authored commit and GitHub-verified merge credit Goutam Adwant; upstream approval closed GH-12037.