Add Google Pub/Sub sink connector
Added a first-party Google Pub/Sub sink connector for SeaTunnel Connector-V2 with JSON and delimited text payload support, authentication modes, registration wiring, docs, and emulator E2E coverage.
apache/seatunnel · #11877
Problem
SeaTunnel did not have a native Google Pub/Sub sink, forcing users to build custom connectors or route data through intermediate systems when publishing rows to GCP-native streaming consumers.
Approach
Implemented a sink-only GooglePubSub connector that publishes one Pub/Sub message per SeaTunnel row, supports Application Default Credentials, service-account keys, and emulator mode, waits for outstanding async publishes during checkpoints and shutdown, and wires the connector into distribution, plugin mapping, CI labels, E2E workflow, and documentation.
Impact and scope
- Expands SeaTunnel's cloud-native connector coverage with a GCP streaming sink.
- Provides at-least-once publishing semantics by surfacing synchronous and asynchronous publish failures before checkpoint completion.
- Includes complete registration and documentation so users can configure the connector without custom packaging.
Validation
- Added unit coverage for option validation, JSON and delimited serialization, checkpoint flush behavior, publish failures, cleanup, factory identifier, and option registration.
- Added emulator E2E coverage that creates a topic and subscription, runs FakeSource to GooglePubSub, pulls the published message, verifies payload, and acknowledges it.
- PR evidence reports connector tests across Java 8 and Java 11 plus emulator E2E runs on Flink, Zeta, and Spark execution paths.