Validate anydoc Markdown identity handoff
Added a design and compatibility proof of concept for passing deterministic anydoc Markdown through SeaTunnel's existing reader while keeping identity tied to the original source URI.
apache/seatunnel · #11813
Document-ingestion design proof
SeaTunnel now has a tested compatibility record showing how anydoc-generated Markdown can preserve the original document and chunk identity.
Problem
A future document-parser backend needs to hand intermediate Markdown to the file connector without replacing source provenance with a temporary Markdown path or destabilizing chunk identifiers and ordering.
Approach
Extracts a package-private Markdown handoff, adds a pinned anydoc fixture, and verifies original URI, document ID, chunk ID, ordering, and spreadsheet-output limitations. It adds no runtime dependency, public API, file format, or production parser backend.
Impact and scope
- Establishes a reproducible compatibility boundary before choosing an external executable, WebAssembly, or sidecar deployment model.
- Protects document provenance and stable chunk identity for future retrieval and indexing workflows.
- Records current tabular-document degradation explicitly instead of overstating parser readiness.
Validation
- Regenerated the fixture with the pinned anydoc CLI and compared it with the checked-in Markdown output.
- All six focused Markdown reader tests and the dependent Maven reactor package passed.
- A SeaTunnel maintainer approved the proof of concept; final checks passed and the GitHub-verified merge commit credits Goutam Adwant.