Prevent recursive FFI session planning
Hardened DataFusion's FFI query-planning boundary by preserving the legacy ABI slot while directing cross-library delegation through a captured, serialized `FFI_QueryPlanner` handle.
apache/datafusion · #24492
Cross-library correctness fix
DataFusion FFI planners now reject an unsafe session-planning path that could recurse until stack exhaustion or return opaque physical-plan nodes.
Problem
Calling `ForeignSession::create_physical_plan` after installing a foreign planner could re-enter that same planner without a bound and exhaust the stack. Its `FFI_ExecutionPlan` result also could not reconstruct built-in nodes with the receiving library's type identities for downcasting.
Approach
Makes direct foreign-session physical planning return an actionable `NotImplemented` error without crossing the FFI boundary. The legacy callback keeps its position and signature for DataFusion 55 ABI compatibility, while documentation and tests establish the captured-planner migration path.
Impact and scope
- Eliminates an unbounded recursion path across dynamically linked DataFusion libraries.
- Preserves local, downcastable physical-plan node identities through the supported serialized planner boundary.
- Maintains the existing FFI struct layout while giving downstream integrations an explicit migration path.
Validation
- Added unit coverage proving zero planner re-entry and a three-library `dlopen` regression covering both rejected direct delegation and successful captured-planner reconstruction.
- Focused integration tests, the extended workspace suite, Clippy, rustdoc, formatting, license, documentation, and final PR checks passed.
- A DataFusion maintainer approved and explicitly thanked Goutam for the change; the GitHub-verified merge commit credits Goutam Adwant and closed GH-24065.