Avoid casting unreachable nested List values
Fixed recursive nested-Struct casts so DataFusion transforms only child values reachable from visible, non-null List rows while continuing to reject invalid values that are actually observable.
apache/datafusion · #24590
Query correctness fix
Sliced and null-parent nested Lists no longer fail because of invalid backing values that are unreachable from the visible result.
Problem
Nested List casts processed the entire backing child array. A slice, SQL `LIMIT`, or null parent could therefore fail when an excluded child value was not castable, even though that value could never appear in the query result.
Approach
Compacted reachable children for List and LargeList arrays, selected and remapped the union of reachable ListView and LargeListView ranges while preserving shared and overlapping values, and retained a zero-copy fast path when every child is reachable.
Impact and scope
- Makes nested casts obey visible array semantics instead of leaking failures from unreachable backing storage.
- Handles slices, null parents, overlapping and out-of-order views, and nested combinations without hiding errors in reachable values.
- Preserves the allocation-free path for ordinary arrays whose complete child storage remains visible.
Validation
- Added 68 focused nested-Struct cast tests covering sliced, null-parent, overlapping, out-of-order, nested, and visible-invalid inputs.
- Added the SQL `LIMIT` regression and passed the extended workspace suite, including 505 SQLLogicTest files and warnings-denied all-feature Clippy.
- Both authored commits and the merge commit are GitHub-verified; the fix received maintainer approval and closed GH-24506 as completed.