Bound planning-time regex compilation
Added a bounded planning policy for DataFusion regex functions and a defaulted UDF extension point that can defer expensive literal evaluation across logical and physical simplification.
apache/datafusion · #24917
Query-planning performance fix
Pathological constant regexes are deferred to execution before they can consume DataFusion's full runtime compilation budget during optimization.
Problem
Pathological constant patterns could spend the regex engine's full 10 MiB runtime compilation budget while a query was still being planned, producing multi-second optimizer delays. Disabling regex folding entirely would also penalize ordinary constant patterns.
Approach
Introduces `ScalarUDFImpl::should_evaluate_const`, applies a 256 KiB planning limit to `regexp_like` and regex operators, and prevents parent expressions from indirectly evaluating deferred subtrees. Normal constants still fold, while expensive patterns retain the existing runtime behavior.
Impact and scope
- Reduced the issue-shaped local planning benchmark from 5.37 seconds to 0.17 seconds across 20 runs, approximately 31.6 times faster.
- Preserves fast constant folding for ordinary regexes while bounding optimizer latency for adversarial or unusually complex literals.
- Provides a backward-compatible hook other immutable UDFs can use to control expensive planning-time evaluation.
Validation
- Added logical optimizer, physical simplifier, scalar-UDF, SQL planner, and integration regressions covering pathological and ordinary patterns, aliases, flags, NULLs, and nonconstant expressions.
- Passed focused crate tests, SQL logic tests, the extended workspace suite, Clippy, rustdoc, formatting, license, typo, Prettier, and workflow checks; no final PR check reported a failure.
- A DataFusion maintainer approved the authored change; the GitHub-verified merge commit credits Goutam Adwant and closed GH-22185.