perf: extend string IN-list pruning benchmark coverage - #24836
Open
goutamadwant wants to merge 1 commit into
Open
perf: extend string IN-list pruning benchmark coverage#24836goutamadwant wants to merge 1 commit into
goutamadwant wants to merge 1 commit into
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #24836 +/- ##
==========================================
- Coverage 81.58% 81.57% -0.01%
==========================================
Files 1123 1123
Lines 406610 406610
Branches 406610 406610
==========================================
- Hits 331716 331708 -8
- Misses 55457 55460 +3
- Partials 19437 19442 +5 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
IN-list pruning benchmark #24707.Rationale for this change
The string IN-list pruning benchmark only covered domain sizes 20, 21, 256, and 1,024 with a fixed 4,096 pruning containers. This did not provide the small-list or container-scaling measurements needed by the follow-up work in #24709 and #24710.
This PR extends the benchmark before making any production threshold change, following the contributor guide's recommendation to isolate benchmark additions from the optimization they are intended to evaluate.
What changes are included in this PR?
The production pruning implementation and its current threshold are unchanged.
What is the testing strategy for this PR?
./dev/rust_lint.shcargo test --locked -p datafusion-pruning(95 tests passed)Extended workspace test suite:
RUST_BACKTRACE=1 cargo test --locked --profile ci --exclude datafusion-examples --exclude datafusion-benchmarks --exclude datafusion-cli --workspace --lib --tests --bins --features avro,json,backtrace,extended_tests,recursive_protection,parquet_encryptionFull 72-case Criterion matrix:
Are there any user-facing changes?
No. This PR only extends benchmark coverage.