Commit d515bb0
authored
Add parquet range-read I/O benchmarks (local + S3) (#165)
Two standalone CLI benchmarks that isolate the raw column-chunk range-read cost (no parquet decode): hybrid_scan computes the byte ranges a column projection touches, then only the reads of those ranges into host/device memory are timed.
- parquet_io_benchmark: io_context (vector-io / device_read_async) vs the cucascade::io::datasource per-range host_read/device_read_async, over uring.
- parquet_s3_io_benchmark: native REST (vector-io host / host->device staging) vs kvikIO RemoteHandle datasource, host/device dest.
- CMake: cucascade_parquet_io_benchmark (CUCASCADE_BUILD_IO) and cucascade_s3_parquet_io_benchmark (CUCASCADE_BUILD_S3_BENCHMARK).
Authors:
- Amin Aramoon (https://github.com/aminaramoon)
Approvers:
- Bobbi Winema Yogatama (https://github.com/bwyogatama)
- https://github.com/felipeblazing
URL: #1651 parent ea9e4a1 commit d515bb0
3 files changed
Lines changed: 1192 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
98 | 98 | | |
99 | 99 | | |
100 | 100 | | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
101 | 115 | | |
102 | 116 | | |
103 | 117 | | |
| |||
123 | 137 | | |
124 | 138 | | |
125 | 139 | | |
| 140 | + | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
| 150 | + | |
| 151 | + | |
| 152 | + | |
| 153 | + | |
| 154 | + | |
| 155 | + | |
126 | 156 | | |
0 commit comments