Repository navigation
feat(WTinyLFU): adapt the window size with a hill climber - #385
Conversation
Add hill-climb=1, the window hill climber of Adaptive Software Cache Management (Middleware'18). Every 10 x cache size bytes of requests it compares the hit ratio with the previous period and moves the window 5% (between 1% and 80%), turning around when the hit ratio dropped. The window already holds whatever the main cache does not use, so a resize only sets the main cache's size, and the main cache evicts down to it when the window grows. SLRU_resize() keeps each segment's configured share, and SLRU_find() keeps an object that no longer fits the next segment in place instead of promoting and evicting it. Off by default, and results without hill-climb=1 are unchanged. Only LRU and SLRU main caches are supported.
SLRU_find() now keeps an object in place only when it fits neither its segment nor the next one, which can only happen after SLRU_resize(). Before, plain SLRU with a smaller later segment (seg-size=9:1) changed slightly. WTinyLFU's 1:4 main behaves the same as before. Also say in the header that growing the window evicts from the main cache, and list hill-climb=0 in the default params.
|
Just tried making the climber step smaller over time, and also the paper's indicator (setting the window directly from how skewed the workload looks, so no climbing). Surprisingly the step barely matters, and the indicator is actually worse, so I would just keep the PR like this. About alibabaBlock_117, my guess was wrong. I tried moving objects into the window instead of throwing them away and it got worse (0.638 to 0.649). The window also stays between 1% and 35% the whole time, so it's not picking a bad size either. So I still don't know what's causing the extra misses there 😅 Numbers (same 26 traces)
The smaller step and the indicator help on Zipf (0.96 pp worse than 1% goes to 0.46 and 0.19), but the indicator gets cluster26 at 0.001 badly wrong (0.12 to 0.42). The indicator is ported from Caffeine's simulator. |
|
Awesome work! |
What does this PR do?
Hi, this is the hill climber I proposed in #21. To be short, basically it's great at fixing a bad 1% window and not so great at leaving a good one alone, so I'd keep it opt-in (I put it behind
-e hill-climb=1). If it's off, results are the same as develop.Every 10x cache size of requests it compares the hit ratio with the last period, moves the window 5% (1% to 80%), and turns around if it dropped. It helps a lot when 1% is way too small, like Twitter cluster26 at 0.01 where the miss ratio goes from 0.43 to 0.08. But when 1% is already the best it kind of wanders off and gets worse, like MSR prxy_0 at 0.01 going from 0.22 to 0.29. On Zipf it's a bit worse too, since there's nothing to climb.
I have to say alibabaBlock_117 at 0.01 surprised me, it is worse than every fixed window. So I don't think the issue is picking a bad size, I think it's the moving itself that costs, because every time the window grows main has to throw away 5% of the cache.
Next I want to try a couple things Caffeine does, move objects into the window instead of throwing them away, and make the step smaller over time so it stops wandering. I can also do the indicator version if you want it (the one that sets the window directly from how skewed the workload looks, instead of climbing).
Closes #21
How it works
window-sizesets where the climber starts, and main has to be LRU or SLRU.SLRU_resize()changes SLRU's size and keeps each segment's share. When the window grows, main evicts down to its new size.SLRU_find()now leaves an object in place if it fits neither its segment nor the next one (only possible after a resize), so a hit can't evict it inside find.How was it tested?
4 unit tests and test_cli.sh cases for bad values. ctest 15/15, test_cli.sh 428/0, the new tests are clean under ASan/UBSan. With it off, WTinyLFU and plain SLRU give the same results as develop (11 fixed-window configs, 32 SLRU runs).
ctest --test-dir _build --output-on-failurepasses-Wall -Wextra -Werror)Results
26 cache_dataset traces with 0.001, 0.01, 0.1, object size ignored, on develop. Miss ratio change vs the fixed 1% window, in pp (negative is better):
cachesim <trace> oracleGeneral wtinylfu 0.001,0.01,0.1 --ignore-obj-size=1 -e hill-climb=1(or-e window-size=<w>).Per-trace miss ratios
Type of change
Checklist
clang-format(or the pre-commit hook fromscripts/setup_hooks.sh)