Repository navigation
More speed up - #501
Merged
Merged
More speed up#501
Conversation
When a decoder needs a copy of a block (e.g. Vulkan, which imports the frame data in GPU memory space), the block is read from the file instead of being copied from the memory map, which is mapped again for each block: no page faults, and no unmapping of these pages later. With vulkan on Intel i7-7800X (6464x4852 RGB 16-bit, 88 MB per block), the copy takes 28 ms instead of 40 ms and the remapping of the file 0.1 ms instead of 6.5 ms.
Callers accepting to wait give the count of jobs given at the same time by other threads (decoded frames: 8, files: the count of I/O threads up to 8), and engines wait for this count of jobs instead of 3, until 3 s after the start of the oldest job instead of 1 s, also when other jobs computed with them are done (instead of a wait of 2 ms), and a job is done as soon as its data is computed. Outputs have 8 threads instead of 4, and 8 more frames in flight (e.g. 16 instead of 8 with the hybrid decoder). The count of jobs computed together in lanes with less CPU time than scalar code is measured on the CPU at the start, instead of 3 on all CPUs: 1 with AVX-512VL on Intel Skylake-X (1 lane is faster than scalar code), 3 expected on AMD Zen 5 and Apple M1 as before. With internal+vulkan on Intel i7-7800X (6464x4852 RGB 16-bit, 72 frames), 6.6 frames are computed together on average instead of 2.7, no frame is computed with scalar code, and the CPU time of the hashes is 3.1 s instead of 6.7-7.4 s, same wall time. --all --no-encode on 10000 DPX files of 6 MiB (Windows, cold): user CPU 11 s instead of 14 s; with --io-threads 4: 44 s instead of 122 s (files waited 1 s for 3 jobs); with --io-threads 2: 62 s instead of 77 s.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up of the edit/hash speed up, mainly lower CPU usage of the hash when decoding.