r/kernel • u/Silent-Degree-6072 • 13d ago
Offloading zram compression to GPU?
Is there a way to offload zram compression & decompression to the GPU?
I have a compilation server with 36 logical cores but only 16GB of RAM (I am unable to upgrade due to the global RAM crisis; DDR4 registered ECC costs a ton). In an effort to squeeze more "RAM" out of my RAM, I decided to start using zram compressed with zstd.
I managed to get 48GB of zram to work (3:1 compression), however it resulted in one of my cores being pinned at 100%, desperately trying to compress the increasing amount of ram being used by 36 compile jobs. Is there a way to offload this to the GPU? I think it would greatly speed up compilation if possible, and would perhaps allow for a greater compression rate (zstd ultra maybe???)
EDIT: The GPU is a GTX 1660 Super with the open-source kernel modules from NVIDIA. It basically never gets used and usage is at a constant 0-1%
2
u/Ok_Green5623 13d ago
I don't think your problem is speed of compression. You will never be able to load your cores with compilation work, because compilation tend to use a lot of ram especially with higher optimization levels. If you need like 32-40GB of RAM to utilize your cores, no matter what you do - you will be heavily throttled by paging in and out. Just limit parallelism and use your CPU at 20-40% before it heats the RAM limit and it will be faster.
1
u/possiblyquestionabl3 13d ago
Are you using zstd for zram? The types of work it does for compression (I'm guessing this is what you want to offload?) isn't very well suited for a GPU - you'll want to have fairly uniform execution (few branches, since the simd programming model will serialize all branches together in most cases) and predictable memory shading.
That said, I believe https://github.com/KhronosGroup/Vulkan-ExtensionLayer/blob/38ced57f063a288ecd4f401523638b7bdb3ac9d2/docs/memory_decompression_layer.md has an example of a DEFLATE shader for to emulate VK_LAYER_KHRONOS_memory_decompression on ICDs / hardware that lacks this support, including an implementation of GInflate (decompression of DEFLATE) - https://github.com/KhronosGroup/Vulkan-ExtensionLayer/blob/38ced57f063a288ecd4f401523638b7bdb3ac9d2/layers/decompression/shaders/GInflate.glsl
16
u/_gaff 13d ago edited 13d ago
https://lwn.net/Articles/1102680/ patch under review will dramatically improve zram performance
Moving data between CPU and GPU is too slow to be used for compression acceleration.
Build a kernel and try this patch to see if it helps your use case maybe? If it does you could probably add a Tested-by with your findings to the series