Simple method to use GPU as a fast RAM based 16 GB object and slices storage VRAM device that has no degradation, implemented in Odin using HIP ROCm and effectivelly freeing host RAM up to 16 GB.
Yesterday night I had this idea crazy ideia of using GPU VRAM as a fast storage device for Odin. The idea is to use ROCm HIP GPU VRAM ( Video RAM ) to store large objects and large slices in VRAM ( 16 GB ) and use it as hyper fast storage device with no deteroration ( unlike NVME or SSDs ). This is particularly useful for systems with limited RAM but with a big GPU VRAM capacity and useful if you need more RAM for a program you are developing and if you can sucessefully use this method. This is fast because the PCIe 5.0 bus is really fast and the transfer rates are very high. The code is implemented in Odin language using ROCm HIP bindings for Odin.
In the Host PC allocation of an object ( struct ) in the Stack and then store it in the GPU VRAM memory.
===> TEST 1
s : My_struct : My_struct{a = 42, b = 3.5, c_arr = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]}
After vram_put_obj( )
After put set all zeros - s : My_struct :
My_struct{a = 0, b = 0, c_arr = [0, 0, 0, 0, 0, 0, 0, 0, 0, 0]}
After get all data from vram_get_obj s : My_struct :
My_struct{a = 42, b = 3.5, c_arr = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]}
After changed the data on the host before vram_put_obj s :
My_struct : My_struct{a = 1000, b = 2000, c_arr = [-1, -1, -1, -1, -1, -1, -1, -1, -1, -1]}
After put again with vram_put_obj changed value s :
My_struct : My_struct{a = 1000, b = 2000, c_arr = [-1, -1, -1, -1, -1, -1, -1, -1, -1, -1]}
After put set all zeros - s : My_struct :
My_struct{a = 0, b = 0, c_arr = [0, 0, 0, 0, 0, 0, 0, 0, 0, 0]}
After get all data second time from vram_get_obj s : My_struct :
My_struct{a = 1000, b = 2000, c_arr = [-1, -1, -1, -1, -1, -1, -1, -1, -1, -1]}
Expected error after vrm_free_obj( ) vram_get_obj( ) :
VRAM_ERR_INVALID_HANDLER_STATE:
ERROR : in vram_get( ) - the obje with this handler has not been put
in the VRAM previouslly or the handler has been previouslly vram_free_obj( ).
In the host PC allocation of an object ( struct ) in the Heap and then store it in the GPU VRAM. memory.
===> TEST 2
Before vram_put_obj( ), the struct s_a is filled data s_a : My_struct :
&My_struct{a = 42, b = 3.5, c_arr = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]}
After vram_put_obj( ) s_a that stored a copy of s_a in the GPU VRAM memory.
In the host free( s_a ) the host object memory after vram_put_obj( ),
so that we can have more memory available on our system.
In the host allocate again the memory for the object.
After get all data from vram_get_obj s_b : My_struct :
&My_struct{a = 42, b = 3.5, c_arr = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]}
- In the host, allocation of a slice in the heap and then store it in the GPU VRAM memory.
- Then delete the object in the host and free memory on the host for different processing phase.
- Then allocate a new slice in a different region of memory and get the slice back from GPU VRAM memory, into host PC memory. This is over PCIe 5.0 so it is really fast.
===> TEST 3
my_slice : [ ]int with 10 element s :
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
After vram_put_slice( my_slice )
After put set all zeros and reduce the size of the slice - my_slice : [ ]int :
[0, 0, 0, 0, 0]
After get all data from vram_get_obj my_slice : [ ]int :
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
After changed the data on the host before vram_put_slice my_slice : [ ]int :
[3, 3, 3, 3, 3]
After put again with vram_put_my_slice changed value my_slice : [ ]int :
[3, 3, 3, 3, 3]
After on the host delete( my_slice ) and allocating a new one in the host
memory in a different region of memory with 10 elements my_slice_b =
[0, 0, 0, 0, 0, 0, 0, 0, 0, 0].
After vram_get_slice( ) my_slice_b =
[3, 3, 3, 3, 3].
After vram_free_slice( )
Expected error after free vram_get_slice :
VRAM_ERR_INVALID_HANDLER_STATE:
ERROR : in vram_get_slice( ) - the obje with this handler slice has not
been put in the VRAM previouslly or the handler has been previouslly
vram_free_slice( ).
# Install ROCm on Linux, I used Omarchy ( ARCH ).
make clean
make gpu_rocm_lib
make opti_x86_rocm
./gpu_vram_store_rocm.exeMIT Open Source License
Best regards,
Joao Carvalho