ZRAM for your SBC

Why do we need swap?

If you browse the web you’ll find many different answers to this question. For a system running a single process, you probably don’t. Over the years I have deployed several Raspberry Pi Zero’s, 3B’s and 4B’s that have no swap enabled. These are all headless systems running just the python app that I wrote for that particular deployment. All of these systems are able to run for an extended period of time (months+) with no issues.

Once we start running graphical OS’s, desktop applications with web browsers, file browsing, etc the need for swap becomes clear. The kernel uses free memory as a file buffer which allows faster access for files that are commonly used. On my Rock 5B you can see how the memory has currently being used:

$ free -h
               total        used        free      shared  buff/cache   available
Mem:           7.5Gi       2.7Gi       3.2Gi       133Mi       1.8Gi       4.8Gi
Swap:          2.0Gi          0B       2.0Gi

Here we can see of 3.2GB of free RAM, but 4.8GB of available. Most of the difference here is due to the file buffer. We also can see that no data has been swapped at this time, and there is plenty of free memory even with the current buffer.

As we start to run more applications and open more files, the memory utilization will start to shift. Even though the kernel will free up space from the buffer when needed, without a swap I have run into application hangs long before I ran out of memory. In other instances during burn in tests I’ve had the system kill my application since it was considered the memory “hog”.

After running a few web browser windows, downloading some files and launching a couple of VSCode consoles, my memory usage now looks like this:

$ free -h
               total        used        free      shared  buff/cache   available
Mem:           7.5Gi       3.9Gi        55Mi       190Mi       3.9Gi       3.7Gi
Swap:          2.0Gi        84Mi       1.9Gi
 

Here you can see that the file buffer has shot up to 3.8GB, used RAM to 3.9GB, and free dropped to 55MB. You can also see some RAM has been swapped (albiet a small amount).

What is ZRAM?

ZRAM creates a virtual object (in memory) to hold paged data in a compressed format. While ZRAM does use some of your memory, on systems that have slow disk I/O (i.e. those using SD cards), this allows the system to take less frequently used memory pages and compress them.

In old kernel versions (i.e. 3.15 and earlier) it was required to run a ZRAM object per CPU core. This was due to the inability for multiple cores to access the same ZRAM device at the same time. If you search for ZRAM, you’ll probably find lots of examples of systems configured with both one ZRAM device for the system as well as those with a ZRAM device per CPU core.

Given that it is 2023, we will assume you are running a 5.x or 6.x Linux kernel and will configure with a single ZRAM device for simplicity. HOWEVER, a lot of the default scripts and tools still assume one device per core, so there may be some adjustments required to ensure proper operations.

You can read more about ZRAM here: https://www.kernel.org/doc/html/next/admin-guide/blockdev/zram.html

Raspberry Pi OS

The classic Raspberry Pi OS by default creates a swap file in /var sized at 100MB. The presense of SOME swap seems to alleviate most of the issues, however since most Raspberry Pi devices run off SD, there is a VERY LARGE performance hit to having the swap on flash.

The swap file is created by the dphys-swapfile service which configures the swap file on startup. Before we configure ZRAM, we first should disable the swapfile. This can be done by simply disabling the service (it will take effect on next boot).

sudo systemctl disable dphys-swapfile

After we disable the swapfile, the next step is to install the ZRAM tools:

sudo apt install zram-tools

The ZRAM configuration will all be done in /etc/defaults/zramswap file. The default configuration will all be commented. I will only edit the size value (configured in MB) to switch from 256MB to 2GB (my Pi4B is an 8GB model). Set the size appropriate for your device (i.e. a Pi Zero with 512MB of RAM I would set to 128MB).

#ALGO=lz4
#PERCENT=50
SIZE=2048
#PRIORITY=100

NOTE: The compression algorithm can be changed to increase the compression or decrease the CPU usage. I’ll leave the default here. You can identify what algorithms are supported by your kernel by running cat /sys/block/zram0/comp_algorithm after you have booted with your ZRAM.

Reload to apply your changes. At this point you should see 2GB of swap when you run free -h. If you don’t see your swap space, check the status of the zramswap service to see what caused the failure.

$ free -h
               total        used        free      shared  buff/cache   available
Mem:           7.6Gi       141Mi       7.3Gi       1.0Mi       195Mi       7.4Gi
Swap:          2.0Gi          0B       2.0Gi
$ sudo systemctl status zramswap.service 
* zramswap.service - Linux zramswap setup
     Loaded: loaded (/lib/systemd/system/zramswap.service; enabled; vendor preset: enabled)
     Active: active (exited) since Fri 2023-06-16 14:08:33 MST; 20min ago
       Docs: man:zramswap(8)
    Process: 565 ExecStart=/usr/sbin/zramswap start (code=exited, status=0/SUCCESS)
   Main PID: 565 (code=exited, status=0/SUCCESS)
        CPU: 122ms

Jun 16 14:08:33 ... systemd[1]: Starting Linux zramswap setup...
Jun 16 14:08:33 ... root[575]: Starting Zram
Jun 16 14:08:33 ... zramswap[575]: <13>Jun 16 14:08:33 root: Starting Zram
Jun 16 14:08:33 ... zramswap[630]: Setting up swapspace version 1, size = 2 GiB (2147479552 bytes)
Jun 16 14:08:33 ... zramswap[630]: no label, UUID=142f8653-e64b-4985-9c89-c129397cea8a
Jun 16 14:08:33 ... systemd[1]: Finished Linux zramswap setup.
pi@rpi-kvm1:~ $ 

Ubuntu 23.04

I am currently running Ubuntu 23.04 on my Radxa Rock 5B, so I’ll use this as my example. First we need to make sure the ZRAM packages are installed:

sudo apt install zram-config zram-tools

After this is installed, then we can configure the /etc/defaults/zramswap just like we did with the Raspberry Pi. One difference though, my kernel didn’t have LZ4 support, so I needed to swap to LZO-RLE:

ALGO=lzo-rle
#PERCENT=50 
SIZE=2048 
#PRIORITY=100

After a reboot, we can check to see that the swap space was created. Check the status of the zramswap service to see what caused the error.

~$ free -h
               total        used        free      shared  buff/cache   available
Mem:           7.5Gi       692Mi       6.1Gi        45Mi       846Mi       6.8Gi
Swap:          2.0Gi          0B       2.0Gi
$ sudo systemctl status zramswap.service 
● zramswap.service - Linux zramswap setup
     Loaded: loaded (/lib/systemd/system/zramswap.service; enabled; preset: enabled)
     Active: active (exited) since Fri 2023-06-16 14:21:15 MST; 9min ago
       Docs: man:zramswap(8)
    Process: 916 ExecStart=/usr/sbin/zramswap start (code=exited, status=0/SUCCESS)
   Main PID: 916 (code=exited, status=0/SUCCESS)
        CPU: 32ms

Jun 16 14:21:15 ... systemd[1]: Starting zramswap.service - Linux zramswap setup...
Jun 16 14:21:15 ... zramswap[923]: <13>Jun 16 14:21:15 root: Starting Zram
Jun 16 14:21:15 ... root[923]: Starting Zram
Jun 16 14:21:15 ... zramswap[936]: Setting up swapspace version 1, size = 2 GiB (2147479552 bytes)
Jun 16 14:21:15 ... zramswap[936]: no label, UUID=14f66260-16d7-4031-894c-59245d11f6be
Jun 16 14:21:15 ... systemd[1]: Finished zramswap.service - Linux zramswap setup.

Debian 11 (Bullseye)

Debian only has a single package to install. I’m using my Rock 5B for testing again, so no LZ4 compression available (this simply depends on the kernel).

sudo apt install zram-tools

The /etc/defaults/zramswap is configured the same way:

ALGO=lzo-rle 
#PERCENT=50  
SIZE=2048  
#PRIORITY=100

Summary

Don’t forget to add some type of swap space! For SBC’s ZRAM is ideal to prevent writing to the SD card for swap. Raspberry Pi OS defaults to using a swap file, so disabling this and enabling ZRAM is ideal. For Ubuntu or other Linux based distributions, be sure to install the ZRAM tools and configure the service. If you are experiencing slow performance and system hangs (in particular in a GUI), be sure to check your swap configuration before you go any further!

If you are looking for more info on the Radxa Rock 5B, check out related articles.