Per-user Redis on shared hosting—and the maxmemory bug that hung sites

On a shared hosting server, letting every site use the same Redis instance leads to the classic problems: one site floods it with keys and other sites suffer evictions, cached data gets mixed across tenants, and one FLUSHALL hurts the whole server. So I designed it as one Redis instance per user, with resources controlled by systemd cgroups — after running it for a while, I hit a bug that I want to cover in the second half of this article.

The usernames and tier numbers in this article are examples. The structure and config are real.

Structure: one systemd template unit for every user

# /etc/systemd/system/[email protected] (ตัดมาเฉพาะส่วนสำคัญ)
[Unit]
Description=Redis per-user (%i)

[Service]
User=%i
Group=%i
ExecStart=/usr/bin/redis-server /usr/local/directadmin/data/users/%i/redis.conf
# กำแพงทรัพยากรจริงอยู่ตรงนี้ — cgroup v2
MemoryHigh=150M
MemoryMax=200M
CPUQuota=50%

Each user's redis.conf — the key point is that there is no TCP port open at all; it uses a UNIX socket in the user's own home directory:

port 0
unixsocket /home/webuser/.redis/redis.sock
unixsocketperm 700
maxmemory 120mb
maxmemory-policy allkeys-lru
# กันยิงคำสั่งอันตรายจากฝั่งแอป
rename-command FLUSHALL ""
rename-command FLUSHDB ""
rename-command CONFIG ""

That gives full isolation: each site has its own instance, socket file permissions of 700 keep others out, and PHP code cannot meddle with CONFIG SET because it has been renamed away (changing the config requires editing the file + restarting).

Then one day, a WooCommerce site hung

One customer reported that clicking to publish a product in WordPress brought up:

RedisException: socket error on read socket

It happened intermittently, not all the time. Restarting Redis fixed it briefly, then it came back — this kind of "comes and goes" behavior is usually a resource problem. Here is what was actually running:

systemctl status redis@webuser
# Memory: 149.8M (high: 150.0M max: 200.0M)  <- ชนเพดาน high ตลอดเวลา

grep maxmemory /usr/local/directadmin/data/users/webuser/redis.conf
# maxmemory 1gb    <- ต้นเหตุ

See it yet? — Redis was told it had 1GB, but the cgroup actually gave it only 150MB

How it hangs: two numbers that do not talk to each other

With the conf pretending there is 1GB available, Redis does not evict anything (it has not reached its own limit). Memory usage keeps growing until it hits the cgroup's MemoryHigh → the kernel hits the brakes on the whole process → PHP cannot wait any longer for a response → that is the "socket error on read socket".

The rule I arrived at: maxmemory must not exceed 80% of cgroup high

Leave 20% for Redis's own overhead (memory allocation during fork, buffers, etc.):

MemoryHigh 150M  ->  maxmemory 120mb
MemoryHigh  50M  ->  maxmemory  40mb

This way, Redis always evicts on its own before the kernel steps in — the site gets a cache miss (a little slower) instead of a socket timeout (the whole page breaks).

Preventing recurrence with a guard script

Fixing one server is not enough. I need to prevent this across the system — cron runs every 15 minutes, checking every instance:

#!/bin/bash
# redis-tier-guard: บังคับ conf maxmemory <= 80% cgroup + restart ตัวที่กำลังโดน throttle
for u in $(systemctl list-units 'redis@*' --state=running --no-legend | sed 's/redis@\([^.]*\).*/\1/'); do
  CONF=/usr/local/directadmin/data/users/$u/redis.conf
  HIGH=$(systemctl show redis@$u -p MemoryHigh --value)         # bytes
  [ "$HIGH" = "infinity" ] && continue
  CAP=$((HIGH * 80 / 100 / 1048576))                             # MB ที่อนุญาต
  CUR=$(grep -oP '^maxmemory \K[0-9]+' <<< "$(grep -i '^maxmemory ' $CONF | sed 's/mb//i')")
  if [ "${CUR:-0}" -gt "$CAP" ]; then
    sed -i "s/^maxmemory .*/maxmemory ${CAP}mb/i" $CONF
    systemctl restart redis@$u
    echo "[FIX] $u: maxmemory ${CUR}mb -> ${CAP}mb (cgroup high $((HIGH/1048576))M)"
    # ตรงนี้ต่อ alert เข้า Telegram/อีเมลของทีมได้
  fi
  # ตัวที่ใช้จริงเกิน 90% ของ high = กำลังจะโดน throttle -> restart ระบาย
  USED=$(systemctl show redis@$u -p MemoryCurrent --value)
  [ "$USED" != "" ] && [ "$USED" -gt $((HIGH * 90 / 100)) ] && systemctl restart redis@$u
done

# /etc/cron.d/redis-tier-guard
*/15 * * * * root /usr/local/sbin/redis-tier-guard.sh >> /var/log/redis-tier-guard.log 2>&1

One decision I spent a long time considering was "why not just alert and let someone fix it?" — the answer is that leaving this alone means a customer's site hangs in the middle of the night, while an auto-fix means restarting the cache once (losing a little cached data). Compared side by side, the auto-fix costs far less. Whenever an instance drifts outside its tier, the system needs to bring itself back into working order.

Personally, I consider this 80% rule applicable to every service running under a cgroup, not just Redis — the number inside the app must always stay below the boundary outside it, or the app will never know it is being squeezed. If you want to use this setup, adjust the tier numbers to suit your server. If I have missed anything in the explanation or made a typo, I will come back and correct it later.