우분투 24.04에서 Redis Cluster를 구성하는 방법

우분투 24.04에서 Redis Cluster를 구성하는 방법

구성 환경

서버 이름	아이피 주소	구분	비고
	192.168.0.231	VIP	Redis Port 6379
cache-232	192.168.0.232	redis-server, haproxy, keepalived	Redis Port 6381, 6382
cache-233	192.168.0.233	redis-server, haproxy, keepalived	Redis Port 6381, 6382
cache-234	192.168.0.234	redis-server	Redis Port 6381, 6382

1. 필요 패키지 설치

sudo apt update
sudo apt install -y curl gnupg lsb-release

2. 커널/시스템 튜닝

Swap 비활성화

• Redis는 메모리 기반 DB라 swap이 발생하면 지연이 심해집니다.

sudo swapoff -a

Overcommit Memory

• Redis는 fork()로 RDB/AOF 스냅샷을 뜨는데, Linux 기본 overcommit_memory=0이면 fork 실패 가능성이 있습니다.

echo "vm.overcommit_memory=1" | sudo tee -a /etc/sysctl.conf

sudo sysctl -p

Transparent Huge Pages (THP) 비활성화

• THP는 큰 페이지 메모리 할당 시 성능에 도움을 주지만, Redis에서는 성능 저하/지연을 유발합니다.

cat /sys/kernel/mm/transparent_hugepage/enabled

echo never | sudo tee /sys/kernel/mm/transparent_hugepage/enabled
echo "echo never > /sys/kernel/mm/transparent_hugepage/enabled" | sudo tee -a /etc/rc.local
sudo chmod +x /etc/rc.local

파일 디스크립터 제한

cat <<EOF | sudo tee -a /etc/security/limits.conf

*       soft    nproc   unlimited
*       hard    nproc   unlimited
*       soft    nofile  65536
*       hard    nofile  65536
EOF

TCP 커널 파라미터

• 네트워크 튜닝으로 동시 연결 안정성 강화

cat <<EOF | sudo tee -a /etc/sysctl.conf

# Redis kernel tuning
net.core.somaxconn=65535
net.ipv4.tcp_max_syn_backlog=8192
net.ipv4.tcp_tw_reuse=1
net.ipv4.tcp_fin_timeout=15
EOF

sudo sysctl -p

3-1. Redis Server 설치

curl -fsSL https://packages.redis.io/gpg | sudo gpg --dearmor -o /usr/share/keyrings/redis-archive-keyring.gpg

echo "deb [signed-by=/usr/share/keyrings/redis-archive-keyring.gpg] https://packages.redis.io/deb $(lsb_release -cs) main" | sudo tee /etc/apt/sources.list.d/redis.list

sudo apt update
sudo apt install -y redis-server

redis-server --version

Redis server v=8.2.1 sha=00000000:1 malloc=jemalloc-5.3.0 bits=64 build=a29002d62758b90f

sudo systemctl disable --now redis-server

3-2. Redis Server 설정

설정 디렉토리

sudo mkdir -p /etc/redis/{6381,6382}

sudo cp /etc/redis/redis.conf /etc/redis/6381/redis.conf

sudo cp /etc/redis/redis.conf /etc/redis/6382/redis.conf

---

cache-232-6381[Port 6381]

cat <<EOF | sudo tee /etc/redis/6381/redis.conf > /dev/null
# 모듈(사용하는 경우만)
loadmodule /usr/lib/redis/modules/rejson.so
loadmodule /usr/lib/redis/modules/redisbloom.so
loadmodule /usr/lib/redis/modules/redistimeseries.so
loadmodule /usr/lib/redis/modules/redisearch.so

# 네트워크
bind 0.0.0.0
port 6381

daemonize no

protected-mode yes
requirepass user_redis_password

# 클러스터 설정
cluster-enabled yes
cluster-config-file nodes.conf
cluster-node-timeout 5000
appendonly yes

# 로그 및 데이터
dir /var/lib/redis/6381
logfile /var/log/redis/6381/redis-server.log
pidfile /run/redis/6381/redis-server.pid
unixsocket /run/redis/6381/redis-server.sock

loglevel debug

maxmemory 512mb
maxmemory-policy volatile-lru

auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mb

save 900 1
save 300 10
save 60 10000
EOF

sudo ln -s /etc/redis/6381/redis.conf /etc/redis/redis-6381.conf

cache-232-6382[Port 6382]

cat <<EOF | sudo tee /etc/redis/6382/redis.conf > /dev/null
# 모듈(사용하는 경우만)
loadmodule /usr/lib/redis/modules/rejson.so
loadmodule /usr/lib/redis/modules/redisbloom.so
loadmodule /usr/lib/redis/modules/redistimeseries.so
loadmodule /usr/lib/redis/modules/redisearch.so

# 네트워크
bind 0.0.0.0
port 6382

daemonize no

protected-mode yes
requirepass user_redis_password

# 클러스터 설정
cluster-enabled yes
cluster-config-file nodes.conf
cluster-node-timeout 5000
appendonly yes

# 로그 및 데이터
dir /var/lib/redis/6382
logfile /var/log/redis/6382/redis-server.log
pidfile /run/redis/6382/redis-server.pid
unixsocket /run/redis/6382/redis-server.sock

loglevel debug

maxmemory 512mb
maxmemory-policy volatile-lru

auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mb

save 900 1
save 300 10
save 60 10000
EOF

sudo ln -s /etc/redis/6382/redis.conf /etc/redis/redis-6382.conf

---

sudo chown -R redis:redis /etc/redis/{6381,6382}

sudo chmod 644 /etc/redis/{6381,6382}/redis.conf

3-3. 디렉토리 생성 및 권한 설정

sudo mkdir -p /var/lib/redis/{6381,6382}

sudo mkdir -p /var/log/redis/{6381,6382}

sudo mkdir -p /run/redis/{6381,6382}

sudo chown -R redis:redis /var/lib/redis /var/log/redis /run/redis

3-4. Redis Server 서비스 등록(systemd)

템플릿 서비스 파일 생성

---

cat /usr/lib/systemd/system/redis-server@.service

# Templated service file for redis-server(1)
#
# Each instance of redis-server requires its own configuration file:
#
#   $ cp /etc/redis/redis.conf /etc/redis/redis-myname.conf
#   $ chown redis:redis /etc/redis/redis-myname.conf
#
# Ensure each instance is using their own database:
#
#   $ sed -i -e 's@^dbfilename .*@dbfilename dump-myname.rdb@' /etc/redis/redis-myname.conf
#
# We then listen exlusively on UNIX sockets to avoid TCP port collisions:
#
#   $ sed -i -e 's@^port .*@port 0@' /etc/redis/redis-myname.conf
#   $ sed -i -e 's@^\(# \)\{0,1\}unixsocket .*@unixsocket /run/redis-myname/redis-server.sock@' /etc/redis/redis-myname.conf
#
# ... and ensure we are logging, etc. in a unique location:
#
#   $ sed -i -e 's@^logfile .*@logfile /var/log/redis/redis-server-myname.log@' /etc/redis/redis-myname.conf
#   $ sed -i -e 's@^pidfile .*@pidfile /run/redis-myname/redis-server.pid@' /etc/redis/redis-myname.conf
#
# We can then start the service as follows, validating we are using our own
# configuration:
#
#   $ systemctl start redis-server@myname.service
#   $ redis-cli -s /run/redis-myname/redis-server.sock info | grep config_file
#
#  -- Chris Lamb <lamby@debian.org>  Mon, 09 Oct 2017 22:17:24 +0100
[Unit]
Description=Advanced key-value store (%I)
After=network.target
Documentation=http://redis.io/documentation, man:redis-server(1)

[Service]
Type=notify
ExecStart=/usr/bin/redis-server /etc/redis/redis-%i.conf
TimeoutStopSec=0
Restart=always
User=redis
Group=redis
RuntimeDirectory=redis-%i
RuntimeDirectoryMode=2755

UMask=007
PrivateTmp=yes
LimitNOFILE=65535
PrivateDevices=yes
ProtectHome=yes
ReadOnlyDirectories=/
ReadWriteDirectories=-/var/lib/redis
ReadWriteDirectories=-/var/log/redis
ReadWriteDirectories=-/run/redis-%i

NoNewPrivileges=true
CapabilityBoundingSet=CAP_SYS_RESOURCE
RestrictAddressFamilies=AF_INET AF_INET6 AF_UNIX
MemoryDenyWriteExecute=true
ProtectKernelModules=true
ProtectKernelTunables=true
ProtectControlGroups=true
RestrictRealtime=true
RestrictNamespaces=true

# redis-server can write to its own config file when in cluster mode so we
# permit writing there by default. If you are not using this feature, it is
# recommended that you replace the following lines with "ProtectSystem=full".
ProtectSystem=true
ReadWriteDirectories=-/etc/redis

[Install]
WantedBy=multi-user.target

---

cat <<'EOF' | sudo tee /etc/systemd/system/redis@.service > /dev/null
[Unit]
Description=Advanced key-value store (%I)
After=network.target
Documentation=http://redis.io/documentation, man:redis-server(1)

[Service]
Type=notify
ExecStart=/usr/bin/redis-server /etc/redis/%i/redis.conf --supervised systemd
TimeoutStopSec=0
Restart=always
User=redis
Group=redis
ExecStop=/usr/bin/redis-cli -p %i shutdown

LimitNOFILE=65535

[Install]
WantedBy=multi-user.target
EOF

서비스 등록 및 실행

Redis Server 서비스 시작

sudo systemctl daemon-reload

• cache-232-6381[Port 6381]

sudo systemctl enable --now redis@6381

• cache-232-6382[Port 6382]

sudo systemctl enable --now redis@6382

상태 확인

systemctl status redis@6381

systemctl status redis@6382

sudo -u redis /usr/bin/redis-server /etc/redis/6381/redis.conf --loglevel debug

redis-cli -s /run/redis/6381/redis-server.sock info | grep config_file

sudo netstat -tulnp | grep -E '6381|6382|16381|16382'

4-1. Redis Cluster 생성

---

redis-cli 설치

sudo apt install -y redis-tools

---

클러스터 생성

export REDISCLI_AUTH=user_redis_password

redis-cli -p 6381 ping

redis-cli --cluster create \
192.168.0.232:6381 \
192.168.0.232:6382 \
192.168.0.233:6381 \
192.168.0.233:6382 \
192.168.0.234:6381 \
192.168.0.234:6382 \
--cluster-replicas 1

• --cluster-replicas 1 : 각 마스터당 1개의 슬레이브를 배치
• 명령 실행 시 확인 메시지에 yes 입력

4-2. 클러스터 상태 확인

redis-cli -c -p 6381 cluster nodes

728x90

Redis 서버 패키지 삭제

sudo apt purge -y redis-server redis-tools

5. HAProxy 설정(cache232, cache233)

HAProxy 설치

sudo apt install -y haproxy

HAProxy 시작

sudo systemctl enable --now haproxy

HAProxy 설정

sudo vim /etc/haproxy/haproxy.cfg

global
    log /dev/log local0
    log /dev/log local1 notice
    chroot /var/lib/haproxy
    stats socket /run/haproxy/admin.sock mode 660 level admin expose-fd listeners
    stats timeout 30s
    user haproxy
    group haproxy
    daemon
    maxconn 4096

defaults
    log global
    mode tcp
    option tcplog
    option dontlognull
    timeout connect 5s
    timeout client 50s
    timeout server 50s

# Redis Cluster VIP Frontend
frontend redis_front
    bind *:6379
    mode tcp
    default_backend redis_cluster

# Redis Cluster Backend
backend redis_cluster
    mode tcp
    balance roundrobin
    option tcp-check

    # Health-check (PING → PONG → QUIT)
    tcp-check send PING\r\n
    tcp-check expect string +PONG
    tcp-check send QUIT\r\n
    tcp-check expect string +OK

    # Redis 노드들 등록
    server cache-232-6381 192.168.0.232:6381 check inter 2s fall 3 rise 2
    server cache-232-6382 192.168.0.232:6382 check inter 2s fall 3 rise 2
    server cache-233-6381 192.168.0.233:6381 check inter 2s fall 3 rise 2
    server cache-233-6382 192.168.0.233:6382 check inter 2s fall 3 rise 2
    server cache-234-6381 192.168.0.234:6381 check inter 2s fall 3 rise 2
    server cache-234-6382 192.168.0.234:6382 check inter 2s fall 3 rise 2

6. Keepalived 설정

Keepalived 설치

sudo apt install -y keepalived

Keepalived 시작

sudo systemctl enable --now keepalived

chk_haproxy 스크립트 생성

cat <<EOF | sudo tee /usr/bin/chk_haproxy.sh > /dev/null
#!/bin/bash

# HAProxy 프로세스 확인
if echo "show info" | socat unix-connect:/run/haproxy/admin.sock stdio | grep -q "Nbproc"; then
    exit 0	# HAProxy가 실행 중이면 성공 (0)
else
    exit 1	# HAProxy가 실행 중이 아니면 실패 (1)
fi

sudo chmod +x /usr/bin/chk_haproxy.sh

네트워크 인터페이스 확인

ip addr show

ip link show

Keepalived 설정

• cache-232(MASTER)

sudo vim /etc/keepalived/keepalived.conf

vrrp_script chk_haproxy {
    script "/usr/bin/chk_haproxy.sh"
    interval 2
    weight 2
    fall 2
    rise 2
}

vrrp_instance VI_1 {
    interface eno1
    state MASTER
    virtual_router_id 51
    priority 101
    advert_int 1

    authentication {
        auth_type PASS
        auth_pass redis_pass
    }

    virtual_ipaddress {
        192.168.0.231/24
    }
    
    track_script {
        chk_haproxy
    }
}

• cache-233(BACKUP)

vrrp_script chk_haproxy {
    script "/usr/bin/chk_haproxy.sh"
    interval 2
    weight 2
    fall 2
    rise 2
}

vrrp_instance VI_1 {
    interface eno1
    state BACKUP
    virtual_router_id 51
    priority 100
    advert_int 1

    authentication {
        auth_type PASS
        auth_pass redis_pass
    }

    virtual_ipaddress {
        192.168.0.231/24
    }

    track_script {
        chk_haproxy
    }
}

HAProxy 및 Keepalived 서비스 재시작 및 확인

• HAProxy 재시작

sudo systemctl restart haproxy

• Keepalived 재시작

sudo systemctl restart keepalived

7. VIP 동작 확인

VIP가 정상적으로 할당되었는지 확인

ip addr show

HAProxy 상태 확인

sudo systemctl status haproxy

스크립트 테스트

• 0이 나와야 정상

/usr/bin/chk_haproxy.sh

echo $?

8. 클러스터 상태 확인

클러스터 상태 확인

redis-cli --cluster check 192.168.0.231:6379

노드 정보 확인

redis-cli -h 192.168.0.231 -p 6379 cluster nodes

클러스터 정보 확인

redis-cli -h 192.168.0.231 -p 6379 cluster info

9. 테스트

클러스터 모드 동작 확인

redis-cli -c -h 192.168.0.232 -p 6381 set mykey1 "Hello, Redis Cluster!"

redis-cli -c -h 192.168.0.234 -p 6382 get mykey1

VIP 연결 확인

redis-cli -h 192.168.0.231 -p 6379 set mykey2 "Hello, Redis VIP!"

redis-cli -h 192.168.0.231 -p 6379 set mykey2

 

참고URL

- 변군이글루 : 우분투 24.04에 최신 버전의 Redis 서버를 설치하는 방법

- 변군이글루 : 우분투 24.04에서 rc.local을 활성화하는 방법

- 변군이글루 : 우분투에서 스왑 메모리를 비활성화하는 방법

- 변군이글루 : HAProxy를 사용하여 고가용성(High Availability) 구성을 구현하는 방법

- 변군이글루 : HAProxy에서 TLS/SSL 인증서를 생성하고 적용하는 방법