Redis is an open source, in-memory key-value store, used as a database, cache, and message broker. Whatever your use case is, on a production we always need a high availability setup to make sure it always running. Redis have seperate instance to provide high availability called Redis Sentinel. Redis Sentinel provide Monitoring, Notification, and Automatic Failover. We need at least 3 Redis Sentinels to provide high availabilty.
Prerequisite
- 3 Ubuntu Machine
Sudo Privileges
Before start, we make sure that we will have no permission issue on the installation & configuration.
sudo su
Use Case
In this tutorial we will install redis with this setup:
____________ ____________
| | | |
| redis1 | | redis2 |
| sentinel1 |------------| sentinel2 |
|____________| | |____________|
master | replica
_____|______
| |
| redis3 |
| sentinel3 |
|____________|
replica
redis1 & sentinel1 is on the same machine with ip address: 192.168.5.100
redis2 & sentinel2 is on the same machine with ip address: 192.168.5.101
redis3 & sentinel3 is on the same machine with ip address: 192.168.5.102
Here we will config redis1 as master, redis2 & redis3 as replica.
Install The Latest Stable Redis
Let’s install the latest stable Redis first. Right now, the latest stable is Redis 6.2.5. In Ubuntu, you can install the latest stable version of Redis from the redislabs/redis package repository. Execute these command bellow on all 3 machine:
Add the repository to the apt index
add-apt-repository ppa:redislabs/redis
Update repository and install redis
apt update && apt install redis
By default, the redis service is not start on boot, you can activate them by using this command
systemctl enable redis-server
Setup Redis Master
Go to redis1 machine and open the redis configuration file
nano /etc/redis/redis.conf
Find these configuration, uncomment, and edit them
bind 0.0.0.0
requirepass "<your-password>" // change the password
masterauth "<your-password>" // change the password
Change <your-password> to use your own password, and you must use the same password for requirepass and masterauth on all 3 redis machines. The configuration bind 0.0.0.0 is needed so the redis can be accessed from other machine.
After that, restart the redis service
service redis-server restart
Setup Redis Replica
Open the redis configuration file
nano /etc/redis/redis.conf
Find these configuration, uncomment, and edit them
bind 0.0.0.0
requirepass "<your-password>" // change the password
replicaof 192.168.5.100 6379
masterauth "<your-password>" // change the password
Change <your-password> to use your own password, and you must use the same password for requirepass and masterauth on all 3 redis machine.
Notice that the difference between redis master and replica is just one configuration line: replicaof 192.168.5.100 6379. You can provide your master redis IP there.
After that, restart the redis service
service redis-server restart
Righ now, you will have redis with 1 master - 2 replicas.
Checking Master - Replica
You can check them by set some key in the redis master (in my case is redis1), and then the key should be available in the redis replica (in my case is redis2 & redis3).
Go to your master redis and run this command to enter the redis console
redis-cli
Authenticate yourself using the requirepass password you set in the redis configuration
auth <your-password>
Set sample key value pair to redis
SET foo bar
Now, go to your replica redis and run this command to enter the redis console
redis-cli
Authenticate yourself using the requirepass password you set in the redis configuration
auth <your-password>
Get the sample key you previously set on the master redis
GET foo
You will get bar
This is confirming that when you write on the master redis, it will be pushed to the replica redis.
Configure Redis Sentinel
Right now you have master - replica setup, but it is not enough, because when the master die, the replica not automatically replace the master. This is what the Sentinels are for.
First, create sentinel configuration file
nano /etc/redis/sentinel.conf
Copy paste this script on sentinel1
daemonize yes
port 26379
bind 0.0.0.0
supervised systemd
pidfile "/run/redis/redis-sentinel.pid"
logfile "/var/log/redis/sentinel.log"
sentinel monitor mymaster 127.0.0.1 6379 2
sentinel auth-pass mymaster <your-password>
sentinel down-after-milliseconds mymaster 5000
sentinel failover-timeout mymaster 60000
sentinel parallel-syncs mymaster 1
Copy paste this script on sentinel2 & sentinel3
daemonize yes
port 26379
bind 0.0.0.0
supervised systemd
pidfile "/run/redis/redis-sentinel.pid"
logfile "/var/log/redis/sentinel.log"
sentinel monitor mymaster 192.168.5.100 6379 2
sentinel auth-pass mymaster <your-password>
sentinel down-after-milliseconds mymaster 5000
sentinel failover-timeout mymaster 60000
sentinel parallel-syncs mymaster 1
Change the <your-password> with your requirepass password on redis configuration.
Notice that the difference between sentinel1 and other sentinel is just the ip of redis master. Because the sentinel1 is in the same machine as redis master, we can use 127.0.0.1 as redis master ip. For other sentinels, we must define the master ip that is 192.168.5.100.
After that, change the ownership of the file to redis
chown redis:redis /etc/redis/sentinel.conf
Create a systemd file
nano /etc/systemd/system/redis-sentinel.service
Copy & paste this script
[Unit]
Description=Redis Sentinel
After=network.target
[Service]
User=redis
Group=redis
Type=notify
ExecStart=/usr/bin/redis-server /etc/redis/sentinel.conf --sentinel
ExecStop=/usr/bin/redis-cli shutdown
Restart=always
[Install]
WantedBy=multi-user.target
Save & Exit, and then reload the daemon
systemctl daemon-reload
Start the sentinel service
service redis-sentinel start
Enable the service so the sentinel will start on boot
systemctl enable redis-sentinel
To see weather the sentinel is configured correctly, look at the sentinel1 log file
tail -f /var/log/redis/sentinel.log
You should see something like this
26139:X 02 Oct 2021 07:28:35.735 # +monitor master mymaster 127.0.0.1 6379 quorum 2
26139:X 02 Oct 2021 07:29:00.775 # +sdown master mymaster 127.0.0.1 6379
26139:X 02 Oct 2021 07:33:01.451 * +sentinel sentinel 8c0f32f16c057d906d18a15c6abea99d73ec509e 192.168.5.101 26379 @ mymaster 127.0.0.1 6379
26139:X 02 Oct 2021 07:33:30.524 * +sentinel sentinel bfc1a7ec8abbdf0f31c5d78737ba12f69c0e7fd7 192.168.5.102 26379 @ mymaster 127.0.0.1 6379
The sentinel1 will monitor the redis master on 127.0.0.1 and also detect other sentinel from other machine. In my case it is from 192.168.5.101 and 192.168.5.102.
Now, if you kill the master machine, the sentinels will choose and replace the master with one of the replica automatically.
Congratulation! Now you can setup high availability Redis using sentinel on Ubuntu!