Redis Sentinel 高可用

李王贵收录于类别 redis

2021-03-28 2021-03-28 约 2168 字预计阅读 5 分钟次阅读条评论

Redis Replication

Replication 原理

副本库通过 slaveof 127.0.0.1 6380 命令, 连接主库, 并发送 SYNC 给主库
主库收到 SYNC, 会立即触发 BGSAVE, 后台保存 RDB, 发送给副本库
副本库接收后会应用 RDB 快照
主库会陆续将中间产生的新的操作,保存并发送给副本库
到此,我们主复制集就正常工作了
再此以后, 主库只要发生新的操作, 都会以命令传播的形式自动发送给副本库.
所有复制相关信息, 从 info 信息中都可以查到. 即使重启任何节点, 他的主从关系依然都在.
如果发生主从关系断开时,从库数据没有任何损坏, 在下次重连之后, 从库发送 PSYNC 给主库
主库只会将从库缺失部分的数据同步给从库应用, 达到快速恢复主从的目的

准备多实例 Redis 服务

实验采用单机多实例的方式进行，创建三个实例。端口 6380-6382, 可以使用 redis 源码包中的脚本（install_server.sh）创建

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78


[[email protected] utils]# bash install_server.sh
Welcome to the redis service installer
This script will help you easily set up a running redis server

Please select the redis port for this instance: [6379] 6380
Please select the redis config file name [/data/redis/etc/6380.conf]
Selected default - /data/redis/etc/6380.conf
Please select the redis log file name [/data/redis/log/redis_6380.log]
Selected default - /data/redis/log/redis_6380.log
Please select the data directory for this instance [/data/redis/6380]
Selected default - /data/redis/6380
Please select the redis executable path [/usr/local/bin/redis-server]
Selected config:
Port           : 6380
Config file    : /data/redis/etc/6380.conf
Log file       : /data/redis/log/redis_6380.log
Data dir       : /data/redis/6380
Executable     : /usr/local/bin/redis-server
Cli Executable : /usr/local/bin/redis-cli
Is this ok? Then press ENTER to go on or Ctrl-C to abort.
Copied /tmp/6380.conf => /etc/init.d/redis_6380
Installing service...
Successfully added to chkconfig!
Successfully added to runlevels 345!
Starting Redis server...
Installation successful!
[[email protected] utils]# bash install_server.sh
Welcome to the redis service installer
This script will help you easily set up a running redis server

Please select the redis port for this instance: [6379] 6381
Please select the redis config file name [/data/redis/etc/6381.conf]
Selected default - /data/redis/etc/6381.conf
Please select the redis log file name [/data/redis/log/redis_6381.log]
Selected default - /data/redis/log/redis_6381.log
Please select the data directory for this instance [/data/redis/6381]
Selected default - /data/redis/6381
Please select the redis executable path [/usr/local/bin/redis-server]
Selected config:
Port           : 6381
Config file    : /data/redis/etc/6381.conf
Log file       : /data/redis/log/redis_6381.log
Data dir       : /data/redis/6381
Executable     : /usr/local/bin/redis-server
Cli Executable : /usr/local/bin/redis-cli
Is this ok? Then press ENTER to go on or Ctrl-C to abort.
Copied /tmp/6381.conf => /etc/init.d/redis_6381
Installing service...
Successfully added to chkconfig!
Successfully added to runlevels 345!
Starting Redis server...
Installation successful!
[[email protected] utils]# bash install_server.sh
Welcome to the redis service installer
This script will help you easily set up a running redis server

Please select the redis port for this instance: [6379] 6382
Please select the redis config file name [/data/redis/etc/6382.conf]
Selected default - /data/redis/etc/6382.conf
Please select the redis log file name [/data/redis/log/redis_6382.log]
Selected default - /data/redis/log/redis_6382.log
Please select the data directory for this instance [/data/redis/6382]
Selected default - /data/redis/6382
Please select the redis executable path [/usr/local/bin/redis-server]
Selected config:
Port           : 6382
Config file    : /data/redis/etc/6382.conf
Log file       : /data/redis/log/redis_6382.log
Data dir       : /data/redis/6382
Executable     : /usr/local/bin/redis-server
Cli Executable : /usr/local/bin/redis-cli
Is this ok? Then press ENTER to go on or Ctrl-C to abort.
Copied /tmp/6382.conf => /etc/init.d/redis_6382
Installing service...
Successfully added to chkconfig!
Successfully added to runlevels 345!
Starting Redis server...
Installation successful!

本例为了使用自定义的目录存放 redis 数据及配置文件对 install_server.sh 脚本中的路径做了修改

Replication 配置

6380 为主库，6381-6382 为从库，配置如下:

主库配置

1
2
3
4
5
6
7
8


[[email protected] log]# redis-cli -p 6380
127.0.0.1:6381> config set requirepass 123
OK
127.0.0.1:6381> auth 123
OK
127.0.0.1:6381> config set masterauth 123
OK
127.0.0.1:6381> CONFIG REWRITE

从库配置

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23


[[email protected] log]# redis-cli -p 6381
127.0.0.1:6381> config set requirepass 123
OK
127.0.0.1:6381> auth 123
OK
127.0.0.1:6381> config set masterauth 123
OK
127.0.0.1:6381> CONFIG REWRITE
OK
127.0.0.1:6381> slaveof 127.0.0.1 6380
OK

[[email protected] log]# redis-cli -p 6382
127.0.0.1:6381> config set requirepass 123
OK
127.0.0.1:6381> auth 123
OK
127.0.0.1:6381> config set masterauth 123
OK
127.0.0.1:6381> CONFIG REWRITE
OK
127.0.0.1:6381> slaveof 127.0.0.1 6380
OK

如果想解除主从关系可以使用 slaveof no one 指令

检查 Replication 状态

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28


[[email protected] etc]# redis-cli -p 6380 -a 123 info replication
# Replication
role:master
connected_slaves:2
slave0:ip=127.0.0.1,port=6381,state=online,offset=3442,lag=1
slave1:ip=127.0.0.1,port=6382,state=online,offset=3442,lag=1
master_repl_offset:3575
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:2
repl_backlog_histlen:3574
[[email protected] etc]# redis-cli -p 6381 -a 123 info replication
# Replication
role:slave
master_host:127.0.0.1
master_port:6380
master_link_status:up
master_last_io_seconds_ago:0
master_sync_in_progress:0
slave_repl_offset:6291
slave_priority:100
slave_read_only:1
connected_slaves:0
master_repl_offset:0
repl_backlog_active:0
repl_backlog_size:1048576
repl_backlog_first_byte_offset:0
repl_backlog_histlen:0

可以在主库写入数据然后到从库读取测试同步

Redis Sentinel 高可用

由于 redis replication 默认状态下如果主库宕机，是不会自动切换主从身份的，需要手动干预，这是生产环境下不允许的。为了解决此问题 redis 引入哨兵模式。

redis sentinel 有以下作用：

监控节点
自动选主，切换主从身份
从库自动指向新的主库
对应用透明
自动处理故障节点

配置 sentinel

准备 sentinel 配置文件

源码包中有示例配置文件可用，直接复制修改即可

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22


mkdir /data/sentinel
cat > /data/sentinel/26379.conf <<EOF
protected-mode no
daemonize yes
logfile "/data/sentinel/26379.log"
port 26379
dir "/tmp"
sentinel myid 994ea01af0f13692c13eeda116a0668084bb5e68

# mymaster 是一个自定义名称，客户端通过 sentinel 连接 redis 时需要使用
# 127.0.0.1 6380  是主节点的 ip 和 端口号
# 1 是 sentinel failover 投票数，默认为 sentinel 节点数/2 + 1, 1个节点时为1
# 为了能正常投票得出结果 sentinel 节点数得为奇数个
sentinel monitor mymaster 127.0.0.1 6380 1
sentinel down-after-milliseconds mymaster 5000
sentinel auth-pass mymaster 123
sentinel config-epoch mymaster 3
sentinel leader-epoch mymaster 3
sentinel known-slave mymaster 127.0.0.1 6382
sentinel known-slave mymaster 127.0.0.1 6381
sentinel current-epoch 3
EOF

启动 sentinel 服务

1

/usr/local/bin/redis-sentinel /data/sentinel/26379.conf

redis-sentinel 是 redis-server 的软链接

连接测试 sentinel

sentinel 管理命令

1
2
3
4
5
6
7


[[email protected] sentinel]# redis-cli -p 26379
# 列出所有被监视的主服务器
127.0.0.1:26379> SENTINEL masters
# 列出所有被监视的从服务器
127.0.0.1:26379> SENTINEL slaves <master name>
# 强制开启 主从切换, 危险
127.0.0.1:26379> SENTINEL failover <master name>

测试主从切换

模拟故障停止主库

1

[[email protected] etc]# redis-cli -p 6380 -a 123 shutdown

查看 sentinel 日志

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20


3963:X 28 Mar 15:31:13.864 # +sdown master mymaster 127.0.0.1 6380
3963:X 28 Mar 15:31:13.864 # +odown master mymaster 127.0.0.1 6380 #quorum 1/1
3963:X 28 Mar 15:31:13.864 # +new-epoch 6
3963:X 28 Mar 15:31:13.864 # +try-failover master mymaster 127.0.0.1 6380
3963:X 28 Mar 15:31:13.864 # +vote-for-leader 994ea01af0f13692c13eeda116a0668084bb5e68 6
3963:X 28 Mar 15:31:13.864 # +elected-leader master mymaster 127.0.0.1 6380
3963:X 28 Mar 15:31:13.864 # +failover-state-select-slave master mymaster 127.0.0.1 6380
3963:X 28 Mar 15:31:13.948 # +selected-slave slave 127.0.0.1:6382 127.0.0.1 6382 @ mymaster 127.0.0.1 6380
3963:X 28 Mar 15:31:13.948 * +failover-state-send-slaveof-noone slave 127.0.0.1:6382 127.0.0.1 6382 @ mymaster 127.0.0.1 6380
3963:X 28 Mar 15:31:14.007 * +failover-state-wait-promotion slave 127.0.0.1:6382 127.0.0.1 6382 @ mymaster 127.0.0.1 6380
3963:X 28 Mar 15:31:14.943 # +promoted-slave slave 127.0.0.1:6382 127.0.0.1 6382 @ mymaster 127.0.0.1 6380
3963:X 28 Mar 15:31:14.943 # +failover-state-reconf-slaves master mymaster 127.0.0.1 6380
3963:X 28 Mar 15:31:15.004 * +slave-reconf-sent slave 127.0.0.1:6381 127.0.0.1 6381 @ mymaster 127.0.0.1 6380
3963:X 28 Mar 15:31:15.994 * +slave-reconf-inprog slave 127.0.0.1:6381 127.0.0.1 6381 @ mymaster 127.0.0.1 6380
3963:X 28 Mar 15:31:15.994 * +slave-reconf-done slave 127.0.0.1:6381 127.0.0.1 6381 @ mymaster 127.0.0.1 6380
3963:X 28 Mar 15:31:16.055 # +failover-end master mymaster 127.0.0.1 6380
3963:X 28 Mar 15:31:16.055 # +switch-master mymaster 127.0.0.1 6380 127.0.0.1 6382
3963:X 28 Mar 15:31:16.055 * +slave slave 127.0.0.1:6381 127.0.0.1 6381 @ mymaster 127.0.0.1 6382
3963:X 28 Mar 15:31:16.055 * +slave slave 127.0.0.1:6380 127.0.0.1 6380 @ mymaster 127.0.0.1 6382
3963:X 28 Mar 15:31:21.064 # +sdown slave 127.0.0.1:6380 127.0.0.1 6380 @ mymaster 127.0.0.1 6382

可以看到 sentinel 检测到主库服务停止，将 6382 选为主库，进行了切换操作， 6381 也自动与 6382 建立主从关系

6380 修复好后，重新启动服务 sentinel 会自动检测到并 6380 加入主从环境中

1
2


[[email protected] etc]# /etc/init.d/redis_6380 start
Starting Redis server...

此时可以从 sentinel 日志中看到以下信息

1
2


3963:X 28 Mar 15:35:05.234 # -sdown slave 127.0.0.1:6380 127.0.0.1 6380 @ mymaster 127.0.0.1 6382
3963:X 28 Mar 15:35:15.229 * +convert-to-slave slave 127.0.0.1:6380 127.0.0.1 6380 @ mymaster 127.0.0.1 6382

sentinel 会自动在 6380 实例的配置文件最后一行加入一行配置

1
2
3
4


[[email protected] etc]# tail -n 3 6380.conf
requirepass "123"
masterauth "123"
slaveof 127.0.0.1 6382

配置多节点 sentinel

此时 redis 的高可用解决了，但 sentinel 只有一个节点存在单点故障，为了解决这个问题，可以通过部署多个 sentinel 节点解决。配置多个 sentinel 节点时需要注意节点数据及配置文件中的 sentinel monitor mymaster 127.0.0.1 6380 1 配置项的最后一个值的配置。

客户端连接 redis

以 python 客户端为例说明, 参考文档: https://pypi.org/project/redis/

安装 redis 库

1

pip install redis

连接至单实例 reids

1
2
3
4
5
6


>>> import redis
>>> r = redis.Redis(host='localhost', port=6379, db=0)
>>> r.set('foo', 'bar')
True
>>> r.get('foo')
b'bar'

Sentinel 模式

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12


>>> from redis.sentinel import Sentinel
>>> sentinel = Sentinel([('localhost', 26379)], socket_timeout=0.1)
>>> sentinel.discover_master('mymaster')
('127.0.0.1', 6379)
>>> sentinel.discover_slaves('mymaster')
[('127.0.0.1', 6380)]

>>> master = sentinel.master_for('mymaster', socket_timeout=0.1, password='123')
>>> slave = sentinel.slave_for('mymaster', socket_timeout=0.1, password='123')
>>> master.set('foo', 'bar')
>>> slave.get('foo')
b'bar'