redis-cluster cluster
redis-cluster cluster
In October 2018, Redis released the stable version 5.0, introducing various new features. One of them is to abandon the Ruby clustering method and change it to the redis-cli method written in C language. The cluster construction method is greatly complicated. reduce.
In versions prior to redis3.0, clustering was generally implemented using the sentinel tool to monitor the status of the master node. If the master node was abnormal, a master-slave switch would be performed and a certain slave would be used as the master. The configuration of the sentinel was slightly complicated, and the performance Performance in various aspects such as high availability is average, especially when access is interrupted at the moment of master-slave switching. Moreover, the sentinel mode has only one master node to provide services to the outside world, which cannot support high concurrency, and the memory of a single master node is not suitable. Set it too large, otherwise the persistent file will be too large, affecting the efficiency of data recovery or master-slave synchronization.
Codes +redis
Redis data partition
Redis cluster uses virtual slot partitioning. All keys are mapped to 0~16383 integer slots according to the hash function. The calculation formula is: Slot=CRC16(key)&16384. Each node is responsible for maintaining a part of the slots and the key value data mapped by the slots.
Each node in a redis cluster is responsible for a subset of hash slots, for example you might have a cluster of 3 nodes with:
Node A contains hash slots from 0 to 5500.
Node B contains hash slots from 5501 to 11000.
Node C contains hash slots from 11001 to 16383.
Redis-cluster construction
The Redis cluster requires at least 3 nodes, because the voting fault tolerance mechanism requires more than half of the nodes to think that a node is down before the node is down, so 2 nodes cannot form a cluster.
To ensure the high availability of the cluster, each node needs to have a slave node, that is, a backup node, so the Redis cluster requires at least 6 servers.
We build three master nodes here, and build a slave node for each master, three masters and three slaves.
Analysis of cluster election principle (ping-pong mechanism)
Election process:
When the slave finds that its master has become FAIL, it attempts to initiate an election in order to become the new master. Since the failed master may have multiple slaves, there is a process of multiple slaves competing to become the master node. The process is as follows:
1. The slave finds that its master becomes FAIL (failure)
2. Add 1 to the cluster currentEpoch (election round mark) recorded by yourself, and broadcast the information to other nodes in the cluster.
3. Other nodes receive the information, and only the master responds, determines the legitimacy of the requester, and sends the result.
4. The slave trying to elect collects the results returned by the master and becomes the new Master after receiving the consent of more than half of the masters.
5. Broadcast Pong messages to notify other cluster nodes.
If this election is unsuccessful, for example, in a cluster consisting of three small masters and slaves A, B, and C, A's master fails, and A's two younger brothers initiate the election. As a result, B's master votes for A's younger brothers A1 and C. The master voted for A's younger brother A2, which will initiate the second election. The election round mark +1 will continue the above process.
Jump relocation:
When the client sends an instruction to a wrong node, the node will find that the slot where the key of the instruction is located is not under its own management. At this time, it will send a special jump instruction to the client to carry the target operation. The node address tells the client to connect to this node to obtain data. After receiving the instruction, the client will not only jump to the correct node for operation, but also update and correct the local slot mapping table cache synchronously. All subsequent keys will use the new slot mapping table.
When one of the masters in the redis-cluster cluster hangs up, it does not affect the client's query and writing.
How many hosts can be connected to the Redis-cluster cluster?
No more than half of the redis service hosts in the cluster can be used
Pseudo cluster construction
lab environment
| Host | IP | Role | port |
|---|---|---|---|
| centos1 | 192.168.100.208 | Master | 6379 |
| centos1 | 192.168.100.208 | Slave | 6380 |
| centos2 | 192.168.100.209 | Master | 6379 |
| centos2 | 192.168.100.209 | Slave | 6380 |
| centos3 | 192.168.100.210 | Master | 6379 |
| centos3 | 192.168.100.210 | Slave | 6380 |
Case implementation
Centos1 configuration
Redis single node construction and syntax structure
[root@centos1 ~]# vim /etc/hosts #Add IP and host name mapping
192,168,100,208 cents1
192.168.100.209 centos2
192,168,100,210 cents3
Create cluster working directory
[root@centos1 ~]# mkdir -p /etc/redis/cluster/6380
[root@centos1 ~]# ll /etc/redis/cluster/
Total usage 0
drwxr-xr-x 2 root root 6 October 19 20:44 6379
drwxr-xr-x 2 root root 6 October 19 20:44 6380
Create data storage directory
[root@centos1 ~]# mkdir -p /data/redis/data/6380
[root@centos1 ~]# ll /data/redis/data/
Total usage 0
drwxr-xr-x 2 root root 6 October 19 20:45 6379
drwxr-xr-x 2 root root 6 October 19 20:45 6380
Generate configuration file
[root@centos1 ~]# cp /usr/src/redis-5.0.4/redis.conf /etc/redis/cluster/6379/
[root@centos1 ~]# cp /usr/src/redis-5.0.4/redis.conf /etc/redis/cluster/6380/
Modify configuration file
[root@centos1 ~]# vim /etc/redis/cluster/6379/redis.conf
daemonize yes //redis runs in the background
bind 192.168.100.208 //Listening address
pidfile /var/run/redis_6379.pid //Port corresponding to the pidfile file
port 6379 //port
logfile "/var/log/redis/redis_6379.log" //Modify the log path
dir "/data/redis/data/6379" //Configure redis rdb data storage location
cluster-enabled yes //Enable the cluster and remove the comment #
cluster-config-file nodes_6379.conf //Cluster configuration, the configuration file is automatically generated when it is first started.
cluster-node-timeout 15000 //Request timeout 15s
appendonly yes //Aof log is enabled, enable it if necessary, it will record a log for each write operation
[root@centos1 ~]# vim /etc/redis/cluster/6380/redis.conf
daemonize yes //redis runs in the background
bind 192.168.100.208
pidfile "/var/run/redis_6380.pid"
port 6380
dir "/data/redis/data/6380"
logfile "/var/log/redis/redis_6380.log" //Modify the log path
cluster-enabled yes //Enable the cluster and remove the comment #
cluster-config-file nodes_6380.conf //Cluster configuration, the configuration file is automatically generated when it is first started.
cluster-node-timeout 15000 //Request timeout 15s
appendonly yes //Aof log is enabled, enable it if necessary, it will record a log for each write operation
Copy redis-server redis-cli to /usr/local/redis
mkdir -p /usr/local/redis
mkdir -p /var/log/redis
[root@centos1 ~]# cp /usr/src/redis-5.0.4/src/redis-server /usr/local/redis/
[root@centos1 ~]# cp /usr/src/redis-5.0.4/src/redis-cli /usr/local/redis/
ln -s /usr/local/redis/* /usr/local/bin/
Close the previous redis (please ignore it if not)
[root@centos1 ~]# ps -ef | grep redis
[root@centos1 ~]# kill -9 989
[root@centos1 ~]# kill -9 1013
[root@centos1 ~]# kill -9 1034
[root@centos1 ~]# kill -9 1079
#Kill the previously started redis process and kill it according to its own process number
Start redis service
[root@centos1 ~]# redis-server /etc/redis/cluster/6379/redis.conf
[root@centos1 ~]# redis-server /etc/redis/cluster/6380/redis.conf
View port
[root@centos1 ~]# netstat -antup | grip 63
Add auto-start at boot
[root@centos2 ~]# echo "redis-server /etc/redis/cluster/6379/redis.conf" >/etc/rc.local
[root@centos2 ~]# echo "redis-server /etc/redis/cluster/6380/redis.conf" >/etc/rc.local
Centos3 configuration
Modify hosts first
[root@centos3 ~]# vim /etc/hosts #Add IP and host name mapping
192,168,100,208 cents1
192.168.100.209 centos2
192,168,100,210 cents3
Add path environment variable
[root@centos3~]# ln -s /usr/local/redis/* /usr/local/bin/
Create working directory
[root@centos3 ~]# mkdir -p /data/redis/data/6380
[root@centos3 ~]# mkdir /var/log/redis
Modify configuration file
[root@cong13 ~]# vim /etc/redis/cluster/6379/redis.conf #Modify port and IP
daemonize yes //redis runs in the background
bind 192.168.100.210 //Listening address
pidfile /var/run/redis_6379.pid //Port corresponding to the pidfile file
port 6379 //port
dir "/data/redis/data/6379" //Configure redis rdb data storage location
logfile "/var/log/redis/redis_6379.log" //Modify the log path
cluster-enabled yes //Enable the cluster and remove the comment #
cluster-config-file nodes_6379.conf //Cluster configuration, the configuration file is automatically generated when it is first started.
cluster-node-timeout 15000 //Request timeout 15s
appendonly yes //Aof log is enabled, enable it if necessary, it will record a log for each write operation
[root@cong13 ~]# vim /etc/redis/cluster/6380/redis.conf #Modify port and IP
daemonize yes //redis runs in the background
bind 192.168.100.210 //Listening address
pidfile /var/run/redis_6380.pid //Port corresponding to the pidfile file
port 6380 //port
dir "/data/redis/data/6380" //Configure redis rdb data storage location
logfile "/var/log/redis/redis_6380.log" //Modify the log path
cluster-enabled yes //Enable the cluster and remove the comment #
cluster-config-file nodes_6380.conf //Cluster configuration, the configuration file is automatically generated when it is first started.
cluster-node-timeout 15000 //Request timeout 15s
appendonly yes //Aof log is enabled, enable it if necessary, it will record a log for each write operation
Start redis service
[root@centos3 ~]# redis-server /etc/redis/cluster/6379/redis.conf
[root@centos3 ~]# redis-server /etc/redis/cluster/6380/redis.conf
View port
[root@centos3 ~]# netstat -anput | grep 63
Add auto-start at boot
[root@centos3 ~]# echo "redis-server /etc/redis/cluster/6379/redis.conf" >/etc/rc.local
[root@centos3 ~]# echo "redis-server /etc/redis/cluster/6380/redis.conf" >/etc/rc.local
View progress
[root@centos3 ~]# ps -ef | grep redis #You can see that it is started with cluster
Create cluster cluster
Redis5.0 no longer uses ruby to build clusters
Use the command redis-cli
Ruby add cluster
redis-trib.rb****environment preparation (this file exists in the redis-4.0.1/src/ directory) //Only need to perform this step on one of them! !
Install ruby
yum -y install ruby rubygems
[root@centos1 ~]# ruby -v
ruby 2.0.0p648 (2015-12-16) [x86_64-linux]
[root@centos1 ~]# gem install -l redis-3.3.0.gem
[root@centos1 ~]# cd redis-4.0.1/src/
[root@centos1 src]# ./redis-trib.rb create --replicas 1 192.168.100.206:6379 192.168.100.207:6379 192.168.100.208:6379 192.168.100.206:6380 192.168.100.207:6380 192.168.100.208:6380
The --replicas parameter specifies how many slave nodes each master node in the cluster is equipped with. Here it is set to 1, and each master node has only one slave node.
Redis5.0 version uses the following method
redis-cli --cluster create --cluster-replicas 1 192.168.100.217:6379 192.168.100.218:6379 192.168.100.219:6379 192.168.100.217:6380 192.168.100.218:6380 192.168.100.219:6380
test:
[root@centos1 ~]# redis-cli -h 192.168.100.208 -p 6379 -c #-c means activate cluster mode
View cluster status
[root@centos1 src]# ./redis-trib.rb check 192.168.100.208:6379
After 5.0, if you want to check the status of the cluster:
redis-cli --cluster check 192.168.100.208:6379
Cluster testing
[root@centos1 ~]# #-c indicates cluster mode. Students can test the difference without adding -c.
192.168.100.208:6379set hehe haha
OK
192.168.100.209:6379get hehe
"haha"
Go to other hosts to view and create.
192.168.100.210:6379get hehe
-Redirected to slot [4618] located at 192.168.100.208:6379 #It will prompt that the data is on 192.168.100.208:6379, and it will be automatically converted to the 192.168.100.208:6379 host.
"haha"
Extend redis-cluster
- First add 192.168.100.211:6379 to the cluster
redis-cli --cluster add-node 192.168.100.211:6379 192.168.100.208:6379
- Check the status of the cluster. 209:6379 was added successfully, but there are no slots.
2. Allocate data slots to the 209:6379Master master node. The method of assigning 0 is to know any master node from the cluster (because only the Master master node has data slots), and then re-shard it.
redis-cli --cluster reshard 192.168.100.208:6379
Check the status of the cluster
- Add slave node to 100.211:6379
redis-cli --cluster add-node 192.168.100.211:6380 192.168.100.208:6379
- Check the status of the cluster and find that the addition was successful, but it is not the slave of 100.211:6379.
- Specify node 100.217:6380 as the slave node of 100.217:6379 to implement master-slave configuration.
- Check the status of the cluster again
shrink
The order of deletion is to delete the Slave node first, and then delete the Master node.
Dynamically delete the Master server 100.211:6379 node
It may be more complicated to delete the Master node. Because there are data slots on the Master node, in order to ensure that data is not lost, these data slots must be migrated to other Master nodes, and then the master node is deleted.
- Re-shard and move the data slot of the Master node to be deleted to other Master nodes to avoid data loss.
redis-cli --cluster reshard 192.168.100.211:6379 --cluster-from ef5c1a86f1b075d2216640c5c36a950a4cd81669 --cluster-to e6ee59a3e8f58d782388e7a6c423506e16811277 --cluster-slots 1365 --cluster-yes
`--cluster-from`Node ID to move out of hash slot
`--cluster-to`Node ID to move into hash slot
`--cluster-slots`Number of hash slots to move
If the number of slots is 3000, 208 209 210 1000 per person
1. **Delete the 100.211:6379 master node, provide the IP address and Port of the node to be deleted, and of course the ID name of the node to be deleted.**
**redis-cli --cluster del-node 192.168.100.211:6379 1876b8cd1153e44e62a1164395a041770a97e03f**
The slots are unbalanced and can be solved using the following methods
redis-cli --cluster rebalance --cluster-use-empty-masters 192.168.100.208:6379