03 – Interaction and Construction Cluster

By Load Balance

Install package

pip install redis

Call the module

from redis import *

　　 This module provides the StrictRedis object (Strict strict), which is used to connect to the redis server, and provides different methods according to different types for interactive operation.

StrictRedis object methods

Create an object through init, specify the parameters host and port to connect to the specified server and port, host defaults to localhost, port defaults to 6379.db defaults to 0.

conn = StrictRedis(host="localhost",port=6379,db=0)

#can be abbreviated as

conn = StrictRedis()

According to different types, there are different examples. The method can be adjusted. It corresponds to the redis command learned before. The parameters required by the method are the same as Command parameters? To

string

  • set
  • setex
  • mset

    • < li>append

  • get
  • mget
  • key

keys

  • exists
  • type
  • delete
  • expire
  • getrange
  • ttl

    • < /ul>

    hash

    • hset
    • hmset
    • hkeys
    • hget
    • hmget
    • hvals
    • hdel

    list

    • lpush< /li>
    • rpush
    • linsert
    • lrange
    • lset
    • lrem

    set

    • sadd
    • smembers
    • srem

    zset

    • zadd
    • zrange
    • zrangebyscore
    • zscore
    • zrem
    • zremrangebyscore

    Learning objectives

    • The ability to use StrictRedis objects to add, delete, modify and check string type data

    Preparation

    h2>

    Create a redis directory on the desktop

    Use pycharm to open Open redis directory

    Create redis_string.py file

    from redis import *

    if __name__=="__main__":

         try:

         #Create a Strict object and establish a connection with the redis server

     conn = StrictRedis()

         except Exception as e:

         print(e)

    string-add

    Method set, add keys and values, return True if the addition is successful, and return False if the addition fails. 

    from redis import *

    if __name__=="main":

         try:

     conn     = StrictRedis()

         #Add key name, value is itheima

     result = conn.set("name","itheima")

         #True for successful printing, False for failure

     print(result)

         except Exception as e:

         print(e)

    String-Get

    ? Method get, add the value corresponding to the key, if the key exists, return the corresponding value, if the key does not exist, return None

    from redis import *

    if __name__=="__main__":

         try:

         #Create a StrictRedis object and establish a connection with the redis server

     conn=StrictRedis()

         #Get the value of the key name

     result = conn.get('name')

         #Output the value of the key, if the key does not exist, return None< /span>

     print(result)

         except Exception as e:

         print(e)

    string-modify

    ? Method set, modify if the key already exists, add if the key does not exist

    from
     span> redis import *

    if __name__=="__main__":

         try:

         #Create a StrictRedis object and establish a connection with the redis server

     conn=StrictRedis()

         #Set the value of the key name, if the key already exists, enter it? Modify, add if the key does not exist

     result = conn.set('name','itcast')

         # output the response result, if the operation is successful, return True, otherwise return False

     print(result)

         except Exception as e:

         print(e)

    string-delete

    ? Method delete, delete the key and the corresponding value, if the deletion is successful, the number of affected keys will be returned, otherwise it will return 0

    from redis import *

    if __name__=="__main__":

         try:

         #Create a StrictRedis object and establish a connection with the redis server

     conn=StrictRedis()

         #Set the value of the key name, if the key already exists, enter it? Modify, add if the key does not exist

     result = conn.delete('name')

         # output the response result, if the deletion is successful, return the affected key Number, otherwise it returns 0

     print(result)

         except Exception as e:

         print(e)

    Get key

    ? Method keys, get keys according to regular expressions

    from redis import *

    if __name__=="__main__":

         try:

         #Create a StrictRedis object and establish a connection with the redis server

     conn=StrictRedis()

         #Get all the keys

     result=conn.keys()

         #output the response result, all the keys form a list, if Return to an empty list if there is no key

     print(result)

         except Exception as e:

         print(e)

    Build master and slave

    Master From the concept

    • ? a master can have multiple slaves, and a slave? can have multiple slaves, and so on, a strong multi-level server cluster architecture is formed
    • Master is used to write data, and slave is used to read data. According to statistics, the read-write ratio of the website is 10:1

    • The read-write separation can be achieved through the master-slave configuration

    share picture

    Both the master and slave are a redis instance (redis service)

    Master-slave configuration

    Configuration master:

    ifconfig

    Modify etc/redis/redis.conf file

    sudo vi redis.conf

    bind 192.    168.26.128

    Restart the redis service

    sudo service redis stop

    redis-server redis.conf

    Configuration from

    Copy etc/redis/redis.conf file

    sudo cp redis.conf ./slave.conf

    Modify redis/slave.config file

    sudo vi slave.conf

    Edit content

    bind 192.168.26.128

    slaveof     192.168.26.128 6379

    port     6378

    redis service

    sudo redis-server slave.conf

    < /div>

    View master-slave relationship

    redis-cli -h 192.168.26.128 info Replication

    Data operation

    h2>

    Execute the ?info command on the master and slave respectively to view the output information and enter the master client

    redis-cli -h 192.168.26.128 -p 6379
     pre>

    Enter client client

    redis-cli -h 192.168.26.128 -p 6378

    Write data on master

    set aa aa

    Read data on slave

    get aa

    Build a cluster

    • Before we have talked about the concept of master-slave, one master can have multiple slaves, if you visit at the same time If the volume is too large (1000w), the main service will definitely be down, and the data service will be down or a natural disaster occurs.
    • Big companies will have many servers (East China, South China, Central China, North China) Region, Northwest Region, Southwest Region, Northeast Region, Taiwan, Hong Kong and Macau Region)

    The concept of cluster

    　　Cluster is a group of independent, interconnected through high-speed network Computers, they form a group and are managed as a single system. When a client interacts with the cluster, the cluster is like an independent server. The cluster configuration is used to improve availability and scalability.

    share picture

    < p>

    When the request comes, it is first processed by the load balancing server, and the request is forwarded to another server.

    redis cluster

    • Classification
      • Software level
      • Hardware level

      • < li>Software level: There is only one computer, and multiple redis services are started on this computer.

    Share pictures

    < /p>

    Hardware level: There are multiple physical computers, and one redis or multiple redis services are activated on each computer.

    share picture

    < p>

    Set up a cluster

    Currently there are two hosts 172.16.179.130 and 172.16.179.131. The IP of this? should be changed to the actual value when using it

    Reference reading

    • Redis cluster construction http://www.cnblogs.com/wuxl360/p/5920330.html
    • [Python] construction redis cluster http://blog.5ibc.net/p/51020.html

    Configuration machine 1

    • In the demo, 172.16.179.130 is the current The ip of the ubuntu machine
    • Enter the Desktop record on 172.16.179.130, create a conf record

    • Create a file 7000.conf under the conf record, and edit the content as follows 

    port 7000

    bind     172.16.179.130

    daemonize yes

    pidfile     7000.pid

    cluster    -enabled yes

    cluster    -config-file 7000_node.conf

    cluster    -node-timeout 15000

    appendonly yes

    Create a file 7001.conf in the conf record, and edit the content as follows

    port 7001

    bind     172.16.179.130

    daemonize yes

    pidfile     7001.pid

    cluster    -enabled yes

    cluster    -config-file 7001_node.conf

    cluster    -node-timeout 15000

    appendonly yes

    Create a file 7002.conf in the conf record and edit the content as follows

    port 7002

    bind     172.16.179.130

    daemonize yes

    pidfile     7002.pid

    cluster    -enabled yes

    cluster    -config-file 7002_node.conf

    cluster    -node-timeout 15000

    appendonly yes

    Summary: The configuration of the three software differs in port, pidfile, and cluster-config-file.

    ?Configure software to start redis service

    redis-server 7000.conf

    redis    -server 7001.conf

    redis    -server 7002.conf

    Check the process as shown in the figure below:

    Share picture

    Configure machine 2

      < li>In the demo, 172.16.179.131 is the ip of the current ubuntu machine
    • Enter the Desktop Record on 172.16.179.131 and create a conf record

    • In the conf record Create a file 7003.conf, and edit the content as follows

      port 7003

      bind       172.16.179.131

      daemonize yes

      pidfile       7003.pid

      cluster      -enabled yes

      cluster      -config-file 7003_node.conf

      cluster      -node-timeout 15000

      appendonly yes

    • Create a file 7004.conf under the conf record and edit the content as follows 

      port 7004

      bind       172.16.179.131

      daemonize yes

      pidfile       7004.pid

      cluster      -enabled yes

      cluster      -config-file 7004_node.conf

      cluster      -node-timeout 15000

      appendonly yes

    • Create a file 7005.conf in the conf? record, and edit the content as follows 

      port 7005

      bind       172.16.179.131

      daemonize yes

      pidfile       7005.pid

      cluster      -enabled yes

      cluster      -config-file 7005_node.conf

      cluster      -node-timeout 15000

      appendonly yes

    • Summary: The configuration of the three software differs in port, pidfile, and cluster -config-file three items

    • Use the configuration file to start the redis service

      redis-server 7003.conf

      redis      -server 7004.conf

      redis      -server 7005.conf

      See the process as shown below

    Share Picture

    Create a cluster

    • The redis installation package contains redis-trib.rb, which is used to create a cluster
    • The next operation will be performed on the 172.16.179.130 machine?

    • Copy the command, This command can be adjusted in any record.

    sudo cp /usr/share/doc/redis-tools/examples/redis- trib.rb /usr/local/bin/

    Install ruby ​​environment, because redis-trib.rb is developed by Ruby

    sudo apt-get install ruby

    Enter "y" at the prompt, and then return to "continue to install"

    Run the following command to create a cluster

    redis-trib.rb create --replicas 1 172.16.179.130:7000 172.16.179.130:7001 172.16.179.130:7002 172.16.179.131:7003 172.16.179.131:7004 172.16.179.131:7005< /pre>

    • Execute? This command may report an error on some machines, mainly because the installed ruby ​​is not the latest version!

    • The defense wall of the celestial dynasty led to the downloading of the latest version, so you need to set the source of the gem

    • The solution is as follows

      • < /ul> 
      -- First check the address of the gem source

      gem source       -l - If it is https://rubygems.org/ it needs to be replaced

      -- The replacement instruction is

      gem sources       --add https://gems.ruby-china.org/ --remove https://rubygems.org/

      -- Install redis dependencies through gem

      sudo gem install redis

      -- Then re-execute the? instruction

      share picture

      redis-trib.rb create --replicas 1 172.16.179.130: 7000 172.16.179.130:7001 172.16.179.130:7002 172.16.179.131:7003 172.16.179.131:7004 172.16.179.131:7005

      Prompt the following master-slave information, enter "yes" and then return? ​​/p>

      share picture

      The prompt is completed and the cluster is successfully built

      Data verification

      • According to the figure above, it can be seen that the currently built master servers are 7000, 7001, and 7003, and the corresponding slave servers are 7004, 7005, 7002

      • Connect 7002 on 172.16.179.131 machine, add parameter -c to connect to the cluster

      redis-cli -h 172.16.179.131 -c -p 7002

      Write data

      set name itheima

      ? I jumped to the 7003 server and wrote the data successfully

      Share pictures

      You can get data at 7003, and if you write data, it will be redirected to 7000 (load balance)

      Share picture

      Which server to write data on: CRC16

      • When designing the redis cluster, it took into account the decentralization and middleware, that is to say, the cluster Each node is equal in relation to each other, and each node stores its own data and the state of the entire cluster. Each node is connected to all other nodes, and these connections remain active, which ensures that we only need to connect to any node in the cluster to get the data of other nodes
      • Redis cluster Instead of using the traditional consistent hash to distribute data, it uses another method called hash slot to distribute the data. The redis cluster allocates 16384 slots by default. When we set a key, we will use the CRC16 algorithm to get the modulus to get the slot to which it belongs, and then assign this key to the nodes in the hash slot interval. The specific algorithm is: CRC16(key )% 16384. So when we saw the set and get during the test, we jumped directly to the node on the 7000 end.
      • The Redis cluster will store the data in a master node, and then in this master and its corresponding salve Data synchronization between. When reading data, it also obtains data from the corresponding master node according to the consistent hash algorithm. Only when a master hangs up, will the corresponding salve node be started and act as a master
      • Note: There must be 3 or more master nodes, otherwise it will fail when creating a cluster And when the number of surviving master nodes is less than half of the total number of nodes, the entire cluster can provide services.

      Python interaction

      The installation package is as follows:

      pip install redis-py-cluster

      redis-py-cluster source code address https://github.com/Grokzen/redis-py-cluster, create a piece of redis_cluster.py, the sample code is as follows:

      from rediscluster import *

      if __name__ == '__main__':

             try:

             # Build all nodes, Redis will use the CRC16 algorithm, Write the key and value to a node

       startup_nodes = [

       {      'host': '192.168.26.128', 'port< span style="color: #800000;">': '7000'},

       {      'host': '192.168.26.130', 'port< span style="color: #800000;">': '7003'},

       {      'host': '192.168.26.128', 'port< span style="color: #800000;">': '7001'},

      ]

             # Constructing a StrictRedisCluster object

       src=StrictRedisCluster(startup_nodes=startup_nodes,decode_responses=True)

             # Set the key as name and the value as itheima data

       result=src.set('name','itheima')

             print(result)

             # Get the key as name

       name = src.get('name')

             print(name)

             except Exception as e:

             print(e)

      pip install redis< /pre>
      from redis import *< /pre>
      conn = StrictRedis(host="localhost< /span>",port=6379,db=0)

      #can be abbreviated as

      conn = StrictRedis()
      

      

      from redis import *

      if __name__=="__main__":

             try:

             #Create a Strict object and establish a connection with the redis server

       conn = StrictRedis()

             except Exception as e:

             print(e)
      

      

      from redis import *

      if __name__=="main":

             try:

       conn       = StrictRedis()

             #Add key name, value is itheima

       result = conn.set("name","itheima")

             #True for successful printing, False for failure

       print(result)

             except Exception as e:

             print(e)
      

      

      from redis import *

      if __name__=="__main__":

             try:

             #Create a StrictRedis object and establish a connection with the redis server

       conn=StrictRedis()

             #Get the value of the key name

       result = conn.get('name')

             #Output the value of the key, if the key does not exist, return None< /span>

       print(result)

             except Exception as e:

             print(e)
      

      

      from redis import *

      if __name__=="__main__":

             try:

                    #创建StrictRedis对象，与redis服务器建?连接

              conn=StrictRedis()

                    #设置键name的值，如果键已经存在则进?修改，如果键不存在则进?添加

              result = conn.set(‘name‘,‘itcast‘)

                    #输出响应结果，如果操作成功则返回True，否则返回False

              print(result)

                except Exception as e:

                    print(e)
      

      

      from redis import *

      if __name__=="__main__":

             try:

                    #创建StrictRedis对象，与redis服务器建?连接

              conn=StrictRedis()

                    #设置键name的值，如果键已经存在则进?修改，如果键不存在则进?添加

              result = conn.delete(‘name‘)

                    #输出响应结果，如果删除成功则返回受影响的键数，否则则返回0

              print(result)

                except Exception as e:

                    print(e)
      

      

      from redis import *

      if __name__=="__main__":

             try:

                    #创建StrictRedis对象，与redis服务器建?连接

              conn=StrictRedis()

                    #获取所有的键

              result=conn.keys()

                    #输出响应结果，所有的键构成?个列表，如果没有键则返回空列表

              print(result)

                except Exception as e:

                    print(e)
      

      

      ifconfig
      

      

      sudo vi redis.conf 

      bind 192.      168.26.128
      

      

      sudo service redis stop

      redis-server redis.conf
      

      

      sudo cp redis.conf ./slave.conf
      

      

      sudo vi slave.conf
      

      

      bind 192.168.26.128

      slaveof       192.168.26.128 6379

      port       6378
      

      

      sudo redis-server slave.conf
      

      

      redis-cli -h 192.168.26.128 info Replication
      

      

      redis-cli -h 192.168.26.128 -p 6379
      

      

      redis-cli -h 192.168.26.128 -p 6378
      

      

      set aa aa
      

      

      get aa
      

      

      port 7000

      bind       172.16.179.130

      daemonize yes

      pidfile       7000.pid

      cluster      -enabled yes

      cluster      -config-file 7000_node.conf

      cluster      -node-timeout 15000

      appendonly yes
      

      

      port 7001

      bind       172.16.179.130

      daemonize yes

      pidfile       7001.pid

      cluster      -enabled yes

      cluster      -config-file 7001_node.conf

      cluster      -node-timeout 15000

      appendonly yes
      

      

      port 7002

      bind       172.16.179.130

      daemonize yes

      pidfile       7002.pid

      cluster      -enabled yes

      cluster      -config-file 7002_node.conf

      cluster      -node-timeout 15000

      appendonly yes
      

      

      redis-server 7000.conf

      redis      -server 7001.conf

      redis      -server 7002.conf
      

      

      port 7003

      bind       172.16.179.131

      daemonize yes

      pidfile       7003.pid

      cluster      -enabled yes

      cluster      -config-file 7003_node.conf

      cluster      -node-timeout 15000

      appendonly yes
      

      

      port 7004

      bind       172.16.179.131

      daemonize yes

      pidfile       7004.pid

      cluster      -enabled yes

      cluster      -config-file 7004_node.conf

      cluster      -node-timeout 15000

      appendonly yes
      

      

      port 7005

      bind       172.16.179.131

      daemonize yes

      pidfile       7005.pid

      cluster      -enabled yes

      cluster      -config-file 7005_node.conf

      cluster      -node-timeout 15000

      appendonly yes
      

      

      redis-server 7003.conf

      redis      -server 7004.conf

      redis      -server 7005.conf
      

      

      sudo cp /usr/share/doc/redis-tools/examples/redis-trib.rb /usr/local/bin/
      

      

      sudo apt-get install ruby
      

      

      redis-trib.rb create --replicas 1 172.16.179.130:7000 172.16.179.130:7001 172.16.179.130:7002 172.16.179.131:7003 172.16.179.131:7004 172.16.179.131:7005
      

      

      -- 先查看??的 gem 源是什么地址

      gem source       -l -- 如果是https://rubygems.org/ 就需要更换

      -- 更换指令为

      gem sources       --add https://gems.ruby-china.org/ --remove https://rubygems.org/

      -- 通过 gem 安装 redis 的相关依赖

      sudo gem install redis

      -- 然后重新执?指令
      

      

      redis-trib.rb create --replicas 1 172.16.179.130:7000 172.16.179.130:7001 172.16.179.130:7002 172.16.179.131:7003 172.16.179.131:7004 172.16.179.131:7005
      

      

      redis-cli -h 172.16.179.131 -c -p 7002
      

      

      set name itheima
      

      

      pip install redis-py-cluster
      

      

      from rediscluster import *

      if __name__ == ‘__main__‘:

              try:

                # 构建所有的节点，Redis会使?CRC16算法，将键和值写到某个节点上

          startup_nodes = [

              {      ‘host‘: ‘192.168.26.128‘, ‘port‘: ‘7000‘},

              {      ‘host‘: ‘192.168.26.130‘, ‘port‘: ‘7003‘},

              {      ‘host‘: ‘192.168.26.128‘, ‘port‘: ‘7001‘},

          ]

                # 构建StrictRedisCluster对象

          src=StrictRedisCluster(startup_nodes=startup_nodes,decode_responses=True)

                # 设置键为name、值为itheima的数据

          result=src.set(‘name‘,‘itheima‘)

                print(result)

                # 获取键为name

          name = src.get(‘name‘)

                print(name)

              except Exception as e:

                print(e)
    , ,

    Leave a Comment

    Your email address will not be published.