HeartBeat + DRBD + MySQL High Available Scheme

By Monitoring

www.521voip.com Focus on cloud computing

Heartbeat+DRBD+mysql high-availability solution

  1. 1.1 Introduction to the plan

This plan The Heartbeat dual-system hot backup software is used to ensure the high stability and continuity of the database, and the data consistency is guaranteed by the DRBD tool. By default, only one mysql is working. When the main mysql server has a problem, the system will automatically switch to the standby machine to continue to provide services. When the main database is repaired, the service will be switched back to continue to be provided by the main mysql.

  1. 1.2 Advantages and disadvantages of the scheme

Advantages: high security, high stability, high availability, automatic switching when failure occurs,

disadvantages: Only one server provides services, and the cost is relatively high. Inconvenient to expand. Split brain may occur.

  1. 1.3 Scheme Architecture Diagram

  • Scenarios for application of 1.4 scheme

    • This scheme is suitable for low database access Large, the number of visits will not grow too fast in the short term, and it is a scenario that requires very high database availability.

    1. 1.5 Plan Actual Combat

    2. 1.5.1 Introduction to actual combat environment

    host name

    Ip

    System

    DRBD disk

    Heartbeat version

    dbserver1

    10.1.1.113

    Centos 5.5 64bit

    /dev/sdb1

    2.1.3

    dbserver2

    10.1.1.108

    Centos 5.5 64bit

    /dev/sdb1

    2.1.3

    1. Installation of 1.5.2DRBD

    According to the official website, if the system kernel (linux) version is lower than 2.6.33, you need to load the DRBD module before installing the software. My kernel version is 2.6.18, and the drbd module will be automatically loaded after installation. The installation command is as follows:

    yum install -y drbd83 kmod-drbd83

    After installation, use the lsmod | grep drbd command to check whether the drbd module is loaded. If it is not loaded, you need to manually run the command to load, the command is as follows:

    insmod drbd/drbd.ko or modprobe drbd

    1. 1.5.3DRBD configuration

    Before configuration, you need to use fdisk to partition /dev/sdb.

    For the configuration of DRBD, you only need to configure the /etc/drbd.conf and hosts files. The content added for the hosts of dbserver1 and dbserver2 is as follows:

    10.1.1.113 dbserver1

    10.1.1.108 dbserver2

    The content of my /etc/drbd.conf file is as follows (the configuration of dbserver1 and dbserver2 is the same):< /p>

    global {usage-count yes; }

    common {syncer {rate 10M;} }

    resource r0 {

    protocol C;

    startup {

    }

    disk {

    on-io-error detach;

    #size 1G;

    }

    net {

    }

    on dbserver1 {

    device /dev/drbd0;

    disk /dev/sdb1;

    address 10.1.1.113:7888;

    meta-disk internal;

    }

    on dbserver2 {

    device /dev/drbd0;

    disk /dev/sdb1;

    address 10.1. 1.108:7888;

    meta-disk internal;

    }

    }

    < ol class="list-paddingleft-2" style="list-style-type:decimal;">

  • 1.5.4DRBD management and maintenance

  • < span style="font-family:Wingdings;">Create DRBD resources

    • After configuring drbd, just You need to use commands to create configured drbd resources on dbserver1 and dbserver2, use the following command:

    drbdadm create-md r0# r0 is the resource name defined in the configuration file

    • Start and stop of DRBD

    /etc/rc.d/init.d/drbd start#Start drbd

    /etc/rc.d/init.d/drbd stop#stop drbd

    /etc/rc.d/init.d/drbd restart#Restart drbd

    • Check DRBD status

    watch -n 1 cat /proc/drbd

    /etc/init.d/drbd status

    You can check the status of drbd in the above two ways

    • Set the current node as the master node, and format and mount it

    drbdadm – –overwrite-data-of-peer primary all

    mkfs.ext3 / dev/drbd0

    mkdir /drbd

    mount /dev/ drbd0 /drbd

    After hanging, you can write some test data to the /drbd directory to see if it can be synchronized to the slave node.

    • Migrate mysql data to drdb (assuming your mysql is already on the server Installed)

    DRBD has been installed and can be synchronized normally, then the next thing we need to do is to The data of the installed mysql is migrated to drbd. For the sake of simplicity, the situation of the newly installed database is not considered, because the new installation only needs to specify the data directory in drbd and put the my.cnf configuration file in drbd. Can. The specific data migration steps are as follows:

    1. a) Close dbserver1 and dbserver2 mysql, /etc/rc.d/init.d/mysqld stop

    2. b) Create a directory for storing database data on dbserver1: mkdir /drbd/dbdata

    3. c) Put the configuration file in the drbd directory in dbserver1:

    mv / etc/my.cnf /drbd/dbdata/

    Delete /etc/my.cnf on dbserver2, rm -f /etc/my.cnf

    Execute the following commands on dbserver1 and dbserver2 to create soft links.

    ln -s /drbd/dbdata/my.cnf /etc/my.cnf

    1. d) Modify the data directory of /etc/my.cnf to point to /drbd/dbdata

    2. e) Move the original mysql data file to /drbd/dbdata

    3. f) Execute chown -R mysql.mysql /drbd/dbdata

    4. g)Start mysql

    • Manually switch DRBD

    Without installation Before configuring drbd, drbd cannot be switched automatically. We can write a process to deepen our understanding of drbd and understand the workflow of heartbeat better. Here are the steps for manual switching:

    1. a) Stop mysql on dbserver1, /etc/rc.d/init.d/mysqld stop.

    2. b) Execute umount /dev/drbd0 on dbserver1.

    3. c) Execute drbdadm secondary all on dbserver1 to switch to slave mode. When both nodes are in secondary mode, dbserver2 can be set to primary.

    4. d) Execute drbdadmprimary all on dbserver2, promote dbserver2 to primary mode, and observe whether /proc/drbd is normal.

    5. e) Execute mount /dev/drbd0 /drbd on dbserver2 to mount the file system.

    6. f) Start mysql of dbserver2, /etc/rc.d/init.d/mysqld start.

    Note: dbserver1 and dbserver2 on the mysql user’s uid Same as gid. Otherwise, after switching, the owner of the mysql data directory will be incorrect and the startup will fail.

    • Master-slave switching

    < p style="text-indent:0px;">To switch from master to slave, you need to uninstall the file system first, and then execute the command to downgrade to slave:

    umount /dev/drbd0

    drbdadm secondary all

    < p style="text-indent:0px;">To switch from master to master, first execute the command to upgrade to master and then hang on the file system:

    drbdadmprimary all if drbdsetup is unsuccessful /dev/drbd0 primary -o

    mount /dev/drbd0 /drbd/

    • Treatment after DRBD split brain

      p>

    When DRBD has a split brain, the disks on both sides of the drbd will be inconsistent. The processing method is as follows:

    Switch to secondary on the node that is determined to be the slave, and discard the data of the resource:

    drbdadm secondary r0

    drbdadm – –discard-my-data connect r0

    Reconnect to the secondary node to be the primary (if this node is currently If the connection status is WFConnection, it can be omitted), use the following command to connect:

    drbdadm connect r0

    1. 1.5.5Heartbeat Installation of

    After DRBD debugging is no problem, you can start to install and configure heartbeat. Both Master and backup servers need to be installed heardheat software. Choose one of the following two installation methods.

    • How to install Rpm package

    yum -y install heartbeat-*

    • Source code compilation and installation method

    wget http://www.ultramonkey.org/download/heartbeat/2.1.3/heartbeat-2.1.3.tar.gz

    tar xzvf heartbeat-2.1.3.tar.gz

    cd heartbeat-2.1.3

    ./configure

    Make

    make install

    p>

    1. 1.5.6 Heartbeat configuration

    < p styl e="margin-left:98px;">The configuration of Heartbeat mainly includes three configuration files, the configuration of authkeys, ha.cf and haresources, let’s look at them separately!

    • Hosts file configuration

    Need to add master and backup hosts in the hosts file to speed up the communication between nodes

    Master and backup The content added to the hosts node is the same. My configuration adds the following content:

    vim /etc/hosts

    # dbserver1 and dbserver2 are the host names of my master and backup

    10.1.1.113 dbserver1

    1. 10.1.1.108 dbserver2

    2. Authkerys Configuration

    This file is used to configure the password authentication method. It supports 3 authentication methods, crc, md5 and sha1, from the left The security is getting higher and higher to the right, and the more resources are consumed. Therefore, if heartbeat is running on a secure network, such as a private network, then the authentication method can be set to crc, and the authkeys configuration of master and backup is the same. My authkeys file configuration is as follows:

    vim /etc/ha.d/authkeys

    auth 1

    1 crc

    • Configuration of ha.cf

    master(dbserver1) ha.cf configuration

    vim /etc/ha.d/ha.cf

    < p style="background:rgb(242,242,242);padding:0px;margin-left:0px;">logfile /var/log/ha-log

    logfacilitylocal0

    keepalive 2

    deadtime 30

    warntime 10

    initdead 60

    udpport 694

    ucast eth0 10.1.1.108

    auto_failback on

    nodedbserver1

    nodedbserver2

    ping 10.1.1.1

    respawn hacluster /usr/lib64/heartbeat/ipfail

    Backup(dbserver2) ha.cf configuration

    vim /etc/ha.d/ha.cf

    logfile /var/log/ha-log

    logfacilitylocal0

    keepalive 2

    deadtime 30

    warntime 10

    initdead 60

    udpport 694

    ucast eth0 10.1.1.113

    auto_failback on

    nodedbserver1

    nodedbserver2

    ping 10.1.1.1

    respawn hacluster /usr/ lib64/heartbeat/ipfail

    • Configuration of haresources

    haresources is used to set the master’s hostname, virtual IP, service and disk mounting, etc. The configuration of master and backup are the same. The following mysqld needs to be made into a service and placed in /etc/rc Under the .d/init.d/ directory, the configuration configuration is as follows:

    vim /etc/ha.d/haresources

    dbserver1 IPaddr::10.1.1.176/24/eth0:1 drbddisk::r0 Filesystem::/dev/drbd0::/drbd::ext3 mysqld

    1. 1.5.7Heartbeat management

    After configuring the heartbeat, you need to remove mysql from the self-starting server, because when the main heartbeat starts, the drdb file system will be mounted and mysql will be started. The mysql on the master stops and unmounts the file system, the file system is mounted from the top, and mysql is started. Therefore, you need to do the following:

    chkconfig mysqld off

    chkconfig –add heartbeat

    chkconfig heartbeat on

    Or you can write the heartbeat startup command to the /etc/rc.local startup file.

    1. 1.5.8Heartbeat+DRBD test

      2. After the environment is set up, careful testing is needed to see if the expected functions are achieved:

      • Stop mysqld on the master and see if you can switch (because heartheat does not check the availability of the service, so you need to implement it through an external script, The method has been described above).

      • Stop the heartheat of the master to see if it can switch normally.

      • Stop the master network or shut down the master system directly to see if it can be switched normally.

      • Start the heartbeat of the master to see if it can switch back normally.

      • Restart the master to see if the switching process is OK.

      Note: Does the switch mentioned here have stopped mysql, whether the file system has been uninstalled, etc.?

      1. 1.5.9 Heartbeat+DRBD monitoring

        2. Because heartbeat cannot monitor the availability of mysql, it needs to be implemented in other ways. It is necessary to monitor the availability of mysql. If a switch occurs, it takes the first time< /p>

    , , , , , ,

    Leave a Comment

    Your email address will not be published.