ZooKeeper principle

1, The global data is consistent: each< span data-mce-="">serversave a copy of the same data, clientwhatever Which server is connected to, the data is consistent

2, reliability: simple, robust, and good performance, If message m is accepted by one server, it will be accepted by all servers.

3, wait-free (wait-free): slow or invalid clients shall not interfere with fast clients The request, so that each client can effectively wait.

4、Sequence: Including global order and partial order:

　　Globally ordered: means that if message a is published before message b on one server, message a will be on all servers Is published before message b;

　　 Partial order: refers to if a message b is the same after message a A sender publishes, a must be ranked before b

5, atomicity: update can only succeed or Failure, no intermediate state

6. Real-time: Zookeeper guarantees that the client will obtain the server’s update information or server failure information within a time interval.

　 But due to network delay and other reasons, Zookeeper cannot guarantee that two clients can get the newly updated data at the same time. If you need the latest data, you should call the sync() interface before reading the data.

cluster role< /span>

In ZooKeeper, there are three roles:

• Leader

< li>

Follower

• Observer

A ZooKeeper cluster can only have one leader at a time, and the others are Follower or Observer.

By default, ZooKeeper has only two roles, Leader and Follower, and no Observer role. In order to use Observer mode,

Add: peerType=observer in the configuration file of any node that wants to become Observer and in the configuration files of all servers,

Configure to observer mode The line of server configuration is appended with: observer, for example: server.1:localhost:2888:3888:observer

All machines in the ZooKeeper cluster select a machine called Leader through a leader election process , Leader server provides read and write services for clients.

Follower and Observer can provide read services, but cannot provide write services. The only difference between the two is that the Observer machine does not participate in the Leader election process, nor does it participate in the write success strategy of over half of the write operation. Therefore, the Observer can improve the read performance of the cluster without affecting the write performance.

ZookeeperPrinciple< /span>

ZookeeperAlthough it is not specified in the configuration filemasterandslave

but ,zookeeperWhen working, there is a node forleader, others arefollower

Leaderis temporarily generated through the internal election mechanism

Zookeeper Election mechanism (new clusterpaxos) p>

Take a simple example to illustrate the entire election process.
Assume there are five servers composed ofzookeepercluster span>,theirid From1-5,At the same time, they are all newly launched,that is, there is no historical data span>,At the point of storing data,all the same.Assuming that these servers are started in order,Let’s take a look What will happen.
1) server1Start,At this time, only its one server is started,The report it sent out has no response,< span style="font-family: 宋体">So its election status has always beenLOOKINGStatus
2) Server2Start,It is the same as the server that was started in the first place span>1To communicate, Exchange their election results with each other,Since neither has historical data,So idhigher value server 2Winner,but because it did not reach more than half of the servers agreed to elect it(More than half of this example is3),so the server1,2 still keep it LOOKINGStatus .
3) Server3Start,According to the previous theoretical analysis< span style="font-family: Calibri">,server3 Become a server1,2,3The Boss in ,< span style="font-family: 宋体">The difference from the above is, At this time, three servers elected it,So it became the election of this < /span>leader.
4) server4Start,According to the previous analysis,Theoretical server 4It should be a server1, 2,3,4the largest of the,But because more than half of the servers in the past have elected servers3,So it can only accept the life of a younger brother.
5) Server5Start,same4 span>Same,Be the younger brother .

not brand new Cluster election mechanism(data recovery) p>

Then, when initializing, the election is carried out according to the above instructions, but whenzookeeperAfter running for a period of time, there is a machinedowndown and restart At the time of election, the election process is relatively complicated.

Need to add dataid、< /span>leader id and logic clock.

dataid: new data id is big, the data will be updated every time it is updatedid.

Leader id: We configured itmyid, one for each machine.

Start incrementing,Each election corresponds to a value,that is: < span style="font-family: 宋体">If in the same election, Then this value should be consistent;the larger the logical clock value< span style="font-family: Calibri">,Explain this electionleader< /span> Progress update.

The election standard becomes:

1, The result of the election with a small logic clock is ignored. Re-vote

2, after unifying the logic clock , DataidBig win

3, dataidUnder the same circumstances,leader idBig win p>

according to this rule leader.

ZookeeperData structure

1, hierarchical directory structure, naming conforms to regular file system specifications (See the picture below)< /span>

2, each node is inzookeeper is called znode, and it has a unique path identifier p>

3, nodeZnodecan contain data and child nodes (butEPHEMERALtype The node cannot have child nodes)

4, the client application can set up a monitor on the node

< p>

Node type

1、< span style="font-family: Calibri">ZnodeThere are two types:

< span style="font-family: 宋体">short (ephemeral) (disconnect and delete by yourself)

persistent (persistent) (disconnect without deleting)

2,ZnodeYesFour formsThe directory node of (default is span>persistent )

PERSISTENT

PERSISTENT_SEQUENTIAL(persistent sequence/test0000000019 )

EPHEMERAL

EPHEMERAL_SEQUENTIAL

< p align="justify">3, createznode set the sequence identifier,znodeA value will be appended to the name, The sequence number is a monotonically increasing counter, maintained by the parent node

4, in a distributed system, the sequence The number can be used to sort all events globally, so that the client can infer the order of events from the sequence number

Zookeeper的ACL(Permission control list)

aclYesAccess control lists , which is the permission control list:

For nodes, you can set relevant read and write permissions, in order to ensure data security

PermissionspermissionsYou can specify different permissions and roles

ZookeeperOneACLand oneZooKeeper node corresponds to it. And, the ACL of the parent node and the ACL are independent of each other. In other words, ACL cannot be inherited by child nodes, and the permissions owned by the parent node There is nothing to do with the permissions used by the child nodes.

ACL composition

< /p>

Zookeeper supports a configurable authentication mechanism. It uses a triplet to define the client’s access permissions:(scheme:expression, perms)

• scheme: represents a certain authority mechanism adopted by the representative
• id: represents the permitted access User
• permissions: permission combination string

< h4>schemeType

• orld :worldThere is only one nextid, that is, there is only one user, which is anyone< span style="font-family: 宋体">, then the combination is writtenworld:anyone:[permissions]
• ip: When set toipSpecifiedipaddress, restricted at this timeip to visit, such asip:192.168.77.130:[permissions]
• auth: login on behalf of authentication, you need a registered user to obtain permission before you can log in to access, The format isauth:userpassword:[permissions]
• digest: You need to encrypt the password to access, the combination form is:

　　　　digest:username:BASE64(SHA1(password)): [permissions]

　　　　authanddigestThe difference is that the former uses a plain text password to log in, while the latter uses a cipher text password to log in. setAcl /path auth:lee:lee:cdrwa andsetAcl /path digest:lee:BASE64(SHA1(password)):cdrwais equivalent,

　　　　 is passingaddauth digest lee:lee can operate the authority of the specified node. In actual situationsinternational situations,digestBe more commonly used.

• super: represents the super administrator and has all permissions li>

permissionsType

• Abbreviation of permission string crdwa :
• CREATE: Create child node permission
• READ: access node/Sub-node permissions
• WRITE: Set node data permissions
• DELETE: Delete child node permissions
• ADMIN: Administrator permissions

ACL command

• getAcl Get the information of a nodeaclauthorization information
• setAcl set a nodeaclPermission information span>
• addauth Enter the authentication and authorization information, and enter the clear text password (login) when registering, but in zkIn the system of , the password exists in an encrypted form