What is idempotence?

For the same business operation, no matter how many times it is called, the result will be the same.

Idempotent design

Let’s take the example of docking Alipay recharge to analyze how to design the payment callback interface?

If the Alipay recharge function has been docked in our system, we need to provide Alipay with a callback interface, and the Alipay callback information will carry (out_trade_no [merchant order number], trade_no [Alipay transaction number]), trade_no It is unique in Alipay, and out_trade_no is unique in the merchant system.

The callback interface can be implemented in the following ways.

Method 1 (normal method)

The process is as follows:

1. Received Alipay payment success request

< p>2. Query whether the current order has been processed according to trade_no

3. If the order has been processed, return directly, if not processed, continue to execute it

4. Open local transaction

p>

5. The local system adds money to the user

6. Set the order status to success

7. Submit local affairs

The above process , For the same order, if Alipay informs multiple times at the same time, what will happen? When multiple notifications reach step 2 at the same time, the query orders are all unprocessed and will continue to be executed downwards. Finally, the user will be paid twice locally.

This method is suitable for stand-alone devices, and the notifications are executed in order. It can only be used for playing by yourself.

Method 2 (jvm locking method)

In method 1, there is a problem with concurrency. At this time, we use the Lock in java to lock. To prevent concurrent operations, the process is as follows:

1. Receive a successful Alipay payment request

2. Call Lock in java to lock

3. Query according to trade_no Whether the current order has been processed

4. If the order has been processed, return directly, if not processed, continue to execute it

5. Open local transactions

6. The local system adds money to the user

7. Set the order status to success

8. Submit local affairs

9. Release the lock

< p>Analysis of the problem: Lock can only work in one jvm. If multiple requests are processed by the same system, the above method of using Lock is no problem. However, most of the Internet systems are deployed in clusters. , Multiple sets of code will be deployed after the same set of codes. If Alipay sends multiple notifications at the same time and forwards them to different machines through load balancing, the above lock will not work. At this time, multiple requests are equivalent to lock-free processing, and the result in mode 1 will appear again. At this point we need distributed locks for processing.

Method 3 (pessimistic locking method)

Use pessimistic locking in the database to achieve. The pessimistic lock is similar to the Lock in the second mode, except that it is implemented by the database. The pessimistic lock in the data is implemented using for update, and the process is as follows:

1. Receive a successful Alipay payment request

2. Open local transactions

3. Query Order information and add pessimistic lock

select * from t_order where order_id = trade_no for update;

4. Determine if the order has been processed

5. If the order has been processed, return directly, if not processed, continue to execute it

6. To the local system Add money to the user

7. Set the order status to success

8. Submit local things

The key point is for update. For update, please explain :

1. When thread A executes for update, the data will lock the current record. When other threads execute this line of code, they will wait for thread A to release the lock before acquiring the lock, and continue to follow up operate.

2. When things are submitted, the locks acquired for update will be automatically released.

Method 3 can normally achieve the effects we need, and can ensure the idempotence of the interface, but there are some disadvantages: 1. If the business processing is time-consuming and concurrent, the subsequent threads will be in a waiting state for a long time , Occupies a lot of threads, leaving these threads in an invalid waiting state. The number of threads in our web service is generally limited. If a large number of threads are in a waiting state due to acquiring the for update lock, it is not conducive to concurrent operation of the system.

Method 4 (Optimistic locking method)

Rely on optimistic locking in the database.

1. Received a successful Alipay payment request

2. Query order information

select * from t_order where order_id = trade_no;< /pre> 

3. Determine if the order has been processed

4. If the order has been processed, return directly, if not processed, continue to execute it

5. Open local things

6. Add money to the user in the local system

7. Set the order status to success, note that this is the key point, pseudo code:

update t_order set status = 1 where order_id = trade_no where status = 0< /span>;

//The above update operation will return the number of rows affected num< /span>

if(num==1){

　　//Indicates successful update

　　Commit the transaction;

}else{

　　//Indicates that the update failed

　　roll back the transaction;

}

Note:

update t_order set status = 1 where order_id = trade_no where status = 0; is achieved by relying on optimistic locking Yes, status=0 is used as a condition to update, similar to the cas operation in java;

When executing this sql, if multiple threads reach this code at the same time, the data will be updated internally. Records will be queued for execution. In the end, one update will be executed successfully. For the other unsuccessful, their num will be 0, and then the commit or rollback operation will be performed according to the num.

Method 5 (Unique Constraint Method)

Rely on the unique constraint in the database to achieve.

We can create a table:

CREATE TABLE `t_uq_dipose` (

`id` bigint(20) NOT AUTO_INCREMENT,

`ref_type` varchar(32) NOT DEFAULT '' COMMENT 'Associated object type ',

`ref_id` varchar(64) NOT DEFAULT '' COMMENT 'Associated object id ',

PRIMARY KEY (`id`),

UNIQUE KEY `uq_1` (`ref_type`,`ref_id`) COMMENT 'Guarantee business uniqueness'

);

For any business, there is a business type (ref_type), and the business has a globally unique order number. When the business comes, check first Whether there are related records in the t_uq_dipose table, if not, continue to release.

The process is as follows:

1. Receive a successful Alipay payment request

2. Query t_uq_dipose (condition ref_id, ref_type) to determine whether the order has been processed< /p>

select * from t_uq_dipose where ref_type = 'recharge order ' and ref_id = trade_no;

3. Determine whether the order has been processed

4. If the order has been processed, return directly, if not processed, continue to execute it

5. Open local transactions

6. Add money to the user for the local system

7. Set the order status to success

8. Insert data into t_uq_dipose, insertion succeeded, submit local affairs, insertion failed, rollback Local affairs, pseudo code:

try {

insert into t_uq_dipose (ref_type,ref_id) values ​​('charge order< /span>',trade_no);

//Submit local affairs:

}catch(Exception e){

//roll back local affairs;

}

Description:

For the same business, ref_type is the same. When concurrent, Insert data will only have one success, the other will violate the unique constraint, enter the catch logic, the current transaction will be rolled back, and finally the most operation will succeed, thus ensuring idempotence operate.

About this method can be written as a general method, but in the case of large business volume, t_uq_dipose Inserting data will become the bottleneck of the system, and you need to consider sub-table operations to solve performance problems.

In the above process, insert the record into t_uq_dipose. It is best to execute at the end. Reason: The insert operation will lock the table. Putting it at the end can minimize the time to lock the table and improve the concurrency of the system.

How can consumers ensure the idempotence of message processing in the message service? Each message has a unique message id, which is similar to the trade_no in the above business, and the idempotence of message consumption can be realized by using the above method.

select * from t_order where order_id = trade_no for update;

select * from t_order where order_id = trade_no;

update t_order set status = 1 where order_id = trade_no where status = 0;

//The above update operation will return the number of rows affected num< /span>

if(num==1){

　　//Indicates successful update

　　Commit the transaction;

}else{

　　//Indicates that the update failed

　　roll back the transaction;

}

CREATE TABLE `t_uq_dipose` (

`id` bigint(20) NOT AUTO_INCREMENT,

`ref_type` varchar(32) NOT DEFAULT '' COMMENT 'Associated object type ',

`ref_id` varchar(64) NOT DEFAULT '' COMMENT 'Associated object id ',

PRIMARY KEY (`id`),

UNIQUE KEY `uq_1` (`ref_type`,`ref_id`) COMMENT 'Guarantee business uniqueness'

);

select * from t_uq_dipose where ref_type = 'Recharge order' and ref_id = trade_no;

 try{

insert into t_uq_dipose (ref_type,ref_id) values ​​('charge order< /span>',trade_no);

//Submit local affairs:

}catch(Exception e){

//roll back local affairs;

}