[Editor’s Note] The first article in this series introduced the microservice architecture pattern. It discusses the advantages and disadvantages of using microservices. Except for some complex microservices, this model is also an ideal choice for complex applications.
When you decide to treat your application as a set of microservices, you need to decide how the application client interacts with the microservices. In a monolithic program, there is usually only a set of redundant or load-balanced service provision points. In the microservice architecture, each microservice exposes a set of fine-grained service provision points. In this article, we look at how it affects client-to-server communication and propose an API Gateway method.
Introduction
Assuming you are developing a native phone for an online shopping application Client. You need to implement a product final page to display product information.
For example, the image below shows the information you see when you swipe the final page of the product on the Amazon Android client.
Although this is a smartphone application, the final page of this product shows a lot of information. For example, not only the basic product information (name, description and price), but also the following:
- Number of items in the shopping cart
- Order history
- User reviews
- Low inventory warning
- Express options
- A variety of recommendations, including the products that are often purchased with this item, the products purchased by other customers who bought the item, and which products the customers who bought the product have viewed.
- Optional shopping options
When using a monolithic application architecture, one The mobile client will obtain this data through a REST request (GET api.company.com/productdetails/productId). A load balancer distributes requests to one of multiple application instances. The application will query various databases and return the request to the client.
In contrast, if the microservice architecture is adopted, the data on the final page will be distributed on different microservices. Here are some microservices that may be related to the data on the final page of the product:
- Shopping cart service–the number of items in the shopping cart
- Order service–order history
- Classification service–Basic product information, such as name, picture and price
- Comment service–User comments
- Inventory service-low inventory warning
- Express service-express options, deadlines, cost calculations from different express APIs
- Recommended Service–Recommended Products
< br style="color: rgb(102, 102, 102); font-family: "Helvetica Neue", STHeiti, "Microsoft YaHei", Helvetica, Arial, sans-serif; font-size: 14px; line-height: 25.2 px;"> We need to decide how mobile clients access these services. Please look at the following ways
Client-to-microservice direct communication
Theoretically, a client can directly initiate a request to any one of multiple microservices. Every microservice will have an external server ( https://serviceName.api.company.nam e). This URL may be mapped to the load balancer of the microservice, which then forwards the request to a specific node. In order to search for product details, the mobile terminal needs to send requests to the above microservices one by one.
Unfortunately, this scheme has many difficulties and limitations. One of the problems is the mismatch between the client’s demand and the number of fine-grained APIs exposed by each microservice. As shown in the figure, the client needs 7 separate requests. In more complex scenarios, more requests may be required. For example, Amazon’s final product page requires hundreds of microservices. Although a client can initiate many requests through the LAN, it would be very inefficient on the public network. This problem is particularly prominent on the mobile Internet. This solution will also cause the client code to be very complicated.
Another problem is that the client’s protocol for directly requesting microservices may not be web-friendly. One service may use Thrift’s RPC protocol, while another service may use the AMQP message protocol. None of them are browsing or firewall friendly, and are best used internally. Applications should use protocols like HTTP or WEBSocket outside the firewall.
Another disadvantage of this scheme is that it is difficult to refactor microservices. Over time, we may need to change the current segmentation scheme of system microservices. For example, we may need to merge two services or split one service into multiple. However, if the client interacts directly with the microservice, this refactoring is difficult to implement.
Due to the above three problems, the direct communication between the client and the server is rarely used in practice.
Using an API Gateway
Generally speaking, a better The solution is to use API Gateway. API Gateway is a server, which can also be said to be the only node that enters the system. This is very similar to the Facade pattern in the object-oriented design pattern. API Gateway encapsulates the architecture of the internal system and provides APIs to each client. It may also have other functions, such as authorization, monitoring, load balancing, caching, request fragmentation and management, static response processing, etc. The figure below shows an API Gateway adapted to the current architecture.
API Gateway is responsible for request forwarding, synthesis and protocol conversion. All requests from the client must first pass through the API Gateway, and then route these requests to the corresponding microservices. API Gateway will often call multiple microservices to process a request and aggregate the results of multiple services. It can convert between web protocol and non-web-friendly protocols used internally, such as HTTP protocol and WebSocket protocol.
API Gateway can provide clients with a customized API. It exposes a coarse-grained API to mobile clients. Take the use scenario of the final page of the product as an example. API Gateway provides a service provision point (/productdetails?productid=xxx) so that mobile clients can retrieve all data on the final page of the product in one request. API Gateway processes this request and returns results by calling multiple services, involving product information, recommendations, reviews, etc.
A good API Gateway example is Netfix API Gateway . The Netflix streaming service provides hundreds of different microservices, including TVs, set-top boxes, smartphones, gaming systems, tablets, etc. At first, the Netflix view provided a suitable for all scenes API. However, they found that this format is not easy to use, because it involves a variety of equipment and their unique needs. Now, they use an API Gateway to provide a fault-tolerant API with corresponding codes for different types of devices. In fact, an adapter needs to call 6 to 8 back-end services on average to process a request. Netflix API Gateway handles billions of requests every day.
The advantages and disadvantages of API Gateway
As you might expect, use API Gateway also has advantages and disadvantages. One of the biggest benefits of API Gateway is to encapsulate the internal structure of the application. Compared to calling the specified service, it is easier for the client to directly interact with gatway. API Gateway provides each client with a specific API, which reduces the number of communications between the client and the server and simplifies the client code.
API Gateway also has some disadvantages. It is a highly available component that must be developed, deployed, and managed. There is another problem, it may become a bottleneck in development. Developers must update API Gateway to provide new service providers to support newly exposed microservices. When updating API Gateway, it must be as lightweight as possible. Otherwise, developers will be queued for updating the Gateway. However, in addition to these shortcomings, for most applications, the use of API Gateway is effective.
Implement an API Gateway
Now that we know how to use API Gateway Motivation and pros and cons, let’s look at what things need to be considered when designing it.
Performance and scalability
Only a few companies need to deal with something like Netflix At that scale, billions of requests need to be processed every day. However, for most applications, the performance and scalability of API Gateway are also very important. Therefore, it makes sense to create an API Gateway that supports synchronous, non-blocking I/O. There are already different technologies that can be used to implement a scalable API Gateway. On the JVM, use a framework based on NIO technology, such as Netty, Vertx, Spring Reactor or JBoss Undertow. Node.js is a non-JVM popular platform, it is a platform built on the basis of Chrome’s JavaScript engine. An alternative plan is NGINX Plus . NGINX Plus provides a mature, scalable, high-performance web server and reverse proxy, all of which are easy to deploy, configure and re-development. NGINX Plus can manage authorization, permission control, load balancing, caching, and provide application health checks and monitoring.
Adopt a reactive programming model
For some requests, API Gateway can It is processed by directly routing the request to the corresponding back-end service. For other requests, it needs to call multiple back-end services and merge the results for processing. For some requests, such as product final page requests, the requests sent to the back-end service are independent of each other. In order to minimize response time, API Gateway should process independent requests concurrently. However, sometimes there are dependencies between requests. API Gateway may need to authenticate the request through the authorization service first, and then route it to the back-end service. Similarly, in order to obtain a customer’s product wish list, it is necessary to obtain the user’s information first, and then return the product information on the list. Such an API component is Netflix Video Grid .
Using the traditional synchronous callback method to implement API merging code will make you enter the nightmare of callback function. This kind of code will be very difficult and difficult to maintain. An elegant solution is to implement a reactive programming model. Similar reaction abstract implementations are in Scala Future , Java8 CompletableFuture and JavaScript Promise . Based on the Microsoft. Net platform, there are Reactive Extensions(Rx) . Netflix created RxJava for the JVM environment to use their API Gateway. Similarly, the JavaScript platform has RxJS, which can run on the browser and Node.js platform. Using reactive programming methods can help quickly implement an efficient API Gateway code.
Service call
A microservice-based application is a distributed System, and must adopt the mechanism of communication between threads. There are two ways to communicate between threads. One is to use an asynchronous mechanism, a message-based method. Such implementation methods include JMS and AMQP. In addition, Zeromq, for example, belongs to direct communication between services. There is also a synchronization mechanism for inter-thread communication, such as Thrift and HTTP. In fact, a system will use both synchronous and asynchronous mechanisms at the same time. Since there are many ways to implement it, API Gateway needs to support multiple communication methods.
Service discovery
API Gateway needs to know the IP of each microservice And port. In traditional applications, you may hard-code these addresses, but in today’s cloud-based microservice applications, this will be a simple problem. Basic services usually use static addresses, which can be specified by operating system environment variables. However, it is not so easy to detect the address of an application service. Application services usually dynamically allocate addresses and ports. Similarly, due to expansion or upgrade, the service instance will also dynamically change. Therefore, API Gateway needs to use the system’s service discovery mechanism, or use server discovery, or The client found . A subsequent article will introduce this part in more detail. If you use the client to discover the service, API Gateway must go to the service registry , which is the database of microservice instance addresses.
Partial processing failed
In the process of implementing API Gateway, another The problem to consider is partial failure. This problem occurs in a distributed system when one service calls another service that times out or is unavailable. API Gateway should not be blocked and wait indefinitely for downstream services. However, how to deal with this failure depends on specific scenarios and specific services. For example, if the recommended service module on the product details page does not respond, API Gateway should return the remaining other information to the user, because this information is also useful. The recommended part can be returned to empty, or the fixed top 10 can be returned to the user. However, if the product information service is not responding, then API Gateway should return an error to the client.
API Gateway should be able to return to the cache when the cache is valid. For example, because the price of a product does not change frequently, API Gateway should return the value in the cache when the price service is unavailable. This type of data can be cached by API Gateway itself, or implemented by external caches such as Redis or Memcached. By returning cached data or default data, API Gateway ensures that system errors do not affect the user experience.
Netflix Hystrix is a very useful library for implementing remote service invocation code. Hystrix records calls that exceed preset limits. It implements circuit break 模式，使得可以将客户端从无响应服务的无尽等待中停止。如果一个服务的错误率超过预设值，Hystrix将中断服务，并且在一段时间内所有请求立刻失效。 Hystrix可以为请求失败定义一个fallback操作，例如读取缓存或者返回默认值。如果你在用JVM，就应该考虑使用Hystrix。如果你采用的非JVM环境，那么应该考虑采用类似功能的库。
总结
对于大多数微服务基础的应用，实现一个API Gateway都是有意义的，它就像是进入系统的一个服务提供点。 API Gateway负责请求转发、请求合成和协议转换。它提供给应用客户端一个自定义的API。 API Gateway可以通过返回缓存或者默认值的方式来掩盖后端服务的错误。在本系列的下一篇文章中，我们将讨论服务间的通信问题。
原文链接：Building Microservices: Using an API Gateway (翻译：陈杰；审校：杨峰)