[Technical blogs] Use CDNs to speed up website visits
2s: most users are willing to wait
10s: the limit for keeping the user’s attention focused on the dialogue
15s: tolerant time limit of most users
– Fiona Fui-Hoon Nah, in A study on tolerable waiting time: how long are Web users willing to wait?, 2007
Website access speed is important for user experience component. Users tend to give a better response to a website that is responsive, and leave an impression that the team’s technical level is low for a website that often freezes. (Example: Meisai official website) Therefore, we have carried out various optimizations on the website in the beta stage, using caching, CDN, optimized loading and other means, which greatly improved the access speed of the website. We plan to first analyze the problem of slow loading, and then introduce our solutions and technologies, and finally compare the optimization effects.
Catalog
- [Technical Blog] Use CDN to speed up website access
- Catalog
- 1. Discover Problem
- 2. Analyze the problem
- 2.1. Webpage loading time and waterfall chart
- 2.2. Detailed analysis
- 2.2.1. Home page
- 2.2.2. Search all course pages
- 2.2.3. Simulate random search pages
li>
- 3. Technology used
- 3.1. Django cache
- 3.2. Database table caching
- 3.3. Web page rendering logic
- 3.4. CDN website cache
- 3.4.1. page rule
- 3.4.2. under attack mode
- 3.4.3. Content compression
- 3.5. Public js/css library
- 3.6. Public image bed
- 4. Acceleration effect
< /ul>
1. Problems found
After our website was attacked (detailed), we decided to use CDN for website protection. After investigating the products of various companies and checking empty budgets, we decided to adopt CloudFlare’s free protection solution and deploy the website on GitHub Student Pack< rt>On the vps brought by capitalist wool. After solving the malicious attack problem, we encountered a new problem: the website loads slowly. The home page takes about 5 seconds to load on average, while the search page even takes about 20 seconds to load when there are a large number of items.
2. Analyze the problem
2.1. Webpage load time and waterfall chart
We Three typical pages were selected and analyzed using the chrome debugging console. They are the home page (a certain page to be visited), search all course pages (the longest time-consuming and occupy the most server resources), and search the page with the “mathematics” keyword (simulation) Random search), the loading time and waterfall chart are as follows.
Home page:
Search all courses:
Simulate random search:
2.2. Detailed analysis
We analyze the reasons for the slow loading of these three pages one by one, mainly through analysis The time-consuming part.
2.2.1. Homepage
As you can see, the slowest loading of homepage elements is the background image and font. In addition, website page loading and js loading also took a long time. Therefore, we plan to use public CDN for static resources and cloudflare acceleration for the website.
2.2.2. Search all course pages
You can see that web page rendering takes the most time. In addition, it takes more time to obtain search results. For a long time (there should be a searchCourse request here, I don’t know why it can’t be intercepted), the search process server occupies a lot.
2.2.3. Simulate random search page
The loading time of this page is still qualified, but in fact it is because the js resource is being used to access the homepage. Loaded, read the cache directly here. In addition, website rendering time is still long.
For the search result page, we plan to solve it from the front-end code (rendering logic) and the back-end (cache) and CDN (cache).
3. The technology used
In order to solve the problem of slow loading of the above pages, we finally adopted the following technologies: Django cache, database table cache, Web page rendering logic, CDN caching of pages and resources, public js/css library, image bed. Let’s introduce them one by one below.
3.1. Django cache
Django comes with a more complete cache system, which is used to make a request for frequent requests or requests that occupy resources. Cache. Our search course interface here is more in line with the needs of caching. The main reasons are: the search results are stable, and the search process will take up more resources (accessing the database, processing data) when preparing the returned json.
Django cache is divided into the following types: Memcached, database, file, local memory, and other open source methods on GitHUb. For specific deployment, you can Google it yourself or refer to this blog. The deployment process is relatively simple.
In our actual application, considering that the local disk space is sufficient, the reading and writing speed is acceptable, and we don’t want to configure other software, we use the local file cache method and set the cache expiration. The time is 15 minutes.
The key code is as follows:
settings.py:
CACHES = {# File-based cache, based on other caches refer to the blog above and simply modify this段可以# When testing, you should use dummy cache'default': {'BACKEND':'django.core.cache.backends.filebased.FileBasedCache','LOCATION': YOUR_CACHE_DIR_PATH, }}
Add a decorator in front of each view function that needs to be cached:
from django.views.decorators.cache import cache_page@cache_page(15*60) # 15 minutedef xxx(request) : yyy
3.2. Database building table cache
Our interface for obtaining scores and rankings is another interface that consumes more resources. Because our ranking adopts some scientific scoring methods (detailed), the execution time of the algorithm increases as the number of scoring items increases. Sometimes it takes about 2 minutes. The resource occupation is more serious than the above interface, and if it is requested by the user Recalculation will cause the gateway to time out, so we use the database to create tables for caching.
We use a timed task (refer to the tutorial for the usage of timed tasks) to update the rankings of courses and teachers every two hours, and write the updated rankings into the RankCache table. For user requests, directly perform table lookup operations on the table. This is equivalent to updating the ranking every two hours, and the user queries the most recently updated cache.
The key code is as follows:
settings.py added:
CRONJOBS = [('* */2 * * *','ratemycourse.view. rankcache.cronjob','>> django_crontab.log')]
and add in the installed app
'django_crontab',
rankcache.py:
The specific ranking algorithm here can refer to the implementation of the blog above
def cronjob(): a=Rankers() # Declare the ranking class a.read_data() # Get the score information from the database a.run_rank() # Calculate the score a.save_to_database() # Write the score to the database rancache table
3.3. Webpage rendering logic
For the search results page, we simply rendered the entire web page at one time in the alpha stage and loaded all the course divs, which resulted in a longer rendering time. At this stage, we have changed to on-demand rendering, rendering only a few course information of the current page each time, reducing rendering time.
The approximate code is as follows:
//2 Get the course number to start loading var course_to_show=(pagenum-1)*course_num_per_page; //3 Load page number Content, use adddiv //clear course_data $("#course_data").html(""); for(var i = course_to_show;i
3.4. CDN website cache
We use cloudflare’s free The package speeds up the website, and the basic configuration process is very simple. Just follow the steps of the website step by step. Here we introduce the special adjustments we made to our website.
3.4.1. page rule
Page rule is a higher-level function provided by cloudflare, similar to IFTTT, which targets certain URLs according to rules The cache selection supports regular matching. Page rule is a higher-level function of cloudflare. The free tier supports 3 items, which are few but enough to use. The operation method is similar to the following figure, enter the regular website address, select add a setting, and then select the specific operation.
Our configuration is as follows:
The specific description is: for all scoring interfaces, set the browser cache expiration time 2h, set the CF edge node cache expiration time 2h, and ignore the querystring for caching.
In fact, this interface can also be used as a Django cache, but we want to try different caching methods, so we adopt the page rule. In fact, the page rule is slightly faster, probably because the cached information is returned directly at the edge node, without the need to reach our server.
3.4.2. under attack mode
Since our website has been attacked before, we have turned on under attack mode. This switch can provide 6 different intensities of js challenge for requesting user's IP and other information to protect the website.
3.4.3. Content compression
cloudflare can compress html, js, css and reduce the size of the transmitted data. In addition, you can also enable Brotli, http2, Mirage, Mobile Redirect and other new protocols, algorithms, and frameworks to further compress website data and speed up transmission. These settings can be modified in the speed tab of the console.
3.5. Public js/css library
We found that loading js, css, font also took a lot of time. Fortunately, for these public libraries, such as jquery and bootstrap, we can load them through public CDNs, which greatly improves the loading speed of these files and saves server traffic. After investigation, we found that Toutiao, Qiniu Cloud, etc. provide this service, for example: Toutiao CDN, Staticfile CDN.
An example is as follows:
// Before modification // After modification
For all public css and js of the website, you can find it in the search page of the above CDN, just copy the corresponding link and replace it.
3.6. Public picture bed
For background pictures, etc., we can use the public picture bed to speed up. We use the aggregation image bed, SM.MS image bed to host our pictures, speed up the access speed. The operation method is similar to the above js and css.
Aggregate image bed can provide multiple backups at one time, and automatically select one to return when requested. Generally, the images returned are uploaded to Ali cdn or Qiniu Cloud. Registration is required before use. Advanced features. Such as api, the maximum number of pictures, etc. need to be charged.
The main feature of SM.MS image bed is that it is free and can be uploaded arbitrarily without registration (100 sheets/minute), provides api, and supports ipv6. The main server should be in Hong Kong, so compared to the aggregation map bed ipv4 access will be slower, some meeting dates may be slower, but campus network ipv6 access is generally very stable.
4. Acceleration Effect
After using the above methods, our web page loading speed has been greatly improved. We compared alpha web pages, alpha+CDN and beta (all the above methods), and the effect is as follows:
Home< /th>
All courses
random search
alpha-CDN
4.87s
8.77s
1.83s
alpha-no CDN
5.4s
15.76s
2.89s
beta
1.94s
1.25s
1.48s
Blue is the original loading time, orange is Only use CDN time, the gray is the load time of applying all the above methods.
Visible, all pages Both have been greatly improved.
The waterfall chart of all pages is as follows, the order is alpha-no CDN, alpha-CDN, beta:
