The Premier League, often referred to as the English Premier League or the EPL outside England, is the top level of the English football league system. Contested by 20 clubs, it operates on a system of promotion and relegation with the English Football League (EFL). Seasons run from August to May with each team playing 38 matches (playing all 19 other teams both home and away). Most games are played on Saturday and Sunday afternoons.
The competition was founded as the FA Premier League on 20 February 1992 following the decision of clubs in the Football League First Division to break away from the Football League, founded in 1888, and take advantage of a lucrative television rights deal. The deal was worth around £1 billion a year domestically as of 2013–14, with Sky and BT Group securing the domestic rights to broadcast 116 and 38 games respectively.
The league is a corporation in which the member clubs act as shareholders, and generates €2.2 billion per year in domestic and international television rights.
Clubs were apportioned central payment revenues of £2.4 billion in 2016–17, with a further £343 million in solidarity payments to English Football League (EFL) clubs.
The Premier League is the most-watched sports league in the world, broadcast in 212 territories to 643 million homes and a potential TV audience of 4.7 billion people.
For the 2018–19 season average Premier League match attendance was at 38,181,second to the Bundesliga’s 43,500,while aggregated attendance across all matches is the highest of any league at 14,508,981.Most stadium occupancies are near capacity.
The Premier League ranks second in the UEFA coefficients of leagues based on performances in European competitions over the past five seasons as of 2019, only behind Spain’s La Liga.
Forty-nine clubs have competed since the inception of the Premier League in 1992: forty-seven English and two Welsh clubs. Seven of them have won the title: Manchester United (13), Chelsea (5), Manchester City (4), Arsenal (3), Blackburn Rovers (1), Leicester City (1) and most recently Liverpool (1). The record of most points in a Premier League season is 100, set by Manchester City in 2017–18.
In real-world enterprise Java development, sometimes we encounter the following problems:
OutOfMemoryError Memory leak Thread deadlock Lock Contention Java process consumes too much CPU …… These problems may be overlooked by many people in daily development (for example, some people encounter the above problems just restart the server or increase the memory, but do not go deep into the root cause of the problem), but the ability to understand and solve these problems is advanced by Java programmers. Essential requirements. This article will introduce some common JVM performance tuning monitoring tools hprof.
Ⅱ the generation of hprof files
The hprof file can be generated in DDMS (the full name of DDMS is DalvikDebug Monitor Service, which is the Dalvik virtual machine debugging monitoring service in Android development environment. Provides test device screen capture, view the running thread of specific process and heap information, Logcat, broadcast status information, Analog phone call, analog receiving and sending SMS, virtual geographic coordinates, etc.) Select the process by clicking the “dump hprof file” button in the upper left corner of the window to generate it directly. It can also be generated by adding code in the program. The following generates hprof file by setting.
We want to automatically generate the heap dump file when the memory overflows. To do this, we add the JVM arguments at runtime: -XX:+HeapDumpOnOutOfMemoryError
Note: The .hprof file generated by dump is placed under the project directory by default. First construct an entity class User, this User class is a general java class, then we construct an ArrayList, and then put an instance of this User class in an infinite loop, because the User class and ArrayList are on the heap, and ArrayList is a Strong reference, so can not be recycled by the GC, (because our List has been used and not destroyed), so once the heap memory occupied by the ArrayList fills the entire heap size, the heap overflows.
package com.deppon.tps;
public class User {
private String name;
private String sex;
private int age;
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
public String getSex() {
return sex;
}
public void setSex(String sex) {
this.sex = sex;
}
public int getAge() {
return age;
}
public void setAge(int age) {
this.age = age;
}
public User( String name,String sex,int age){
this.name=name;
this.sex=sex;
this.age=age;
}
}
Then we create a main method is inside the Test1 class, create an ArrayList, and infinitely add a User class object to it is:
package com.deppon.tps;
import java.util.ArrayList;
import java.util.List;
public class Test1 {
public static void main(String[] args) {
List<User> persons = new ArrayList<User> ();
while( 1>0){
persons.add( new User("liuhai","male",25));
}
}
When running the above code, the heap overflows and a heap dump file is generated (here the VM parameter specifies that if the heap overflows, a heap dump is generated)
Use the MAT (Memory Analyze Tool) tool — plugin to download, install and open the hprof file. and analyze this hprof file (java_pid32430.hprof). We found that it did detect the memory leak problem, as follows:
As shown, it is clear here that there is a collection type in the main() method, and then each element in the collection is an object of com.charles.research.User, and each object’s Shallow Heap and Retained Heap size is 24bytes. Because the ArrayList always exists, when the object is long enough, the heap is filled and overflows. Here we created 76,430,681 User objects, each occupying 24 bytes, so it occupies a total of 24 * 7634068 = 1332217632 bytes, which is about 174.72M of heap space. Here, the S hallow Size and Retained Size of the User object are both 24 bytes?
What do we want to look at for Shallow Size and Retained Size?
Shallow Size is the size of the memory occupied by the object itself, and does not contain the object it references. The Shallow Size for regular objects (non-array) is determined by the number and type of its member variables, and the ShallowSize of the array is determined by the array type and array length, which is the sum of the array element sizes.
Retained Size=The current object size + the sum of the sizes of the objects that the current object can be directly or indirectly referenced. (Indirect reference meaning: A->B->C, C is an indirect reference), and excludes objects directly or indirectly referenced by GC Roots
So, our User class here, because our machine is a 32-bit WIN7 system, so the object header occupies 8 bytes, it contains a String object reference (name), occupies 4 bytes, contains a String object reference (sex), occupies 4 bytes, contains an int type (age), occupying 4bytes, so it occupies a total of 8+4+4+4=20bytes, because the number of digits is to be filled, so the final size is 24bytes. This is the size of the object itself (Shallow Heap).
Tip: In order to explain the completion, you can also experiment. If we add a String member to User, then the User class size is still 24 bytes, because the reference to this new String object is just 4 bytes to fill the pit. Adding a String member, the size of the User class is directly increased from 24 bytes to 32 bytes, because another pit needs to be filled.
And our User class does not introduce other classes (not including String, because String is directly referenced by the Root GC), so recycling the memory occupied by User is to recycle the User itself, so the Retained Heap size is equivalent to the Shallow Heap size.
Arthas is an open source tool for Java developed by Alibaba recently. It mainly for diagnosing the problems of java.
I. Overview
This tool can help you do the following things:
Which jar package loaded this class from?
Why do you code throws various kinds of Exceptions?
If you encounter problems on product environment , you can’t debug well. Can you only add System.out on your code and republish it again and again to find problem by adding logs?
Why hasn’t the online code been executed here? Is it because there is no commit in the code? Or is it the wrong branch code?
There is a problem with the data processing of a user online, but it can not debug online and can not be reappear offline.
Is there a global perspective to see how the system works?
What is the way to monitor the real-time running state of JVM?
After downloading, decompress, as shown in the following:
Download the latest “bin.zip” package file. After decompression, there is a file named “as.bat” in the bin directory. For the time being, this script accepts only one parameter “pid”, so it can only diagnose Java processes on one machine.
The startup command is:
as.bat <pid>
Note: When I started Windows 10, I encountered the following problems
D:\download\arthas-packaging-3.0.4-bin > telnet 'telnet'is not an internal or external command, nor is it a runnable program Or batch files.
The solution is: Control Panel – > Start or Close Windows Function – > Check Telnet Function
1.2 Linux Installation Mode
Install Arthas:
curl -L https://alibaba.github.io/arthas/install.sh | sh
Start Arthas:
./as.sh
After successful startup, you will see the following interface.
III. Common orders
3.1 Basic Command
Help — View command help information CLS – Clear the current screen area Session — View information about the current session Reset – Reset Enhanced Classes, which will be restored to all classes enhanced by Arthas. When the Arthas server is closed, all enhanced classes will be reset. Version — Outputs the version number of Arthas loaded by the current target Java process Quit – Exit the current Arthas client, other Arthas clients are unaffected Shutdown – Close the Arthas server and all Arthas clients quit Keymap – Arthas shortcut list and custom shortcut keys
JVM correlation
Dashboard – Real-time Data Panel for Current System Thread — View thread stack information for the current JVM JVM — View the current JVM information Sysprop — Viewing and modifying JVM system properties New! Getstatic — View static properties of classes
Class / classloader correlation
SC — View class information loaded by JVM SM — View method information for loaded classes Dump – dump loaded class byte code to a specific directory Redefine – Load external. class files, redefine to JVM JAD — Decompiles the source code that specifies the loaded class Classloader – View the inheritance tree, urls, class loading information of classloader, and use classloader to getResource
Monitor/watch/trace correlation
Note that these commands are implemented by bytecode enhancement technology, which inserts some aspect into the method of the specified class to achieve data statistics and observation. Therefore, when online and pre-sent, please try to identify the classes, methods and conditions that need to be observed, and execute shutdown command at the end of diagnosis or execute reset command for enhanced class.
Monitor — Method Execution Monitoring Watch – Method Executes Data Observation Trace — Invoke paths within methods and output time-consuming paths on each node on the method path Stack — Outputs the call path for the current method to be invoked TT – Method executes the time-space tunnel of data, records the input and return information of each call of the specified method, and can observe these calls at different times.
Options
Options — View or set the Arthas global switch
Pipeline
Arthas supports the use of pipes to further process the results of the above commands, such as SM org. apache. log4j. Logger | grep Grep — Search for results that satisfy conditions Plaintext — Remove the color from the result of the command WC – Statistical output by line
Web Console
Connect Arthas through websocket.
Other features
Asynchronous command support Execution results are logged Batch processing support Usage Description of ognl Expressions
3.2 Use examples
First, in the window, enter help to see all available commands provided (the communication is essentially through telnet protocol), as follows:
Attach success.
Connecting to arthas server... current timestamp is 1537266148
Trying 127.0.0.1...
Connected to 127.0.0.1.
Escape character is '^]'.
,---. ,------. ,--------.,--. ,--. ,---. ,---.
/ O \ | .--. ''--. .--'| '--' | / O \ ' .-'
| .-. || '--'.' | | | .--. || .-. |`. `-.
| | | || |\ \ | | | | | || | | |.-' |
`--' `--'`--' '--' `--' `--' `--'`--' `--'`-----'
wiki: https://alibaba.github.io/arthas
version: 3.0.4
pid: 25206
timestamp: 1537266148841
$ help
NAME DESCRIPTION
help Display Arthas Help
keymap Display all the available keymap for the specified connection.
sc Search all the classes loaded by JVM
sm Search the method of classes loaded by JVM
classloader Show classloader info
jad Decompile class
getstatic Show the static field of a class
monitor Monitor method execution statistics, e.g. total/success/failure count, average rt, fail rate, etc.
stack Display the stack trace for the specified class and method
thread Display thread info, thread stack
trace Trace the execution time of specified method invocation.
watch Display the input/output parameter, return object, and thrown exception of specified method invocation
tt Time Tunnel
jvm Display the target JVM information
dashboard Overview of target jvm's thread, memory, gc, vm, tomcat info.
dump Dump class byte array from JVM
options View and change various Arthas options
cls Clear the screen
reset Reset all the enhanced classes
version Display Arthas version
shutdown Shut down Arthas server and exit the console
session Display current session information
sysprop Display, and change the system properties.
redefine Redefine classes. @see Instrumentation#redefineClasses(ClassDefinition...)
$
Here we mainly talk about watch, which monitors variables.
Here’s an example (I’m based on Windows 10, JDK 8 environment, Linux is the same). (For the watch command, I pretend I don’t know how to use it, and immediately enter watch help to see what’s going to happen.)
,---. ,------. ,--------.,--. ,--. ,---. ,---.
/ O \ | .--. ''--. .--'| '--' | / O \ ' .-'
| .-. || '--'.' | | | .--. || .-. |`. `-.
| | | || |\ \ | | | | | || | | |.-' |
`--' `--'`--' '--' `--' `--' `--'`--' `--'`-----'
wiki: https://alibaba.github.io/arthas
version: 3.0.4
pid: 11924
timestamp: 1537326702039
$ watch -help
USAGE:
watch [-b] [-e] [-x <value>] [-f] [-h] [-n <value>] [-E] [-M <value>] [-s] class-
pattern method-pattern express [condition-express]
SUMMARY:
Display the input/output parameter, return object, and thrown exception of specif
ied method invocation
The express may be one of the following expression (evaluated dynamically):
target : the object
clazz : the object's class
method : the constructor or method
params[0..n] : the parameters of method
returnObj : the returned object of method
throwExp : the throw exception of method
isReturn : the method ended by return
isThrow : the method ended by throwing exception
#cost : the execution time in ms of method invocation
Examples:
watch -Eb org\.apache\.commons\.lang\.StringUtils isBlank params[0]
watch -b org.apache.commons.lang.StringUtils isBlank params[0]
watch -f org.apache.commons.lang.StringUtils isBlank returnObj
watch -bf *StringUtils isBlank params[0]
watch *StringUtils isBlank params[0]
watch *StringUtils isBlank params[0] params[0].length==1
watch *StringUtils isBlank '#cost>100'
WIKI:
https://alibaba.github.io/arthas/watch
OPTIONS:
-b, --before Watch before invocation
-e, --exception Watch after throw exception
-x, --expand <value> Expand level of object (1 by default)
-f, --finish Watch after invocation, enable by default
-h, --help this help
-n, --limits <value> Threshold of execution times
-E, --regex Enable regular expression to match (wildcard matching b
y default)
-M, --sizeLimit <value> Upper size limit in bytes for the result (10 * 1024 * 1
024 by default)
-s, --success Watch after successful invocation
<class-pattern> The full qualified class name you want to watch
<method-pattern> The method name you want to watch
<express> the content you want to watch, written by ognl.
Examples:
params[0]
'params[0]+params[1]'
returnObj
throwExp
target
clazz
method
<condition-express> Conditional expression in ognl style, for example:
TRUE : 1==1
TRUE : true
FALSE : false
TRUE : 'params.length>=0'
FALSE : 1==2
$
Here, we monitor the return value of method UUID (). The monitoring results are as follows:
Recently, I have been working for a subject storehouse system. Since duplicate subject are added to the storehouse, it is necessary to query the duplicate subject, and delete all duplicate subject leave only one, so that the duplicate subject cannot be taken when the test is taken.
First wrote a small example:
Single field operation
This is the table in the database:
Group By:
Select Repeat Field From Table Group By Repeat Field Having Count(*)>1
See if there is duplicate data:
GROUP BY <column name sequence>
HAVING <group conditional expression>
Query out: those groups that satisfy the group conditional expression (the number of repetitions is greater than 1) in the having clause according to the dname grouping
There is no difference between count(*) and count(1).
Query all duplicate data:
Select * From Table Where Repeat Field In (Select Repeat Field From Table Group By Repeat Field Having Count(*)>1)
Delete all duplicate subject:
Change the above query select to delete (this will cause an error)
DELETE
FROM
dept
WHERE
dname IN (
SELECT
dname
FROM
dept
GROUP BY
dname
HAVING
count(1) > 1
The following error will occur: [Err] 1093 – You can’t specify target table ‘dept’ for update in FROM clause
The reason is: update this table and query this table at the same time, query this table and update the table, can be understood as deadlock. Mysql does not support this update to query the same table operation
Solution: Query the columns of data to be updated as a third-party table, and then filter the updates.
query duplicate subject in the table (according to depno, except for the one with the smallest rowid)
the first method:
SELECT
*
FROM
dept
WHERE
dname IN (
SELECT
dname
FROM
dept
GROUP BY
dname
HAVING
COUNT(1) > 1
)
AND deptno NOT IN (
SELECT
MIN(deptno)
FROM
dept
GROUP BY
dname
HAVING
COUNT(1) > 1
)
The above is correct, but the query is too slow, you can try the following method:
The second method:
According to the dname grouping, find out the deptno minimum. Then query deptno and do not contain the one just found out. This will query all the duplicate data (except the smallest line of deptno)
SELECT *
FROM
dept
WHERE
deptno NOT IN (
SELECT
dt.minno
FROM
(
SELECT
MIN(deptno) AS minno
FROM
dept
GROUP BY
dname
) dt
)
Delete redundant duplicate subject in the table and leave only one:
the first method:
DELETE
FROM
dept
WHERE
dname IN (
SELECT
t.dname
FROM
(
SELECT
dname
FROM
dept
GROUP BY
dname
HAVING
count(1) > 1
) t
)
AND deptno NOT IN (
SELECT
dt.mindeptno
FROM
(
SELECT
min(deptno) AS mindeptno
FROM
dept
GROUP BY
dname
HAVING
count(1) > 1
) dt
)
The second method (corresponding to the second method of query above, just change select to delete):
DELETE
FROM
dept
WHERE
deptno NOT IN (
SELECT
dt.minno
FROM
(
SELECT
MIN(deptno) AS minno
FROM
dept
GROUP BY
dname
) dt
)
Operation of multiple fields:
If you already have been learn to delete a single field is there, multiple fields are also very simple.
DELETE
FROM
dept
WHERE
(dname, db_source) IN (
SELECT
t.dname,
t.db_source
FROM
(
SELECT
dname,
db_source
FROM
dept
GROUP BY
dname,
db_source
HAVING
count(1) > 1
) t
)
AND deptno NOT IN (
SELECT
dt.mindeptno
FROM
(
SELECT
min(deptno) AS mindeptno
FROM
dept
GROUP BY
dname,
db_source
HAVING
count(1) > 1
)
to sum up: In fact, there are still a lot of things to optimize in the above methods. If the amount of data is too large, the implementation is very slow. You can consider adding optimization:
Add an index to the frequently queried field
Change * to the field you need to query, don’t query it all out.
Small tables drive large tables with IN, large tables drive small tables with EXISTS. The case where IN is suitable is the case where the amount of external data is small, not the case where the external data is large, because IN will traverse all the data of the outer table, assuming 100 tables of a, and 10000 of b, the number of traversals is 100*10000 times, and exists Then, it is executed 100 times to judge whether the data in the a table exists in the b table, and it only executes the number of a.length. As for which efficiency is high, it depends on the situation, because in is compared in memory, and exists is the database query operation.
Logstash converts the @timestamp time to the local time. If the local time zone is the Beijing time zone, add the following filter to solve the problem.
Introduce a very useful Java process monitoring tool, which can print out the average time and maximum time consumption of each method in the Java class, which is very helpful for troubleshooting delays in Java programs. The name of this tool is: Greys-Anatomy.
Greys positioning is a professional JVM business problem location tool. Since it is a JVM, most of the things we face are Java programmers. I hope to share all the skills and ideas I have when writing software, so that more Java programmers can participate in or benefit from development.
Main characteristics View class and method information that has been loaded by the JVM
1, method execution monitoring
Call volume, success failure rate, response time
2, the method performs data operations
Input parameters, return value, exception information record and view; support action playback
3, performance overhead rendering
Tracking method call trajectories in a specified path, taking time
After the installation is complete, you can write a shell script, similar to the following (my-gresy.sh, assuming the java program is in the tomcat container):
Shell scripts monitor server processes and ports
Recently learned shell programming, wrote a script that can monitor the port, PID, program name, etc. used by the current server; it can be used to find out if there are any ports that are not commonly used to be intercepted, and then judge whether it has been “engaged” by the hacker;
code show as below:
#tcp part
Port1=`netstat -an|grep LISTEN|egrep "0.0.0.0|:::"|awk '/^tcp/ {print $4}'|awk -F: '{print $2$4}'|sort -n`
Echo "TCP state:"
Echo "--------------------------------"
Echo "PORT PID COMMAND"
For a in $port1
Do
b=`lsof -n -i:$a|grep TCP|grep LISTEN|grep IPv4|awk '{printf("%d\t%s\n"),$2,$1}'`
Echo "$a $b"
Done
Echo "--------------------------------"
#udp part
Echo ""
Port2=`netstat -an|grep udp|awk '{print $4}'|awk -F: '{print $2}'|sed '/^$/d'|sort -n`
Echo "UDP state:"
Echo "--------------------------------"
Echo "PORT PID COMMAND"
For a in $port2
Do
b=`lsof -n -i:$a|grep UDP|grep IPv4|awk '{printf("%d\t%s\n"),$2,$1}'`
If [ -n "$b" ];then
Echo "$a $b"
Fi
Done
Echo "--------------------------------"
Exit 0
How to access static php files under the yii framework without having to create multiple actions, here is a simple record, I hope to leading to a better implementation: 1, configured in main.php
‘post/.html’=>’post/page/’ This line of code is the most important
2, implement a postController
<?php
class PostController extends Controller{
public function actions() {
return array (
'page' => array (
'class' => 'CViewAction'
)
);
}
}
3, add the post/pages directory in the corresponding views directory, and then add a static php file (such as 12345.php) in the pages directory It can be accessed via http://domainname/post/12345.html. If there is a subdirectory (such as 20120920/123456.php), you can pass
