JVM performance tuning monitoring tool hprof use detailed

Ⅰ Problem

In real-world enterprise Java development, sometimes we encounter the following problems:

OutOfMemoryError
Memory leak
Thread deadlock
Lock Contention
Java process consumes too much CPU
……
These problems may be overlooked by many people in daily development (for example, some people encounter the above problems just restart the server or increase the memory, but do not go deep into the root cause of the problem), but the ability to understand and solve these problems is advanced by Java programmers. Essential requirements. This article will introduce some common JVM performance tuning monitoring tools hprof.

Ⅱ the generation of hprof files

The hprof file can be generated in DDMS (the full name of DDMS is DalvikDebug Monitor Service, which is the Dalvik virtual machine debugging monitoring service in Android development environment. Provides test device screen capture, view the running thread of specific process and heap information, Logcat, broadcast status information, Analog phone call, analog receiving and sending SMS, virtual geographic coordinates, etc.) Select the process by clicking the “dump hprof file” button in the upper left corner of the window to generate it directly. It can also be generated by adding code in the program. The following generates hprof file by setting.

We want to automatically generate the heap dump file when the memory overflows. To do this, we add the JVM arguments at runtime:
-XX:+HeapDumpOnOutOfMemoryError

jvm performance

Note: The .hprof file generated by dump is placed under the project directory by default.
First construct an entity class User, this User class is a general java class, then we construct an ArrayList, and then put an instance of this User class in an infinite loop, because the User class and ArrayList are on the heap, and ArrayList is a Strong reference, so can not be recycled by the GC, (because our List has been used and not destroyed), so once the heap memory occupied by the ArrayList fills the entire heap size, the heap overflows.

package com.deppon.tps;
public class User {
	private String name;
	private String sex;
	private int age;
	public String getName() {
		return name;
	}
	public void setName(String name) {
		this.name = name;
	}
	public String getSex() {
		return sex;
	}
	public void setSex(String sex) {
		this.sex = sex;
	}
	public int getAge() {
		return age;
	}
	public void setAge(int age) {
		this.age = age;
	}
    public User( String name,String sex,int age){
        this.name=name;
        this.sex=sex;
        this.age=age;
    }
}

Then we create a main method is inside the Test1 class, create an ArrayList, and infinitely add a User class object to it is:

package com.deppon.tps;
import java.util.ArrayList;
import java.util.List;
public class Test1 {
	public static void main(String[] args) {
		List<User> persons = new ArrayList<User> ();
        while( 1>0){
            persons.add( new User("liuhai","male",25));
        }
}

When running the above code, the heap overflows and a heap dump file is generated (here the VM parameter specifies that if the heap overflows, a heap dump is generated)

Use the MAT (Memory Analyze Tool) tool — plugin to download, install and open the hprof file. and analyze this hprof file (java_pid32430.hprof). We found that it did detect the memory leak problem, as follows:

jvm performance

As shown, it is clear here that there is a collection type in the main() method, and then each element in the collection is an object of com.charles.research.User, and each object’s Shallow Heap and Retained Heap size is 24bytes. Because the ArrayList always exists, when the object is long enough, the heap is filled and overflows. Here we created 76,430,681 User objects, each occupying 24 bytes, so it occupies a total of 24 * 7634068 = 1332217632 bytes, which is about 174.72M of heap space. Here, the S hallow Size and Retained Size of the User object are both 24 bytes?

What do we want to look at for Shallow Size and Retained Size?

Shallow Size is the size of the memory occupied by the object itself, and does not contain the object it references. The Shallow Size for regular objects (non-array) is determined by the number and type of its member variables, and the ShallowSize of the array is determined by the array type and array length, which is the sum of the array element sizes.

Retained Size=The current object size + the sum of the sizes of the objects that the current object can be directly or indirectly referenced. (Indirect reference meaning: A->B->C, C is an indirect reference), and excludes objects directly or indirectly referenced by GC Roots

So, our User class here, because our machine is a 32-bit WIN7 system, so the object header occupies 8 bytes, it contains a String object reference (name), occupies 4 bytes, contains a String object reference (sex), occupies 4 bytes, contains an int type (age), occupying 4bytes, so it occupies a total of 8+4+4+4=20bytes, because the number of digits is to be filled, so the final size is 24bytes. This is the size of the object itself (Shallow Heap).

Tip: In order to explain the completion, you can also experiment. If we add a String member to User, then the User class size is still 24 bytes, because the reference to this new String object is just 4 bytes to fill the pit. Adding a String member, the size of the User class is directly increased from 24 bytes to 32 bytes, because another pit needs to be filled.

And our User class does not introduce other classes (not including String, because String is directly referenced by the Root GC), so recycling the memory occupied by User is to recycle the User itself, so the Retained Heap size is equivalent to the Shallow Heap size.

Ⅲ set jvm parameters in tomcat

  1. Open the /tomcat_home/bin/catalina.sh file
  2. Plus: JAVA_OPTS=”$JAVA_OPTS -server -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=D:\heapdump”
jvm performance

Note: When -XX:HeapDumpPath is not set, the files that are dumped are in the /tomcat_home/bin directory.

Windows system

  1. Open the /tomcat_home/bin/catalina.bat file
  2. Plus: set JAVA_OPTS=%JAVA_OPTS% -server -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=D:\heapdump

As shown below:

jvm performance

Ali Open Source Java Diagnostic Tool-Arthas

Arthas is an open source tool for Java developed by Alibaba recently. It mainly for diagnosing the problems of java.

I. Overview

This tool can help you do the following things:

  1. Which jar package loaded this class from?
  2. Why do you code throws various kinds of Exceptions?
  3. If you encounter problems on product environment , you can’t debug well. Can you only add System.out on your code and republish it again and again to find problem by adding logs?
  4. Why hasn’t the online code been executed here? Is it because there is no commit in the code? Or is it the wrong branch code?
  5. There is a problem with the data processing of a user online, but it can not debug online and can not be reappear offline.
  6. Is there a global perspective to see how the system works?
  7. What is the way to monitor the real-time running state of JVM?

II. Installation Method

1.1 Windows Installation Mode

Download address: http://search.maven.org/classic/#search%7Cga%7C1%7Cg%3A%22com.taobao.arthas%22%20AND%20a%3A%22arthas-packaging%22

After downloading, decompress, as shown in the following:

Download the latest “bin.zip” package file. After decompression, there is a file named “as.bat” in the bin directory. For the time being, this script accepts only one parameter “pid”, so it can only diagnose Java processes on one machine.

The startup command is:

as.bat <pid>

Note: When I started Windows 10, I encountered the following problems

D:\download\arthas-packaging-3.0.4-bin > telnet 'telnet'is not an internal or external command, nor is it a runnable program Or batch files.

The solution is: Control Panel – > Start or Close Windows Function – > Check Telnet Function

1.2 Linux Installation Mode

Install Arthas:

curl -L https://alibaba.github.io/arthas/install.sh | sh

Start Arthas:

./as.sh

After successful startup, you will see the following interface.

III. Common orders

3.1 Basic Command

Help — View command help information
CLS – Clear the current screen area
Session — View information about the current session
Reset – Reset Enhanced Classes, which will be restored to all classes enhanced by Arthas. When the Arthas server is closed, all enhanced classes will be reset.
Version — Outputs the version number of Arthas loaded by the current target Java process
Quit – Exit the current Arthas client, other Arthas clients are unaffected
Shutdown – Close the Arthas server and all Arthas clients quit
Keymap – Arthas shortcut list and custom shortcut keys

JVM correlation

Dashboard – Real-time Data Panel for Current System
Thread — View thread stack information for the current JVM
JVM — View the current JVM information
Sysprop — Viewing and modifying JVM system properties
New! Getstatic — View static properties of classes

Class / classloader correlation

SC — View class information loaded by JVM
SM — View method information for loaded classes
Dump – dump loaded class byte code to a specific directory
Redefine – Load external. class files, redefine to JVM
JAD — Decompiles the source code that specifies the loaded class
Classloader – View the inheritance tree, urls, class loading information of classloader, and use classloader to getResource

Monitor/watch/trace correlation

Note that these commands are implemented by bytecode enhancement technology, which inserts some aspect into the method of the specified class to achieve data statistics and observation. Therefore, when online and pre-sent, please try to identify the classes, methods and conditions that need to be observed, and execute shutdown command at the end of diagnosis or execute reset command for enhanced class.

Monitor — Method Execution Monitoring
Watch – Method Executes Data Observation
Trace — Invoke paths within methods and output time-consuming paths on each node on the method path
Stack — Outputs the call path for the current method to be invoked
TT – Method executes the time-space tunnel of data, records the input and return information of each call of the specified method, and can observe these calls at different times.

Options

Options — View or set the Arthas global switch

Pipeline

Arthas supports the use of pipes to further process the results of the above commands, such as SM org. apache. log4j. Logger | grep
Grep — Search for results that satisfy conditions
Plaintext — Remove the color from the result of the command
WC – Statistical output by line

Web Console

Connect Arthas through websocket.

Other features

Asynchronous command support
Execution results are logged
Batch processing support
Usage Description of ognl Expressions

3.2 Use examples

First, in the window, enter help to see all available commands provided (the communication is essentially through telnet protocol), as follows:

Attach success.
Connecting to arthas server... current timestamp is 1537266148
Trying 127.0.0.1...
Connected to 127.0.0.1.
Escape character is '^]'.
  ,---.  ,------. ,--------.,--.  ,--.  ,---.   ,---.                           
 /  O  \ |  .--. ''--.  .--'|  '--'  | /  O  \ '   .-'                          
|  .-.  ||  '--'.'   |  |   |  .--.  ||  .-.  |`.  `-.                          
|  | |  ||  |\  \    |  |   |  |  |  ||  | |  |.-'    |                         
`--' `--'`--' '--'   `--'   `--'  `--'`--' `--'`-----'                          
                                                                                

wiki: https://alibaba.github.io/arthas
version: 3.0.4
pid: 25206
timestamp: 1537266148841

$ help
 NAME         DESCRIPTION                                                                                                                                                                      
 help         Display Arthas Help                                                                                                                                                              
 keymap       Display all the available keymap for the specified connection.                                                                                                                   
 sc           Search all the classes loaded by JVM                                                                                                                                             
 sm           Search the method of classes loaded by JVM                                                                                                                                       
 classloader  Show classloader info                                                                                                                                                            
 jad          Decompile class                                                                                                                                                                  
 getstatic    Show the static field of a class                                                                                                                                                 
 monitor      Monitor method execution statistics, e.g. total/success/failure count, average rt, fail rate, etc.                                                                               
 stack        Display the stack trace for the specified class and method                                                                                                                       
 thread       Display thread info, thread stack                                                                                                                                                
 trace        Trace the execution time of specified method invocation.                                                                                                                         
 watch        Display the input/output parameter, return object, and thrown exception of specified method invocation                                                                           
 tt           Time Tunnel                                                                                                                                                                      
 jvm          Display the target JVM information                                                                                                                                               
 dashboard    Overview of target jvm's thread, memory, gc, vm, tomcat info.                                                                                                                    
 dump         Dump class byte array from JVM                                                                                                                                                   
 options      View and change various Arthas options                                                                                                                                           
 cls          Clear the screen                                                                                                                                                                 
 reset        Reset all the enhanced classes                                                                                                                                                   
 version      Display Arthas version                                                                                                                                                           
 shutdown     Shut down Arthas server and exit the console                                                                                                                                     
 session      Display current session information                                                                                                                                              
 sysprop      Display, and change the system properties.                                                                                                                                       
 redefine     Redefine classes. @see Instrumentation#redefineClasses(ClassDefinition...)                                                                                                       
$ 

Here we mainly talk about watch, which monitors variables.

First, paste my test code:

package com.oct.tail;

import java.util.UUID;

/**
 * @Author Ryan
 * @Date 2018/9/18  9:58
 * @desc
 */
public class OtherTestCase {

    /**
     *
     * @return
     */
    public static String uuid(){
       return UUID.randomUUID().toString().replaceAll("-", "");
    }

    public static void main(String[] args) {

        while(true){
            System.out.println("uuid = " + uuid());

            try {
                Thread.sleep(1000);
            } catch (InterruptedException e) {
                e.printStackTrace();
            }
        }
    }
}

Here’s an example (I’m based on Windows 10, JDK 8 environment, Linux is the same). (For the watch command, I pretend I don’t know how to use it, and immediately enter watch help to see what’s going to happen.)

 ,---.  ,------. ,--------.,--.  ,--.  ,---.   ,---.
 /  O  \ |  .--. ''--.  .--'|  '--'  | /  O  \ '   .-'
|  .-.  ||  '--'.'   |  |   |  .--.  ||  .-.  |`.  `-.
|  | |  ||  |\  \    |  |   |  |  |  ||  | |  |.-'    |
`--' `--'`--' '--'   `--'   `--'  `--'`--' `--'`-----'


wiki: https://alibaba.github.io/arthas
version: 3.0.4
pid: 11924
timestamp: 1537326702039

$ watch -help
 USAGE:
   watch [-b] [-e] [-x <value>] [-f] [-h] [-n <value>] [-E] [-M <value>] [-s] class-
 pattern method-pattern express [condition-express]

 SUMMARY:
   Display the input/output parameter, return object, and thrown exception of specif
 ied method invocation
   The express may be one of the following expression (evaluated dynamically):
           target : the object
            clazz : the object's class
           method : the constructor or method
     params[0..n] : the parameters of method
        returnObj : the returned object of method
         throwExp : the throw exception of method
         isReturn : the method ended by return
          isThrow : the method ended by throwing exception
            #cost : the execution time in ms of method invocation
 Examples:
   watch -Eb org\.apache\.commons\.lang\.StringUtils isBlank params[0]
   watch -b org.apache.commons.lang.StringUtils isBlank params[0]
   watch -f org.apache.commons.lang.StringUtils isBlank returnObj
   watch -bf *StringUtils isBlank params[0]
   watch *StringUtils isBlank params[0]
   watch *StringUtils isBlank params[0] params[0].length==1
   watch *StringUtils isBlank '#cost>100'

 WIKI:
   https://alibaba.github.io/arthas/watch

 OPTIONS:
 -b, --before                Watch before invocation
 -e, --exception             Watch after throw exception
 -x, --expand <value>        Expand level of object (1 by default)
 -f, --finish                Watch after invocation, enable by default
 -h, --help                  this help
 -n, --limits <value>        Threshold of execution times
 -E, --regex                 Enable regular expression to match (wildcard matching b
                             y default)
 -M, --sizeLimit <value>     Upper size limit in bytes for the result (10 * 1024 * 1
                             024 by default)
 -s, --success               Watch after successful invocation
 <class-pattern>             The full qualified class name you want to watch
 <method-pattern>            The method name you want to watch
 <express>                   the content you want to watch, written by ognl.
                             Examples:
                               params[0]
                               'params[0]+params[1]'
                               returnObj
                               throwExp
                               target
                               clazz
                               method

 <condition-express>         Conditional expression in ognl style, for example:
                               TRUE  : 1==1
                               TRUE  : true
                               FALSE : false
                               TRUE  : 'params.length>=0'
                               FALSE : 1==2


$

Here, we monitor the return value of method UUID (). The monitoring results are as follows:

$
$
$ watch -f com.oct.tail.OtherTestCase uuid returnObj
Press Ctrl+C to abort.
Affect(class-cnt:1 , method-cnt:1) cost in 18 ms.
ts=2018-09-19 11:13:48;result=@String[26c80eb505664dbcb14f8d810fb4811c]
ts=2018-09-19 11:13:49;result=@String[fc03c43864f94372b646ce6253d90646]
ts=2018-09-19 11:13:50;result=@String[55ff41e0d66347c2bc75ab8ff4ffda4e]
ts=2018-09-19 11:13:51;result=@String[c504388c0aa74458a41a1b3a77c3d536]
ts=2018-09-19 11:13:52;result=@String[18d59c09ffde4c7aab15feb88b3e433f]
ts=2018-09-19 11:13:53;result=@String[c19dd8c1e5f8442696c8f886e81e74d5]
ts=2018-09-19 11:13:54;result=@String[d37a74aa502f4897aa1ed84dc69b83d8]
ts=2018-09-19 11:13:55;result=@String[cc11753b6f424c1e9a6a1ab36f334349]
ts=2018-09-19 11:13:56;result=@String[75a9b3c0bed4426d9363168912f16d74]
ts=2018-09-19 11:13:57;result=@String[f13022118e5a4115800a6eacc480e6a8]

It works so well that I can hardly believe it.