JAVA visualVM troubleshooting Tool for Weblogic/FusionMiddleware Infrastructure

There are lot of variety of free troubleshooting and debugging tools exist but which ones are really useful when analyzing issues with WebLogic Server? 

In this blog I will talk out visualvm which is shipped with JDK itself.

VisualVM most recent version, is 1.3.6, This tool consists of an user-friendly visual interface for monitoring running JVMs. Multiple instances of WLS or Java can be monitored at the same time by just connecting to running local or remote JVMs. VisualVM needs to run on Oracle Sun JDK 6+, the JDK, not the JRE. VisualVM can be started as follow to specify the JDK to run it it on.

You can launch visualvm locally and remotely

For invoking Locally , you can set the display variable with a x-server
<JAVA_HOME>/bin/visualvm

But default it will identify all the java processes on the machine

For Invoking a remote jvm process from your local you need to enable jmx port on that domain
setDomainEnv.sh

export JAVA_OPTIONS="-Dcom.sun.management.jmxremote.port=3333 -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.authenticate=false"

and restart the domain

RightClick on Remote and first add host

You can now browse around the jvm statistics

If you install this plugin VisualGC, you get the error "Not Supported for this JVM", when you try to start Visual GC, try the following steps

1.  Start jstatd on the remote host. You need to specify a policy file, otherwise you will get an AccessControlException: access denied exception. Create a policy file e.g. /tmp/tools.policy containing: 

grant codebase "file:${java.home}/../lib/tools.jar" {

   permission java.security.AllPermission;

};

Then start jstatd using the command below: 

jstatd -J-Djava.security.policy=/tmp/tools.policy

You can write a small script that will run in background as below
runjstad.sh

/pkgs/JDK/jdk160_31/bin/jstatd -J-Djava.security.policy=/var/tmp/tools.policy  > /tmp/jstat.log 2>&1 &

2.   Start Visual VM. Add a Remote Host and then add a "jstatd connection" to it. Your JVM should appear in the list. You can then click on it and look at the "Visual GC" tab for garbage collection information.

This will list all java the processes on that host

Note: if you run jstat on the remote host , No need to use jmx port , You just need to add the host on the visualvm and it will automatically identify the all the java process on the host

Example visualGC screen:

This is very useful tool when troubleshooting weblogic performance issues

We can actually see the Live GC patterns