Node Manager Basics and Setup

about Node Manager

Node Manager is an important utility when running WebLogic in a production environment. Most people pay little attention to the Node Manager, but when problem arises, it is a little bit late to figure out what Node Manager is and what it does.

This article is not a full introduction to Node Manager, but will describe some best practices for common real life situations and issues, and share some tips and tricks gathered over the years.

If you would like a full introduction to Node Manager first, with all the nice diagrams showing how everything is connected, then you should look at the official documentation, which by the way are getting pretty good, before coming back to read the rest of this article -  http://docs.oracle.com/middleware/1213/wls/NODEM/overview.htm#NODEM112

So, what is Node Manager?

Node Manager is a utility used to start, stop and restart WebLogic and Coherence servers on the host Node Manager is installed. Since you could connect to Node Manager from a remote server, you will use Node Manager on the remote server as a tool to remotely start and stop a WebLogic instance.

There are two version of Node Manager, the Java version and the script based version. Unless you have very specific requirements, use the Java version, which also happens to be the one that most people think of when they talk about Node Manager. This article will be about the Java version, but if you want to know more about the difference of the two versions, have a look at the official documentation - http://docs.oracle.com/middleware/1213/wls/NODEM/overview.htm#NODEM114

Per host vs per domain Node Manager

Up until WebLogic 12.1.2 the default was one Node Manager per host. It was possible to do it otherwise, but it was not documented very well.

From 12.1.2 the default is one Node Manager per domain, and the configuration and logs are now located under $DOMAIN_HOME/nodemanager instead of under wlserver_10.3/common/nodemanager

It is still possible to have one Node Manager per host, for those who prefers that.

If you are using Node Manager for an earlier release (before 12.1.2) or just want to stick with one Node Manager per host, you should consider moving the configuration and log files away from wlserver_10.3/common/nodemanger and have them outside the $ORACLE_HOME. There are several reasons for that, one important reason is to keep configuration files away from the binaries, so the configuration is kept in case an upgrade overwrites everything under wlserver_10.3/common/nodemanager. Another important reason is that you want to have logfiles in a location that you monitor for disk space, since logfiles tends to grow, and you might not have taken that into consideration when deciding how much space you needed for the installation.

One special case where you actually needed more than one Node Manager for a host in 11g was when you followed the Enterprise Deployment Guide (EDG) for SOA. In an enterprise setup, the Administration Server is installed in a different $DOMAIN_HOME than the managed servers. This makes it easier to start the Administration Server on another host in case of a host failure (Administration Server domain directory should be on a SAN or a shared disk).

The reason for using one Node Manager for the Administration Server domain directory and one for the Managed Servers domain directory is that the nodemanager.domains file in 11g could only can have one path per domain.

In 12c there is possible to have one primary domain path and several alternate domain paths for each domain in nodemanager.domains, so it should be enough with one Node Manager per domain from 12.1.3 http://docs.oracle.com/middleware/1213/wls/NODEM/java_nodemgr.htm#NODEM163

Since one Node Manager per domain could mean several Node Manager instances on one host, every Node Manager instance on that host must have a unique ip:port combination.

How to start Node Manager?

The simple answer is to run startNodeManager.sh from $DOMAIN_HOME/bin, but except for testing, you should not do that.

Node Manager should be started as a system service when the server boots, so a startup script is needed. In the WebLogic 12.1.3 documentation there are init.d and xinetd scripts provided for Linux - http://docs.oracle.com/middleware/1213/wls/NODEM/java_nodemgr.htm#NODEM247

For Windows there is the installNodeMgrSvc.cmd script included in the $DOMAIN_HOME/bin directory.

Node Manager will run as the user who started it. When Node Manager starts a WebLogic instance, either an Administration Server or a Managed Server, the WebLogic instance will run as the same user as the user who runs Node Manager.

There is currently some flaws in the documentation, both in the init.d script for Linux and in the description of installNodeMgrSvc.cmd for Windows. Oracle has been notified, and the documentation should be updated soon.

By using the init.d script from the documentation, Node Manager will run as root, and by starting it as a Windows service, it will run as System.

Running WebLogic as root/System could be a security risk, and when you are going to patch WebLogic, you will most likely do that as the user you installed WebLogic as. However, since some files have changed ownership/rights to root/System, you might not be able to patch without a lot of hassle.

On Linux there is a post-bind function in WebLogic, so it is possible to configure WebLogic to change the user from root to something less hazardous, but this is not possible in Windows.

So, what should we do?

For Linux, make sure Node Manager is started as a non-root user, by changing the line in the init.d file in the documentation from

$PROGRAM_START >> $OUT_FILE 2>&1  &

to

su -c $NMUSER $PROGRAM_START >> $OUT_FILE 2>&1  &

The variable NMUSER could be declared somewhere between

#### BEGIN CUSTOM ENV

and

#### END CUSTOM ENV

as

NMUSER=oracle

Replace oracle with the user you want to run Node Manager and WebLogic.

For Windows you could go to services and properties for the WebLogicNodeManager service. On the “Log On” tab, select “This account” and select the user account that you want to run Node Manager (and WebLogic) as.

​

Username and password

To log into Node Manager, you have to use a username and password. Unfortunately, the default username is weblogic, which also happens to be the default username for the WebLogic Administration server. However, these two weblogic users are not the same, it is just a coincident that their names are the same. There is no relation between those two user accounts. But since they have the same name, many people think it is the same account (and I don’t blame them, how are you supposed to know it's not), which might lead to problems when changing password, since you then have changed password for just one of the accounts, not both. And when you try the new password on the “wrong” weblogic account, you won't be allowed access. You could end up spending the better part of the night to figure out what is wrong with your password.

The recommendation is to use a different username for the Node Manager account, I often use the name nodemanager.

​

To avoid any extra confusion, the operating system user which Node Manager runs as, has nothing to do with which user you use to log into Node Manager.

​

Verify connection between Administration Server and Node Manager

To verify that the Administration Server is able to communicate with Node Manager, check the Administration Console

​

If status is not Reachable, then username/password could be wrong, or SSL is enabled on one side and not the other, or if Node Manager is on a remote host, there could be network or firewall issues.

Create an encrypted credentials file

When using scripts to connect to Node Manager, you have to log in. Passing credentials in clear text is not a good idea, since it often also means that the credentials are available in clear text in script files or in a property file.

Node Manager support the use of an encrypted credential file. You should still protect this file, but it is better than clear text.

If you follow this advice, no one should need to log into Node Manager by providing the password by hand. Which means that there is no reason to have a password that is easy to remember, or easy to type. Make the password long and random, you will only type it a few times (to create the credentials file).

Creating the encrypted credential file is different in 11g and 12c. The 11g method still works in 12c, but this method is deprecated and might stop working.

11g method - source setDomainEnv.sh before running this command to get the paths correctly (everything on one line)

$ java weblogic.Admin -username nodemanager -userconfigfile /u01/app/oracle/config/nmUserFile -userkeyfile /u01/app/oracle/config/nmKeyFile STOREUSERCONFIG

12c method - source setDomainEnv.sh before running this command to get the paths correctly

$ java weblogic.WLST

wls:/offline> nmConnect( 'nodemanager','welcome1','localhost',5556,'mydomain', '/u01/ config/user_projects/domains/mydomain', 'plain')

wls:/mydomain/serverConfig> storeUserConfig( '/u01/app/oracle/config/nmUserFile','/u01/app/oracle/config/nmKeyFile', 'true')

Usage of the encrypted credentials file

As in the 12c method above, nmConnect usually takes a series of literal parameters. But when you are using an encrypted credential file, you don’t want to use literal parameters, at least not for username/password. The alternative nmConnect format is below:

nmConnect(userConfigFile=nmUserFile, userKeyFile=nmKeyFile, host=nmHost, port=nmPort, domainName=domain, domainDir=domainPath, nmType=nmType)

In this example, all parameters are replaced with variables. Of course, you could still have literal values, but using variables in scripts are often considered best practice.

Calling the WLST scripts with -loadProperties <path to properties file>, you have access to variables from the property file directly in your WLST scripts. An example properties file is below:

adminUrl=t3://wls12c.dev.sysco.no:7001

nmHost=wls12c.dev.sysco.no

nmPort=5556

nmUserFile=/u01/app/oracle/config/nmUserFile

nmKeyFile=/u01/app/oracle/config/nmKeyFile

nmType=plain

domain=mydomain

domainPath=/u01/config/user_projects/domains/mydomain

Change password

Even though very few needs to know the Node Manager password if you use the encrypted credential file, you should still change the Node Manager password from time to time.

The easiest and recommended way is to use the Administration Console to change the password.

It is possible, but not recommended, to change the password directly in nm_password.properties. If you do that, you have to restart Node Manager after changing the password. If you change the username and/or password from the Administration Console, you do not have to restart Node Manager.

Remember, after you have changed the password, you have to regenerate the nmUserFile with the new password (just follow the instructions for creating a new credential file).

http://docs.oracle.com/cloud/latest/fmw121300/NODEM/java_nodemgr.htm#NODEM161

How to add a new Node Managers (hosts)

The recommended way to add a new host, is to use the pack command on the domain on an existing server and do an unpack on the new server.

It is also possible to use the WLST command nmEnroll() to add the new host to the domain.

Some important files

nodemanager.properties

nodemanager.properties contains most of the configuration for Node Manager. The file is created first time Node Manager is started.

StartScriptEnabled=true tells Node Manager to use the start script to start WebLogic. By default, startWeblogic.sh (or .cmd) is used, unless StartScriptName is set to something else.

Be careful about StopScript, it might not be what you think. It is not the opposite of StartScript, a script used to stop the WebLogic instance. It is a cleanup script, which will be called after the WebLogic instance has stopped.

ListenAddress and ListenPort are important when running more than one Node Manager on a server. Every instance of Node Manager needs an unique ListenAddress:ListenPort combination. If everyone tries to use the default, only the first instance to start will succeed, the rest will fail.

SecureListener is default set to true, which means Node Manager can only be access by using SSL. If you do not have enabled SSL on the Administration Server, you must set SecureListener=false (and restart Node Manager if it is running).

If you think using SSL is a good idea (which it is), then you should replace the DemoTrust certificates with new ones (every installation are shipped the same DemoTrust certificates, so they are not secure at all).

Be aware that several property names are deprecated, but still works in 12.1.3, and that some of the new ones introduced in 12.1.3 is not yet supported in WLST offline mode and not yet supported by pack/unpack. Examples of deprecated property names are StartScriptEnabled and StartScriptName, from 12.1.3 use weblogic.StartScriptEnabled and weblogic.StartScriptName instead.

Not all possible properties are in the nodemanager.properties file, for a complete list of Node Manager properties, see the documentation:

http://docs.oracle.com/middleware/1213/wls/NODEM/java_nodemgr.htm#NODEM178

nm_password.properties

Encrypted Node Manager username and password file. Used to authenticate connection between Node Manager and clients (i.e., Administration Server, when Administration Server connects to Node Manager to start or stop a Managed Server, and when a WLST script connects to Node Manager to do some tasks, i.e., start or stop a Managed Server).

The Configuration Wizard puts the Node Manager username and password for the initial connection into nm_password.properties on the local server. To get it distributed to other servers, you have to use the WLST command nmEnroll(). nmEnroll() will either create a new nm_password.properties file, it if does not exists, or update an exisiting nm_password.properties file.

If you change the nm_password.properties manually, which is not recommended by the way, you have to restart Node Manager. If you change the username and/or password from the Administration Console, you do not have to restart Node Manager.

nodemanager.domains

10.3.6

domain-name=domain-directory

12.1.3

domainName=primaryDomainPath;alternateDomainPath1;alternateDomainPath2

<servername>.out logfiles

Generated by Node Manager, located together with the Administration Server or Managed Servers .log files.

No daily / by size rotation (at least not in 11g), so this file could be big. Rotated when the server restarts.

Summary

Node Manager is a very useful utility, and although you do not have to use Node Manager, it might do the operation of WebLogic somewhat easier.

Best practices for starting and stopping WebLogic instances are using WLST scripts together with Node Manager, and some high availability functions, like whole server migration, requires Node Manager.

Node Manager is not very difficult, but like any other software, it helps to know a little about it. I hope that this article has given you some more information of Node Manager, and at least has given you some new pointers to start searching from next time you encounter strange problems while starting or stopping your WebLogic servers.