WebLogic Server

Introduction

This document has instructions for configuring the Hyperic Agent to monitor WebLogic Server (WLS), and information to help you understand and solve problems with the WLS auto-discovery process.

Configuration Overview

This section is an overview of how you configure Hyperic to monitor a WebLogic Server domain. See the following section for detailed instructions.

  1. The Hyperic Agent auto-discovers the WLS Administration Server. For this to happen:

    • The Hyperic Agent and the WLS plugin must be installed on Administration Server host. (The WLS plugin is part of the standard agent installation; you do not need to install it separately.)

    • For WLS 9.0 and later, the agent must run on a 1.5 JRE.

    • The agent must have correct permissions to access the admin server configuration file.

    • If the admin server requires SSL, the agent must be configured accordingly.

  2. You add the auto-discovered admin server to Hyperic inventory. In terms of the Hyperic inventory model, the inventory type of the admin server is "server". For information about the inventory model, see Resources, Resource Types and Inventory Types

  3. In Hyperic, you edit configuration properties for the newly-added server with the admin server's username and password.

  4. The agent determines the Managed Servers in the WLS domain from the domain configuration file and adds them to inventory. In terms of the Hyperic inventory model, the inventory type of the Managed Server is "server".

    • The agent adds a Managed Server as a child of an existing platform in inventory, which corresponds to the machine that hosts the Managed Server.

  5. After the Administration and Managed Servers are added to inventory, you tailor the monitoring settings, as desired.

Configure WebLogic Server Monitoring

Perform the steps in the follow sections for each WebLogic Server domain you want to monitor.

Specify Agent JRE

The Hyperic Agent must run on a 1.5 JRE to monitor WebLogic Server 9.0 and higher. For earlier versions, a 1.4 JRE is supported.

Hyperic recommends you run the agent using the same JRE as WebLogic Server.

On Unix-based platforms define the location of the WebLogic Server JRE using the HQ_JAVA_HOME environment variable, for example:

 
HQ_JAVA_HOME=/usr/local/bea/jdk150_04 

For Hyperic Agent v4.x on Windows platforms you must specify HQ_JAVA_HOME as a system variable. For more information, see "Configure JREs for Hyperic Components" in Getting Started with vFabric Hyperic.

Run Agent with Required Permissions

In order to discover the WLS Administration Server, the Hyperic Agent must run under an account with permissions that allow it to obtain the Administration Server's process arguments, and its current working directory.

On Unix-based platforms, to ensure that the Hyperic Agent has the permissions it needs, either:

  • Run the Hyperic Agent as the same user id that runs the Administration Server, or

  • Run the Hyperic agent as root

See also: http://jira.hyperic.com/browse/HHQ-511

After the agent has discovered the Administration Server, it no longer needs the permissions required for process process discovery. (The agent uses JMX to discover the WLS Managed Servers.)  After discovery of the admin server, you can update the permissions to the Hyperic Agent directory. As superuser, run this command in the agent directory:

 
chown user -R 

where user is the username under which you will henceforth run the agent, for example, hqadmin.

Note:  If you cannot run the agent as the Administration Server user or as root, follow the directions in Add Administration Server Manually

Configure WLS Location

To ensure the Hyperic Agent can find the WebLogic Server installation, define the installation path in agent.properties. For example:

 
weblogic.installpath=/usr/local/bea/weblogic92 

Note: if you add the Administration Server to inventory manually, you specify the installation path from Hyperic user interface, rather than in the properties file.

Configure SSL

If the Administration Server is configured to use one-way SSL, add the properties in One-Way SSL to agent.properties.

If the Administration Server is configured to use two-way SSL, add the properties in One-Way SSL and also the properties in Two-Way SSL to agent.properties.

One-Way SSL

In one-way SSL the Hyperic Agent needs the public key of the Administration Serves's cert in order to trust it and to decrypt/encrypt communications. This public key is stored in the TrustKeyStore.

Configure these properties in agent.properties:

  • weblogic.security.TrustKeyStore=IdentityKeyStore.jks

    • where IdentityKeyStore.jks is the Administration Server's identity keystore.

  • weblogic.security.CustomTrustKeyStoreFileName=${installpath.escaped}/DemoTrust.jks

    • where DemoTrust.jks is the Administration Server's certs keystore.

  • weblogic.security.SSL.ignoreHostnameVerification=true

Two-Way SSL

In Two-Way SSL, each side presents an SSL certificate to the other - the client must trust the server cert and the server must trust the client cert. The process is:

  1. HQ Agent initiates a connection with the Administration Server.

  2. The Administration server presents its certificate and asks the HQ Agent for its certificate.

  3. The HQ Agent presents its certificate (client2certs.pem), using the private key (clientkey.pem and clientkey) to encrypt the communication.

To enable the HQ Agent to use Two-Way-SSL with a Weblogic Administration Server, you specify the client cert the HQ Agent presents to the Administration Server by adding these properties to agent.properties.

  • weblogic.auth.method=ssl2ways

  • weblogic.ssl2ways.key=ClientKey.pem (client private key)

  • weblogic.ssl2ways.key.pass=ClientKey (passphrase for client private key)

  • weblogic.ssl2ways.cert=Client2Certs.pem (client certificate)

Weblogic docs: Using Two-Way SSL Authentication

Restart Agent

Restart the agent. 

Note: If you have not modified agent.properties, this step is not necessary. 

Verify Auto-Discovery of Administration Server

After completing the previous steps, the Hyperic Agent should auto-discover the Administration Server. The Administration Server must be running for the agent to discover it.

Check the Auto-Discovery portlet in the dashboard to verify that the Administration Server was discovered. The agent scans the platform upon startup, so the Administration Server should appear in the portlet within a few minutes. The name of the server will be:

PlatformIdentifier ServerType DomainName AdminServerName

where:

  • PlatformIdentifier - is the first, or left-most, label of the FQDN of platform that hosts the admin server. (This  comes from the platform FQDN property in Hyperic.)

  • ServerType - is "Weblogic Admin n.n", where n.n is the version of the admin server.  (This is the server type property for the admin server in Hyperic.)

  • DomainName - is the name of the WebLogic Server domain that contains the admin server.

  • AdminServerName - is the name configured for the admin server in WebLogic Server.

If the admin server was not discovered, see Solve Administration Server Auto-Discovery Problems

Configure Monitoring

In this step, you configure the Administration Server in Hyperic with credentials that allow it to access the domain config.xml, which specifies the Managed Servers in the domain. You can specify monitoring options at this time, or do it later.

Required Administration Server Configuration

These steps are required to enable discovery of the Managed Servers in the domain.

  1. Navigate to the Admin Server in the Hyperic user interface.

    1. Choose Browse from the Resources tab and click Servers

    2. To narrow the list of servers displayed, enter WebLogic in the Keyword box and click images/download/attachments/79038211/accept.png .

    3. Select the admin server from the list to display its Inventory page.

  2. Choose Configure Server from the Tools menu.

  3. In the "Shared" section, enter the WLS username and password in the admin.username and admin.password fields.

Click OK and proceed directly to Verify Managed Server Auto-Discovery, or if desired, configure monitoring options for the new server, as described in the next paragraph. 

Configure Log Event Tracking and Service Monitoring

In the "Monitoring Section" of the server configuration page you specify desired log tracking options, and whether you want services on the server to be auto-discovered.  

Note:  By default, services running on the server will NOT be autodiscovered.

  1. Enable Log Tracking - Check to enable log tracking.

  2. Track event log level - Events greater than or equal to selected level will be tracked. The levels, in decreasing order of severity, are:

    • Error (maps to WLS "Error")

    • Warn (maps to WLS "Warning")

    • Info (maps to WLS "Info" and "Notice"

    • Debug (maps to WLS "Debug")

  3. Log Pattern Match - Enter a substring or regular expression to track only matching messages. See: http://download.oracle.com/javase/1.4.2/docs/api/java/util/regex/Pattern.html.

  4. Log Pattern Exclude - Enter a substring or regular expression to exclude matching messages.

  5. Log Files - Enter a comma delimited list of log files to track. Relative file specifications are resolved relative to the server's installation path.

  6. Enable Config Tracking - Check to enable configuration tracking.

    WebLogic Server Configuration Tracking

    In Hyperic 4.6.5, the WebLogic Server plugin uses a new plugin support class —-- org.hyperic.hq.product.FileChangeTrackPlugin --— that enables more detailed change tracking than available in previous versions of Hyperic. This plugin tracks the change type ("add", "delete", "modify", or "rename") and the actual changes in text files. For information about the new support class, see FileChangeTrackPlugin. For information about what files are tracked by default, see Configuration Files Tracked by Default below.

  7. Auto-Discover Applications, Entity EJBs, and other services? - Enable if you wish to auto-discover and monitor WLS services.

  8. Click OK.

Verify Managed Server Auto-Discovery

After you complete the steps in Required Administration Server Configuration the Manager Servers currently running in the WLS domain should appear in Hyperic inventory and metric collection should commence.

Check the Recently Added portlet in the Dashboard for the Managed Servers. 

If the Managed Servers have not been added to inventory, see Solve Managed Server Auto-Discovery Problems.

Complete Monitoring Configuration

Perform the steps in Configure Log Event Tracking and Service Monitoring for the Managed Servers, and for the admin server, if you did not previously do so.

To view or modify the default metric collection settings for the WLS servers and their services, access the metric template for your WebLogic Server version:

  1. Click Administration  in the Masthead,

  2. Select the Monitoring Defaults link in the HQ Server Settings section of the Administration page.

  3. On the Monitoring Defaults page, click Edit Metric Template for your WebLogic Server version.

Troubleshoot WebLogic Server Auto-Discovery Problems

The sections below contain information and tips that will help you understand and solve problems you may encounter with auto-discovery of WLS components.

Solve Administration Server Auto-Discovery Problems

Read this section if the admin server does not appear in the Auto-Discovery portlet, as described above in Verify Auto-Discovery of Administration Server.

Test Auto-Discovery

You can check the results of the Admin Server auto-discovery process.

The Hyperic Plugin Development Kit (PDK) supports stand-alone invocation of plugins from a command shell. You can run auto-discovery, collect metrics, and perform supported control actions at the command line.

This is an example execution of the WLS plugin's "discover" function, including the results:

 
% java -jar pdk/lib/hq-product.jar \ 
-Dplugins.include=weblogic -Dadmin.username=weblogic -Dadmin.password=weblogic -m discover 
1 servers detected 

Server: bear Weblogic Admin 9.2 medrec MedRecServer [/usr/local/bea/weblogic92/samples/domains/medrec/servers/MedRecServer] 
AIID...../usr/local/bea/weblogic92/samples/domains/medrec/servers/MedRecServer 
config... 
product..{admin.username=weblogic, server.log_track.files=/usr/local/bea/weblogic92/samples/domains/medrec/servers/MedRecServer/logs/MedRecServer.log, admin.url=t3://bear:7011, domain=medrec, server=MedRecServer, admin.password=weblogic} 
metric...null 
control..{program=/usr/local/bea/weblogic92/samples/domains/medrec/startWebLogic.sh}

Note:  Command usage is documented in Running and Testing Plugins from Command Line.

The result above shows that a resource of type "server" was auto-discovered. The name of the newly discovered server in Hyperic is:

bear Weblogic Admin 9.2 medrec MedRecServer

The Hyperic naming convention for the WLS Administration Server is explained above in Verify Auto-Discovery of Administration Server.

Check Network Access

Check to see if the location of the Hyperic Agent with respect to the WLS admin server is causing the problem.  In the Hyperic User Forum thread "WebLogic 8.1 Admin Server configuration stalls" at http://forums.hyperic.com/jiveforums/thread.jspa?threadID=6366&tstart=0 the user's admin server was on a different subnet than the Hyperic components.

Verify Permissions

You can check whether auto-discovery failed because of a permissions problem. 

This section show the query that the WLS plugin uses to obtain the Administration Server's arguments and its current working directory (cwd).  Two examples are shown. The first example shows the results that are returned when the query fails to return the Admin Server arguments and cwd. The second example shows the results when the query is successful.

Note:  To run this query in your environment, replace "java" with "HQ_JAVA_HOME"/bin/java", where "HQ_JAVA_HOME" is the location of the JRE the agent uses - the same JRE as the Administration Server uses, if you followed the recommendation in Specify Agent JRE.

If Plugin Lacks Necessary Permissions

In the results below, the value returned for the current working directory (cwd) is "???", indicating that  the user running the command lacked permission to obtain that information.

 
% java -jar pdk/lib/sigar.jar pargs State.Name.eq=java,Args.-1.eq=weblogic.Server 
pid=2317 
exe=??? 
cwd=??? 
0=>/usr/local/bea/jrockit90_150_04/bin/java<= 
1=>-jrockit<= 
2=>-Xms256m<= 
3=>-Xmx512m<= 
4=>-Xverify:none<= 
5=>-da<= 
6=>-Dplatform.home=/usr/local/bea/weblogic92<= 
7=>-Dwls.home=/usr/local/bea/weblogic92/server<= 
8=>-Dwli.home=/usr/local/bea/weblogic92/integration<= 
9=>-Dcom.bea.medrec.xml.incoming=incoming<= 
10=>-Dlog4j.configuration=file:/usr/local/bea/weblogic92/samples/domains/medrec/log4jConfig.xml<=
11=>-Dweblogic.management.discover=true<= 
12=>-Dwlw.iterativeDev=<= 
13=>-Dwlw.testConsole=<= 
14=>-Dwlw.logErrorsToConsole=<= 
15=>-Dweblogic.ext.dirs=/usr/local/bea/patch_weblogic920/profiles/default/sysext_manifest_classpath<=
16=>-Dweblogic.Name=MedRecServer<= 
17=>-Djava.security.policy=/usr/local/bea/weblogic92/server/lib/weblogic.policy<= 
18=>weblogic.Server<= 

If Plugin Has Necessary Permissions

If you run the query as the weblogic user or as root, the Administration Server's current working directory (cwd) is returned:

 
% java -jar pdk/lib/sigar.jar pargs State.Name.eq=java,Args.-1.eq=weblogic.Server 
pid=2317 
exe=/usr/local/bea/jrockit90_150_04/bin/java 
cwd=/usr/local/bea/weblogic92/samples/domains/medrec 

Add Administration Server Manually

If you are unable to run the agent with sufficient permissions to auto-discover the WLS Administration Server, or you have problems with the auto-discovery process, you can add the admin server to inventory manually.

  1. In the Hyperic user interface, navigate the the platform that hosts the Administration Server.

  2. Choose New Server from the Tools menu.

  3. On the New Server page, supply the required values and click OK. For example,

     
    Name: [Weblogic Admin 9.2 medrec MedRecServer] 
    Server Type: [Weblogic Admin 9.2] 
    Install Path: [/usr/local/bea/weblogic92/samples/domains/medrec/servers/MedRecServer] 

  4. Select the server to display its Inventory page.

  5. Choose Configure Server from the Tools menu.

  6. In the Configuration Properties page, enter values for the properties in the "Shared" section, using the values that were returned by the WebLogicConfig command above:

     
    admin.url [t3://bear:7011] 
    admin.username [weblogic] 
    admin.password [weblogic] 
    server [MedRecServer] 
    domain [medrec] 
    jvm.runtime [JVMRuntime] (choose JRockit or JVM (Sun)) 

  7. Click OK.

Solve Managed Server Auto-Discovery Problems

Read this section if your Managed Servers were not discovered after you performed the Required Administration Server Configuration.

To add a WLS Managed Server to inventory, the Hyperic Agent needs to locate the "platform" in Hyperic upon which the Managed Server runs. Because a Managed Server is a "server" in the Hyperic inventory hierarchy, adding it to inventory means making it the child of a platform. The agent determines the address of machine that hosts the Managed Server from config.xml and looks for a platform in inventory whose FQDN matches that address. If there is no platform in inventory with that FQDN, the agent will fail to add the Managed Server to inventory.  

How the Plugin Discovers Managed Servers 

For each Managed Server defined in config.xml, the plugin:

  • Connects to the host specified by the Managed Server's ListenAddress property as specified in config.xml.

  • Obtains inventory attributes for the Managed Server.

  • Creates a server in Hyperic inventory under the platform whose fully qualified domain name (FQDN) matches the Managed Server's ListenAddress. 

If a Managed Server's ListenAddress is specified in config.xml as an IP address, rather than a domain name, the plugin will not find the the platform in inventory. By default the plugin searches for the platform by FQDN only - it doesn't search the IP addresses associated with platforms in inventory.   

For example, if a Managed Server whose ListenAddress=10.0.0.102 runs on a platform whose FQDN in Hyperic is  appserver2.hyperic.com, the plugin will not find that platform in inventory, and will will fail to add the Managed Server to inventory. In this event, the plugin will log a message like this to the agent.log file in its /log directory:

 
[WeblogicRuntimeDiscoverer] Discovered server (Weblogic 9.2 medrec bear) hosted on another platform (fqdn=10.0.0.102) 

Check if the Agent Can Access config.xml

You can check whether the Agent is able to access the config.xml file, which specifies the domain configuration, including the Managed Servers in the domain.

This section shows an example execution of the "WebLogicConfig" query, which the WLS plugin uses to obtain the Administration Server's config.xml .

The plugin assumes that config.xml is in the Administration Server's /config directory).

Note:  To run this query in your environment, replace "java" with "HQ_JAVA_HOME"/bin/java", where "_HQ_JAVA_HOME"_is the location of the JRE the agent uses - the same JRE as the Administration Server uses, if you followed the recommendation in Specify Agent JRE.

 
% java -jar pdk/lib/hq-product.jar weblogic WeblogicConfig /usr/local/bea/weblogic92/samples/domains/medrec/config/config.xml 
/usr/local/bea/weblogic92/samples/domains/medrec/config/config.xml... 
9.2 
medrec 
admin=MedRecServer 
-- listing properties -- 
admin.url=t3://bear:7011 
version=9.2 
domain=medrec 
server=MedRecServer 

Resolve Managed Server IP Address/FQDN Mismatch

If your Managed Server's ListenAddress is specified as an IP addess (rather than a domain name) in config.xml  the Hyperic Agent will fail to add the Managed Server to inventory. There are two ways to solve this problem.

  • Option 1 - Configure the Hyperic Agent to add all the Managed Servers in the WLS domain to the Hyperic platform that hosts the Hyperic Agent, instead of adding each Managed Server to the Hyperic platforms that actually hosts it.  To do this, add this property to agent.properties:

     
    weblogic.discover.fqdn=same 

  • Option 2 - For each Managed Server, define the mapping between its ListenAddress as defined in config.xml and its Hyperic FQDN. This method will allow the plugin to add the Managed Server to the platform in Hyperic for the machine that actually hosts it. To do this, for each Managed Server in the domain, add a line like this to agent.properties:

     
    weblogic.discover.fqdn.ListenAddress=FQDN.fqdn 
    where ListenAddress is the value of that property as defined for the Managed Server in config.xml and FQDN is the 

    • where:

      • ListenAddress is the value of that property as defined for the Managed Server in config.xml, and

      • FQDN is the platform's fully qualified domain name in Hyperic. 
        For example:

         
        weblogic.discover.fqdn.10.0.0.4=appserver2.hyperic.com 

        Restart the agent after making changes to agent.properties.

Configuration Files Tracked by Default

For WebLogic Server, the default value of the Configuration Files field, which specifies which files to track is:

;true;.*\.jar|.*\.dll|.*\.class|.*\.jsp|.*\.php|.*\.pl|.*\.js|.*\.py|.*\.pyc|
.*\.cgi|.*\.so.*\.xml|.*\.cfg|.*\.properties|.*\.ini|.*\.conf|.*\.config|.*\.ldift|
.*\.sql|.*\.ddl|.*\.sh;,;true;.*\.jar|.*\.dll|.*\.class|.*\.jsp|.*\.php|.*\.pl|
.*\.js|.*\.py|.*\.pyc|.*\.cgi|.*\.so|.*\.sql|.*\.sh.*\.xml|.*\.cfg|.*\.properties|
.*\.ini|.*\.conf|.*\.config|.*\.lok|.*\.ldift;

When configuration tracking is enabled, this filter causes Hyperic to track files in any directory in the WlsHome directory and any directories below it, files with one of the following extensions:

.jar, .dll, .class, .jsp, .php, .pl, .js, .py, .pyc, .cgi, .so, .sql, .sh, .xml, .cfg, .properties, .ini, .conf, .config, .ldift, .ddl, .lok, .ldift

For information about how to enable configuration tracking and how to specify which files to track, see Set Up Configuration Tracking for a Resource in vFabric Hyperic Administration.

Metrics