Troubleshoot installation

Troubleshoot installation

+
Search Tips | Advanced Search

This topic describes troubleshooting the installation of the WAS ND product.
If you are looking for troubleshooting information for the Web server plug-ins for WAS, see Troubleshooting Web server plug-ins installation and removal. This topic does not describe the plug-ins.
Use this topic after installing your WAS product. The successful installation of the ND product is a two-part process:

The first step is using the installation wizard to install a shared set of core product files.

The second step is to create a deployment manager profile, an appserver profile, or a custom profile.

If an installation is not successful, use this troubleshooting information to correct the problems.

Overview

Use this topic to help interpret the log files and diagnose possible problems when the installation is unsuccessful. The installer program records the following indicators of success in the logs:

INSTCONFSUCCESS
INSTCONFPARTIALSUCCESS
INSTCONFFAILED

Procedure

Run the installver command to calculate and compare checksums for all installed components to the bill of materials list for WAS.
Compare the output from the installver command to the installation log files that are described in the next step.

Check the installation log files for errors after installing:
The files...
app_server_root/logs/install/log.txt
app_server_root/logs/manageprofiles/profile_create.log
profile_root/logs/pctLog.txt

...record installation and profile creation status.
If the error happens early in the installation, look for the log.txt file in the system temporary area. The installation program copies the log from the temporary area to the logs directory at the end of the installation.
During installation, a single entry in...
app_server_root/logs/install/log.txt

...points to the temporary log file, either...
%TEMP%\log.txt

...on Windows platforms, or...
/tmp/log.txt

...on platforms such as AIX or Linux. The installation program copies the file from the temporary directory to...
app_server_root/logs/install/log.txt

...at the end of the installation.
If the installation fails and the log.txt file has only this one pointer to the temporary directory, open the log.txt file in the temporary directory. The log might have clues to the installation failure.
Uninstall creates...
app_server_root/logs/uninstall/log.txt

Log more information when InstallShield MultiPlatform (ISMP) cannot start the installation wizard.
Certain events can prevent the installer from starting the installation wizard. Such an event is not enough disc space to launch the installation wizard, for example. If your installation fails and there is no information in the installation logs, use the -log parameter to record entries for events that cause the installer program to fail to start the installation wizard. The syntax of the install command for logging such events is:
install -options /path/to/responsefile -silent -log # !/path/to/log/file @ALL

Unix:
install -options "/usr/IBM/WAS/silentFiles/myresponsefile.txt" -silent -log # !/usr/IBM/WAS/myOptionFiles/log.txt @ALL

Windows:
install.exe -options "C:\IBM\WebSphere\silentFiles\myresponsefile.txt" -silent -log # !C:\IBM\WebSphere\silentFiles\log.txt @ALL

The following examples show how to use the -log parameter when creating a deployment manager profile. The command is in...
app_server_root/bin/ProfileCreator

The name of the command varies per platform according to this pattern, pctplatform:

pctAIX.bin
64-bit kernels: pctAIXPPC64.bin
pctHPUX.bin
64-bit Intel architecture (IA64) kernels: pctHPUXIA64.bin
pctLinux.bin
31-bit compatibility mode on 64-bit zSeries kernels: pctLinux390.bin
64-bit exploitation mode on 64-bit zSeries kernels: pctLinux390z.bin
Power platforms: pctLinuxPPC.bin
pctSolaris.bin
pctWindows.exe

The first example shows the command for AIX systems:
./pctAIX.bin -options /usr/IBM/WAS/silentFiles/responsefile.pct.NDstandAloneProfile.txt -silent -log !/usr/IBM/WAS/silentFiles/pctlog.txt @ALL

For Linux on 32-bit platforms...
./pctLinux.bin -options /opt/IBM/WAS/silentFiles/responsefile.pct.NDstandAloneProfile.txt -silent -log # !/opt/IBM/WAS/silentFiles/log.txt @ALL

For Windows platforms:
pctWindows.exe -options "C:\IBM\WebSphere\silentFiles\responsefile.pct.NDstandAloneProfile.txt" -silent -log # !C:\IBM\WebSphere\silentFiles\log.txt @ALL

Log file names and locations

The following information shows the log files for all of the installable components on the WAS ND v6.1 disc.

Log files for IBM HTTP Server

The following table shows the installation log locations when installing IBM HTTP Server.

Windows:

C:\IBM HTTP Server\log.txt
C:\IBM HTTP Server\ihsv61_install.log

AIX and Linux:

/usr/IBMHttpServer/ log.txt
/usr/IBMHttpServer/ihsv61_install.log

Solaris
/opt/IBMHttpServer/log.txt
/opt/IBMHttpServer/ihsv61_install.log

Log files for Application Client for WAS

The following table shows the installation log locations when installing the Application Client.

Windows system log path name Operating system log path name on systems such as AIX or Linux

C:\IBM\WebSphere\ AppClient\logs\log.txt

/usr/IBM/WAS/AppClient/logs/log.txt

/opt/IBM/WAS/AppClient/logs/log.txt

Product Log files

The following table shows the installation logs, content, and indicators of success and failure for WAS ND v6.1:

Log Content Indicators
app_server_root /logs/install/log.txt Logs all installation events

Return code
Meaning
0 Success
1 Failure
2 Partial Success

app_server_root/logs/wasprofiles/wasprofile_create_profile.log Traces all events that occur during the creation of the named profile
Created when using the Profile Management tool or the manageprofiles command

INSTCONFFAIL

Total profile creation failure.

INSTCONFSUCCESS

Successful profile creation.

INSTCONFPARTIALSUCCESS

Profile creation errors occurred but the profile is still functional. Additional information identifies the errors.

app_server_root/logs/install/ installconfig.log.gz Logs the activities of ANT configuration scripts that run at the end of the installation procedure
Gzip file

Configuration action failed:

Unsuccessful ANT script configuration.

Configuration action succeeded:

Successful ANT script configuration.

profile_create.log

The profile_create.log file is an XML file that contains a record of the events that occur during the creation of the last profile. In addition to the date tag at the beginning of the file, other tags of interest in the log files include the sequence tag, the level tag, the method tag, and the message tag:

The sequence tag records the sequence of events that occur during the creation of the profile.

The level tag is an early indicator of event status:

INFO

Indicates a normal event.

WARNING

Indicates an event that occurred with errors that do not prevent the creation of the profile.

ERROR

Indicates an event that prevents the creation of the profile.

The method tag indicates the name of the routine that recorded the event.

The message tag describes the event and contains any data returned by the method.

The following stanza is an example of how an event is documented in each log file:
<record> <date>2004-09-08T11:51:39</date> <millis>1094658699225</millis> <sequence>0</sequence> <logger>com.ibm.ws.profile.WSProfile</logger> <level>INFO</level> <class>com.ibm.ws.profile.WSProfile</class> <method>getRegistryFile</method> <thread>10</thread> <message>Returning registry file at: C:\IBM\WebSphere\AppServer\properties\profileRegistry.xml </message> </record>

Log files created during the creation of the Application Server profile

In addition to the logs created within the core product files, the following logs are created in...
profile_root/logs

Both the Profile Management tool and the manageprofiles command create the log when creating an appserver profile:

activity.log

Compiled activity.log from various installation activities. "data" formated.

amjrte_config.log

Tivoli Access Manager configuration log for its Java Runtime Environment

collect_metadata.log

Collects metadata information about managed objects in the system to evaluate and prevent potential installation conflicts

createDefaultServer.log

A log from wsadmin recording the creation of the server1 process in the default profile

createshortcutforprofile.log

Windows tool log for creating menu entries and shortcuts

defaultapp_config.log

JACL script log from configuring default application resources

defaultapp_deploy.log

Application DefaultApplication installation log

node_name Service.log

Start and stop events for server1

filetransfer_config.log

Application filetransfer installation log

hamanager_config.log

Configuration log for the high availability application

ivt_config.log

Application ivtApp installation log

mejb_config.log

Application ManagementEJB installation log

pctLog.txt

Log created when using the Profile Management tool to create a profile.
This log is not created when using the manageprofiles command directly.
This log is not created during product installation.

query_config.log

Application Query installation log

samples_config.log

Configuration log for the PlantsByWebSphere Samples application

samples_install.log

Installation log for the SamplesGallery and PlantsByWebSphere Samples applications

scheduler.cal_config.log

Application SchedulerCalendars installation log

SIBDefineChains.log

Creation log for service integration bus endpoints, inbound channels and channel chains, outbound thread pool, and outbound channel and channel chains

SIBDeployRA.log

Deployment log for the service integration bus function

webui_config.log

Application console installation log

winservice_config.log

Service log for the Windows service created for server1

The following logs are created in profile_root/logs/server1...

startServer.log Log of start server events
stopServer.log Log of stop server events
SystemErr.log Record system errors
SystemOut.log Log of all activity within the system
trace.log Log of all traced events within the system
The following logs are created in profile_root/logs/ffdc...

server1_exception.log First failure data capture log for server1 errors
server1_numeric_identifier.txt Any first failure data capture logs

Log file info for Web server plug-ins is covered in Troubleshooting Web server plug-ins installation .

Determine whether the installation problem is caused by a failing ANT script.
The file...
app_server_root/logs/instconfig.log

...indicates ANT configuration problems that could prevent WAS ND v6.1 from working correctly. The log file is not present on systems such as AIX or Linux.

Verify that no files exist in...
app_server_root/classes

IBM Support sometimes queues work for customers and provides test or debugging fixes. A common location for the fixes is in the app_server_root/classes directory.
By default, the directory...
app_server_root/classes

...is picked up first in the class path to let it override other classes.
Putting a fix in the directory lets you verify that the fix does indeed solve your problem. After verifying that the fix solves the problem, you are supposed to delete the fix from the app_server_root/classes directory to return the system to a working state.
If you do not remove such fixes from...
app_server_root/classes

...you can experience errors.

Uninstall WAS, if possible, and reinstall after turning on tracing if the error logs do not contain enough information to determine the cause of the problem.

Report the stdout and stderr logs to the console window, by adding the -is:javaconsole parameter to the install command:

install -is:javaconsole

Capture the stream to a file with the following commands:
install -is:javaconsole > captureFileName.txt 2>&1

Capture additional information to a log of your choice with the -is:log file_name option.

Turn on additional installation logging by passing the parameter...
-W Setup.product.install.logAllEvents="true" parameter

...to the install command:
install -W Setup.product.install.logAllEvents="true"

If you have successfully created an appserver profile, use the First steps console or the command line method to start the appserver.

Verify whether the server starts and loads properly by looking for a running Java process and the Open for e-business message in the SystemOut.log and SystemErr.log files.
If no Java process exists or if the message does not appear, examine the same logs for any miscellaneous errors. Correct any errors and retry. You can find the SystemOut.log and SystemErr.log files in...

profile_root/logs/server_name

Use the First steps console or the command line method to stop the appserver if it is running and to start the deployment manager if one exists.
To stop server1 from the command line:
profile_root/bin/stopServer.sh server1

If you enable security, specify the -user and the -password parameters of the command.
To start the deployment manager from the command line:
profile_root/bin/startManager.sh

Verify that the server starts and loads properly by looking for a running Java process and the Server dmgr open for e-business message in the profile_root/logs/server_name/SystemOut.log file.

For Windows press Ctrl+Alt+Delete and type T to open the Task Manager. Click the Processes tab and the Image Name column header to sort by image name. Look for processes named java.exe.
For *nix open a command window and issue the top command to see a display of running processes. If the top command is not available on your system, use the ps command:
ps -ef | grep java

If no Java process exists or if the message does not appear, examine the same logs for any miscellaneous errors. Correct any errors and try again to start the deployment manager.

Start the Snoop servlet to verify the ability of the Web server to retrieve an application from the Application Server.
Test your environment by starting your Application Server, your Web server, and using the snoop servlet with an IP address.

Start the Application Server. In a ND environment, the Snoop servlet is available in the cell only if you included the DefaultApplication when adding the Application Server to the cell. The -includeapps option for the addNode command migrates the DefaultApplication to the cell. If the application is not present, skip this step.Change directories to...
profile_root/bin

...and run

./startServer.sh server1

Start the IBM HTTP Server or the Web server that you are using.
Use a command window to change the directory to the IBM HTTP Server installed image, or to the installed image of your Web server. Issue the appropriate command to start the Web server, such as these commands for IBM HTTP Server:
To start the IBM HTTP Server from the command line:
For *nix...
cd IBMHttpServer/bin
./apachectl start

For Windows...
cd IBMHttpServer\bin
apache

Point your browser to...
http://localhost:9080/snoop

...to test the internal HTTP transport provided by the Application Server. Point your browser to...
http://Host_name_of_Web_server_machine/snoop

...to test the Web server plug-in.
The HTTP Transport port is 9080 by default and must be unique for every profile. The port is associated with a virtual host named default_host, which is configured to host the installed DefaultApplication and any installed Samples. The snoop servlet is part of the DefaultApplication. Change the port to match your actual HTTP Transport port.

Verify that snoop is running.
Either Web address should display the Snoop Servlet - Request/Client Information page.

Remote IBM HTTP Server only:

Verify that the automatic propagation function can work on a remote IBM HTTP Server by using the following steps. This procedure is not necessary for local Web servers.

Create a...
user=adminUser, password=adminPassword

...in...
IHS_root /conf/admin.passwd

For example...
c:\ws\ihs60\bin\htpasswd -cb c:\ws\ihs60\conf\admin.passwd adminUser adminPassword

Use the console of the deployment manager or the Application Server to enter the User ID and password information that you created for the administrative user of IBM HTTP Server. Go to Servers > Web server > Web_server_definition > Remote Web server administration. Set the following values: admin Port=8008, User Id=adminUser, Password=adminPassword.

Set the correct read/write permissions for the httpd.conf file and the plugin-cfg.xml file. See the IHS_root /logs/admin_error.log file for more information.

Automatic propagation of the plug-in configuration file requires the IBM HTTP administrative server to be up and running. If you are managing an IBM HTTP Server using the console, the following error might display:
"Could not connect to IHS Administration server error"
Perform the following procedure to correct the error:

Verify that the IBM HTTP Server administration server is running.

Verify that the Web server host name and the port that is defined in the console matches the IBM HTTP Server administration host name and port.

Verify that the fire wall is not preventing you from accessing the IBM HTTP Server administration server from the WAS administrative console.

Verify that the user ID and password that is specified in the WebSphere Application Server console under remote managed, is created in the admin.passwd file, using the htpasswd command.

If you are trying to connect securely, verify that you export the IBM HTTP Server administration server keydb personal certificate into the WebSphere Application Server key database as a signer certificate. This key database is specified by the com.ibm.ssl.trustStore directive in the sas.client.props file in the profile where your console is running. This consideration is primarily for self-signed certificates.

If you still have problems, check the IBM HTTP Server admin_error.log file and the WAS logs (trace.log file) to determine the cause of the problem.

Start the console.

Start the Application Server.

Point your browser to http://localhost:9060/ibm/console.
The HTTP Admin port is 9060 by default and must be unique for the console of each stand-alone Application Server. The port is associated with a virtual host named admin_host, which is configured to host the console, which is installed by default as a system application. Change the port to match your actual HTTP Admin port.
If you have problems accessing the console after installation, check the installAdminConsole.log file for a failure indication. Clean up the system temporary directory and reinstall the console using the wsadmin scripting facility.

Type any ID and click OK at the administrative console window.

The server starts. The console starts. You can access the console through the browser. The administrative console accepts your login.

Federate the base Application Server into the cell.
To add the base Application Server into the cell:

Deployment manager console method:

Click System administration > Nodes > Add Node and follow the wizard. The default SOAP port for the Application Server is 8880. You can use localhost as the value of the Host name field, if the Application Server is on the same machine.

Command-line method assuming the SOAP port of the dmgr is 8879:

profile_root/bin/addNode.sh localhost 8879 -includeapps

If you enable security, specify the -user and the -password parameters of the command.

Verify that the Application Server was incorporated into the cell. The command window displays a sequence of messages when you issue the addNode command:
Tool information is being logged in file profile_root\logs\addNode.log Begin federation of node AppServer01 with Deployment Manager at localhost:8879. Successfully connected to Deployment Manager Server: localhost:8879 Servers found in configuration: Server name: server1 Stopping all server processes for node AppServer01 Creating node agent configuration for node: AppServer01 Reading configuration for node agent process: nodeagent Adding node AppServer01 configuration to cell: AdvancedDeploymentCell Performing configuration synchronization between node and cell. Launching node agent process for node: AppServer01 Node agent launched. Waiting for initialization status. Node agent initialization completed successfully. Process ID is: 3012

Node AppServer01 has been successfully federated.

The last message is an indicator of success. A second Java process is running, which is the nodeagent process. The stdout.log file and stderr.log file in the node_name directory each contain relevant messages.

Resolve any IP address caching problems.
By default, the Java 2 SDK caches the IP address for the domain name service (DNS) naming lookup. After resolving the host name successfully, the IP address stays in the cache. By default, the cache entry remains forever.
This default IP caching mechanism can cause problems, as described in the following problem scenarios.
Problem scenario 1
Suppose the Application Server at host1.ibm.com has an initial IP address of 1.2.3.4. When a client at host2.ibm.com conducts a DNS lookup of host1.ibm.com, the client stores the 1.2.3.4 address in the cache. Subsequent DNS name lookups return the cached value, 1.2.3.4.
The cached value is not a problem until the host1.ibm.com IP address changes, to 5.6.7.8, for example. The client at host2.ibm.com does not retrieve the current IP address, but always retrieves the previous address from the cache.
If this scenario occurs, the client cannot reach host1.ibm.com unless you stop and restart the client process.
Problem scenario 2
Suppose the Application Server at host1.ibm.com has an initial IP address of 1.2.4.5. Although the IP address of the appserver does not change, a network outage can record an exception code as the IP address in the cache, where it remains until the client is restarted on a working network.
For example, if the client at host2.ibm.com disconnects from the network because of an unplugged cable, the disconnected lookup of the Application Server at host1.ibm.com fails. The failure causes the IBM Developer Kit to put the special exception code entry into the IP address cache.
Subsequent DNS name lookups return the exception code, which is java.net.UnknownHostException.
IP address caching and WAS process discovery
If you change the IP address of a federated WAS node, processes running in other nodes cannot contact the changed node until you stop and restart them.
If a deployment manager process starts on a disconnected node, it cannot communicate with cell member processes until you stop and restart the deployment manager process. For example, plugging in an unplugged network cable does not restore proper addresses in the IP cache until the deployment manager process is restarted.
Use the IP address cache setting
You can always stop and restart a deployment manager process to refresh its IP address cache. However, this process might be expensive or inappropriate.
The parameters...

networkaddress.cache.ttl (public, JDK1.4)
sun.net.inetaddr.ttl (private, JDK1.3)

...control IP caching. The value is an integer that specifies the number of seconds to cache IP addresses. The default value, -1, specifies to cache forever. A value of zero (0) is a specification to never cache.
Use a zero (0) value is not recommended for normal operation. If you do not anticipate network outages or changes in IP addresses, use the cache forever setting. The never caching setting introduces the potential for DNS spoofing attacks.

For more information about the Java 2 SDK
The Java 2 Platform Standard Edition (J2SE) 1.4.2 Web site and the Java 2 Platform Standard Edition (J2SE) 1.5 Web site describe the private sun.net.inetaddr.ttl property, which works in both J2SE 1.4.2 (WAS V5.1 and V6.0) and J2SE 1.5 (WAS V6.1).

Restore the original copy of the etc/issue file if the file is modified.
The prereqChecker program in the installation wizard uses the file to verify the version of the operating system. If you cannot restore the original version, ignore the Operating System Level Check message about the operating system being unsupported. The installation can continue successfully despite the warning.

Results

This procedure results in debugging errors that might occur during installation.

What to do next

The Installation problems contains more detailed debugging and reporting instructions. See Installation component troubleshooting tips for more information about troubleshooting the installation.
For current information available from IBM Support on known problems and their resolution, see the IBM Support page.
IBM Support has documents that can save you time gathering the information that resolve a problem. Before opening a PMR, see the IBM Support page.

Troubleshooting Web server plug-ins installation and removal
Installation component troubleshooting tips
Installation problems
Installation either completes with errors or warnings, or hangs
Diagnosing a failing ANT configuration script
Web server installation and startup problems
Messages issued during installation and profile creation

	pctAIX.bin
	64-bit kernels: pctAIXPPC64.bin
	pctHPUX.bin
	64-bit Intel architecture (IA64) kernels: pctHPUXIA64.bin
	pctLinux.bin
	31-bit compatibility mode on 64-bit zSeries kernels: pctLinux390.bin
	64-bit exploitation mode on 64-bit zSeries kernels: pctLinux390z.bin
	Power platforms: pctLinuxPPC.bin
	pctSolaris.bin
	pctWindows.exe

startServer.log	Log of start server events
stopServer.log	Log of stop server events
SystemErr.log	Record system errors
SystemOut.log	Log of all activity within the system
trace.log	Log of all traced events within the system

server1_exception.log	First failure data capture log for server1 errors
server1_numeric_identifier.txt	Any first failure data capture logs

Troubleshoot installation

Overview

Procedure

Log files for IBM HTTP Server

Log files for Application Client for WAS

Product Log files

profile_create.log

Log files created during the creation of the Application Server profile

Results

What to do next