Troubleshoot WebSphere DataPower XC10 appliance
- Troubleshooting and support
- Working with log files
- Downloading auditing data
- Analyzing log and trace data
- Monitor the hardware temperatures
- Monitor your Ethernet interface status details with a serial connection
- Verifying outbound connections from the appliance
- Using the command line interface to run operations on your appliance
- Troubleshooting cache integration
- Troubleshooting installation
- Troubleshooting administration
- Release notes
1. Troubleshooting and support for WebSphere DataPower XC10 appliance
To isolate and resolve problems with your IBM products, we can use the troubleshooting and support information. This information contains instructions for using the problem-determination resources that are provided with your IBM products, including appliance.
1.1. Techniques for troubleshooting problems
Troubleshooting is a systematic approach to solving a problem. The goal of troubleshooting is to determine why something does not work as expected and how to resolve the problem. Certain common techniques can help with the task of troubleshooting.
The first step in the troubleshooting process is to describe the problem completely. Problem descriptions help you and the IBM technical-support representative know where to start to find the cause of the problem. This step includes asking yourself basic questions:
- What are the symptoms of the problem?
- Where does the problem occur?
- When does the problem occur?
- Under which conditions does the problem occur?
- Can the problem be reproduced?
The answers to these questions typically lead to a good description of the problem, which can then lead you to a problem resolution.
What are the symptoms of the problem?
When starting to describe a problem, the most obvious question is "What is the problem?" This question might seem straightforward; however, we can break it down into several more-focused questions that create a more descriptive picture of the problem. These questions can include:
- Who, or what, is reporting the problem?
- What are the error codes and messages?
- How does the system fail? For example, is it a loop, hang, crash, performance degradation, or incorrect result?
Where does the problem occur?
Determining where the problem originates is not always easy, but it is one of the most important steps in resolving a problem. Many layers of technology can exist between the reporting and failing components. Networks, the data grid, and servers are only a few of the components to consider when we are investigating problems.
The following questions help you to focus on where the problem occurs to isolate the problem layer:
- Is the problem specific to one platform or operating system, or is it common across multiple platforms or operating systems?
- Is the current environment and configuration supported?
- Do all users have the problem?
- (For multi-site installations.) Do all sites have the problem?
If one layer reports the problem, the problem does not necessarily originate in that layer. Part of identifying where a problem originates is understanding the environment in which it exists. Take some time to completely describe the problem environment, including the operating system and version, all corresponding software and versions, and hardware information. Confirm that we are running within an environment that is a supported configuration; many problems can be traced back to incompatible levels of software that are not intended to run together or have not been fully tested together.
When does the problem occur?
Develop a detailed timeline of events leading up to a failure, especially for those cases that are one-time occurrences. We can most easily develop a timeline by working backward: Start at the time an error was reported (as precisely as possible, even down to the millisecond), and work backward through the available logs and information. Typically, you need to look only as far as the first suspicious event that you find in a diagnostic log.
To develop a detailed timeline of events, answer these questions:
- Does the problem happen only at a certain time of day or night?
- How often does the problem happen?
- What sequence of events leads up to the time that the problem is reported?
- Does the problem happen after an environment change, such as upgrading or installing software or hardware?
Responding to these types of questions can give you a frame of reference in which to investigate the problem.
Under which conditions does the problem occur?
Knowing which systems and applications are running at the time that a problem occurs is an important part of troubleshooting. These questions about your environment can help you to identify the root cause of the problem:
- Does the problem always occur when the same task is being performed?
- Does a certain sequence of events need to happen for the problem to occur?
- Do any other applications fail at the same time?
Answering these types of questions can help you explain the environment in which the problem occurs and correlate any dependencies. Remember that just because multiple problems might have occurred around the same time, the problems are not necessarily related.
Can the problem be reproduced?
From a troubleshooting standpoint, the ideal problem is one that can be reproduced. Typically, when a problem can be reproduced you have a larger set of tools or procedures at your disposal to help you investigate. Consequently, problems that we can reproduce are often easier to debug and solve.
However, problems that we can reproduce can have a disadvantage: If the problem is of significant business impact, you do not want it to recur. If possible, recreate the problem in a test or development environment, which typically offers you more flexibility and control during your investigation.
- Can the problem be recreated on a test system?
- Are multiple users or applications encountering the same type of problem?
- Can the problem be recreated by running a single command, a set of commands, or a particular application?
1.2. Searching knowledge bases
We can often find solutions to problems by searching IBM knowledge bases. We can optimize your results by using available resources, support tools, and search methods.
We can find useful information by searching the information center for appliance. However, sometimes you need to look beyond the information center to answer your questions or resolve problems.
To search knowledge bases for information that you need, use one or more of the following approaches:
- Search for content using the IBM Support Assistant (ISA).
ISA is a no-charge software serviceability workbench that helps you answer questions and resolve problems with IBM software products. We can find instructions for downloading and installing ISA on the ISA website.
- Find the content that you need using the IBM Support Portal.
The IBM Support Portal is a unified, centralized view of all technical support tools and information for all IBM systems, software, and services. The IBM Support Portal lets you access the IBM electronic support portfolio from one place. We can tailor the pages to focus on the information and resources that you need for problem prevention and faster problem resolution. Familiarize yourself with the IBM Support Portal by viewing the demo videos (https://www.ibm.com/blogs/SPNA/entry/the_ibm_support_portal_videos) about this tool. These videos introduce you to the IBM Support Portal, explore troubleshooting and other resources, and demonstrate how we can tailor the page by moving, adding, and deleting portlets.
- Search for content about appliance by using one of the following additional technical resources:
- WebSphere DataPower XC10 appliance release notes
- WebSphere DataPower XC10 appliance Support website
- WebSphere DataPower XC10 appliance forum
- Search for content using the IBM masthead search. Use the IBM masthead search by typing your search string into the Search field at the top of any ibm.com page.
- Search for content by using any external search engine, such as Google, Yahoo, or Bing. If you use an external search engine, your results are more likely to include information that is outside the ibm.com domain. However, sometimes we can find useful problem-solving information about IBM products in newsgroups, forums, and blogs that are not on ibm.com.
Include "IBM" and the name of the product in your search if we are looking for information about an IBM product.
1.3. Getting fixes
A product fix might be available to resolve your problem.
To find and install fixes:
- Obtain the tools required to get the fix. Use the Update Installer to install and apply various types of maintenance packages for WebSphere eXtreme Scale or WebSphere eXtreme Scale Client. Because the Update Installer undergoes regular maintenance, use the most current version of the tool.
- Determine which fix you need. See the Recommended fixes for WebSphere DataPower XC10 appliance to select the latest fix. When you select a fix, the download document for that fix opens.
- Download the fix. In the download document, click the link for the latest fix in the "Download package" section.
- Apply the fix. Follow the instructions in the "Installation Instructions" section of the download document.
- Subscribe to receive weekly e-mail notifications about fixes and other IBM Support information.
1.3.1. Getting fixes from Fix Central
Use Fix Central to find the fixes that are recommended by IBM Support for a variety of products, including WebSphere DataPower XC10 appliance. With Fix Central, we can search, select, order, and download fixes for your system with a choice of delivery options. A WebSphere DataPower XC10 appliance product fix might be available to resolve your problem.
To find and install fixes:
- Obtain the tools that are required to get the fix. If it is not installed, obtain your product update installer. We can download the installer from Fix Central. This site provides download, installation, and configuration instructions for the update installer.
- Select as the product, and select one or more check boxes that are relevant to the problem to resolve.
- Identify and select the fix that is required.
- Download the fix.
- Open the download document and follow the link in the "Download Package" section.
- When downloading the file, ensure that the name of the maintenance file is not changed. This change might be intentional, or it might be an inadvertent change that is caused by certain web browsers or download utilities.
- Apply the fix.
- Follow the instructions in the "Installation Instructions" section of the download document.
- For more information, see the "Installing fixes with the Update Installer" topic in the product documentation.
- Optional: Subscribe to receive weekly e-mail notifications about fixes and other IBM Support updates.
1.4. Contacting IBM Support
IBM Support provides assistance with product defects, answers FAQs, and helps users resolve problems with the product. After trying to find your answer or solution by using other self-help options, such as release notes, we can contact IBM Support. Before contacting IBM Support, your company or organization must have an active IBM maintenance contract, and you must be authorized to submit problems to IBM. For information about the types of available support, see the Support portfolio topic in the "Software Support Handbook".
To contact IBM Support about a problem:
- Define the problem, gather background information, and determine the severity of the problem. For more information, see the Getting IBM support topic in the Software Support Handbook.
- Gather diagnostic information.
- Submit the problem to IBM Support in one of the following ways:
- Use IBM Support Assistant (ISA):
- Online through the IBM Support Portal: We can open, update, and view all of your service requests from the Service Request portlet on the Service Request page.
- By phone: For the phone number to call in your region, see the Directory of worldwide contacts web page.
If the problem that you submit is for a software defect or for missing or inaccurate documentation, IBM Support creates an Authorized Program Analysis Report (APAR). The APAR describes the problem in detail. Whenever possible, IBM Support provides a workaround that we can implement until the APAR is resolved and a fix is delivered. IBM publishes resolved APARs on the IBM Support website daily, so that other users who experience the same problem can benefit from the same resolution.
1.5. Exchanging information with IBM
To diagnose or identify a problem, you might need to provide IBM Support with data and information from your system. In other cases, IBM Support might provide you with tools or utilities to use for problem determination.
To submit diagnostic information to IBM Support:
- Open a problem management record (PMR).
- Collect the diagnostic data that you need. Diagnostic data helps reduce the time that it takes to resolve your PMR. We can collect the diagnostic data manually or automatically:
- Collect the data manually.
- Collect the data automatically.
- Compress the files using the .zip or .tar file format.
- Transfer the files to IBM. Use one of the following methods to transfer the files to IBM:
- IBM Support Assistant
- The Service Request tool
- Standard data upload methods: FTP, HTTP
- Secure data upload methods: FTPS, SFTP, HTTPS
If we are using a z/OS product and you use ServiceLink / IBMLink to submit PMRs, we can send diagnostic data to IBM Support in an e-mail or by using FTP.
All of these data exchange methods are explained on the IBM Support website.
Receiving information from IBM Support
Occasionally an IBM technical-support representative might ask you to download diagnostic tools or other files. Use FTP to download these files.
Ensure that your IBM technical-support representative provided you with the preferred server to use for downloading the files and the exact directory and file names to access.
To download files from IBM Support:
- Use FTP to connect to the site that your IBM technical-support representative provided and log in as anonymous. Use your e-mail address as the password.
- Change to the appropriate directory:
- Change to the /fromibm directory.
cd fromibm- Change to the directory that your IBM technical-support representative provided.
cd nameofdirectory
- Enable binary mode for the session.
binary
- Use the get command to download the file that your IBM technical-support representative specified.
get filename.extension
- End your FTP session.
quit
Subscribing to Support updates
To stay informed of important information about the IBM products that you use, we can subscribe to updates.
By subscribing to receive updates about the product, we can receive important technical information and updates for specific IBM Support tools and resources. We can subscribe to updates by using one of two approaches:
- Social media subscriptions
- The following RSS feed is available for the product:
- RSS feed for WebSphere DataPower XC10 appliance forum
For general information about RSS, including steps for getting started and a list of RSS-enabled IBM web pages, visit the IBM Software Support RSS feeds site.
- My Notifications
- With My Notifications, we can subscribe to Support updates for any IBM product. My Notifications replaces My Support, which is a similar tool that you might have used in the past. With My Notifications, we can specify to receive daily or weekly e-mail announcements. We can specify what type of information we want to receive, such as publications, hints and tips, product flashes (also known as alerts), downloads, and drivers. My Notifications enables you to customize and categorize the products about which we want to be informed and the delivery methods that best suit your needs.
To subscribe to Support updates:
- Subscribe to the RSS feed for the WebSphere DataPower XC10 appliance forum.
- On the subscription page, click the RSS feed icon.
- Select the option to use to subscribe to the feed.
- Click Subscribe.
- Subscribe to My Notifications by going to the IBM Support Portal and click My Notifications in the Notifications portlet.
- Sign in using your IBM ID and password, and click Submit.
- Identify what and how we want to receive updates.
- Click the Subscribe tab.
- Select the appropriate software brand or type of hardware.
- Select one or more products by name and click Continue.
- Select your preferences for how to receive updates, whether by e-mail, online in a designated folder, or as an RSS or Atom feed.
- Select the types of documentation updates that we want to receive, for example, new information about product downloads and discussion group comments.
- Click Submit.
Until you modify your RSS feeds and My Notifications preferences, you receive notifications of updates that you have requested. We can modify your preferences when needed; for example, if you stop using one product and begin using another product.
2. Working with the WebSphere DataPower XC10 appliance log files
Logs can be viewed directly from the appliance using the user interface or they can be downloaded to your local file system for review.
Log data is stored directly on the appliance. We can provide the trace.zip file to the IBM Support team.
- Navigate to...
Appliance | Troubleshooting | Logging
- Click either...
- View current error file to view the error log
- View current trace file to view trace log
A new Web browser window is opened for the log viewer. The log viewer is used to view the trailing 10 lines of the log you selected. New log entries are appended into the log viewer as they occur. The log viewer has several actions that control the behavior of the log viewer.
- Click Pause to stop new log entries from being appended.
- Click Restart for new entries to be appended.
- Click Clear to clear all the data from the log viewer.
- Click Download log files to save all the available logs to your file system. If you need to view information regarding events that have already happened, then use this link. A window is presented allowing you to open the compressed file or save it to your file system. The trace.zip file contains all the log files to provide to the support team.
A set of CSV files are included in the trace.zip file that we can use to track historical data for the servers on the appliance. Within the trace.zip file, the CSV files are in the server_name/logs directory. The files are named: jvmstats.log, mapstats.log, and ogstats.log. We can collect the following information from the CSV files: memory usage by the map or server, map hit rates, transaction times, throughput, and so on.
- In the Configure trace levels section, we can view or modify the trace levels. We can edit the trace levels for the Administrative Console or the Data Grid. For the administrative console, we can change the output of the default logger to one of the following trace levels:
- OFF
- SEVERE
- WARNING
- INFO
- FINE
- Add a trace string. Click Add trace setting and enter in a valid trace string. The trace level for a new trace string is set to INFO by default.
- Remove a trace string Click the remove icon () next to a trace string to remove that trace string.
- Modify a trace level. Click the trace level and select a new trace level in the drop-down menu. Click Save to commit the new trace level for the specified trace string.
After you have completed these steps, you have reviewed all the available log data.
3. Downloading auditing data
Audit activity is captured by the appliance User activity for a set of auditable objects is preserved. The audit data that captures user activity for specific auditable objects is stored on the appliance. See the following table for information about the auditable objects included in your audit data.
Objects Add/Create Delete Update Other Assimilation Y Y N/A Group Y Y Y Session N/A N/A N/A Time out, log in User Y Y Y Zone Y Y Y Zone activation Y Y Y To ensure data integrity, audit data must be retrieved from the appliance when it is needed. After you have downloaded the audit data, we are responsible for using your own methods to ensure the data integrity. Use the following steps to retrieve your audit data from the appliance.
- Open the Auditing section in the user interface and click...
Applicance | Troubleshooting Auditing
- Download the audit data. We can either download all the data, or a filtered version of the data.
- Download all data
Click Download all data to download the audit.zip file. The audit.zip file contains all the data that is currently available on the appliance. If we are looking for the audit data in a comma-separated value (CSV) format or only a subset of the available audit data, then we can adjust the date range and download the filtered data.
- Download filtered data
Adjust the date range and set the time zone settings. The date range and time zone settings are used to generate the filtered data. These settings are only applicable to the filtered data and are not applicable when downloading all data. The default value for the time zone setting is the time zone associated with the appliance. The default date range is to include data from the last 30 days. By adjusting the time zone and the date range, you specify the audit events that are included.
Click Download filtered data to download the CSV formatted audit data from the date range specified. The audit.csv file is a comma-separated value formatted file that can be downloaded to the file system. We can import the CSV formatted data into vendor software for additional formatting and viewing.
4. Analyzing log and trace data
Use the log analysis tools to analyze how your runtime is performing and solve problems that occur in the environment.
We can generate reports from the existing log and trace files in the environment. These visual reports can be used for the following purposes:
- To analyze runtime environment status and performance:
- Deployment environment consistency
- Logging frequency
- Run topology versus configured topology
- Unplanned topology changes
- Quorum status
- Partition replication status
- Statistics of memory, throughput, processor usage, and so on
- To troubleshoot problems in the environment:
- Topology views at specific points in time
- Statistics of memory, throughput, processor usage during client failures
- Current fix pack levels, tuning settings
- Quorum status
4.1. Log analysis overview
Use the xsLogAnalyzer tool to help troubleshoot issues in the environment.
All failover messages Chart and list of failover messages, including affected servers. All eXtreme Scale critical messages IDs along with the associated explanations and user actions, which can save you the time from searching for messages. All exceptions top five exceptions, including the messages and how many times they occurred, and what servers were affected by the exception. Topology summary Displays a diagram of how your topology is configured according to the log files. Use this summary to compare to your actual configuration, possibly identifying configuration errors. Topology consistency: ORB comparison table Displays ORB settings in the environment. Use this table to help determine if the settings are consistent across your environment. Event timeline view Displays a timeline diagram of different actions that have occurred on the data grid, including life cycle events, exceptions, critical messages, and first-failure data capture (FFDC) events.
4.2. Running log analysis
We can run the xsLogAnalyzer tool on a set of log and trace files from any computer.
- You must have a WXS client installation.
- Collect log and trace files from the appliance.
- To create custom scanners in your generated report, create a scanner specifications properties file and configuration file before we run the tool.
- Run the xsLogAnalyzer tool.
The script is in the following locations :
- In a stand-alone installation: wxs_install_root/ObjectGrid/bin
- In an installation that is integrated with WebSphere Application Server: was_root/bin
If your log files are large, consider using the -startTime, -endTime, and -maxRecords parameters when we run the report to restrict the number of log entries that are scanned. Using these parameters when we run the report makes the reports easier to read and run more effectively. We can run multiple reports on the same set of log files.
xsLogAnalyzer.sh|bat -logsRoot c:\myxslogs -outDir c:\myxslogs\out -startTime 11.09.27_15.10.56.089 -endTime 11.09.27_16.10.56.089 -maxRecords 100
- -logsRoot
- Absolute path to the log directory that we want to evaluate (required).
- -outDir
- Existing directory to write the report output. If you do not specify a value, the report is written to the root location of the xsLogAnalyzer tool.
- -startTime
- Start time to evaluate in the logs. The date is in the following format: year.month.day_hour.minute.second.millisecond
- -endTime
- End time to evaluate in the logs. The date is in the following format: year.month.day_hour.minute.second.millisecond
- -trace
- Trace string, such as ObjectGrid*=all=enabled.
- -maxRecords
- Maximum number of records to generate in the report. The default is 100. If you specify the value as 50, the first 50 records are generated for the specified time period.
- Open the generated files. If you did not define an output directory, the reports are generated in a folder called report_date_time. To open the main page of the reports, open the index.html file.
- Use the reports to analyze the log data. Use the following tips to maximize the performance of the report displays:
- To maximize the performance of queries on the log data, use as specific information as possible. For example, a query for server takes much longer to run and returns more results than server_host_name.
- Some views have a limited number of data points that are displayed at one time. We can adjust the segment of time that is being viewed by changing the current data, such as start and end time, in the view.
4.3. Create custom scanners for log analysis
We can create custom scanners for log analysis. After you configure the scanner, the results are generated in the reports when we run the xsLogAnalyzer tool. The custom scanner scans the logs for event records based on the regular expressions specified.
- Create a scanner specifications properties file that specifies the general expression to run for the custom scanner.
- Create and save a properties file. The file must be in the loganalyzer_root/config/custom directory. We can name the file as: you like. The file is used by the new scanner, so naming the scanner in the properties file is useful, for example: my_new_server_scanner_spec.properties.
- Include the following properties in the my_new_server_scanner_spec.properties file:
include.regular_expression = REGULAR_EXPRESSION_TO_SCANThe REGULAR_EXPRESSION_TO_SCAN variable is a regular expression on which to filter the log files.Example: To scan for instances of lines that contain both the "xception" and "rror" strings regardless of the order, set the include.regular_expression property to the following value:
include.regular_expression = (xception.+rror)|(rror.+xception)This regular expression causes events to be recorded if the string "rror" comes before or after the "xception" string.Example: To scan through each line in the logs for instances of lines that contain either the phrase "xception" or the phrase "rror" strings regardless of the order, set the include.regular_expression property to the following value:
include.regular_expression = (xception)|(rror)This regular expression causes events to be recorded if the either the "rror" string or the "xception" string exist.
- Create a configuration file that the xsLogAnalyer tool uses to create the scanner.
- Create and save a configuration file. The file must be in the loganalyzer_root/config/custom directory. We can name the file as scanner_nameScanner.config, where scanner_name is a unique name for the new scanner. For example, you might name the file serverScanner.config
- Include the following properties in the scanner_nameScanner.config file:
scannerSpecificationFiles = LOCATION_OF_SCANNER_SPECIFICATION_FILEThe LOCATION_OF_SCANNER_SPECIFICATION_FILE variable is the path and location of the specification file that you created in the previous step. For example: loganalyzer_root/config/custom/my_new_scanner_spec.properties. We can also specify multiple scanner specification files by using a semi-colon separated list:scannerSpecificationFiles = LOCATION_OF_SCANNER_SPECIFICATION_FILE1;LOCATION_OF_SCANNER_SPECIFICATION_FILE2
- Run the xsLogAnalyzer tool.
After we run the xsLogAnalyzer tool, the report contains new tabs in the report for the custom scanners that you configured. Each tab contains the following views:
- Charts
- A plotted graph that illustrates recorded events. The events are displayed in the order in which the events were found.
- Tables
- A tabular representation of the recorded events.
- Summary reports
4.4. Troubleshooting log analysis
- Problem: Out of memory conditions occur when we are using the xsLogAnalyzer tool to generate reports. An example of an error that might occur follows: java.lang.OutOfMemoryError: GC overhead limit exceeded.
Solution: The xsLogAnalyzer tool runs within a JVM. We can configure the JVM to increase the heap size before we run the xsLogAnalyzer tool by specifying some settings when we run the tool. Increasing the heap size enables more event records to be stored in JVM memory. Start with a setting of 2048M, assuming the operating system has enough main memory. On the same command-line instance in which we are planning to run the xsLogAnalyzer tool, set the maximum JVM heap size:
java -XmxHEAP_SIZEm
The HEAP_SIZE value can be any integer and represents the number of megabytes that are allocated to JVM heap.For example, you might run java -Xmx2048m. If the out of memory messages continue, or you do not have the resources to allocate 2048m or more of memory, limit the number of events that are being held in the heap. We can limit the number of events in the heap up by passing the -maxRecords parameter to the .xsLogAnalyzer command
- Problem: When you open a generated report from the xsLogAnalyzer tool, the browser hangs or does not load the page.
Cause: The generated HTML files are too large and cannot be loaded by the browser. These files are large because the scope of the log files that we are analyzing is too broad.
Solution: Consider using the -startTime, -endTime, and -maxRecords parameters when we run the xsLogAnalyzer tool to restrict the number of log entries that are scanned. Using these parameters when we run the report makes the reports easier to read and run more effectively. We can run multiple reports on the same set of log files.
5. Monitor the hardware temperatures
Temperature sensors constantly measure the temperature of various internal components of the appliance. These temperatures can conveniently be monitored on a single panel using the user interface.
You must be assigned the Appliance administration permission to perform these steps. The IBM WebSphere DataPower XC10 appliance monitors various internal component temperatures to ensure that the appliance is operating at a safe temperature.
- Navigate to Appliance > Troubleshooting.
- Expand Hardware Temperatures.
- Review your hardware temperatures.
The most recent reading for each temperature sensor is displayed using three significant digits.
- If the temperature of the component is at a typical operating temperature, the green icon () is displayed.
- If the temperature of the component exceeds the safe operating level, then the yellow warning icon () is displayed.
After you have successfully completed these steps, you have viewed the available temperature readings to confirm your appliance is operating at a safe temperature.
Monitor your Ethernet interface status details with a serial connection
If network trouble is encountered on your IBM WebSphere DataPower XC10 appliance, then we can view additional details about the activity and status of your Ethernet interfaces. This topic explains how to view these details with a serial connection with the appliance, but these details are also available in the user interface.
You must have physical access to the appliance and be able to log in as xcadmin.
- Establish a serial connection with the appliance as the xcadmin user.
The serial connection must be between an ASCII terminal or a PC running terminal emulation software to the serial port on the appliance. If we are using a PC for the serial console, use a PC-based serial communications program for Windows or Linux. Use either the supplied serial cable or PL-2303 USB-to-serial cable to make the connection to the appliance.
If we are using the PL-2303 USB-to-serial cable, download and install a driver for the cable before proceeding. The serial console settings are 9600.8.n.1.
- Display the Ethernet interface status. Use the following command to display the Ethernet interface status:
netif status <interface>
Where <interface> is the name of the Ethernet interface to query. Example output follows:
Console> netif status mgt0 mgt0 generic MTU:1500 flags:UP BROADCAST RUNNING MULTICAST inet addr:127.0.0.1 mask:255.255.255.128 inet6 addr: 2001:DB8:0:0:0:0:0:0/32 mask: ffff:ffff:ffff:ffff:: ethernet MAC: 00:1a:64:88:a0:6c autoneg:on duplex:Full port:TP speed:100Mbps statistics collisions:0 multicast:13 rx_bytes:101069093 rx_compressed:0 rx_crc_errors:0 rx_dropped:0 rx_errors:0 rx_fifo_errors:0 rx_frame_errors:0 rx_length_errors:0 rx_missed_errors:0 rx_over_errors:0 rx_packets:1515625 tx_aborted_errors:0 tx_bytes:10045272 tx_carrier_errors:0 tx_compressed:0 tx_dropped:0 tx_errors:0 tx_fifo_errors:0 tx_heartbeat_errors:0 tx_packets:20104 tx_window_errors:0After completing these steps, you have reviewed the available status details for the Ethernet interfaces on your appliance.
7. Verifying outbound connections from the appliance
Using the outbound connection function, we can confirm a network address is reachable from the appliance.
You must be assigned the Appliance administration permission to perform these steps. The outbound connection function can aid in the troubleshooting effort to isolate a problem by confirming a target network address is reachable from the appliance. This tool is not relevant when debugging every issue, but serves as a convenient method to confirm a target network address is available and that connectivity to that network address is not hindered by a firewall or an issue with the network.
- Navigate to Appliance > Troubleshooting.
- Expand Outbound Connections.
- Enter an IP address or fully-qualified host name. The network address entered in this field is used as the target address when the ping command is issued.
- Click Ping to perform the ping command to the network address you provided.
- If the connection attempt to the network address you entered is successful, then you see the following icon: .
- If the connection attempt to the network address you entered is not successful, then you see the following icon: .
After successfully completing these steps, you have determined if a target address is currently reachable over the network from the appliance.
8. Using the command line interface to run operations on your appliance
We can run commands to restart the appliance hardware, reset the appliance to its original factory settings, or shut down the appliance. Use the command-line interface to reset the appliance configuration data, restart the appliance, or shut down the appliance.
- Establish a connection with the appliance using the xcadmin user. Use a serial connection or we can connect with the command-line interface using a Secure Shell (SSH).
- Serial connection: The serial connection must be between an ASCII terminal or a PC running terminal emulation software to the serial port on the appliance. If we are using a PC for the serial console, use a PC-based serial communications program for Windows or Linux. Use either the supplied serial cable or PL-2303 USB-to-serial cable to make the connection to the appliance.
If we are using the PL-2303 USB-to-serial cable, download and install a driver for the cable before proceeding.
- Remote login: To connect to the appliance with SSH, provide the URL of your appliance to your SSH client.
- After you have established a connection with the appliance, we can issue one of the following commands.
- clear-all
- Resets the configuration data for the appliance, including removing all the data grids and new users. All of the internal processes that are running on the appliance are restarted. This command runs without restarting the appliance hardware.
- clear-logs
- Deletes all of theWebSphere DataPower XC10 appliance log files.
- clear-tls-config
- Resets the Transport Layer Security (TLS) configuration. Run this command if clear-all is not an option because you do not want to lose all configuration data, but TLS configuration becomes corrupted or we want to restore the default TLS values. In most situations, use the user interface for TLS changes or use the clear-all command to revert to the original configuration of the appliance.
Run the clear-tls-config command on each appliance in the collective. After running the command on each appliance, restart the processes in each appliance in the collective.
If the collective is successfully communicating, use the device restart command. The collective is communicating properly when all appliances in the collective are accessible through the user interface and can be seen as started in the Collective panel. However, if the TLS configuration is preventing the collective from communicating and the device restart command does not bring the appliance back up, we can use the force-recycle command to forcibly stop and start all the processes in the appliance without saving any data.
- device RESET
- Resets the appliance back to factory settings of the network, licensing, and default user IDs. All the data in the data grids is deleted. Reinitialize the appliance after running this command.
- device restart
- Restarts the appliance hardware. The data grid, configuration, and user data is saved before the device shuts down and is available when the appliance restarts.
- device shutdown
- Shuts down the appliance hardware. The data grid, configuration, and user data is saved before the device shuts down.
- file get <file_location> <file_name_on_appliance>
- Retrieve a file and saves the file to the appliance.
- firmware upgrade <firmware_file>
- Installs the specified firmware upgrade file.
- force-recycle
- Restarts the WebSphere DataPower XC10 appliance processes without saving any data. Because data loss can occur, run this command only if we are not worried about data loss or you have tried the device restart command and the appliance did not become available.
- get-dns-servers
- Display the IP addresses of the configured domain name servers.
- list ethernet-interface
- List the available Ethernet interfaces on the appliance.
- net-test available
- Tests whether a network interface exists with an active carrier.
- net-test dns <host_name>
- Performs domain name server look up on a specified host name.
- net-test ping <host_name>
- Tests connectivity to a specified host name or IP address.
- net-test tcp<host_name> <port>
- Tests the ability to open a TCP connection to port at a specified hostname or IP address.
- netif show <interface>
- Display the Ethernet interface configuration information.
- netif status <interface>
- Display the Ethernet interface status.
- show version
- Display the current version of appliance firmware that is installed.
- set-dns-servers <valid DNS server>
- Set the DNS server for the appliance.
- start-progress
- We can issue this command when the startup is in progress. The command displays the percentage of the startup process that has completed.
- status cpu-usage
- Display the processor load percentage for the past minute, 5 minutes, and 15 minutes.
If you reset the appliance, we can reinitialize your appliance.
9. Troubleshooting cache integration
Use this information to troubleshoot issues with the cache integration configuration, including HTTP session and dynamic cache configurations.
- Problem: HTTP session IDs are not being reused.
Cause: We can reuse session IDs. If you create a data grid for session persistence in Version 7.1.1 or later, session ID reuse is automatically enabled. However, if you created prior configurations, this setting might already be set with the wrong value.
Solution: Check the following settings to verify that you have HTTP session ID reuse enabled:
- The reuseSessionId property in the splicer.properties file must be set to true.
- The HttpSessionIdReuse custom property value must be set to true. This custom property might be set on one of the following paths in the WebSphere Application Server administrative console:
- Servers > server_name > Session management > Custom properties
- Dynamic clusters > dynamic_cluster_name > Server template > Session management > Custom properties
- Servers > Server Types > WebSphere application servers > server_name, and then, under Server Infrastructure, click Java and process management > Process definition > JVM > Custom properties
- Servers > Server Types > WebSphere application servers > server_name > Web container settings > Web container
If you update any custom property values, reconfigure eXtreme Scale session management so the splicer.properties file becomes aware of the change.
- Problem:
When we are using a data grid to store HTTP sessions and the transaction load is high, a CWOBJ0006W message displays in the SystemOut.log file.
CWOBJ0006W: An exception occurred: com.ibm.websphere.objectgrid.ObjectGridRuntimeException: java.util.ConcurrentModificationExceptionThis message occurs only when the Web application modifies a List object that was set as an attribute on the HTTPSession.
Solution: Clone the attribute that contains the modified List object and put the cloned attribute into the session object.
10. Troubleshooting installation
Use this information to troubleshoot issues with the installation and updates.
- Problem: When we run the installation command from a remote computer, such as \\mymachine\downloads\, the following message displays: CMD.EXE was started with the above path as the current directory. UNC paths are not supported. Defaulting to Windows directory. As a result, the installation does not complete correctly.
Solution: Map the remote computer to a network drive. For example, in Windows, we can right-click My computer and choose Map Network Drive and include the uniform naming conventions (UNC) path to the remote computer. We can then run the installation script from the network drive successfully, for example: y:\mymachine\downloads\WXS\install.bat.
- Problem: The installation completes unsuccessfully.
Solution: Check the log files to see where the installation failed. When the installation completes unsuccessfully, the logs are in the wxs_install_root/logs/wxs directory.
- Problem: A catastrophic failure occurs during the installation.
Solution: Check the log files to see where the installation failed. When the installation fails when it is partially completed, the logs can generally be found in the user_root/wxs_install_logs/ directory.
- Problem: If we are installing the WebSphere eXtreme Scale Client on Windows, you might see the following text in the results of the installation:
Success: The installation of the following product was successful: WebSphere eXtreme Scale Client. Some configuration steps have errors. For more information, refer to the following log file: <WebSphere Application Server install root>\logs\wxs_client\install\log.txt" Review the installation log (log.txt) and review the dmgr augmentation log.Solution: If you see a failure with the iscdeploy.sh file, we can ignore the error. This error does not cause any problems.
- Problem:
If you have a full installation and try to apply WebSphere eXtreme Scale Client only maintenance with the update installer, you see the following message:
Prerequisite checking has failed. Click Back to select a different package, or click Cancel to exit. Failure messages are: Required feature wxs.client.primary is not found.If you have WebSphere eXtreme Scale Client installed and try to apply a full maintenance package with the update installer, you see the following message:
Prerequisite checking has failed. Click Back to select a different package, or click Cancel to exit. Failure messages are: Required feature wxs.primary is not found.Solution: The maintenance package that you install must match the type of installation. Download and apply the maintenance package that applies to the installation type.
11. Troubleshooting administration
Use the following information to troubleshoot administration, including starting and stopping servers, using the xscmd utility, and so on.
- Problem: When we are running a xscmd command, the following message is printed to the screen:
java.lang.IllegalStateException: Placement service MBean not available. [] at com.ibm.websphere.samples.objectgrid.admin.OGAdmin.main(OGAdmin.java:1449) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:60) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37) at java.lang.reflect.Method.invoke(Method.java:611) at com.ibm.ws.bootstrap.WSLauncher.main(WSLauncher.java:267) Ending at: 2011-11-10 18:13:00.000000484Cause: A connection problem has occurred with the catalog server.
Solution: Verify that the catalog servers are running and are available through the network. This message can also occur when you have a catalog service domain defined, but less than two catalog servers are running. The environment is not available until two catalog servers are started.
- Problem: When we are running a xscmd command, the following message is printed to the screen:
CWXSI0066E: Unmatched argument argument_name was detected.Cause: You entered a command format that the xscmd utility did not recognize.
Solution: Check the format of the command. You might encounter this issue when running regular expressions with the -c findbyKey command.
12. Release notes
Access last-minute updates, limitations, and known problems
The release notes are available on the product support site as technotes. To see a list of all the technotes for WebSphere DataPower XC10 appliance, go to the Support Web page. Clicking the links provided here will result in a search of the Support Web page for the relevant release notes, which will be returned as a list.
- To see a list of the release notes, go to the Support Web page.
Access system and software requirements
The hardware and software requirements are documented on the following pages:
- System requirements
Access product documentation
For the entire information set, go to the Library page.
Access the product support Web site
To search for the latest technotes, downloads, fixes, and other support-related information, go to the Support page.
Contacting IBM Software Support
If you encounter a problem with the product, first try the following actions:
- Follow the steps described in the product documentation
- Look for related documentation in the online help
- Look up error messages in the message reference
If we cannot resolve your problem by any of the preceding methods, contact IBM Technical Support.