WAS v8.5 > Reference > Troubleshooting tips

Troubleshooting tips for the dynamic cache service

The dynamic cache service works within an application server JVM, intercepting calls to cacheable objects. This article describes some common runtime and configuration problems and remedies.


Servlets are not cached

Recommended response

Enable servlet caching. On the web container settings page of the dmgr console, select the Enable servlet caching check box.


Cache entries are not written to disk

Explanation Recommended response
Cache entries are written to disk when the cache is full and new entries are added to the memory cache. Cache entries also are written to disk when Flush to disk is enabled in the dmgr console and the server is stopped.

Verify that Disk offload is enabled on the Dynamic cache service settings page of the dmgr console. Also verify that cache entries written to disk are serializable and do not have the PersistToDisk configuration set to false.


Some servlets are not replicated or written to disk

Recommended response
Ensure the attributes and response are serializable. If we do not want to store the attributes, use the following property in your cache policy:

    <property name="save-attributes">false</property>


Dynamic cache service does not cache fragments on the Edge

Recommended response
Set the EdgeCacheable property to true in the cache policy for those entries that are to be cached on the Edge.

    <property name="EdgeCacheable">true</property>


Dynamic cache invalidations are not sent to the IBM HTTP Server plug-in

Explanation Recommended response
The DynaCacheEsi.ear file is required to send invalidations to external caches.

Install the DynaCacheEsi.ear file using the dmgr console.


Cache entries are evicted often

Problem Explanation Recommended response
The cache is full and new entries are added to the cache. Cache entries are evicted when the cache is full and new entries are added to the cache. A least recently used (LRU) eviction mechanism removes the least recently used entry to make space for the new entries.

Enable Disk offload on the Dynamic cache service settings page of the dmgr console so the entries are written to disk. We can also increase the cache size to accommodate more entries in the cache.


Cache entries in disk with timeout set to 0 expire after one day

Explanation Recommended response
Maximum lifetime of an entry in disk cache is 24 hours. A timeout of 0 in the cache policy configures these entries to stay in disk cache for one whole day, unless they are evicted earlier. Set the timeout for the cache policy to a number less than 0.


Unable to monitor cache entries on the Edge

Explanation Recommended response

Use the dynamic cache monitor to monitor the contents of the memory cache, disk cache and external caches, like the Edge cache. For the ESI processor's cache to be visible in the cache monitor, the DynaCacheEsi.ear application must be installed and the esiInvalidationMonitor property must be set to true in the plugin-cfg.xml file.

Set the esiInvalidationMonitor property in the plugin-cfg.xml file to true. Refer to the Displaying cache information article for more information about monitoring dynamic cache service activity.


Fine tune cache for my environment

Recommended response
Use the Tivoli Performance viewer to study the caching behavior for the applications. Also consider performing the following actions:

  • Increase the priority of cache entries that are expensive to regenerate.

  • Modify timeout of entries so they stay in memory as long as they are valid.

  • Enable disk offload to store LRU evicted entries.

  • Increase the cache size.


Cleaning the disk cache files after installing the fix pack or a new release

Symptom Problem Recommended response
If the server is configured to use the disk cache, you must delete the disk cache files because the disk cache files are not compatible to the previous version. Failure to remove the old disk cache files results in a ClassCastException error in the systemerr.log file when we access the cache from the disk.

IBM recommends using the HPEL log and trace infrastructure. With HPEL, one views logs using the LogViewer command-line tool in PROFILE/bin.

To delete the disk cache, perform the following steps:

  1. Note your disk offload location. If we do not know the disk cache offload location, perform the following steps:

    1. Click Servers > Application servers > server_name > Container services > Dynamic cache service in the dmgr console navigation tree.
    2. The location is specified in the Disk offload field. If the location is not specified, the default directory profile_root/temp/your_node/server_name/_dynacache is used.

  2. Verify the you stop the server and delete all the files under the offload location.

  3. If we use the WebSphere Application Server, Network Deployment product, delete the disk cache files for each server.


Set the flush attribute to true on every <jsp:include> tag in the cacheable JSP file

Ensure you set the flush attribute to true on every <jsp:include> tag in the cacheable JSP file
Symptom Problem Description Recommended response
When you obtain the JSP file from the dynamic cache, a part of the page is not displayed. The flush attribute is set to false on the <jsp: include> tag in the JSP file. When the cacheable JSP file includes another JSP file and if the flush attribute is set to false on the <jsp: include> tag, any data written to the parent output stream prior to the<jsp: include> tag are not cached. Set flush=true on every <jsp: include> tag in the cacheable JSP file.


File system I/O activity when the application server is idle

Symptom Explanation Possible response Recommended response
You experience file system I/O activity when the application server is idle. WAS is polling the Hierarchical File System (HFS) for the cachespec.xml file, even when there are no cache policies defined. Dynamic cache queries the file system every 30 seconds to check for updates to the cache policy file. By checking for cache policy file updates, dynamic cache can automatically update any changed cache policies.

The dynamic cache is enabled by default because several system components rely on it for performance reasons. We can disable the dynamic cache service using a wsadmin script or through the dmgr console. Refer to the Using the dynamic cache service topic for more information about enabling the dynamic cache service.

CAUTION:

By disabling dynamic cache you might experience slower performance of security because it uses the distributed map functionality of the dynamic cache service.

Set flush=true on every <jsp: include> tag in the cacheable JSP file.


Dynamic cache limitation when using the JSTL <c:import> tag to include a fragment

Problem Description Recommended response
When a cacheable fragment is included using the JSP Standard Tag Library (JSTL) <c:import> tag, part of the page content disappears and part of the page content displays twice on a cache hit. Dynamic cache relies on flushing the content prior to and after including a fragment so the parent content preceding the include is not lost, and also to prevent pulling the child content into the parent fragment.

However, in the case of the JSTL <c:import> tag, the flush=true attribute, which flushes the parent writer prior to the child fragment is actually invoked, is not supported. Also, JSTL buffers the responses, so the child writer is not flushed following the child fragment is done. Subsequently, the child response is pulled into the parent.

Restriction: Dynamic cache will return multiple include statements when the JSTL <c:import> tag is used in a cached JSP file.

To avoid this problem, surround the <c:import> statement with out.flush method statements as follows:

    <% out.flush(); %> <c:import url="DNCParent2.jsp" /> <% out.flush(); %>


Service integration bus messages are repeated

The service integration bus messages are repeated in the logs of the cluster members hosting a production application
Problem Recommended response
In a multi-cell environment, the following messages are repeated in the logs of the cluster members hosting the production application: 
[time_stamp] CWSIT0007W: It is not possible to contact
 the bootstrap server at 
 9.9.9.9:7299:BootstrapBasicMessaging because of exception: 
 com.ibm.websphere.sib.exception.SIResourceException: 
 CWSIC1001E: 
A client attempted to connect with a remote messaging  engine (9.9.9.9:7299 - BootstrapBasicMessaging) but  the connection cannot be completed. Ensure the  messaging engine is started: exception com.ibm.ws.sib
 .jfapchannel.JFapConnectFailedException: CWSIJ0063E: A 
 network connection to host name 9.9.9.9, port 7299
 cannot be established...
[time_stamp] 00000023 SystemOut O RemoteInvalidator
 unable to connect to ...
 First, ensure each service integration bus member in the remote cell is started. Next, ensure the SIB_ENDPOINT_ADDRESS port is specified correctly for each service integration bus member in the remote cell or core group. If the wrong port is specified, delete the outbound configuration "--setup=dynacacheOutSIB --delete..." and reconfigure it using the correct port. When everything is working correctly, a message similar to the following message displays in the logs: 
[time_stamp] 0000000a RemoteInvalid I   
DYNA1075I: The "RemoteJMSInvalidator" external 
cache group is connected. The configured JNDI name of the queue connection factory is 
"DynacacheDestination-cell2" and the JNDI name of 
the targeted destination is 
"jms/DynacacheOutBoundConnectionFactory-cell1".


Platform messaging component cannot authenticate the user ID

Explanation Recommended response
In a multi-cell environment, a service integration bus member has the following error: 
[time_stamp] 00000022 SibMessage   E  [:] CWSII0050E: 
The Platform Messaging Component can not authenticate 
the user ID.
This message might be misleading because the dynamic cache service does not use secure buses. 		This message might indicate one of the following problems:

The service integration bus server is receiving a request for a bus destination that does not exist. Ensure the correct remote cell name was used when executing "--setup=dynacacheOutSIB" and "--setup=dynacacheECA" in the sending cell.

No service integration bus members are available. This problem often occurs because the wrong cluster is specified when doing the inbound setup "--setup=dynacacheInSIB". Ensure the service integration bus cluster is specified for the "--cluster" option and not some other cluster.


The messaging engine's unique ID does not match that found in the data store

Explanation Recommended response
In a multi-cell environment, a service integration bus member has the following error: 
[DynacacheBus-edgeaphid10Cell01-cg2:edgeaphid10Node01.
 cg2SIBServer-DynacacheBus-edgeaphid10Cell01-cg2] 
CWSIS1535E: The messaging engine's unique id does not  match that found in the data store. 
 ME_UUID=D520787E8CA7F18A, ME_UUID(DB)=980C0B42B3A904F3
[time_stamp] 0000002f SibMessage    I   
[DynacacheBus-edgeaphid10Cell01-cg2:edgeaphid10Node01.
 cg2SIBServer-DynacacheBus-edgeaphid10Cell01-cg2] 
CWSIS1546I: The messaging engine, 
 ME_UUID=D520787E8CA7F18A, INC_UUID=7228ea45e216f3ef,
 has lost an existing lock or failed to gain an initial
 lock on the data store.
[time_stamp] 0000002f SibMessage    I   
[DynacacheBus-edgeaphid10Cell01-cg2:edgeaphid10Node01.
 cg2SIBServer-DynacacheBus-edgeaphid10Cell01-cg2] 
CWSIS1519E: Messaging engine edgeaphid10Node01.
 cg2SIBServer-DynacacheBus-edgeaphid10Cell01-cg2 
 cannot obtain the lock on its data store, which ensures 
 it has exclusive access to the data.
[time_stamp] 0000002d SibMessage    E   
[DynacacheBus-edgeaphid10Cell01-cg2:edgeaphid10Node01.
 cg2SIBServer-DynacacheBus-edgeaphid10Cell01-cg2] 
CWSIS0002E: The messaging engine encountered an exception  while starting. Exception: com.ibm.ws.sib.msgstore.
 PersistenceException: 
 CWSIS1501E: The data source has produced an unexpected
 exception: com.ibm.ws.sib.msgstore.persistence.
 DatasourceWrapperStateException: New connections cannot  be provided because the persistence layer has been  stopped at com.ibm.ws.sib.msgstore.persistence.impl.
 PersistentMessageStoreImpl.start
 (PersistentMessageStoreImpl.java:188)
 at com.ibm.ws.sib.msgstore.impl.MessageStoreImpl.
   start(MessageStoreImpl.java:1175)
 at com.ibm.ws.sib.admin.impl.JsMessagingEngineImpl.
  start(JsMessagingEngineImpl.java:491)

Stop the server, delete the directory in WAS_INSTALL_ROOT\profiles\AppSrv01\filestores\com.ibm.ws.sib that corresponds to the service integration bus member, and restart the server.


dynacacheJMSSIB script can not locate DynacacheMessageHandler.ear file

Explanation Recommended response
In a multi-cell environment, the dynacacheJMSSIB script can not locate DynacacheMessageHandler.ear file, and the following message appears in the log: 
WASX7017E: Exception received while running file 
"../../../util/dynacacheJMSSIB.py"; exception 
information: 
com.ibm.ws.scripting.ScriptingException: WASX7115E: 
Cannot read input file "/opt/WAS/6.1/cf270928.19
 /profiles/AppSrv01/logs/DynacacheMessageHandler.ear"
 Run the dynacacheJMSSIB script from WAS_INSTALL_ROOT/profiles/PROFILE_NAME/bin so that it can locate ../../../util/dynacacheJMSSIB.py and ../../../installableApps/DynacacheMessageHandler.ear at the appropriate relative paths.


Related concepts:

Cache monitor


Related


Troubleshooting the dynamic cache service
Display cache information
Tasks: Using the dynamic cache service to improve performance
Troubleshoot applications with HPEL
Install enterprise application files with the console
Use the dynamic cache service


Reference:

Web container settings
Dynamic cache service settings


+

Search Tips   |   Advanced Search