Dynamic cache service multi-cell and multi-core group invalidation
Dynamic cache service multi-cell and multi-core group invalidation enables caches created in different cells to be synchronized without the cache entries being replicated across cell boundaries. When this function is enabled for either servlet and object cache instances, data gets cached in both cells as a result of the cells handling the same traffic. When a cache entry is invalidated, using the Java Message Service (JMS), the ID of the invalidated cache entry is sent to a peer cell. The receiving cell uses a service integration bus to process the invalidation. This function can also be used to propagate invalidations to other core groups in the same cell.
The dynamic cache external cache adapter, ExternalCacheAdapter (ECA), is used to capture invalidations from a server. JMS/service integration bus is used to send and receive invalidations in the remote cell or core group. Additional application servers are created to host the invalidation application. These servers automatically become members of the service integration bus. These application servers process the invalidations, which consist of JMS messages, from the remote cell and communicate them to the cluster members in the local cell.
The external cache adapter, com.ibm.websphere.servlet.cache.RemoteInvalidator, is used to send the IDs of invalidated caches to a service integration bus in a remote cell. Instead of specifying a host:port for the address, this type of external cache adapter uses the JMS queue connection factory JNDI name, and queue destination name of the remote service integration bus as the address:
Adapter bean name = com.ibm.websphere.servlet.cache.RemoteInvalidator
Address = JMS_QUEUE_CONNECTION_FACTORY_JNDI_NAME, REMOTE_QUEUE_DESTINATION_NAME
Before we can use this function, ensure that:
- Each cell or core group contains a cluster hosting a production application. The same production application has to be used in both cells or core groups to ensure invalidation IDs are processed correctly.
- A separate cluster of two servers is created in each cell or core group to host the service integration bus. The members of this cluster cannot have a RemoteJMSInvalidator external cache group defined. If a RemoteJMSInvalidator external cache group is defined for these cluster members, an infinite loopback condition occurs where invalidations are repeatedly send back and forth between cells.
- A datasource is defined and is accessible from both service integration bus cluster members. If the database is shared by both cells, ensure a unique database name is used when defining each the datasource for each cell.
- Each cell or core group contains a replication domain containing both the service integration bus cluster and all of the clusters hosting the production applications.
If we are using this function for object cache instances, we must also set the following JVM custom properties to true. These properties can be specified in either the cacheinstances.properties file or the properties map when we create a cache instance, or we can use the administrative console to add this property to a JVM.:
- com.ibm.ws.cache.CacheConfig.interCellReplication
- enableInterCellInvalidation
Configuration considerations
The dynacacheJMSSIB.py script is provided in the WAS_INSTALL_ROOT/util/ directory. Use this script to configure the external cache adapter and service integration bus infrastructure in both cells. The script must be executed from the WAS_INSTALL_ROOT/profiles/PROFILE_NAME/bin directory for it to work correctly. For example:
- WAS_INSTALL_ROOT/profiles/Dmgr01/bin)
The configuration of the servlet or object cache instances should be available to both clusters in a core group. Use either a properties file, a factory or the administrative console to configure a servlet or object cache instance. This configuration must be set at the cell scope, and must be available to both the service integration bus and production clusters in the core group. The easiest way to ensure this availability is to create and define the cache instances on both the web production and the service integration bus clusters when we install the application.
Following is an example of the statements needed to add to a cacheinstances.properties file for a cache instance:
- cache.instance.0=/services/cache/instance_one
cache.instance.0.cacheSize=1000
cache.instance.0.enableInterCellInvalidation=true
Following is an example of the statements needed to specify when using the DistributedMapFactory to create a DistributedMap for a cache instance:
- Properties p = new Properties();
==>p.put("com.ibm.ws.cache.CacheConfig.enableInterCellInvalidation", "true");
DistributedMap map1 = DistributedMapFactory.getMap("myMap", p);
(ZOS) To set these properties in the administrative console, click Servers > Application servers >server, and then, in the Server Infrastructure section, click Java and process management > Process definition > Control > Java virtual machine > Custom Properties, or click Resources > Cache instances > Object cache instances > cache_instance_name.
To set these properties in the administrative console, click Servers > Application servers > server, and then, in the Server Infrastructure section, click Java and process management > Process definition > Java virtual machine > Custom Properties, or click Resources > Cache instances > Object cache instances > cache_instance_name.
The dynacacheJMSSIB.py script configures each cell in 3 steps:
- Configuration of the Inbound JMS/service integration bus, which
consists of the following actions:
- The creation of a service integration bus and the specification of a service integration bus destination.
- The creation of the JMS queue.
- The activating the JMS.
- Installation of the WAS_INSTALL_ROOT/installables/DynacacheMessageHandler.ear file, which is a message driven bean, on the service integration bus cluster.
- Configuration of the Outbound JMS/service integration bus, which consists of configuring the JMS queue connection factory.
- Configuration of the external cache group, which consists of configuring the external cache group, and external cache adapter on each cluster member hosting the production applications.
Tune considerations
The following JVM custom properties are can be used to tune the dynamic cache service multi-cell and multi-core group invalidation:
- dynacache.jms.connRetryInterval
- Number of seconds that a cluster member waits before attempting to reconnect to a service integration bus server. The default is 30
- dynacache.jms.numStoredInvalidations
- Maximum number of invalidation IDs a cluster member can store while waiting for a service integration bus server to become available. After the threshold is reached, the oldest invalidations is removed. The default is 10000.
- dynacache.jms.cacheInstance
- Cache instance to use for processing invalidations. Setting this property to "*" causes invalidation IDs to be processed on all cache instances. By default, the baseCache cache instance is used. This property is only applicable to the service integration bus servers.
- dynacache.jms.invProcessingDelay
- Number of seconds the service integration bus server queues invalidation IDs before processing them. The default is 20.