Hints and tips for configuring file store size

Preserving the appropriate amount of space within the log file, permanent store file, and temporary store file of a file store helps to ensure that operations and transactions behave as expected.

The default settings for the minimum and maximum sizes are most suitable if we are not using a dedicated disk for our file store because they protect the file store from other disk users, and also protect other disk users from the file store. The default settings are less appropriate if we have a disk dedicated to file store use, and in this case we might want to consider setting the maximum size set to unlimited.

Input/Output performance typically poses a significant challenge for file stores. Performance improvements can be achieved using dedicated disks for the file store to avoid contention.

If our file store is not on a dedicated disk, consider making the minimum and maximum sizes the same. Initially, the message store reserves the amount of space defined by the minimum setting. Therefore, if we use the same value for the maximum and minimum settings, it reduces the likelihood of other applications using the disk space that we intended for the file store usage. Setting the maximum size to a value other than unlimited reduces the likelihood of the file store using disk space that we intended for other applications usage. Other applications can include, for example, the application server itself.

The most significant performance gains can be derived using disk hardware configured to ignore the flags set by the operating system to indicate that a synchronous write is required when a reliable battery backed storage device is used. Such a device can effectively maintain the logs in memory and store files so that the writes do not have to wait for physical disk movements. Rather the data written is stored in a cache, and the assurance that the data is not lost in the event of a failure comes from the disk vendor on the basis that the cache is reliable either due to the battery or some other reason.

Log file size

Data is first written to the log file sequentially, that is, new records are added to the end of the file. When the end of the log file is reached, old records at the beginning of the log file are overwritten by new records and this process repeats. Later, the data is written to the permanent store file and temporary store file. The exception is a short-lived data, which is only written to the log file.

The log file size lays the limitation on the maximum size of messages that can be sent. A LogFileFullException is thrown if the log file is too small to handle large messages. The exception occurs because a single large message or multiple large messages are sent before the existing messages from the log file are moved to the permanent or temporary store files. The size of the log file needs to be increased if this exception is encountered; otherwise, the default size is sufficient. Only testing using the maximum message size and load expected can accurately determine whether the default size is too small. It is not always beneficial to simply allocate a larger log file size because the size of the log file is used by algorithms that control the internal workings of the file store. An overly large log file results in increased memory usage.

Store File size

The permanent store file and temporary store file each have a minimum reserved size and a maximum size. When they are created, the permanent and temporary store files consume their minimum reserved sizes, plus the size of the log. If the maximum size is set to a larger value than the minimum reserved size, the files grow up to the maximum size as required.

The default store file sizes are sufficient. However, testing only with specific configurations can reveal what size store file might be required. This testing involves filling each queue point with messages of the largest expected size (until the message high limit threshold is reached) when the store file is set to unlimited. The store file size reached during this test, plus the log file size is usually a safe size to use as the maximum store file size.

The store file maximum size is not intended to be the limit that prevents further messages being sent. This is because it is not only message data stored in the store file. Queue data and protocol state data is also stored in the store file. Instead, the high threshold of the queues must be used to prevent any more messages being sent.

Once a store file is full, there is a remote chance that the store might not be able to receive more data even if all the messages in the store are consumed. This can happen if the log file size is over half the size of the maximum store file size. A rule of thumb to minimize the occurrence of this is to ensure that the minimum store file size is double the log file size and that the maximum is twice the minimum plus the size of the log file.

For example, if the log file size is 100 MB then the minimum store file size can be 200 MB (double the size of the log file), and the maximum size can be 500 MB (double the size of the minimum store file plus the log file).

The default configuration for file store attributes is intended to be sufficient to be used in typical messaging workloads without further administration. To improve the performance or availability of the log file or the two store files, we can configure the file store attributes to control where these files are placed. Similarly, we can configure the attributes that control the sizes of the log file and two store files, for example to handle workloads with many active transactions, large messages, or a large volume of message data resident in the messaging engine.

This method of improving performance cannot be guaranteed on a compressing file system, for example, on an NT file system with the Compress this directory option selected. We must avoid configuring a file store to use a compressing file system for production use.

The following table shows the minimum and default values for file store attributes.

Attribute	Description	Minimum and default values
Log size	Size of the log file, in megabytes	Minimum: 10 MB Default: 100 MB
Minimum permanent store size	The minimum number of megabytes reserved by the permanent store file. The minimum store file size must always be at least as big as the log file.	Minimum: 0 Default: 200 MB
Maximum permanent store size	The maximum size in megabytes of the permanent store file. Consider making the maximum store file size double the size of the log file. The maximum size of the store file must always be larger than the log file. If the log file size is the same as the maximum store file size then the messaging engine will not start.	Minimum: 50 MB Default: 500 MB
Minimum temporary store size	The minimum number of megabytes reserved by the temporary store file. The minimum store file size must always be at least as big as the log file.	Minimum: 0 Default: 200 MB
Maximum temporary store size	The maximum size in megabytes of the temporary store file. Consider making the maximum store file size double the size of the log file. The maximum size of the store file must always be larger than the log file. If the log file size is the same as the maximum store file size then the messaging engine will not start.	Minimum: 50 MB Default: 500 MB
Unlimited permanent store size	Indicates whether the permanent store file is unlimited in size	Default: false
Unlimited temporary store size	Indicates whether the temporary store file is unlimited in size	Default: false
Log directory	Name of the directory containing the log file	Default: ${USER_INSTALL_ROOT}/filestores/com.ibm.ws.sib/<me_name>-<me_uuid>/log
Permanent store directory	Name of the directory containing the permanent store file	Default: ${USER_INSTALL_ROOT}/filestores/com.ibm.ws.sib/<me_name>-<me_uuid>/permanentStore
Temporary store directory	Name of the directory containing the temporary store file	Default: ${USER_INSTALL_ROOT}/filestores/com.ibm.ws.sib/<me_name>-<me_uuid>/temporaryStore

Modifying file store configuration

IBM File System Locking Protocol Test for WAS

Foreign bus [Settings]