Plan your page sets and buffer pools

Information to help you with planning the initial number, and sizes of our page data sets, and buffer pools.

This topic contains the following sections:

• Plan the page sets
  • Page set usage
  • Number of page sets
  • Size of page sets
  • Plan for z/OS data set encryption

• Calculate the size of our page sets
  • Page set zero
  • Page set 01 - 99
  • Calculating the storage requirement for messages

• Enable dynamic page set expansion
• Defining your buffer pools

Plan the page sets

Page set usage

For short-lived messages, few pages are normally used on the page set and there is little or no I/O to the data sets except at startup, during a checkpoint, or at shutdown.

For long-lived messages, those pages containing messages are normally written out to disk. This operation is performed by the queue manager in order to reduce restart time.

Separate short-lived messages from long-lived messages by placing them on different page sets and in different buffer pools.

Number of page sets

Use several large page sets can make the role of the IBM MQ administrator easier because it means that we need fewer page sets, making the mapping of queues to page sets simpler.

Use multiple, smaller page sets has a number of advantages. For example, they take less time to back up, and I/O can be carried out in parallel during backup and restart. However, consider that this adds a significant performance cost to the role of the IBM MQ administrator, who is required to map each queue to one of a much greater number of page sets.

Define at least five page sets, as follows:

• A page set reserved for object definitions (page set zero)
• A page set for system-related messages
• A page set for performance-critical long-lived messages
• A page set for performance-critical short-lived messages
• A page set for all other messages

Defining your buffer pools explains the performance advantages of distributing your messages on page sets in this way.

Size of page sets

Define sufficient space in your page sets for the expected peak message capacity. Consider for any unexpected peak capacity, such as when a build-up of messages develops because a queue server program is not running. We can do this by allocating the page set with secondary extents or, alternatively, by enabling dynamic page set expansion. For more information, see Enable dynamic page set expansion. It is difficult to make a page set smaller, so it is often better to allocate a smaller page set, and allow it to expand when needed.

When planning page set sizes, consider all messages that might be generated, including non-application message data. For example, trigger messages, event messages and any report messages that the application has requested.

The size of the page set determines the time taken to recover a page set when restoring from a backup, because a large page set takes longer to restore.

Note: Recovery of a page set also depends on the time the queue manager takes to process the log records written since the backup was taken; this time period is determined by the backup frequency. For more information, see Plan for backup and recovery. Note: Page sets larger than 4 GB require the use of SMS extended addressability.

Planning for z/OS data set encryption

We can apply the z/OS data set encryption feature to page sets for queue managers running at IBM MQ for z/OS Version 9.1.4 or later.

We must allocate these page sets with EXTENDED attributes, and a data set key label that ensures the data is AES encrypted.

See the section, confidentiality for data at rest on IBM MQ for z/OS with data set encryption. for more information.

Calculate the size of our page sets

For queue manager object definitions (for example, queues and processes), it is simple to calculate the storage requirement because these objects are of fixed size and are permanent. For messages however, the calculation is more complex for the following reasons:

• Messages vary in size.
• Messages are transitory.
• Space occupied by messages that have been retrieved is reclaimed periodically by an asynchronous process.

Large page sets of greater than 4 GB that provide extra capacity for messages if the network stops, can be created if required. It is not possible to modify the existing page sets. Instead, new page sets with extended addressability and extended format attributes, must be created. The new page sets must be the same physical size as the old ones, and the old page sets must then be copied to the new ones. If backward migration is required, page set zero must not be changed. If page sets less than 4 GB are adequate, no action is needed.

Page set zero

Page set zero is reserved for object definitions.

For page set zero, the storage required is:

    (maximum number of local queue definitions x 1010)
       (excluding shared queues)
 +  (maximum number of model queue definitions x 746)
 +  (maximum number of alias queue definitions x 338)
 +  (maximum number of remote queue definitions x 434)
 +  (maximum number of permanent dynamic queue definitions x 1010)
 +  (maximum number of process definitions x 674)
 +  (maximum number of namelist definitions x 12320)
 +  (maximum number of message channel definitions x 2026)
 +  (maximum number of client-connection channel definitions x 5170)
 +  (maximum number of server-connection channel definitions x 2026)
 +  (maximum number of storage class definitions x 266)
 +  (maximum number of authentication information definitions x 1010)
 +  (maximum number of administrative topic definitions x 15000)
 +  (total length of topic strings defined in administrative topic definitions)

Divide this value by 4096 to determine the number of records to specify in the cluster for the page set data set.

You do not need to allow for objects that are stored in the shared repository, but we must allow for objects that are stored or copied to page set zero (objects with a disposition of GROUP or QMGR).

The total number of objects that we can create is limited by the capacity of page set zero. The number of local queues that we can define is limited to 524 287.

Page sets 01 - 99

For page sets 01 - 99, the storage required for each page set is determined by the number and size of the messages stored on that page set. (Messages on shared queues are not stored on page sets.)

Divide this value by 4096 to determine the number of records to specify in the cluster for the page set data set.

Calculating the storage requirement for messages

This section describes how messages are stored on pages. Understanding this can help you calculate how much page set storage we must define for the messages. To calculate the approximate space required for all messages on a page set we must consider maximum queue depth of all the queues that map to the page set and the average size of messages on those queues.

Note: The sizes of the structures and control information given in this section are liable to change between major releases. For details specific to your release of IBM MQ, refer to SupportPac MP16 - WebSphere MQ for z/OS Capacity planning & tuning and IBM MQ Family - Performance Reports

We must allow for the possibility that message "gets" might be delayed for reasons outside the control of IBM MQ (for example, because of a problem with your communications protocol). In this case, the "put" rate of messages might far exceed the "get" rate. This can lead to a large increase in the number of messages stored in the page sets and a consequent increase in the storage size demanded.

Each page in the page set is 4096 bytes long. Allowing for fixed header information, each page has 4057 bytes of space available for storing messages.

When calculating the space required for each message, the first thing we must consider is whether the message fits on one page (a short message) or whether it needs to be split over two or more pages (a long message). When messages are split in this way, we must allow for additional control information in your space calculations.

For the purposes of space calculation, a message can be represented as the following:

The message header section contains the message descriptor and other control information, the size of which varies depending on the size of the message. The message data section contains all the actual message data, and any other headers (for example, the transmission header or the IMS bridge header).

A minimum of two pages are required for page set control information which, is typically less than 1% of the total space required for messages.

Short messages

A short message is defined as a message that fits on one page.

From IBM WebSphere MQ Version 7.0.1, small messages are stored one on each page.

Long messages

If the size of the message data is greater than 3596 bytes, but not greater than 4 MB, the message is classed as a long message. When presented with a long message, IBM MQ stores the message on a series of pages, and stores control information that points to these pages in the same way that it would store a short message. This is shown in Figure 1:



Very long messages

Very long messages are messages with a size greater than 4 MB. These are stored so that each 4 MB uses 1037 pages. Any remainder is stored in the same way as a long message, as described above.

• Enable dynamic page set expansion
  Page sets can be extended dynamically while the queue manager is running. A page set can have 119 extents, and can be spread over multiple disk volumes.
• Defining your buffer pools
  Use this topic to help plan the number of buffer pools we should define, and their settings.

Parent topic: Plan for the queue manager