Designing publish/subscribe clusters

There are two basic publish/subscribe cluster topologies: direct routing and topic host routing. Each has different benefits. When you design your publish/subscribe cluster, choose the topology that best fits your expected network requirements.

For an overview of the two publish/subscribe cluster topologies, see Publish/subscribe clusters. To help you evaluate your network requirements, see Plan for distributed publish/subscribe network and Publish/subscribe clustering: Best practices.

In general, both cluster topologies provide the following benefits:

• Simple configuration on top of a point-to-point cluster topology.
• Automatic handling of queue managers joining and leaving the cluster.
• Ease of scaling for additional subscriptions and publishers, by adding extra queue managers and distributing the additional subscriptions and publishers across them.

However, the two topologies have different benefits as the requirements become more specific.

Direct routed publish/subscribe clusters

With direct routing, any queue manager in the cluster sends publications from connected applications direct to any other queue manager in the cluster with a matching subscription.

A direct routed publish/subscribe cluster provides the following benefits:

• Messages destined for a subscription on a specific queue manager in the same cluster are transported directly to that queue manager and do not need to pass through an intermediate queue manager. This can improve performance in comparison with a topic host routed topology, or a hierarchical topology.
• Because all queue managers are directly connected to each other, there is no single point of failure in the routing infrastructure of this topology. If one queue manager is not available, subscriptions on other queue managers in the cluster are still able to receive messages from publishers on available queue managers.
• It is very simple to configure, especially on an existing cluster.

Things to consider when using a direct routed publish/subscribe cluster:

• All queue managers in the cluster become aware of all other queue managers in the cluster.
• Queue managers in a cluster that host one or more subscriptions to a clustered topic, automatically create cluster sender channels to all other queue managers in the cluster, even when those queue managers are not publishing messages on any clustered topics.
• The first subscription on a queue manager to a topic string under a clustered topic results in a message being sent to every other queue manager in the cluster. Similarly, the last subscription on a topic string to be deleted also results in a message. The more individual topic strings being used under a clustered topic, and the higher the rate of change of subscriptions, the more inter-queue manager communication occurs.
• Every queue manager in the cluster maintains the knowledge of subscribed topic strings that it is informed of, even when the queue manager is neither publishing nor subscribing to those topics.

For the above reasons, all queue managers in a cluster with a direct routed topic defined will incur an additional overhead. The more queue managers there are in the cluster, the greater the overhead. Likewise the more topic strings subscribed to, and the greater their rate of change, the greater the overhead. This can result in too much load on queue managers running on small systems in a large or dynamic direct routed publish/subscribe cluster. See Direct routed publish/subscribe performance for further information.

When you know that a cluster cannot accommodate the overheads of direct routed clustered publish/subscribe, we can instead use topic host routed publish/subscribe. Alternatively, in extreme situations, we can completely disable clustered publish/subscribe functionality by setting the queue manager attribute PSCLUS to DISABLED on every queue manager in the cluster. See Inhibiting clustered publish/subscribe. This prevents any clustered topic from being created, and therefore ensures that your network does not incur any overheads associated with clustered publish/subscribe.

Topic host routed publish/subscribe clusters

With topic host routing, the queue managers where clustered topics are administratively defined become routers for publications. Publications from non-hosting queue managers in the cluster are routed through the hosting queue manager to any queue manager in the cluster with a matching subscription.

A topic host routed publish/subscribe cluster provides the following extra benefits over a direct routed publish/subscribe cluster:

• Only queue managers on which topic host routed topics are defined are made aware of all other queue managers in the cluster.
• Only the topic host queue managers need to be able to connect to all other queue managers in the cluster, and will typically only connect to those where subscriptions exist. Therefore there are significantly fewer channels running between queue managers.
• Cluster queue managers that host one or more subscriptions to a clustered topic automatically create cluster sender channels only to queue managers that host a cluster topic that maps to the topic string of the subscription.
• The first subscription on a queue manager to a topic string under a clustered topic results in a message being sent to a queue manager in the cluster that hosts the clustered topic. Similarly, the last subscription on a topic string to be deleted also results in a message. The more individual topic strings being used under a clustered topic, and the higher the rate of change of subscriptions, the more inter-queue manager communication occurs, but only between subscription hosts and topic hosts.
• More control over the physical configuration. With direct routing, all queue managers have to participate in the publish/subscribe cluster, increasing their overheads. With topic host routing, only the topic host queue managers are aware of other queue managers and their subscriptions. You explicitly choose the topic host queue managers, therefore we can ensure that those queue managers are running on adequate equipment, and we can use less powerful systems for the other queue managers.

Things to consider when using a topic host routed publish/subscribe cluster:

• An extra hop between a publishing queue manager and a subscribing queue manager is introduced when the publisher or the subscriber is not located on a topic hosting queue manager. The latency caused by the extra hop can mean that topic host routing is less efficient that direct routing.
• On large clusters, topic host routing eases the significant performance and scaling issues that we can get with direct routing.
• We might choose to define all your topics on a single queue manager, or on a very small number of queue managers. If you do this, make sure the topic host queue managers are hosted on powerful systems with good connectivity.
• We can define the same topic on more than one queue manager. This improves the availability of the topic, and also improves scalability because IBM MQ workload balances publications for a topic across all hosts for that topic. Note, however, that defining the same topic on more than one queue manager loses message order for that topic.
• By hosting different topics on different queue managers, we can improve scalability without losing message order.

• Direct routing in publish/subscribe clusters
  Publications from any publishing queue manager are routed direct to any other queue manager in the cluster with a matching subscription.
• Topic host routing in publish/subscribe clusters
  Publications from non-hosting queue managers in the cluster are routed through the hosting queue manager to any queue manager in the cluster with a matching subscription.
• Publish/subscribe clustering: Best practices
  Using clustered topics makes extending the publish/subscribe domain between queue managers simple, but can lead to problems if the mechanics and implications are not fully understood. There are two models for information sharing and publication routing. Implement the model that best meets your individual business needs, and performs best on your chosen cluster.
• Defining cluster topics
  Cluster topics are administrative topics with the cluster attribute defined. Information about cluster topics is pushed to all members of a cluster, and combined with local topics to create portions of a topic space that spans multiple queue managers. This enables messages published on a topic on one queue manager to be delivered to subscriptions of other queue managers in the cluster.
• Inhibiting clustered publish/subscribe
  Introducing the first direct routed clustered topic into a cluster forces every queue manager in the cluster to become aware of every other queue manager, and potentially causes them to create channels to each other. If this is not desirable, we should instead configure topic host routed publish/subscribe. If the existence of a direct routed clustered topic might jeopardize the stability of the cluster, due to scaling concerns of each queue manager, we can completely disable clustered publish/subscribe functionality by setting PSCLUS to DISABLED on every queue manager in the cluster.
• Publish/subscribe and multiple clusters
  A single queue manager can be a member of more than one cluster. This arrangement is sometimes known as overlapping clusters. Through such an overlap, clustered queues can be made accessible from multiple clusters, and point-to-point message traffic can be routed from queue managers in one cluster to queue managers in another cluster. Clustered topics in publish/subscribe clusters do not provide the same capability. Therefore, their behavior must be clearly understood when using multiple clusters.
• Design considerations for retained publications in publish/subscribe clusters
  There are a few restrictions to consider when designing a publish/subscribe cluster to work with retained publications.
• REFRESH CLUSTER considerations for publish/subscribe clusters
  Issuing the REFRESH CLUSTER command results in the queue manager temporarily discarding locally held information about a cluster, including any cluster topics and their associated proxy subscriptions.

Parent topic: Plan for distributed publish/subscribe network

Related information

• Publish/subscribe cluster scenario
• Configure a publish/subscribe cluster
• Tuning distributed publish/subscribe networks
• Distributed publish/subscribe troubleshooting