Cluster node availability

If a node is unavailable when you update the cluster configuration, it contacts the primary master to get the updated configuration information when it comes back online. If the primary master is offline at the same time as the secondary master, the primary master comes back online with read-only databases until the secondary master is available.

A node can become unavailable for a number of reasons, including a shutdown request, system failure, or networking failure. If a cluster node is not available during a cluster configuration change, it contacts the primary master for up-to-date information when it restarts. There might be a slight delay where the restored node tries to use the old policy and configuration information before it retrieves the missed updates.

The relationship between the primary and secondary nodes can be temporarily affected if both nodes are shut down simultaneously, and only one is powered back up.  Until the other node is up, the databases on the newly powered up node are in read-only mode.  When you power up the other node, the databases on the primary node become writable.

We can then shut down the secondary node without affecting the write capability on the primary server.  It is only if both master nodes are offline at the same time the restored primary master becomes read-only until the secondary master is back online.

This situation can be serious if the secondary node fails and the primary node stops for any reason.  In this case, the primary node is not writable when it restarts until a secondary node is either started, or removed from the cluster. If the secondary node is removed, the primary master can operate as a single master in the cluster. You must address a failed primary or secondary master as soon as possible to avoid this situation. The above discussion about cluster node availability applies only to the configuration database and an embedded runtime database.

Parent topic: Cluster configuration rules