Administration guide > Configure the deployment environment > Configuring deployment policies > Controlling shard placement with zones


Use zones for effective shard placement

Zones represent logical groupings of physical servers...


Typical use case

The classic use case for zones is when you have two or more geographically dispersed data centers and you want to ensure there is a full set of asynchronous replica shards for the data grid on each.

With this strategy, you can recover from the failure of the local data center transparently, with no loss of data. Data centers themselves have high speed, low latency networks. However, communication between one data center and another has higher latency.

Synchronous replicas are used in each data center where the low latency minimizes the impact of replication on response times. Using asynchronous replication reduces impact on response time. The geographic distance provides availability in case of local data-center failure.

Three configuration settings in WebSphere eXtreme Scale (WXS) control shard placement:

The following sections explain the different options, presented loosely from least to most complicated.


Disable development mode

To activate the first WXS shard placement policy, in the deployment XML file, set...

developmentMode="false"


Policy 1: Shards for the same partition are placed in separate physical servers

Consider a simple example of a data grid with one replica shard. With this policy, the primary and replica shards for each partition are on different physical servers. If a single physical server fails, no data is lost. The primary or replica shard for each partition are on different physical servers that did not fail, or both are on some other physical server that did not fail.

The high availability and simplicity of this policy make it the most efficient setting for all production environments. In many cases, applying this policy is the only step required for effectively controlling shard placement in the environment.

In applying this policy, a physical server is defined by an IP address. Shards are placed in container servers. Container servers have an IP address, for example...

startOgServer.sh -listenerHost

Multiple container servers can have the same IP address.

Since a physical server has multiple IP addresses, consider the next step for more control of the environment.


Define zones to group container servers

Container servers are assigned to zones using...

startOgServer.sh -zone

In a WAS environment, zones are defined through node groups with a specific name format...

ReplicationZone<Zone>


Policy 2: Shards for the same partition are placed in separate zones

Consider extending the example of a data grid with one replica shard by deploying it across two data centers. Define each data center as an independent zone. Use a zone name of DC1 for the container servers in the first data center, and DC2 for the container servers in the second data center. With this policy, the primary and replica shards for each partition would be in different data centers. If a data center fails, no data is lost. For each partition, either its primary or replica shard is in the other data center.

With this policy, you can control shard placement by defining zones. You choose the physical or logical boundary or grouping of interest. Then, choose a unique zone name for each group, and start the container servers in each of the zones with the name of the appropriate zone. Thus WXS places shards so that shards for the same partition are placed in separate zones.


Specify zone rules

Zone rules are specified in the zoneMetadata element of the WXS deployment policy descriptor XML. A zone rule defines a set of zones in which shards are placed. A shardMapping element assigns a shard to a zone rule. The shard attribute of the shardMapping element specifies the shard type:

P Primary shard
S Synchronous replica shards
A Asynchronous replica shards.

If more than one synchronous or asynchronous replica exist, then provide shardMapping elements of the appropriate shard type. The exclusivePlacement attribute of the zoneRule element determines the placement of shards in the same partition in zones. The exclusivePlacement attribute values are:

true Shards cannot be placed in the same zone as another shard from the same partition).

Remember: For the "true" case, have at least as many zones in the rule as you have shards using it. Doing so ensures that each shard can be in its own zone.

false Shards from the same partition can be placed in the same zone.

The default setting is true.

For more information, see Zone definition examples for deployment file.


Extended use cases

The following are various use cases for shard placement strategies:


Rolling upgrades

Consider a scenario in which to apply rolling upgrades to the physical servers, including maintenance that requires restarting the deployment. In this example, assume that you have a data grid spread across 20 physical servers, defined with one synchronous replica. You want to shut down 10 of the physical servers at a time for the maintenance.

When you shut down groups of 10 physical servers, no partition has both its primary and replica shards on the servers you are shutting down. Otherwise, you lose the data from that partition.

The easiest solution is to define a third zone. Instead of two zones of 10 physical servers each, use three zones, two with seven physical servers, and one with six. Spreading the data across more zones allows for better failover for availability.

Rather than defining another zone, the other approach is to add a replica.


Upgrade WXS

When you are upgrading WXS software in a rolling manner with data grids that contain live data, the catalog service software version must be greater than or equal to the container server software versions.

Upgrade all the catalog servers first with a rolling strategy.


Changing data model

A related issue is how to change the data model or schema of objects that are stored in the data grid without causing downtime. It would be disruptive to change the data model by stopping the data grid and restarting with the updated data model classes in the container server classpath, and reloading the data grid. An alternative would be to start a new data grid with the new schema, copy the data from the old data grid to the new data grid, then shut down the old data grid. Each of these processes are disruptive and result in downtime.

To change the data model without downtime, store the object in one of these formats:

Write serializers to go from plain old Java object (POJO) to one of these formats on the client side. Schema changes become easier.


Virtualization and cloud computing

By default, WXS insures that two shards for the same partition are never placed on the same IP address. When you are deploying on virtual images, such as VMware, many server instances, each with a unique IP address, can be run on the same physical server.

To ensure that replicas can only be placed on separate physical servers, you can use zones to solve the problem. Group the physical servers into zones, and use zone placement rules to keep primary and replica shards in separate zones.


Zones for wide-area networks

You might want to deploy a single WXS data grid over multiple buildings or data centers with slower network connections. Slower network connections lead to lower bandwidth and higher latency connections. The possibility of network partitions also increases in this mode due to network congestion and other factors.

To deal with these risks, the WXS catalog service organizes container servers into core groups that exchange heartbeats to detect container server failure. These core groups do not span zones. A leader within each core group pushes membership information to the catalog service. The catalog service verifies any reported failures before responding to membership information by heartbeating the container server in question.

If the catalog service sees a false failure detection, the catalog service takes no action. The core group partition heals quickly. The catalog service also heartbeats core group leaders periodically at a slow rate to handle the case of core group isolation.


Parent topic:

Controlling shard placement with zones