For up-to-date product documentation, see the IBM MobileFirst Foundation Developer Center.

Capacity considerations

Capacity is the single-most common question. How much RAM do you need? How much disk space? How many nodes? The answer is always: it depends.

IBM MobileFirst™ Analytics gives you the opportunity to collect many heterogeneous event types, including raw client SDK debug logs, server-reported network events, custom data, and much more. It is a big data system with big data system requirements.

The type and amount of data that you choose to collect, and how long you choose to keep it, has a dramatic impact on your storage requirements and overall performance. As an example, consider the following questions.

• Are raw debug client logs useful after a month?
• Are you using the Alerts feature in MobileFirst Analytics? If so, are you querying on events that occurred in the last few minutes or over a longer range?
• Are you using custom charts? If so, are you creating these charts for built-in data or custom instrumented key/value pairs? How long do you keep the data?

The built-in charts on the MobileFirst Analytics Console are rendered by querying data that the MobileFirst Analytics Server already summarized and optimized specifically for the fastest possible console user experience. Because it is pre-summarized and optimized for the built-in charts, it is not suitable for use in alerts or custom charts where the console user defines the queries.

When we query raw documents, apply filters, perform aggregations, and ask the underlying query engine to calculate averages and percentages, the query performance necessarily suffers. It is this use case that requires careful capacity considerations. After your query performance suffers, it is time to decide whether you really must keep old data for real-time console visibility or purge it from the MobileFirst Analytics Server. Is real-time console visibility truly useful for data from four months ago?

Indicies, Shards, and Nodes

The underlying data store is Elasticsearch. You must know a bit about indices, shards and nodes, and how the configuration affects performance. Roughly, we can think of an index as a logical unit of data. An index is mapped one-to-many to shards where the configuration key is shards. For more information, see Configuration guide. The MobileFirst Analytics Server creates a separate index per document type. If your configuration does not discard any document types, you have a number of indices that are created that is equivalent to the number of document types that are offered by the MobileFirst Analytics Server.

If you configure the shards to 1, each index only ever has one primary shard to which data is written. If you set shards to 10, each index can balance to 10 shards. However, more shards have a performance cost when you have only one node. That one node is now balancing each index to 10 shards on the same physical disk. Only set shards to 10 if you plan to immediately (or nearly immediately) scale up to 10 physical nodes in the cluster.

The same principle applies to replicas. Only set replicas to something greater than 0 if you intend to immediately (or nearly immediately) scale up to the number of nodes to match the math.

For example, if you set shards to 4 and replicas to 2, we can scale to 8 nodes, which is 4 * 2.

Parent topic: MobileFirst Analytics Server installation guide