Product overview > Cache > Database integration



Data pre-loading and warm-up

In many scenarios that incorporate the use of a loader, you can prepare the data grid by pre-loading it with data.

When used as a complete cache, the data grid must hold all of the data and must be loaded before any clients can connect to it. When used a sparse cache, you should warm up the cache with data so that clients can have immediate access to data when they connect.

There are two approaches for pre-loading data into the grid: Using a Loader plug-in or a client loader, as described in the following sections.


Loader plug-in

The loader plug-in is associated to each map and responsible for synchronizing a single primary partition shard with the database. The preloadMap method of the loader plug-in is invoked automatically when a shard is activated. So if you have 100 partitions, 100 loader instances exist, each loading the data for its partition. When run synchronously, all clients are blocked until the preload has completed.

Figure 1. Loader plug-in

Loader plug-in

See Use a loader for more information.


Client loader

A client loader is a pattern for using one or more clients to load the grid with data. Using multiple clients to load grid data can be effective when the partition scheme is not stored in the database. You can invoke client loaders manually or automatically when the data grid starts. Client loaders can optionally use the StateManager to set the state of the data grid to pre-load mode, so that clients are not able to access the grid while it is pre-loading the data. WebSphere eXtreme Scale includes a Java™ Persistence API (JPA)-based loader that you can use to automatically load the data grid with either the OpenJPA or Hibernate JPA providers. For more information about cache providers, see JPA cache plug-in.

Figure 2. Client loader

Client loader that is using a JPA provider


Parent topic:

Database integration: Write-behind, in-line, and side caching


Related concepts

Sparse and complete cache

Side cache and in-line cache

In-line caching

Write-behind caching

Loaders

Map preloading

Database synchronization techniques

Invalidate stale cache data

Indexing