Program guide > Programming with system APIs and plug-ins > Persistent store plug-ins

Use a loader with entity maps and tuples

The entity manager converts all entity objects into tuple objects before they are stored in an WebSphere eXtreme Scale map. Every entity has a key tuple and a value tuple. This key-value pair is stored in the associated eXtreme Scale map for the entity. When you are using an eXtreme Scale map with a loader, the loader must interact with the tuple objects.

eXtreme Scale includes loader plug-ins that simplify integration with relational databases. The Java™ Persistence API (JPA) Loaders use a Java Persistence API to interact with the database and create the entity objects. The JPA loaders are compatible with eXtreme Scale entities.


A tuple contains information about the attributes and associations of an entity. Primitive values are stored using their primitive wrappers. Other supported object types are stored in their native format. Associations to other entities are stored as a collection of key tuple objects that represent the keys of the target entities.

Each attribute or association is stored using a zero-based index. You can retrieve the index of each attribute using the getAttributePosition or getAssociationPosition methods. After the position is retrieved, it remains unchanged for the duration of the eXtreme Scale life cycle. The position can change when the eXtreme Scale is restarted. The setAttribute, setAssociation and setAssociations methods are used to update the elements in the tuple.

Attention: When you are creating or updating tuple objects, update every primitive field with a non-null value. Primitive values such as int should not be null. If you do not change the value to a default, poor performance issues can result, also affecting fields marked with the @Version annotation or version attribute in the entity descriptor XML file.

The following example further explains how to process tuples. For more information about defining entities for this example, see Entity manager tutorial: Order entity schema. WebSphere eXtreme Scale is configured for using loaders with each of the entities. Additionally, only the Order entity is taken, and this specific entity has a many-to-one relationship with the Customer entity. The attribute name is customer, and it has a one-to-many relationship with the OrderLine entity.

Use the Projector to build Tuple objects automatically from entities. Using the Projector can simplify loaders when you are using an object-relational mapping utility such as Hibernate or JPA.

public class Order
    @Id  String orderNumber;
    java.util.Date date;
    @OneToOne(cascade=CascadeType.PERSIST) Customer customer;
    @OneToMany(cascade=CascadeType.ALL, mappedBy="order") @OrderBy("lineNumber") List<OrderLine> lines;

public class Customer {
    @Id String id;
    String firstName;
    String surname;
    String address;
    String phoneNumber;

public class OrderLine
    @Id @ManyToOne(cascade=CascadeType.PERSIST) Order order;
    @Id int lineNumber;
    @OneToOne(cascade=CascadeType.PERSIST) Item item;
    int quantity;
    double price;

A OrderLoader class that implements the Loader interface is shown in the following code. The following example assumes that an associated TransactionCallback plug-in is defined.

public class OrderLoader implements {

    private EntityMetadata entityMetaData;
    public void batchUpdate(TxID txid, LogSequence sequence) 
            throws LoaderException, OptimisticCollisionException {
        public List get(TxID txid, List keyList, boolean forUpdate)    
            throws LoaderException {
        public void preloadMap(Session session, BackingMap backingMap) 
            throws LoaderException {


The instance variable entityMetaData is initialized during the preLoadMap method call from the eXtreme Scale. The entityMetaData variable is not null if the Map is configured to use entities. Otherwise, the value is null.

batchUpdate method

The batchUpdate method provides the ability to know what action the application intended to perform. Based on an insert, update or a delete operation, a connection can be opened to the database and the work performed. Because the key and values are of type Tuple, they must be transformed so the values make sense on the SQL statement.

The ORDER table was created with the following Data Definition Language (DDL) definition, as shown in the following code:


The following code demonstrates how to convert a Tuple to an Object:

public void batchUpdate(TxID txid, LogSequence sequence)
            throws LoaderException, OptimisticCollisionException {
        Iterator iter = sequence.getPendingChanges();
        while (iter.hasNext()) {
            LogElement logElement = (LogElement);
            Object key = logElement.getKey();
            Object value = logElement.getCurrentValue();

            switch (logElement.getType().getCode()) {
            case LogElement.CODE_INSERT:

1)              if (entityMetaData!=null) {

// The order has just one key orderNumber
2)                  String ORDERNUMBER=(String) getKeyAttribute("orderNumber", (Tuple) key);
// Get the value of date
3)                  java.util.Date unFormattedDate = (java.util.Date) getValueAttribute("date",(Tuple)value);
// The values are 2 associations. Lets process customer because
// the our table contains as primary key
4)                  Object[] keys= getForeignKeyForValueAssociation("customer","id",(Tuple) value);
                    //Order to Customer is M to 1. There can only be 1 key
5)                  String CUSTOMER_ID=(String)keys[0];
// parse variable unFormattedDate and format it for the database as formattedDate
6)                   String formattedDate = "2007-05-08-"; // formatted for DB2
// Finally, the following SQL statement to insert the record
            case LogElement.CODE_UPDATE:
            case LogElement.CODE_DELETE:

// returns the value to attribute as stored in the key Tuple
 private Object getKeyAttribute(String attr, Tuple key) {
        //get key metadata
        TupleMetadata keyMD = entityMetaData.getKeyMetadata();
        //get position of the attribute
        int keyAt = keyMD.getAttributePosition(attr);
        if (keyAt > -1) {
            return key.getAttribute(keyAt);
        } else { // attribute undefined
            throw new IllegalArgumentException("Invalid position index for  "+attr);

// returns the value to attribute as stored in the value Tuple
    private Object getValueAttribute(String attr, Tuple value) {
        //similar to above, except we work with value metadata instead
        TupleMetadata valueMD = entityMetaData.getValueMetadata();

        int keyAt = valueMD.getAttributePosition(attr);
        if (keyAt > -1) {
            return value.getAttribute(keyAt);
        } else {
            throw new IllegalArgumentException("Invalid position index for  "+attr);
// returns an array of keys that refer to association.
    private Object[] getForeignKeyForValueAssociation(String attr, String fk_attr, Tuple value) {
        TupleMetadata valueMD = entityMetaData.getValueMetadata();
        Object[] ro;

        int customerAssociation = valueMD.getAssociationPosition(attr);
        TupleAssociation tupleAssociation = valueMD.getAssociation(customerAssociation);

        EntityMetadata targetEntityMetaData = tupleAssociation.getTargetEntityMetadata();

        Tuple[] customerKeyTuple = ((Tuple) value).getAssociations(customerAssociation);

        int numberOfKeys = customerKeyTuple.length;
        ro = new Object[numberOfKeys];

        TupleMetadata keyMD = targetEntityMetaData.getKeyMetadata();
        int keyAt = keyMD.getAttributePosition(fk_attr);
        if (keyAt
< 0) {
            throw new IllegalArgumentException("Invalid position index for  " + attr);
        for (int i = 0; i
< numberOfKeys; ++i) {
            ro[i] = customerKeyTuple[i].getAttribute(keyAt);

        return ro;


  1. Ensure that entityMetaData is not null, which implies that the key and value cache entries are of type Tuple. From the entityMetaData, Key TupleMetaData is retrieved, which really reflects only the key part of Order metadata.

  2. Process the KeyTuple and get the value of Key Attribute orderNumber

  3. Process the ValueTuple and get the value of attribute date

  4. Process the ValueTuple and get the value of Keys from association customer

  5. Extract CUSTOMER_ID. Based on relationship, an Order can only have one customer, we will only have one key. Hence the size of keys is 1. For simplicity, we skipped parsing of date to correct format.

  6. Because this is an insert operation, the SQL statement is passed onto the data source connection to complete the insert operation.

Transaction demarcation and access to database is covered in Write a loader.

get method

If the key is not found in the cache, call the get method in the Loader plug-in to find the key.

The key is a Tuple. The first step is to convert the Tuple to primitive values that can be passed onto the SELECT SQL statement. After all the attributes are retrieved from the database, convert into Tuples. The following code demonstrates the Order class.

public List get(TxID txid, List keyList, boolean forUpdate)    throws LoaderException {
        System.out.println("OrderLoader: Get called");
        ArrayList returnList = new ArrayList();

1)        if (entityMetaData != null) {
            int index=0;
            for (Iterator iter = keyList.iterator(); iter.hasNext();) {
2)                Tuple orderKeyTuple=(Tuple);

                // The order has just one key orderNumber
3)                String ORDERNUMBERKEY = (String) getKeyAttribute("orderNumber",orderKeyTuple);
                //We need to run a query to get values of

5)                //1) Foreign key: CUSTOMER_ID
6)                //2) date
                // Assuming those two are returned as
7)                              String  CUSTOMER_ID = "C001"; // Assuming Retrieved and initialized
8)                java.util.Date retrievedDate = new java.util.Date(); 
                                // Assuming this date reflects the one in database

                // We now need to convert this data into a tuple before returning

                //create a value tuple
9)                TupleMetadata valueMD = entityMetaData.getValueMetadata();
                Tuple valueTuple=valueMD.createTuple();

                //add retrievedDate object to Tuple
                int datePosition = valueMD.getAttributePosition("date");
10)                valueTuple.setAttribute(datePosition, retrievedDate);

                //Next need to add the Association
11)                int customerPosition=valueMD.getAssociationPosition("customer");
                TupleAssociation customerTupleAssociation = 
                EntityMetadata customerEMD = customerTupleAssociation.getTargetEntityMetadata();
                TupleMetadata customerTupleMDForKEY=customerEMD.getKeyMetadata();
12)                int customerKeyAt=customerTupleMDForKEY.getAttributePosition("id");

                Tuple customerKeyTuple=customerTupleMDForKEY.createTuple();
                customerKeyTuple.setAttribute(customerKeyAt, CUSTOMER_ID);
13)                valueTuple.addAssociationKeys(customerPosition, new Tuple[] {customerKeyTuple});

14)                int linesPosition = valueMD.getAssociationPosition("lines");
                TupleAssociation linesTupleAssociation = valueMD.getAssociation(linesPosition);
                EntityMetadata orderLineEMD = linesTupleAssociation.getTargetEntityMetadata();
                TupleMetadata orderLineTupleMDForKEY = orderLineEMD.getKeyMetadata();
                int lineNumberAt = orderLineTupleMDForKEY.getAttributePosition("lineNumber");
                int orderAt = orderLineTupleMDForKEY.getAssociationPosition("order");

                if (lineNumberAt
< 0 || orderAt
< 0) {
                    throw new IllegalArgumentException(
                                        "Invalid position index for lineNumber or order "+ 
                                        lineNumberAt + " " + orderAt);
                // Assuming two rows of line number are returned with values 1 and 2

                Tuple orderLineKeyTuple1 = orderLineTupleMDForKEY.createTuple();
                orderLineKeyTuple1.setAttribute(lineNumberAt, new Integer(1));// set Key
                orderLineKeyTuple1.addAssociationKey(orderAt, orderKeyTuple);

                Tuple orderLineKeyTuple2 = orderLineTupleMDForKEY.createTuple();
                orderLineKeyTuple2.setAttribute(lineNumberAt, new Integer(2));// Init Key
                orderLineKeyTuple2.addAssociationKey(orderAt, orderKeyTuple);

16)                valueTuple.addAssociationKeys(linesPosition, new Tuple[] 
                                    {orderLineKeyTuple1, orderLineKeyTuple2 });

                returnList.add(index, valueTuple);


        }else {
            // does not support tuples
        return returnList;

  1. The get method is called when the ObjectGrid cache could not find the key and requests the loader to fetch. Test for entityMetaData value and proceed if not null.

  2. The keyList contains Tuples.

  3. Retrieve value of attribute orderNumber.

  4. Run query to retrieve date (value) and customer ID (foreign key).

  5. CUSTOMER_ID is a foreign key that must be set in the association tuple.

  6. The date is a value and should already be set.

  7. Since this example does not perform JDBC calls, CUSTOMER_ID is assumed.

  8. Since this example does not perform JDBC calls, date is assumed.

  9. Create value Tuple.

  10. Set the value of date into the Tuple, based on its position.

  11. Order has two associations. Start with the attribute customer which refers to the customer entity. You must have the value of ID to set in the Tuple.

  12. Find the position of ID on the customer entity.

  13. Set the values of the association keys only.

  14. Also, lines is an association that must be set up as a group of association keys, in the same way as you did for customer association.

  15. Since set up keys for the lineNumber associated with this order, run the SQL to retrieve lineNumber values.

  16. Set up the association keys in the valueTuple. This completes the creation of the Tuple that is returned to the BackingMap.

This topic offers the steps create tuples, and a description of the Order entity only. Complete similar steps for the other entities, and the entire process that is tied together with the TransactionCallback plug-in. See Plug-ins for managing transaction life cycle events for details.

Parent topic:

Plug-ins for communicating with persistent stores

Related concepts

Write a loader

JPAEntityLoader plug-in

Write a loader with a replica preload controller

Related reference

JPA loader programming considerations