Entry-Attribute-value data model

The way data is organized and stored differs greatly from system to system:

• Databases store information in rows, typically with a fixed number of columns, each carrying a single value for that record;

• Directories maintain object-oriented entries that can contain a varying number of attributes. These in turn hold zero, one or multiple values¹;

• Lotus Domino databases contain Documents that are made up of Fields, each of which can be defined as single- or multi-valued;

• Still other systems represent their data content as nodes, objects, records, formatted byte streams or key-value sets.

In order for communication to meaningful to all participants, data formats have to be compatible or they must be translated to suit each system involved. This is called data marshalling and is often the first hurdle an integration specialist faces - and one which can quickly consume a sizeable chunk of project resources to overcome. IBM Tivoli Directory Integrator Connectors handle this for you by automatically converting source-specific types to a consistent, canonical representation. Individual data values are translated to relevant Java objects, with comparable native types being represented in the same way. For example, lines read from files, LDAP string attributes, Domino text fields and RDBMS CHAR and VARCHAR columns are all converted to java.lang.String by their respective Connectors.

These marshalled values are then accumulated into Attributes: specialized Java objects defined by Tivoli Directory Integrator. As noted above, some sources permit only a single value per column or field, while others allow several values to be stored under the same attribute name. The Tivoli Directory Integrator Attribute supports both single-valued and multi-valued implementations, and can even hold no values at all if necessary, for example when representing a nullable column in a database.

All the Attributes that make up a single unit of data (that is, record, message, document, and so forth) are collected in another Tivoli Directory Integrator object called an Entry. An Entry can hold any number of Attributes, or none at all.

The Entry-Attribute-value data model

Each data flow has a primary Entry 'bucket' called its Work Entry. Whenever a Connectors reads in data, it creates Attributes and puts these in the Work Entry. Any Connector configured for output uses Attributes already found in the Work Entry to drive changes to target systems.

This two-stage approach provides for almost unlimited flexibility in how data is transferred, transformed, filtered and enriched. It also means that we can initially build your data flow entirely with input Connectors and then interactively examine the data with the CE as it is read and manipulated before you even have to consider connections to output systems.

As we will see later on, the Tivoli Directory Integrator Entry handles complex hierarchical data just as easily as it does flat schema.

Parent topic: Simplify and solve

¹ Consider for a moment the fact that you probably have multiple email addresses, all of which can be stored in the multi-valued attribute entitled 'mail' in your company's employee directory