EasyETL Guide

'ETL' stands for Extract, Transform and Load, and boils down to getting data from one place, changing it as needed and then putting it someplace else. 'EasyETL' is a feature of Tivoli Directory Integrator (TDI) that lets we do this quickly and interactively in just a few keystrokes.

Common ETL examples are:

• Exporting database records, Notes Documents, directory entries or even incoming mail or MQ messages to a file;
• Loading data from file into a system or data store;
• Migrating data directly from one system to another, or between versions of the same system in the case of software/schema updates.

The first step in creating a new EasyETL task is to choose your Input Source and select the attributes that we want to transfer. Already at this point, TDI lets we run your EasyETL job and collect the data read into our copy buffer for pasting. If the data needs to be transformed or computed then EasyETL lets you add transformation, re-run the ETL job and copy/paste the transformed data. And we can also choose an Output Target and have the EasyETL job write the data directly there.

Once your EasyETL solution is working as desired, TDI can generate the command line assets (batch-files or scripts) for launching and scheduling the integration task. Finally, EasyETL leverages the change detection features of TDI to quickly turn your ETL job into a data synchronization task.

The good news for TDI users is that each EasyETL solution is a TDI Project with a single AssemblyLine, and it can be opened in the full-featured development environment. However, once you change it there it's no longer available as EasyETL.

Using EasyETL

Welcome screen

As shown in the screenshot above, the topmost link here opens up the EasyETL² workbench. Click on this link now to open the TDI EasyETL workbench.

Figure 2. EasyETL Workbench

The EasyETL workbench shows you three things:

• The Project Navigator that lists your ETL jobs. We can right-click on any Project to do things like running it or creating command line assets to launch it;

• A Simple AssemblyLine³ editor for each open project;

• Various Views as tabs along the bottom part of the screen. By default you get three Views:

  • the Console output from the test TDI Server;

  • the Server status view where we can monitor both the Server and any running EasyETL Projects;

  • and the Data Collector where the resulting data for each cycle is displayed in a tabular list.

We will find more information on each section as you work through this document.

• Creating a Project
  

• Detecting Changes
  

¹ We can return to the Welcome page at any time by selecting Help > Welcome from the main menu.

² EasyETL is a TDI perspective, and you can switch between perspectives using the menu selection: Windows > Open Perspective > Other... If we have made changes to a perspective and would like to reset it back to default, simply select Windows > Reset Perspective.

³ An ‘AssemblyLine' is the implementation of a data flow in TDI, so when we create or open an EasyETL Project then the underlying AssemblyLine is presented in the editor.

Note also that the term ‘AssemblyLine' is abbreviated as ‘AL' in this and other TDI literature.