EasyETL Guide

‘ETL' stands for Extract, Transform and Load, and boils down to getting data from one place, changing it as needed and then putting it someplace else. ‘EasyETL' is a feature of Security Directory Integrator that lets you do this quickly and interactively in just a few keystrokes. Common ETL examples are:

• Exporting database records, Notes® Documents, directory entries or even incoming mail or MQ messages to a file;
• Loading data from file into a system or data store;
• Migrating data directly from one system to another, or between versions of the same system in the case of software/schema updates.

Security Directory Integrator EasyETL lets you solve these and other scenarios in a few intuitive steps, resulting in solutions suitable for both one-off data movement needs, as well as for mission critical data flows in your infrastructure.

The first step in creating a new EasyETL task is to choose your Input Source and select the attributes that you want to transfer. Already at this point, SDI lets you run your EasyETL job and collect the data read into your copy buffer for pasting. If the data needs to be transformed or computed then EasyETL lets you add transformation, re-run the ETL job and copy/paste the transformed data. And we can also choose an Output Target and have your EasyETL job write the data directly there.

Once your EasyETL solution is working as desired, SDI can generate the command line assets (batch-files or scripts) for launching and scheduling the integration task. Finally, EasyETL leverages the change detection features of SDI to quickly turn your ETL job into a data synchronization task.

Note: The good news for SDI users is that each EasyETL solution is an SDI Project with a single AssemblyLine, and it can be opened in the full-featured development environment. However, once you change it there it's no longer available as EasyETL.

Using EasyETL

Launch the SDI Config Editor and select the workspace to use. When SDI starts the first time it opens in the Welcome page¹

Welcome screen

As shown in the screenshot above, the topmost link here opens up the EasyETL² workbench. Click this link now to open the Security Directory Integrator EasyETL workbench.

Figure 2. EasyETL Workbench

The EasyETL workbench shows you three things:

• The Project Navigator that lists your ETL jobs. We can right-click on any Project to do things like running it or creating command line assets to launch it;
• A Simple AssemblyLine³ editor for each open project;
• Various Views as tabs along the bottom part of the screen. By default you get three Views:
  • the Console output from the test SDI Server;
  • the Server status view where we can monitor both the Server and any running EasyETL Projects;
  • and the Data Collector where the resulting data for each cycle is displayed in a tabular list.

You will find more information on each section as you work through this document.

• Create a Project
  
• Detecting Changes
  

¹ We can return to the Welcome page at any time by selecting Help > Welcome from the main menu.² EasyETL is a Security Directory Integrator perspective, and we can switch between perspectives using the menu selection: Windows > Open Perspective > Other... If you have made changes to a perspective and would like to reset it back to default, simply select Windows > Reset Perspective.³ An ‘AssemblyLine' is the implementation of a data flow in SDI, so when you create or open an EasyETL Project then the underlying AssemblyLine is presented in the editor.

Note also that the term ‘AssemblyLine' is abbreviated as ‘AL' in this and other SDI literature.