Extract, Transform and Load
Extract, Transform and Load
ETL integration scenarios are implemented by orchestrating the steps of ETL using the Orchestration capabilities of frends so that the initial activity in the sequence is the extraction or parsing of the incoming data, adding in logic or activites to transform the data and finally loading or sending the data to the target system or interface:
Green = Extract Phase
Blue = Transform Phase
Purple = Load Phase
Orange = Error Handling
Supported Extract Formats
Frends implements the extract functionality utilizing the standard Frends Tasks and has out-of-the-box support for:
- XML
- JSON
- CSV
- Fixed Width
- Flat File
- Custom text format
- EDIFACT
Supported Transform Functionality
To transfrom the parsed or extracted data you can utilize the native frends low-code approach to programmatically map and apply the transformations or choose from other well known and standardized approaches such as XSLT or JSONMap.
Frends Low-Code Approach
Using this approach the source data or Extract phase always generates JSON which can then be accessed using the low-code approach:
Example of the low-code transformation in the above process.
Usually utilizing the low-code approach requires you to loop through the dataset using loops in the process orchestration. The main benefits of utilizing the low-code approach are:
- Combine transform and load phases into a single step
- Complete visibility into the processing and every step taken
- Complete control over data manipulation using C#
- Blazing fast executions through compiled code
XSLT or JSONMap
Alternative approach to utilizing the low-code approach is to utilize Standard frends tasks for XSLT or JSONMap transformations where the input for the data is the extracted data and the output of the task is the transformed data ready to be loaded: