Breaking news: MOSAIC Grand release

ETL

MOSAIC Data Integration

Extract, Enrich, Transform, Load

Processes

Extract Enrichment Transformation Loading of data

The MOSAIC platform provides an opportunity to organize an effective and clear data integration process, including
• data extract from source systems, with control of the load on the data sources
enrichment of data from user sources
• data transformation
loading data into target databases

We propose the E2TL approach, which is the most optimal in the innovative approach of building virtual cubes, consisting of data from multiple sources, and the concept of “process data where it is”, implemented in MOSAIC.

Missions and solutions

Allows you to set up a flexible process for extracting, transforming and loading data, covering all typical and individual scenarios for accessing data:
Uploading data on a schedule. For example, getting data from data sources should start every day at 3 am.
Uploading data when it is updated in the data source. For example, extracting data from sources when the data freshness is no worse than the specified one.
Uploading a part of the data depending on the results of processing the previous part of the data.
Uploading a piece of data by event initiated by the process of loading another piece of data.
Uploading data on special conditions. For example, extracting data from a data source, provided that some attributes have appeared in another source.

In addition, monitoring of the availability of data sources and automatic processing of non-critical errors is provided.

• Connection to data sources through native self developped connectors
• Connect to any other data sources via ODBC or heterogeneous links
Controlling the load on data sources through the use of Mass Service Theory with a queue of data requests, requests priorities, dynamic resource quotas, etc.

Manual data entry or batch data entry from files of various types, including MS Excel with a complex structure, customizable by users, is used to:
• collecting data from branches / departments that are not integrated into the standard IT processes of the Company
• input of indicators, the automated calculation of which is not desirable or is not performed for various reasons
• enrichment of data from sources, access to which is possible only in manual mode
• data collection when building an automated interface is prohibited or impossible due to information security
At the same time, manual data input is the same standard process as automated data loading from source systems, and supports advanced capabilities:
• Calculation of the values of custom indicators based on the loaded data by using a configurable SQL query
• Automatic control of input data via custom data checks
• Editing entered data
• Search uploaded data
• Confirmation or cancellation of the entered data by the responsible employees
• Access rights management
• Notifications of loading or changing data status
• Logging of all changes
• Internal reporting

Data transformation algorithms can be configured by both administrators and users with basic skills in SQL scripts, No code development tools or in the MOSAIC data structure editor.

At the moment, the MOSAIC platform can upload data to Oracle, MS SQL, PostgreSQL, MySQL, MariaDB, Hadoop.
The list of possible types of data receivers is rapidly expanding at the request of our customers.