UniQ Data Store
UniQ Data Store is a pipeline used to import, convert, enrich, and process data from banking systems. It uses a Persistent Item Cache (non-relational) database to transform data into a standard relational database management system, such as Oracle or SQL Server.
The solution covers the following items:
The solution is based on Python & Django technology, RabbitMQ for data transport, Celery as a task manager, and Flower as a monitoring tool
Main Functions of UniQ DS :
- Security Management and Access rules based on Roles and User profiles
- Consumes standard CSV files extracted from Core Baking System using Non-Relational Data basis (PIC)
- Processes the extracted data into three categories: Referential, Transactional, and Accounting data for better validation and control.
- Cleans the Received files and processes them into Relational Database Tables such as SQL server & Oracle.
- Facilitates the orchestration of data transfer from the original database (PIC) to the target database.
- Handles multi-valued and sub-valued fields by creating additional columns and tables in the target database.
- Provides asynchronous data transfer with high performance
- Allows the Monitoring and Control of the different tasks.
- Enables exploration of the target database to visualize, extract, and report selected data.
- Exports data from the target database to other applications within the bank's information systems.
Benefits of UniQ DS
- Simple to implement using standard functions and technology tools.
- Extracts the data from the source data based with standard tools
- Consumes CSV files from the source database.
- Automates daily data transfer.
- Addresses issues with Local Customer-Specific Tables.
- Solves problems with Multi-Valued and Sub-Valued Fields
- Can integrate data from different sources.
- Provides the bank with a unified operational database.
UniQ DS Pain Killer
- Simplifies access to data using standard technology tools such as Power BI, Tableau, Click Sense, etc.
- Allows the bank to use data from the relational database to support other applications within the bank.
