Data Warehouse Automation is the latest buzzword in Data Warehousing.
So what is Data Warehouse Automation ?
Data Warehouse Automation describe the automation of the following:
- Simplified capture of the Data Warehouse Design.
- Automated Build (i.e. Generate Code)
- Automated Deployment of code to the Server
- Automated Batch execution of the ETL code on the Server.
- Automated Monitoring and Reporting of the Batch execution.
Generally this is achieved through a Data Warehouse Automation tool. You could also manually create scripts to generate code and automate the Build, Deploy Run processes.
Why do you need it ?
In the Past, Data warehousing has taken too long and the result have been too inflexible. This led to much frustration in the business community. As a result businesses have turned to other solutions like Big Data and Self-Service BI. However a Data Warehouse provides additional benefits over other solutions (i.e. Self Service BI). For example:
- The ability to keep history,
- A single source of the truth,
- End user productivity,
- Reduced risk of reliance on key individuals,
- Data augmentation,
- Query performance etc.
The problem isn’t the Data Warehouse concept, its still an extremely useful method of managing information. The problem has been our execution of Data Warehouse development as an industry.
This is where Data Warehouse Automation comes in. It doesn’t throw out the idea of a Data Warehouse in search of a better way, instead it directly addresses the real problem, being the execution of Data Warehouse development.
Lets face it, Data is Dirty. There is no quick and easy way to solve that, no matter what tool you use. It takes skill and experience to derive usable information from dirty data. If you are going to invest that time, why not invest it in a data warehouse. A Data Warehouse is:
- A long term asset,
- Fit for multiple purposes,
- Supports multiple reports, dashboards and analysis.
Benefits of Data Warehouse Automation
- It’s fast. Dramatically reduce your development time.
- It’s flexible. Respond to changing business requirements quickly and easily.
- Stay focused. On what really matters. Concentrate on reporting and analytics instead of stuck in ETL code.
- Quality.Data Warehouse Automation tools produce tested, high performance, complete and readable code. At least Dimodelo Data Warehouse Studio does!
- Consistency and Continuity. It produces consistent code, naming standards etc. Developers come and go, but as long as they keep using the same tool, its easy for one developer to understand the work of another.
Who
Data Warehouse Automation tools are used by professional Data Warehouse developers to enhance their productivity. It’s even possible for a technical data analyst with some training and support to maintain their own Data Warehouses.
Dimodelo Solutions provide a Data Warehouse Automation tool for the Microsoft platform call Dimodelo Data Warehouse Studio.
When
When do we want it … Right now! Certainly that is what the business has been telling IT for years. Seriously there are mature data warehouse automation tools on the market right now.
Where
You need the flexibility to deliver a Data Warehouse, on your desktop (development environment), on-premise or in the Cloud. Your solution should be able to deploy to multiple environments and support all three scenario simultaneously.
Further Reading:
– Check out our other Articles on Data Warehouse Automation
– Look at Dimodelo Data Warehouse Studio, a Data Warehouse Automation tool