Is Data Warehouse Automation good for BI Professionals?
As professionals, anything that makes our job significantly easier, can be conceived as a threat.
I must be honest, when Microsoft started on its journey of ‘Self Service BI’, PowerPivot, PowerView etc, I perceived this as a threat, and as a watering down of my skillset.
However there is two truths that you can’t get away from:
- Data is Dirty. That is, its often incomplete, inaccurate, doesn’t join together well, isn’t at the right level of aggregation etc. etc.
- Someone, Somewhere, somehow needs to deal with it.
This means you need skill and experience to sort out the mess. The idea that anyone can do it, or that a self service BI tool can solve this problem is bogus. In fact these tools rely on the fact that data isn’t dirty, and someone has cleaned it up prior to reporting.
But Self Service BI has one significant advantage over Data Warehousing, and that’s time to market. You can create a single self service BI report much faster than creating a data warehouse to support said report. However the two solutions are not equivalent. A Data Warehouse serves multiple purposes, and multiple reports, and provides advantages like History, Single Version of the Truth, End User Productivity, Reduced reliance on key individuals, Data Augmentation, Query Performance etc. A single self service BI report contains all the logic, and data cleansing inside the report, which isn’t available to other reports. In truth there are concepts of shared models in Self Service BI, but they are rarely implemented, don’t have the same advantages of a Data Warehouse, and quite frankly, will likely take just as long to build as a Data Warehouse. As a professional you know that although Self Service BI tools are great at providing immediate gratification, the allure of Self Service BI has short term gain for long term pain. Does anyone remember Spread-marts? I once worked for a business that had 17000 spreadsheets on its file system. Self Service BI has the potential to deliver a similar result over time.
So why is Data Warehouse Automation good for the BI professional ?
It brings implementation time down to the equivalent time frame of Self Service BI. This provides the BI professional ammunition when fighting the battle of Self Service BI vs Corporate BI in the organisation. But why the battle? Self Service BI tools have obvious value to the business, and the two concepts should be able to work together. In-fact, this is the subject of an up coming post on agile analytics. Data Warehouse Automation allows corporate BI to keep pace and complement Self Service BI. Working side by side providing the Data Warehouse benefits to Self Service BI efforts.
Data Warehouse Automation allows you to offer an alternative/complementary solution that is a quality, quick to implement, and flexible.
Some BI professionals are resistant to Data Warehouse Automation. Typically these objections include:
- I’m a professional, and the idea that this tool can do my job is upsetting.
This takes me back to truths 1 and 2. The tool can never replace you, it will enhance you. Lets not forget the end game here. No business ever asked for a data warehouse, what they want is information and insight to drive efficiency, growth, profit etc. This is the true value proposition of your profession. Having your head stuck in ETL is no good for anyone. Data Warehouse Automation frees you to concentrate on high value reporting and analytics. The thirst for information is not diminishing.
- The tool doesn’t do it the way I do it
Ok. Probably not. There are as many opinions of how a data warehouse should be constructed as data warehouse professionals! But does the tool produce a correct result that adheres to common standards. In many cases the answer will be yes, and isn’t that good enough. Again, is it producing the result for your Business ? That’s the end game.
- I can do it better. Its a black box, I can’t affect the end result
In some cases that might be true. There are always edge cases, where you have specific needs. However you may find that some tools (like Dimodelo Data Warehouse Studio) allow you to write you own custom code, where you need it, and produce your own generation templates, so the code is generated exactly as you wish. Being a black box is partly the point of the tool. It aims to obfuscate the underlying technical details so you can work at the design level. It takes away the boring monotonous stuff and lets you work on the creative design end of process.
- Its too expensive
But is it value for money? What is the return on investment, and how quickly do you get that return? Certainly that’s how a business manager will look at it.
As technicians we sometimes get caught up in the technology, and loose sight of the value we provide to the business. If you stop delivering that value, that’s when life gets difficult.
Further Reading:
– The Who, What, Where, When and Why of Data Warehouse Automation.
