When faced with a new Enterprise Data Warehouse development, it’s hard to know where to start. Many people make the mistake of starting with some reporting requirements. Unfortunately, this leads to narrow silos that can’t accommodate new requirements. Initially, it’s better to anchor your high-level design to the core business processes of the business. If…
Attempting to deliver your Data Warehouse in a single iteration, “waterfall” style, would be very difficult. Today’s agile development process demands that the tasks be broken down into smaller iterative phases that incrementally deliver value. This means you need to prioritize your Data Warehouse development. In the case of a Data Warehouse, the business value…
A “Data Warehouse Bus Matrix” describes the high-level design of a Data Warehouse. At a glance, it shows all the facts and dimensions of a data warehouse and their relationships in a table-like ‘matrix’. It’s useful as a tool to design, plan, estimate and communicate your data warehouse. The “Data Warehouse Matrix” comes from the…
What is a Semantic Layer A semantic layer exists to present data to users as a set of related and commonly understood business entities, terms and metrics. A semantic layer is typically the “top” layer of a data warehouse/lakehouse. It is accessible to end users and report developers, who use it as the source for…
Here at Dimodelo Solutions, we are passionate about data warehouses and the benefits they can bring an organisation. But we are equally passionate about the art and practice (the how) of building these critical information assets. This is why we developed our Metadata Driven Data Warehouse tool “Dimodelo Data Warehouse Studio” (formally Dimodelo Architect). What…
“The report of my death has been grossly exaggerated.” – Mark Twain. Have you heard “The Data Warehouse is dead”? This claim appears from time to time, usually accompanied by the latest next big thing in Data processing (Self Service BI, Big Data, Data Virtualization etc). However, what these claims fail to understand is a…
This quick tip describes how to get Row Count in Azure Synapse Analytics (formerly Azure Data Warehouse). Azure Data Warehouse doesn’t support @@ROWCOUNT or ROWCOUNT_BIG. The link offers an alternative. This article modified that alternative to return the row count value as an OUTPUT parameter so a variable can be set to the row count value….
How to prepare a file for ingestion into Azure SQL Data Warehouse using Polybase, when there are NULLs in the data? There are only a few web pages that talk about how Polybase handles NULL values… And most of them are misleading. For example: Polybase to treat the NULL string as Null values and not…
To run ETL on Azure Data Warehouse, it’s a good idea to create a user in a higher resource class. There are a number of ways to authenticate to Azure DW. See – Controlling and granting database access to SQL Database and SQL Data Warehouse The easiest (from a development perspective) is to create an…
In this episode, I had the pleasure of speaking with Klaus Lindinger, Digital Innovation Officer at dataWerks & Partner at Infoteam Sales Process Consulting, based in Frankfurt. Over the past 30 years Klaus has worked with some of the world’s largest software brands. His first introduction to business intelligence, believe it or not, was collecting…
