Attempting to deliver your Data Warehouse in a single iteration, “waterfall” style, would be very difficult. Today’s agile development process demands that the tasks be broken down into smaller iterative phases that incrementally deliver value. This means you need to prioritize your Data Warehouse development. In the case of a Data Warehouse, the business value…
A “Data Warehouse Bus Matrix” describes the high-level design of a Data Warehouse. At a glance, it shows all the facts and dimensions of a data warehouse and their relationships in a table-like ‘matrix’. It’s useful as a tool to design, plan, estimate and communicate your data warehouse. The “Data Warehouse Matrix” comes from the…
What is a Semantic Layer A semantic layer exists to present data to users as a set of related and commonly understood business entities, terms and metrics. A semantic layer is typically the “top” layer of a data warehouse/lakehouse. It is accessible to end users and report developers, who use it as the source for…
Here at Dimodelo Solutions, we are passionate about data warehouses and the benefits they can bring an organisation. But we are equally passionate about the art and practice (the how) of building these critical information assets. This is why we developed our Metadata Driven Data Warehouse tool “Dimodelo Data Warehouse Studio” (formally Dimodelo Architect). What…
“The report of my death has been grossly exaggerated.” – Mark Twain. Have you heard “The Data Warehouse is dead”? This claim appears from time to time, usually accompanied by the latest next big thing in Data processing (Self Service BI, Big Data, Data Virtualization etc). However, what these claims fail to understand is a…
This quick tip describes how to get Row Count in Azure Synapse Analytics (formerly Azure Data Warehouse). Azure Data Warehouse doesn’t support @@ROWCOUNT or ROWCOUNT_BIG. The link offers an alternative. This article modified that alternative to return the row count value as an OUTPUT parameter so a variable can be set to the row count value….
How to prepare a file for ingestion into Azure SQL Data Warehouse using Polybase, when there are NULLs in the data? There are only a few web pages that talk about how Polybase handles NULL values… And most of them are misleading. For example: Polybase to treat the NULL string as Null values and not…
To run ETL on Azure Data Warehouse, it’s a good idea to create a user in a higher resource class. There are a number of ways to authenticate to Azure DW. See – Controlling and granting database access to SQL Database and SQL Data Warehouse The easiest (from a development perspective) is to create an…
This post is the second in a series of detailed cases studies discussing the ETL strategies that can be used when a Persistent staging layer is included in your Data Warehouse. It’s intended as a reference for developers using Dimodelo Data Warehouse Studio, our Data Warehouse Automation tool to quickly build a Data Warehouse. It’s most…
Why you Need a Persistent Layer 1. All history, all of the time, instead of some history some of the time Including a persistent layer in your architecture is a paradigm shift in how you see the data warehouse. In the popular Kimball methodology, without the persistent layer, the data warehouse layer was responsible for…
