Data Modeling

This category contains 17 posts

When Data Modeling goes too far

One thing I have struggled with when I have created Operational Data Stores is the tendency to create generic tables that promote re-use. I find these are usually tables like Address and Person. In an enterprise environment there may be many applications or Systems of Record that store Person or Address information. There is the … Continue reading

How to create 10,000 Extract, Transform, and Load automated tests using 4 tables #agile #data

The thing I love about my chosen profession is the ability to learn new things and improve on lessons learned from past projects. Recently I was able to take on a problem that I have experienced on multiple past projects. “How can we easily create automated tests for a Data Migration or Extract, Transform, and … Continue reading

Business Case for creating a Data Model and Data Normalization

I recently came across a great article by @datachick on “Why having a Data Model is important?” on DataVersity. Highly recommended read. Karen Lopez provides a great list of items on why she finds a Data Model helpful. I do believe there is something missing from the list of items though. All of the items … Continue reading

12 Rules to create a Dimensional Model from a Normalized Model in an #agile way

Recently I have been on a project where I have been fortunate enough to develop both a normalized Operational Data Store and a Dimensional Data Warehouse. In addition to this, I have been fortunate to be able to recreate the Dimensional Data Warehouse three times over. This has been because the project has been done … Continue reading

Adaptive Data Model – #Agile or Anathema?

I have seen the concept of an Adaptive Data Model proposed as an Agile method to Data Modelling lately. (Most recently in Ken Collier’s excellent book – “Agile Analytics”) The theory is that you can be more Agile using an ¬†Adaptive Data Model instead of a traditional Data Model of the business domain. Definition An … Continue reading

The state of #Agile according to Data Modellers

DataVersity released their snapshot survey on Data Modelling and the results aren’t good for Data Modellers who want more adoption of Agile by the Data Modelling community. Before we get into the details, DataVersity is a great source of references and webinars for all things data. You can find them by clicking on this link: … Continue reading

#Agile Enterprise Data Model – #holistic and #iterative

I’ve recently been working on a Data Warehouse project that I am trying to do in an Agile way. Unlike Agile Software Development, Agile Data projects have some complexities that cause you to adapt the common Agile methods and practices. I am a big fan of User Story Mapping and creating a User Story Map … Continue reading

Why #Dimensional Modeling matters

I’ve recently completed a data modeling initiative on a major project. After doing this I’ve come to two major conclusions: The coverage area in Insurance is probably the most devious and twisted area of data that I have ever modeled. Dimensional Modeling should be done on every model to ensure you can simply model the … Continue reading

#SDEC12 Conference Review #Agile

Well another Software Development and Evolution conference has come and gone. (You’ve always wondered what SDEC stood for didn’t you?) It was a lot of work and effort to make it all happen, but in the end it was very enjoyable. I learned an immense amount and cant’t wait until next year. My Highlights The … Continue reading

#Data #Hoarding and #Norm-aholics

Are you a sufferer of Data Hoarding? This is an affliction that affects thousands of businesses every day. Common symptoms are terabytes of data being kept around just in case you ever need them. The data ends up being stored in a multitude of disconnected tables just piled on top of one another with no … Continue reading