Death of the Data Warehouse

Posted by Paul on August 8, 2011

Where would we be without the monolithic Data Warehouse? All of those operational, financial and commercial systems; tied, twisted and teased into one massive, steroid-dependent database. Fit for a few reports and up for much debate. Destined to never agree with any of the functions that it was initially brought in to serve.

The big, ugly beast surely needs to be put down. The days of the huge database are over. It is un-scalable; it is un-maintainable, and, above all, it is un-remarkable.

The Death of the Data Warehouse is here.

However, when you look again at the surly beast dragging its knuckles across the enterprise landscape you can’t help but to look back at its face and feel pity. In that face there is still a little bit of you. We were all part of a dream to build that slick, intelligence and speedy leopard and we know we are to blame in the reality of the overweight oath drooling and groaning its way through middle-age.

So, do you put that beast down, maybe put in on an exercise regime, or re-engage with it for one last campaign? I think my old friend is a little leaner than he first appears, there is still a twinkle in his eye that maybe only I can see. Come on, let’s saddle-up. Let’s remind ourselves of what we can do together. It’s showtime!

Category Categories: Data Warehousing  |  Comments 1 comment

Master Data – Why did it have to happen?

Posted by Paul on July 14, 2011

Master Data Management (MDM) has played an interesting role in recent years within the Data industry. Primarily, it has capitalised upon the failings of many Data Warehouses to bring value-for-money over the course of their lifetimes. It appeals to people who have built these systems with a bias towards the facts in the warehouse rather than the reference data.

Although in many cases MDM does bring an element of the complete solution that is needed, we should understand why we are hoisting up another solution in the face of failure of the first. When encountering reference data in building data warehouses, did architects and designers simply ignore the fact that reference data was important and we rushed by some tactical solution into hard-coding dimensions and hierarchies?

I don’t think that is a good enough defence.

Those responsible for the design of Data Warehouses need to always consider how the system fits together. It is their duty to design and construct a system that allows reference data to be managed by the business and not IT. This is not a revolutionary standpoint that was only realised with the dawning of the MDM discipline, it is simply good design and common-sense.

Alas, I shall try and avoid to re-iterate how common-sense is not too common in our industry, but we should have known better, surely?

But as a wise-colleague used to have to say on more than one occasion: ” We are where we are…”. MDM is good because it formalises part of the Data Warehouse that previously was missed in many instances by those within our community that simply grasp the importance of managing data in a clear, simple way by those people who deserve better from us: the business.

Lesson learnt. What will be the next lesson?

Category Categories: General, MDM  |  Comments No comments

Predicting

Posted by Paul on May 18, 2011

As we take the next critical step to funding this massive undertaking, I wonder what the future for me; the company and those lives we will affect, will look like in six months; two years and beyond.

Predicting the future is something I have never felt comfortable in doing; however, I have begun to realise that this is not because of the fear of predicting something and it turning out to be wrong. Inevitably some, if not many, of my predictions for this project will be wrong. The fear comes from making a decision on too little information.

I’ve been lucky that I have had the opportunity to talk to many experts in the variety of fields that my knowledge doesn’t cover and they have helped considerably to shape the ways in which I see our company growing and shaping over the coming months and years.

So, for me, predicting the future is a perfectly cold; rational activity that is simply based on the facts we know, and giving a reasonable best guess for those facts we don’t know. In some ways it takes the passion from the vision – it’s the engine that will make this business sore. But in others it’s the critical part, without which we couldn’t paint the path that leads us to our dreams and vision.

Category Categories: General  |  Comments No comments

The Beginning: Prototypes and Plans

Posted by Paul on April 27, 2011

Well, having decided there is a real need for a new way in which we manage our online lives, I am now two months into a prototype of such a solution – and I’m exhausted!

Prototyping is going well: the concepts work and the selection of Ruby on Rails seems to have proved to not be such a bad choice – admittedly there were times when I was pulling my greying hair out but overall it has served me well.

The pulling together of the business plan has been more challenging – at times it has been a burden too tough to tackle; especially since it is not something you write ever day. However, several days ago it all suddenly became a lot clearer to me. I suddenly realised the whole picture and how to draw that picture for others. It was a real relief.

So, now the dull scraping sound of a corpse being  dragged along the ground has been replaced by the light tinkling of bells on the feet of pixies – yes, what a wonderful world.

Let’s take that plan and fly!

(Actually, it’s quite sunny outside – maybe I’ll have a little beer before I plough into that plan…)

 

Category Categories: General  |  Comments No comments

Where do I start?

Posted by Paul on April 14, 2011

I’m hauling myself into the world of blogging; peeling back the mysterious arts of online authoring and trying to put those few, irrational thoughts and opinions that manifest in this dear head of mine into perfectly concise; edible and comprehensible English.

It’s all doomed!

But just maybe, there will be a window or two that we come across in these scribbled archives through which thoughts and ideas shine a little brighter and maybe a little longer than they would have without our etchings.

 

Category Categories: General  |  Comments No comments