Clickstream

Academics Say Enterprise Data modeling Sucks Big Rocks

I was browsing MIS Quarterly archives and stumbled across an academic paper that confirmed a long-held belief - Strategic Data Planning, aka Enterprise Data modeling, aka many other TLAs is an expensive waste of time and, ahem, "not the best way to develop a data architecture." From their abstract:

"In spite of strong conceptual arguments for the value of strategic data planning as a means to increase data integration in large organizations, empirical research has found more evidence of problems than of success." [emphasis mine]

A lot of data warehouse projects used to start off with a big data-model-in-the-sky effort. Most people now understand that data warehouse and data integration efforts are use-based more than they are data-model-driven.

Sadly, companies like IBM were foisting this crap on organizations as recently as three years ago. I have proof in the form of three big, useless binders that were supposed to help a project I was working on. I suspect IBM and many other companies continue to foist away today.

I have to include some choice quotes from the paper. How many times have we heard "try harder" or its relative "work smarter, not harder" in our work? In the case of SDP, do both!

"Although some previous researchers have questioned the basic efficacy of the SDP method, there has usually been the implicit assumption that if problems in the implementation of the approach are identified and addressed, the expected benefits will follow. In a sense, practitioners have been encouraged to "try harder." This paper suggests that "trying harder" may not be the answer in many situations."

This suggestion of how these types of projects are staffed rings true. I call it the "Island of Misfit Toys" staffing model.

"Proposition #13: The time required by individuals may self-select the wrong participants.
Prior research and the cases in this article suggest that the SDP methodology requires a major time commitment from its participants, often as much as half a year of effort. Because the time of the most insightful and capable individuals is usually in high demand, the methodology may self-select the wrong people--those who can be spared because they are not involved in other critical business issues. The result could be less insightful and less creative solutions." [there's an understatement!]

The conclusion is a real zinger:

"The assumption is that given the right participants and cooperation from the rest of the organization, the resulting architecture will be correct enough to serve as a blueprint for more detailed data and systems design. Several findings from these nine cases seem to challenge that basic assumption. First, the participants themselves express uneasiness about the correctness of their architectures. Second, systems developers complain that the architectures are too high-level to be useful in designing systems, or they contain errors that are only apparent when a more detailed analysis is conducted. And third, other organizations using quicker, easier methods (such as "stealing" an architecture, or putting six IS professionals in a room for a week to create one), are able to develop similar architectures at a fraction of the cost."[6 people, 1 week? Ouch!]

Admittedly, Strategic Data Planning has evolved since this article was written. It's too bad that for many practitioners it is still largely an academic exercise with little practical use, like the IBM project I mentioned.

The problem is that 'strategic' and 'planning' were left behind by most practitioners as they focused on 'data'. With proper focus on results (the strategic and planning parts) rather than work products (data) some usefulness can be salvaged from SDP.

I made "Strategic data planning: Lessons from the field" available so you can enjoy it as much as I did, but you should still support your local library.

Posted by Mark Tuesday, February 01, 2005 1:02:00 AM | permalink |

Comments:

What does more recent research (i.e. less than 13 years old) indicated on this subject?

# posted by  Anonymous : 6:27 AM

 

Good question.

In practice, it's all but gone. I hardly ever hear anyone mention EDM any more. Just the odd case that crops up here and there.

The value has really fallen to defacto models that are getting built up inside the metadata repositories of DI/ETL products, and in the user-based models.

Really EDM was aiemd at too many problems. It didn't solve the IT problems of systems/data integration because it's the wrong approach. It didn't solve the user/IT problem of finding data which is being addressed through data warehousing, BI, portals and various metadata-driven tools like EII.

EDM is much like the data modeller's version of "ERP will solve the integration problem." That didn't work either.

# posted by  Mark : 12:55 PM

 

the information above provided in very much useful one.Quality education is offered in our institute,informatica

# posted by  Unknown : 4:40 AM

 

View Modern Hindi Songs Lyrics s by popularity along with songs featured in, albums, videos and song meanings. We have 1 albums and 58 song lyrics in our directory.

# posted by  mak : 10:59 PM

 

Post a Comment