Clickstream

My Latest Article on Real-time Data Integration is Out

I have a feature article on real-time data integration in the current issue of Intelligent Enterprise. I was originally writing a longer piece on how to select the appropriate technologies for real-time data integration but ended up publishing this condensed version.

The basic idea is that there are only a few very basic conceptual models people use when talking about data integration: point-to-point, hub-and-spoke, and bus (not the diesel kind). These models impose a certain view of the applications and how they are integration, and provide the language and ideas for convincing others that data integration is basic IT infrastructure.

I think this point is missed repeatedly. Integration is almost always viewed as a point solution between the system being built Right Now and systems that already exist. To avoid the huge maintenance trap of spaghetti code and system connections that one-off integration creates, it has to be exposed and managed as a class of infrastructure just like the networks on which it exists.

Below the conceptual model that defines the "what", there are interaction models that define the "how". How does one application communicate with another? How does data get from point A to point B? This really boils down to the mechanism: publish and subscribe, direct or brokered messaging, and so forth.

All this stuff is at the architecture level. It's abstract and the missing part is what the components of the architecture are. The article discusses them, but not in detail.

Once you have all this in your head, it's pretty easy to evaluate vendor's all-encompassing products, point solutions, and ways to roll your own, which is probably the best overall, provided you're given the time to properly design it. Sorry, but XP won't cut it when laying in infrastructure. You sometimes have to design before coding.

Posted by Mark Monday, June 16, 2003 1:28:00 PM | permalink |