Clickstream

Future of Business Intelligence Is Coming From Outside the BI Market

My keynote for the Orlando conference was all about the future. Instead of trying to predict exactly what we would be doing in five years' time, I talked about why it's hard to know that, why the people who do it (us industry analysts) are so often wrong, what we can look at to guess the shape of the future in our market, and some examples of smart people doing work that's already in the next generation of data warehousing.

The slides are posted below. I'll have a transcript posted once I have it cleaned up for public consumption.

The future of business intelligence is most likely going to come from (or be heavily influenced by) consumer web sites, consumer electronics and games. I'm not seeing a lot of real innovation in this market from the major players. They seem to still be adjusting to the architecture change from client-server to basic web 1.0.

At least I know some of my assumptions about the future are right. Cindi Howson mentioned that most customers are still running desktop versions of BI software, so the BI vendors are still having to pay too much attention to the past. While sharing a taxi with Claudia Imhoff I checked my thoughts about the younger generation against her experiences with her daughter. Surprisingly, I got a few things right. She also had some great observations about what people need versus what they want and what IT gives them. See her blog for of her thoughts.

| View | Upload your own

Labels: BI, business intelligence, data federation, data warehouse, future, information management, web2.0

Posted by Mark Friday, November 02, 2007 5:18:00 PM | permalink | (2) comments

Five Minute Data Integration Podcasts

Podcasts we did on defining data integration technologies just went up at SearchDataManagement today. Each describes a particular technology, what it's used for and when its appropriate. We did four of them on ETL, EII, EDR and EAI, with plans to add several more on topics like master data management and data profiling. It's always strange to hear yourself as other people hear you.

Labels: data federation, data integration, EII, etl, podcast, replication

Posted by Mark Friday, August 31, 2007 12:53:00 PM | permalink | (0) comments

Slides for Data Federation/EII Uses and Abuses

I just uploaded the slides for the data federation/Enterprise Information Integration uses and abuses webcast to SlideShare (embedded below). You can also follow this link to the archived webcast if you want to listen to the talk. Minimal text on the pages so that is probably your best option if you want the details.

Labels: data federation, data integration, EII, events, webcast

Posted by Mark Saturday, August 04, 2007 2:20:00 PM | permalink | (18) comments

List of Data Integration Webcast Links for This Year

Here are links to replays for all the webcasts I've done during the past year.

The Hybrid Data Warehouse: Extending the BI Environment with Data Federation
Webcast: 3/21/07

Challenges and Techniques When Hand-coding ETL
Webcast: 6/26/07

Extract-Transform-Load (ETL) Market Overview and Directions
Webcast: 6/13/07

Enterprise Information Integration Technologies: Common Uses and Abuses
Webcast: 2/14/07

Enterprise Integration Series: Evaluation Criteria for Selecting ETL Tools
Webcast: 8/2/06

Labels: data federation, data integration, EII, etl, events, webcasts

Posted by Mark Monday, July 09, 2007 11:58:00 AM | permalink | (0) comments

When to Consolidate and When to Federate

The will be on data consolidation versus virtualization and the fact that we're starting to need both in the data warehouse (hence the "hybrid data warehouse" in the title).

On-demand access to current data or operational details requires some rethinking of infrastructure. If we want to physically consolidate the data then we need to rethink the extract mechanisms, schemas, storage models and even delivery interfaces to client applications.

The assumption is no longer true that data should always be stored in a central warehouse database. We have technologies that open up access to whole new realms of data: spreadsheets, files, web pages, messages, XML, images, standard industry formats. Centralizing this content isn’t required to manage the schema and relationships for these things – often it can’t.

Data federation / EII tools provide an option that lets us centralize most of the data, but provide access to information that can't easily be consolidated. Unlike direct access to the native sources, these tools allow us to mediate access and provide a single point of management and reuse.

The webcast will go into some more detail on methods of consolidation and where federated access makes sense. Wednesday at 9:00 AM, Pacific.

Labels: data federation, data integration, EII, events

Posted by Mark Sunday, March 18, 2007 7:11:00 PM | permalink | (0) comments

Data Federation and EII are Underutilized Integration Tools

I'm working my way through the current EII and data federation products as part of research I'm doing for on-demand/real-time data warehousing. The new features in these products - ease of use, non-relational sources, ERP connectors, performance / scaling enhancements, multiple input and output protocols - really make them worth a second look. It's underutilized in current environments that have to deal with on-demand or "right time" data access and multi-format data.

This coming Wednesday (February 14) I'll be doing a short webcast for TDWI on EII covering a little bit of how EII and federation tools work and how they can be used both in and outside the BI environment.

In the past I haven't been a big fan of EII for a few good (I think) reasons:

1. EII/federation vendors are wedged between big ETL vendors on one side and big EAI vendors on the other side (long term product viability).
2. One class of products tended to focus on XML-only interchange, and generally sucked for relational data, particularly in the areas of performance, scalability and ease-of-use.
3. The other class of products focused on distributed queries but didn't work well for anything else, and were mostly limited to relational output.
4. The IT usage scenarios were/are not quite mature enough to provide a solid market.
5. Too expensive relative to the value based on use cases.

I now find myself liking the latest releases of the products. The vendors who survived the mini-hype wave circa 2005 have largely dealt with points 2, 3 and 4 but points 1 and 5 are still true.

Long term viability of standalone vendors is still an issue, though I expect the best few may survive. We're more likely to see EII/federation slip into the data integration stacks or platforms of major vendors and squeezing the smaller niche products out. We're seeing some of that happen already.

The better products still cost too much unless you have the perfect use case. The ROI simply isn't there for real-time data delivery when companies already have alternatives that meet "good enough" criteria; technologies like replication, EAI/queuing software, and CDC/ETL combinations. But...

At the same time, I'm seeing more use cases, and more urgency around some of the existing uses. For a time dashboard hype was lifting EII vendors, but the value/cost ratio of the dashboard use case was poor. Many EII vendors are now chasing the SOA market because EII shows more value there than as an adjunct to support dashboards. Try using EAI tools to make distributed multi-format queries some time. It's not fun.

I'll be writing up more as I run products through their paces. Vendors I'll be doing in-depth work with include IBM, Composite, BEA, Business Objects and several others. Look here over the coming months for more detailed technical writeups. I'm going to be tying portals and mashup servers into the mix so this will be fun.

Labels: data federation, data integration, EII

Posted by Mark Sunday, February 11, 2007 10:30:00 PM | permalink | (0) comments