Clickstream

Ten Mistakes to Avoid When Selecting and Deploying ETL Tools

TDWI published my ETL "Ten Mistakes" paper and it's now up on their web site . Unfortunately, site registration is required. A PDF will be on the way in the near future and I'll post a link to it as soon as I have it. In the meantime, I'll pretend I'm Charles Dickens and serialize them here. A note to get things started:

You'd be surprised by how many companies just look at the two or three tools Gartner listed as "up and to the right", watch some demos, then spend a few hundred K$. Following a formal process is vital. I also think that companies spend too much time looking for the best tool, rather than the one that best fits their specific needs. A lot of time is wasted evaluating features that will likely never be used.

Foreword

Data integration and extraction form the foundation for business intelligence and data warehousing projects. Making the right decisions when selecting and deploying this technology is key to a successful long-term implementation. We’ll take a look at 10 of the most common mistakes organizations make when selecting and deploying extract, transform, and load (ETL) products.

The proper frame for evaluating ETL products is “best fit” rather than finding a single “best product.” All products have their strengths and weaknesses; the goal of an evaluation is to identify those strengths and weaknesses and match them up with what is important for your specific project.

To do this requires that you follow a formal process of qualifying, comparing, and selecting vendors, and that you take into account both the immediate needs and the future evolution of your systems. If you avoid the following 10 mistakes, you should be on your way to finding the technologies that best fit your organization’s needs.

1. Failing to Follow a Formal Process

One of the most common errors that organizations make when selecting ETL products is not following a formal evaluation process. A list of criteria isn’t enough; it’s vital to take a series of steps to make sure you find the product that best fits your needs. The following process works well for most companies:

1. Identify the selection committee members who will determine the evaluation criteria and rate the vendors.
2. Determine the requirements that all products must meet in order to be considered.
3. Research a list of vendors that meet your minimum set of requirements.
4. Based on your research, narrow the list to three to five vendors that meet the minimum requirements. This is the shortlist that you’ll evaluate more thoroughly.
5. Create a list of detailed evaluation criteria, including qualitative criteria such as the look and feel of the interface or how easy the product is to use. Define your method for rating product performance on these criteria, as well as a method for ranking the results.
6. Meet with vendors and rate the products against your list of criteria based on their presentations and demonstrations. At this point, you may further eliminate products from consideration.

   Be flexible. After meeting with vendors and seeing demonstrations, you may discover criteria you had not thought to include in your evaluation. If everyone agrees that the criteria are reasonable, add them to your list and check back with the other vendors. You’ll find that evaluation is an iterative process, particularly if you haven’t seen most of the products before.

7. Conduct a proof of concept and rate the products, or re-rate the products now that you have more detailed information.
8. Rank the vendors and make your first and second choices. Choosing a second-place vendor can save you time if you fail to negotiate a deal with the first vendor.

Following a formal process ensures that your criteria are well thought out and prioritized before you begin looking at products. The process also ensures that products are examined at the same level of detail and compared equally. By creating a short project plan for the process, you’ll be able to keep the evaluation focused and complete it quickly.

Link to TDWI page

Posted by Mark Wednesday, July 12, 2006 11:45:00 AM | permalink |