Sunday, February 22, 2009

ClearQuest integration with Team Foundation Server

Should CQ and TFS ever be integrated?

As a rule of thumb, the answer is NO. Team Foundation Server (TFS) is an Application Lifecycle Management solution that covers 9 out of 10 ALM disciplines listed in the previous related article.  On top of that, TFS implements most, if not all, of the major Change Management features of ClearQuest (CQ).  On the other hand, ClearQuest can be deployed on Windows or Linux.  It may be deployed as a Unified Change Management solution, as a part of Team Unified Platform suite, or, in the future, as a part of a Jazz based ALM.

When CQ and TFS should not be integrated?

When an organization owns and operates both the TFS and CQ at the same time, it incurs redundancy (most likely unnecessarily) and related administrative and support costs.  Whether the move is from TFS to ALM with CQ or vice versa, the goal is to migrate to one or the other product and not to make the redundancy and its overhead permanent.  This is a fiscal, operational, or a common sense argument, not a technical one.  However every sane CIO/COO/CTO or a manager with budgetary responsibilities should ask such a question and get an answer rather quickly.  Adopting a longer than 6 months outlook and keeping the “migration vs. integration” discussion aligned with organization’s overall IT strategy would likely help with making better decisions.

Plausible scenarios for integration

Obviously, there are circumstances that fall outside of the “rule of thumb” constraints.  Below are several common examples where integration, as opposed to migration from CQ to TFS, may still be necessary:

  1. There is significant development taking place outside of Windows platform and outside of TFS supported ALM activities: SCM in Linux or UNIX environments; UNIX/Linux automated build, testing, and deployment.  In other words, the TFS ALM features are largely unavailable in the non-Windows environment where CQ solution is already successfully deployed.
  2. It is more expensive to migrate to TFS the CQ environment with its own extended/external relationships to ClearCase, RequisitePro, linked design model elements or code in Rational Solution Architect, Rational Portfolio Manager, etc.

High level roadmap on how CQ and TFS can integrate

Manual synchronization
  • Very Basic, using existing out of the box TFS and CQ features.  I would not recommend this method: it is quite laborious, very error prone, and not sustainable in the long term.
TFS provides ClearQuest converter command-line tool out of the box.  The tools is intended for migration of CQ repository items into TFS.  Theoretically one can setup updates from CQ to TFS as follows:
  1. Define a date based queries in ClearQuest that returns new unsynchronized records for each record type of interest.
  2. Follow the steps to analyze and migrate records outlined in MSDN
  3. Optionally add TFS integration fields omitted by the converter
  4. Run the CQ converter tool (with date based queries in CQ) to copy the new items into TFS
ClearQuest ships with the ClearQuest Import utility (Windows GUI app) that allows importing CSV file records into ClearQuest database repository.  In order to setup synchronization from TFS to CQ:
  1. Open Work Items query for each item type of interest in Excel.  Save the results as a CSV file.
  2. Open the CSV file and change the column names to be the same as the expected CQ import schema record fields.  It may be necessary to add calculated fields to satisfy CQ import record schema.  Save the CSV.
  3. Run CQ Import utility, specifying whether the records are new or being updated.
NOTE: such import/export does not transfer change history
Batch synchronization
  • Building custom synchronization
One should take advantage of existing work done in the Migration and Synchronization Toolkit.  The customized solution would rely on using ClearQuest COM API (cqole.dll) and TFS API.  On one hand such solution would have to re-implement functions of TFS CQ converter, plus synchronization checks whether to insert or update new records.  On the opposite side, such solution will use CQ API objects to insert and update CQ records from TFS.
Visual Studio Plug-In

Another way to customize synchronization between TFS and CQ from a single IDE is to create a Visual Studio plug-in.  One scenario that may be more efficient is on-demand synchronization, where VSTS plug-in accesses 1:1 (shadow copies) related records in TFS and CQ and respectively updates them at the time of access.  VSTS IDE extension would likely be the nicest way to leverage and use CQ and TFS integration, but it would also be the most expensive.  If the long term use of such integration and productivity gains can justify it -- more power to you.

Parting thoughts

Perhaps more important than integration between CQ and TFS are the questions of future migrations and integrations between TFS and Jazz (Team Concert, etc.).  These questions will be part of my product research agenda for IBM Jazz ALM suite.

Monday, February 9, 2009

MarketScope for Application Life Cycle Management

Gartner had a report issued in December of 2008 about Application Life Cycle Management market and the main players. Those interested may get the full report here. The report contains several noteworthy observations in regards to IBM Jazz and Microsoft Team Foundation.

In the IBM related section of the research note, it is a bit surprising that many of the same issues raised in the ClearCase/ClearQuest UCM vs. Team Foundation Server post are also raised. Specifically, Gartner analysts point out that IBM Rational “has been hampered by a mass of products built with different technologies amassed through acquisition.” On account of “pre-Jazz” ALM products, the analysts state:

IBM’s pre-Jazz solutions have been able to handle some of the largest and most complex development projects, but they carry a tag of being difficult to learn, expensive to administer and operate, and they now seem somewhat outdated when viewed in terms of current development trends, such as agile.

The history of Rational and IBM acquisition did result in overlapping products, including the ALM domain. On top of that, the current release of the Jazz based products, namely Team Concert, Requirements Composer, and Quality Manager is not yet an integrated suite. According to IBM Rational product roadmap, those products should work as an ALM suite in the future. However, they do not in this release.  Having said this, Gartner analysts have put a great deal of credit into the future ALM Jazz implementations.

We rate IBM as a Strong Positive because of its current market strengths and breadth of portfolio. This gives the company a broad base to sell into. In the midterm, gaps in the project and portfolio management offerings will handicap some broader integrations. The open development model of Jazz, coupled with IBM’s aggressive introduction of new tools and renovated Rational offerings, sets a strong pace. Jazz is a solid architectural foundation for further innovation.

In the Microsoft section, there are a few statements that did not quite make sense, for instance: “gaps remain in some management and execution tool areas.”  What about MSBuild? How about MS Project and Project Server integration? Configuration management? Change management? Requirements management features of TFS?  Regarding Microsoft, analysts come to this conclusion:

We rank Microsoft as Positive. The company has a strong reach in the market, and moderate pricing for a large, stable provider. It has a proven scalable architecture, but it needs to complete the toolset. Furthermore, Microsoft needs to mature its sales force so that it can more consistently reach enterprise buyers.

Without familiarity with other vendors and ALM offerings outside of IBM and Microsoft, it would be difficult for me to validate Gartner’s research and conclusions about the other products. The report repeats some marketing language from the companies, but leaves the reader with unanswered questions about its conclusions.