Once upon an ERP project, somewhere near production deployment, a nasty show-stopper bug tore down the dream of production use of ERP in scheduled time. Well, nothing out of the ordinary in that. Bugs should be found before production deployment.  And this tiny little bug was not even the nastiest sort, a mere data mapping flaw that revealed itself only with certain rare message and content. Why am I even writing about it? Well, in this particular customer case, these show-stoppers started to be more of a habit than a rare accident. What really made the customer to hit the roof was the fact that bugs were always found in either deployment testing or in integration testing and yes – sometimes they were not found at all. Or actually they were found, but not by the testing steps. They were found by the end users in production.

Before I go further with the story, I think I should open up how we manage/execute testing of an integration solution in general. Testing happens in steps:

1.      Unit Tests during implementation

2.      System Testing  with partial processes (e.g. 2 units) with mock-ups emulating  interfaces (a.k.a Component Testing)

3.      Integration Tests in customer’s test environment

4.      Load and Stress Testing (if requirements define so)

5.      Production Deployment tests (in cyclic-deployment) when putting solutions online.

With this customer Integration Competency Center is responsible of defining processes and continuously evaluating their execution. Among other indicators, quality of development process execution is measured by the amount of bugs found in integration and deployment tests. I was participating as an external supplier member in a weekly ICC meeting when this issue of continuous bugs was brought up and problem solving was assigned to me.

I was surprised how that well-defined development process with lots of built-in testing could end up with such a lousy quality. Closer study revealed that persons, who were writing the business use cases at the customer side, provided us with only one or two successful test cases with test materials. This oversight led to situation where all testing steps using that material failed to cover the whole solution sufficiently and bugs were able to hide themselves easily in non-tested work-flow paths or maps. Pointing out the fault was really easy. But hey, wait a minute! Why ICC did not find out that well-defined deployment process was just missing adequate amount of test cases? The reason for this was also obvious: in that incarnation of ICC, no one was responsible of testing. What they were missing was a Test Manager, whose mission among other things is to whip out the test cases and test material.

Ever worked in an organization in which processes are well-defined and also realize in daily work? I have. So I emphasized the meaning of the process definition in our Integration Competency Center Guide.  Application integration has similar processes as software development. Well done process definition carefully designs inputs, actions and outputs of each step that process must take to achieve its goal.  In addition ICC defines also roles, responsibilities and tasks that are required to execute those steps. Nothing shall fall in the gap between these steps or roles, not even the test material. As underground in London so kindly reminds us: mind the gap.