Okay, we’re going to play a quick game of yes and no. You know the game, I ask a question and you answer with either yes or no.
When engaging with a client who wishes to have their product(s) performance tested, do they actually understand what performance testing means and the effort that it involves?
Does the client fully understand the business risks it is trying to mitigate through performance testing?
Does the client have an expectation as to the depth of testing and the deliverables that will be presented during and upon completion of testing activities?
Does the client have the appropriate timeframe and budget put aside for testing?
Now, if the answer to all of the above is YES, then you’re either in a great starting position to draft your high level test plan and quotation for the work OR you’re in bed, dreaming (probably the latter).
How can we help you understand performance testing?
In most, if not all engagements I’ve been involved in, the answers to most of the above questions have been a resounding “NO”. Performance testing whether intentional or not, is rarely given the consideration it should in comparison to functional testing. When it is considered, it’s often too late in the project lifecycle to make the difference it should.
While we cannot magically make prospective clients suddenly understand and think about performance testing earlier in the product life cycle, we can assist them to make the best decisions once they do. This benefits them and us, so we’re not quoting for (or putting in) effort in areas where it isn't required or indeed wanted.
As a consultancy in the field of testing, we are up against domestic competitors as well as offshore competitors. The latter appear (on paper anyway) a much more cost-effective option. As a result, we need to provide the best service possible, in the quickest timeframe and demonstrate exactly what the customer is getting for their money.
At the earliest opportunity, we must ensure that the potential client’s knowledge of performance testing is sufficient in order for them to be able to make informed and educated decisions about not only the level of testing they require, but the management to drive the testing/project forward. We ask the following questions at the start of each engagement.
Q: What risks are the business looking to mitigate by performance testing?
This informs our high level test approach
Q: Does the business have long terms plans for performance testing or is this a “one-off” exercise?
This also informs the high level test approach
Q: Does the client want a ‘quick and nasty’ testing exercise in order to simply prove an application will perform?
This helps to determine the level of testing, potential management and details of deliverables that could be supplied.
Q: What In-House test tools (RAID, Bug Tracking) if any, does the client have?
This has a bearing on the management and effort required to control the project.
In most cases, we find that potential clients need to be educated about performance testing.
We need to help them understand the complexities of performance testing and explain what performance testing can and can’t tell them about their application or project.
As a consultancy we need to forewarn them about the likely problems/snags/bugs that we run into with each and dare I say, every project.
By clarifying this information, it doesn’t just save a lot of confusion and misunderstanding when we start work, but also helps to keep overall project costs down.