I have read and heard about a ‘shift left’ in testing, however I was a little unsure of its impact on performance testing. In an attempt to fill my knowledge void I spoke to two of my colleagues, Richard and James, to get their views on the topic.
What is shift left?
When talking to Richard, he defined ‘shift left’ as testing earlier and ideally more frequently within the software development lifecycle i.e. Testing in the left side of a project Gantt chart, rather than “right” at the end. With earlier testing, bugs can be found sooner. This in turn means that they are cheaper to fix. When done properly, application development methods using “shift-left” techniques can potentially offer a faster return on investment by reducing the time between software releases. James echoed Richard’s sentiments about identifying issues earlier and the cost ramifications of doing so whilst also touching upon the idea of developer’s performance testing their own code.
How does shift left affect performance testing?
Richard argued that agile development practices often mean that formal performance testing is overlooked and is often done as an afterthought. He talked about developer’s tendencies to “forward fix” performance related problems and identified the increased risks to applications and ultimately businesses who follow this this approach. As the chart below demonstrates, fixing bugs in production or late in the development cycle is expensive. The antidote to finding bugs late is to test early! He was keen to suggest this is as true for performance related bugs as it is for functional defects.
James explained that using a “shift-left” approach means that software can be tested before it has been completely developed. He discussed how performance test tools (both open and closed source) are perfectly capable of running tests against incomplete components, although some inexperienced testers don't always realise this. Whilst there is still a danger that you'll have to re-work your tests as the application nears completion, the earlier interaction between both testers and developers may help the application to be built correctly
Both testers concluded that testing should be done sooner in the application lifecycle.
What tools are developers using at the moment?
James has a strong preference for open source tools and he was able to give an insight into the tools currently being used by developers to test their code. He reeled off a list of open source tools including WebTest, FunkLoad and The Grinder, however his preferred tool was Gatling. As a developer, James said the feel of this tool was one which suits developers. He also elaborated on "point-and-click testing" open source tools such as JMeter. Whilst he likes JMeter, he likened some point and click tools such as this to a crutch; “they’ll help you walk, but they won't help you run”.
It’s this point which made Richard suggest that developers should use “best of breed” test tools, such as HP LoadRunner allowing the use of a common toolset throughout the testing lifecycle. Both Richard and James stated that good testers should be able to test applications before they’re finished. As an example, Richard described how developers could use HP LoadRunner to test standalone web components. By using a more advanced tool like LoadRunner compared to a tool such as JMeter, Richard argued that developers get better access to analytics and in-test performance monitoring to truly understand the performance impact of changes that they make.
Could HP LoadRunner in the Cloud be suited to ‘shift left’ development?
James explained that when LoadRunner was first released, it felt a lot like the tool developers were using it at the time. However software development tools and techniques have changed a lot since then (for example, few developers use C nowadays). Cheaper and open source alternatives to LoadRunner are available and are well worth a try.
This may be the case but Richard still believes that LoadRunner is an ideal tool for developers as well as specialist testers. The new LoadRunner “Community Edition” license which allows 50 vUser tests with no license cost, compares favourably with open-source pricing. Using the Community Edition together with an offering like our low-cost HP LoadRunner in the Cloud allows ad-hoc developer-driven testing with occasional bursts up to higher test volumes using industry-leading tools. As discussed before, LoadRunner isn’t always as expensive as you think, and as the industry leading test tool it should have a place in both your tester’s and developer’s toolkits. Early testing and promoting collaboration between test and development teams can only be beneficial if your goal is to produce better quality software more quickly.
We want to hear from you!
But don’t just take Richard’s word for it, if you’d like to try LRITC, just get in touch and we can demonstrate the versatility and low cost of our cloud-based performance test solution. If you’re still an open-source aficionado, tell us and we’ll keep you informed when James publishes more of his open-source themed blog posts.