Last Friday saw the release of Gatling 2.0.0-RC1. Gatling is a relatively new open-source performance testing framework. We've been trialing it on a project for a client, keeping a close eye on its development progress, and even contributing a few bug fixes and improvements.
So what can you expect from Gatling 2.0?
Stuff that's been there since 1.5
- An easy-to-write Domain Specific Language for writing tests...
Gatling's scripting langugage is designed to be easy to read and easy to write. Here's a simple script:
scenario("Look at Prices") .exec( http("Request 1: Our Prices") .get("/our-prices.html") .check( css("h1").is("Our prices") )
And they even provide a handy cheat sheet.
- ...But it's really just Scala under the hood
Gatling's DSL is really just Scala code. You don't need to know Scala to use Gatling (they chose a small subset that's enough for most testing needs), but if you do, then you can do anything in Gatling that you can do in Scala.
This can come in handy for mixed-ability teams. Real world applications often need bespoke code to handle project-specific quirks. This way, your senior testers can handle the bespoke Scala code, and your junior testers can handle the rest in the DSL.
- Lightweight concurrency based on Akka
In most performance tools, virtual users are expensive. If you need to do 1000 things an hour, it uses fewer resources to have 10 users doing 100 things an hour, than 1000 users doing 1 thing an hour.
In Gatling, users are very lightweight, thanks to its use of Akka for concurrency, and there's no difference between these two. You can design your tests around how the system will be used, not what your test framework can handle.
- Higly scalable HTTP support
Gatling uses asynchronous HTTP calls for maximum scalability, and supports all the usual web technologies - HTML, JSON, XML, CSS selectors, XPath, etc.
- Pull in test data from external sources
Gatling can extract test data from CSV files, but it can also pull data straight out of your database, or for even more advanced data flow scenarios, from Redis.
Stuff That's New in 2.0
- Improved browser emulation
Gatling 2.0 significantly beefs up the browser emulation in its HTTP support. It now supports downloading linked resources in HTML documents, and can emulate the connection, cookie and caching behaviour of most common browsers.
- Create test scripts from HAR files
Gatling 2.0 can now take a HAR file, and generate a test script from it.
New protocols - WebSockets and MQ/JMS
Gatling 2.0 includes support for WebSockets and MQ.
Gatling 2.0's graphing capabilities have been improved, with their new interactive hybrid time-percentile graphs (some example graphs are shown in this blog post - minus interactivity, unfortunately).
Create Detailed, up-to-date documentation
The Gatling team have put a lot of effort into the documentation for Gatling 2.0 (high quality documentation was a key milestone for the RC1 release), which you can read online.
Create New load injection DSL
Want to start by ramping your users up to 10 iterations per second over 30 minutes, then ramp up to 30 users over 5 minutes, then hold steady for 30 minutes? No problem. Fun fact: if you use one of the
randomized profiles, you're running my code!
IDE integration, via Maven and SBT plugins
Gatling now has high quality plugins for both Maven and SBT. If you're using an IDE with built-in support for Maven or SBT, like IntelliJ, then you get IDE integration for free.
Gatling 2.0 can output test results to Graphite, a real-time graphing framework, for real-time graphing of test results.
What's coming up in the future?
- One highly requested feature for Gatling is support for distributed (controller-injector style) testing. The Gatling team have held off on it until now, since Gatling can inject more load from a single box than most other frameworks, and they provide instructions for duck-tape distributed testing for those who really need it. But, distributed testing's in the works, and should certainly be possible - many of the supporting components, like Akka, are designed with distributed computing in mind. Distributed testing was de-scoped from 2.0, but is expected to be a major focus of 2.1.
- In Gatling 2.0, IDE integration is optional. So it's possible to download a version of Gatling that has IDE integration (via Maven or SBT), and a separate version that just runs from the console. For 2.1, they plan to move towards an integrated model, similar to Play!, with a single version that's usable from the console, from SBT, or from an IDE.
- There are rumours that a more sophisticated results analysis tool may be in the works for Gatling - the standard reporting is nice, but there's still some stuff that we have to do with bespoke scripts, like comparing test runs. Details on this are sketchy, and there may even be more than one group working on analysis tools.
- The Gatling team are keen to add support for new protocols - ServerSentEvents, LDAP, JDBC and TCP are high on their hit list. There's no roadmap for these, and in all likelihood, these will be added when someone with the skills to implement them needs them.
What's probably not coming up?
A few performance testing tools (including LoadRunner in the closed-source space, and WebTest in the open-source space) feature full browser emulation. In principle, this wouldn't be that hard to support in Gatling, via HtmlUnit or Java Chromium Embedded. However, Gatling's core is optimised for scalability, scalability, and scalability. Full browser emulation doesn't scale so well, so probably won't make it into Gatling core. Still, never say never - Gatling's easily extensible, and someone might just come along and write a plugin.
As a small step in this direction, we developed a small Gatling plugin for our client, which allows UI-based scripting on complex form-based pages. This allowed the client to simplify some of their test scripts.
Can I try it out?
If you want to try it out, you can either download the console-only version from the official site. Or, if you want to use it from an IDE, you can clone this GitHub repository to get started with Maven, or follow these instructions to get started with SBT. If you come across any bugs, issues are tracked on GitHub.
I'd like to thank the Gatling team for all their hard work on Gatling 2.0. Apologies if I've missed any really cool features, or mis-represented anyone's contribution.