I attended the recent Round Table event hosted by Re-Think Recruitment.
Simon Reekie from www.laterooms.com led the discussion where those who gathered for the early evening event got stuck into the issues surrounding the increasing array of channels and devices over which organisations communicate with their users and customers; in turn we looked at how to go about testing in an increasingly complex and fast-moving environment.
Increasingly development for such a dynamic environment has to make compromises such as the trade-off between providing a unique experience using device specific features; and providing a consistent experience across all devices.
There are also challenges around:
- the test environments required to do a test
- test data requirements
- increasing use of external calls to web-service providers leading to increasing use of subbing
Some interesting options emerged, which in combination may make some in-roads to a strategy. The 6 emerging themes were:
- Use of analytics to identify the mix of end-user devices
adopt a risk based approach where you test the 20% of devices that account for 80% of the workload
- Focus on the core activities, check on each of the main devices and analyse any differences
- Rely on compliance to standards; if a device complies to a standard, and the application is written to comply to that standard then arguably we can infer all devices of that standard will comply
- Consider extending the testing remit to include feedback from early-adopters; is the risk to image by going with a product that isn't considered fully ready worth it, or is it better to keep testing to try and remove all defects prior to release (some would argue this is impossible, or at the very least subject to diminishing returns for effort expended)
- Look at the dividing line between front-end and back-end and how this differs from channel-to-channel and device-to-device; this can help understand the differences and commonalities and hence steer test strategy
- Do a soft-launch to a sub-brand that enables any poor feedback to have limited brand impact, thus avoiding impact on the premium brand
Discussion was good and many continued the debate in Brown's bar which was a short walk away.
Incidentally, for the pedants reading the article, the table was actually rectangular - a point that was spotted as a non-compliance by at least a couple of attendees.