Juha Suomalainen // September 26 2018

Continuous UX research as part of product development process

As a user experience specialist and a researcher, I have noticed that many companies see user research and usability testing as separate functions from the actual development process. Companies may want to conduct one or more separate rounds of user tests either with a ready-made product or a high-fidelity prototype, but do not see UX research as a similar tool as, e.g., quality assurance. Testing in a rather late phase of the product development process may lead into a situation where the client argues that it is too late or too expensive to implement the improvements suggested.

Luckily this is not always the case. I think that it is not too exaggerated to say that I was privileged to join a project with Nordea bank where UX research was integrated at an early phase of the product development process – I worked closely together with the designers, product owners, marketing, business development, and even customer service, to develop a new product for the customers

A major advantage in having UX research integrated to the product development process is that the researchers can gather a wide knowledge on the user needs and the product or service itself.  User feedback and usability problems collected and analyzed during the whole development process not only validate that you are actually heading in the right direction, but also reduce the possibility of having surprises and costly changes at the end of the project.

What I personally like in continuous projects is that people actually get to know each other. It was great to see that we could not just deliver the findings from the tests, but also it felt like that we could unite the silos inside the customer’s organization. This enabled us to utilize the findings from previous test rounds in evaluating the current design. This is only possible if there is a factor of trust between the development team and researchers.

How did we do it?

In this particular project we conducted frequent light evaluations. Typically, it was two days of testing with seven test users and a debriefing workshop with the customer on the second day to discuss on the findings. Based on this experience I recommend having tests every other week, which lends more time for prototyping and test planning.

Our UX tests are usually performed in our purpose-built UX labs in Helsinki, but can be conducted, e.g., at client premises as well. Client stakeholders are invited to observe the tests in our facilities but can observe the tests basically anywhere – even on a mobile device using our Digitalist live stream. In this case we offered the possibility to observe the tests through the live stream and join the debriefing workshop via Skype, since some client stakeholders were located in Copenhagen and Oulu. To be honest, I was truly surprised how little this option was used.

In agile projects, it is highly recommendable that at least the core team of designers and developers are following the tests in the observation room as it allows for active discussion already during the test day. The main findings can be discussed at the end of each test session, which gives the designers and developers a possibility to get back to work immediately – there is no need to wait for a report to be delivered.

Why is it beneficial to have a partner to conduct UX research

The main thing that this case also effectively demonstrates is that UX research does not have to be a separate function but rather it should be integrated into the current development process. This does not interfere with the current way of working but supports it.


Feedback from the client

As a long-time customer of Digitalist in using their usability testing and user research facilities and services in Helsinki I am happy to report that we were very satisfied with all aspects of the cooperation.

Nordea Digital used Digitalist as a mid-term solution to shift from irregular, need-based testing to a continuous, in-house test setup. The arrangements and service kick-off were flexible and fast, which, in turn, allowed us to start getting feedback within a couple of weeks after the green light was given by our management.

The shift from practically no testing to regular testing took some effort internally to set up. Designers and POs, while supportive of testing, suddenly faced a new situation which took some getting used to. In the beginning, everyone wanted to have the opportunity to watch tests and get a briefing on the results, but few realized they need to set aside the time to participate. This was a new way of working for us. You can only be engaged if you are ready to engage.

What I found most surprising in the process was the commitment and integration of Digitalist staff into our development. While retaining a neutral position in terms of user feedback, they actively suggested improvements and solutions to discovered issues which helped our designers take their design to the next level.

Another surprise was the effectiveness and clock speed of the setup: we were able to run weekly testing with several work streams getting all of their testing needs met. By maintaining a small backlog of tests, we always had something waiting to be tested in case of a delay in a particular stream, and so no tests had to be cancelled throughout the entire test period.

Looking back at the testing experience, it was a very effective and efficient process. The investments made in early-stage testing have very tangibly translated into better design and improved user satisfaction. The standards for service and test quality in the Digitalist setup are our elusive benchmarks in setting up internal testing.

Panu Kanervo, Creative Director, Service and Strategic Design at Nordea

By Juha Suomalainen, Design Researcher, User Insight & Vision