‘Involving zero users in service design leads to zero insights’

Creating user centric services isn’t possible without  involving users. “Involving users in service design should be as normal as brushing your teeth”, says Maureen Wijsman-de Hond. She’s responsible for the municipality of Rotterdam’s UX-team, which she built from scratch. Maureen shares Rotterdam’s user testing journey and gives valuable insights into the necessity of user testing.

Maureen Wijsmand-de Hond

She fell in love with user testing during the User Needs First conference of 2018. An inspiring session from Auke Molendijk and a pursuit to get closer to the citizens of Rotterdam led Maureen to set up Rotterdam’s UX lab. “I wanted to be more in touch with citizens. I’m convinced that you can only create good government services together with users. And having direct contact with the people of Rotterdam helps me to see what impact my work has.”

The first step towards Rotterdam’s UX team

With this in mind, Maureen started experimenting with user testing. She for instance used eye-tracking technology with webcams to test websites with users. What parts of the website do users look at most? What parts aren’t looked at all or barely? What can be learned from where users navigate to with their eyes? “This led to software developers gaining valuable insights into how they should develop websites and applications. They acquired many new user stories to work with. This helped them improve applications.”

User testing needs to be experienced

The valuable findings from such experiments have convinced more and more colleagues in Rotterdam of the importance of involving users. It’s when they see the impact it has firsthand, that they begin to understand why user testing is such an essential component in service design. “That’s one of the challenges we always face with user testing. It requires time, effort and money to set it up and conduct it properly. And that needs some convincing of stakeholders. But it’s always when they see the results of user testing with their own eyes that they realize why it’s so important and worth investing in. Besides, a well designed and tested service also leads to cost reduction. There’s also no need for re-work, fewer complaints, and many other benefits.”

Getting this message across can be challenging for UX-professionals. In Rotterdam, the message is gaining momentum. What started out as Maureen single-handedly experimenting in ‘her’ UX lab, has grown into a professional UX team of 8 colleagues. “The team consists of both researchers and designers. This mix of people helps us look at users and government services from different angles. Colleagues of the municipality are becoming increasingly aware of our existence and our value. We’re now researching, designing and testing all over the organization. We do this both digitally and in person, at one of our offices, public locations, and in the homes of citizens.”

Low-hanging fruit

There’s also often a misconception that interventions with user testing are complicated. Often, the most impactful improvements can be made with simple changes. Maureen gives a few examples. “One intervention that comes to mind is that of letters to citizens about lead pollution in certain areas of the city (in people’s gardens). The letter introduced the topic and explained what was going on and then continued to inform people on whether they should take precautions or be more aware. For instance, if they had to be careful with gardening fruits and vegetables or making sure their children didn’t play with soil from the garden.”

The problem however was that people got worried when reading the letter. “For a lot of people, there was nothing to worry about and they didn’t need to change anything in their daily routine. But because of the way we introduced the topic and focused on informing the people about the health risks of lead pollution, we needlessly had people worried. All we really had to say first was: ‘You have nothing to worry about.’ That’s what people wanted to hear. And if they did have to be more aware, only then should we have explained the situation more in-depth.”

Testing with vulnerable citizens

Another example is user testing with “vulnerable citizens”. “We excel at involving specific groups in service design, like people with limited sight or who are completely blind, deaf people, individuals with cognitive limitations, or physically handicapped people, to name a few.”

For example, the UX team experimented with their chat functionality on the website. A group of blind individuals and people with limited sight were recruited to test how accessible the chat functionality was. The result: it wasn’t. And the solution was simple. “Visually impaired people often navigate through websites with a screen reader. We made the mistake of labelling the chat functionality as ‘button’. So in the front-end, it looked just fine, but the back-end was flawed. And the screen reader picked up on that, reading out the chat button just as ‘button’ to our users, which made it useless. We only discovered this once we tested with these users. The solution was a simple change of the label from ‘button’ to ‘Chat with us here’. But however simple this solution seemed, it helped thousands of people in Rotterdam.”

Maureen believes that recruiting vulnerable users is essential for proper service design. “If you design services for people who struggle with them the most, you can be sure other people can use these services too. That’s why you need to develop and test services with these groups. Furthermore, testing with different users also enriches your own take on service design. I learn something from citizens each day. It makes me more human.”

In the Netherlands, there are many foundations and organizations with ambassadors representing different vulnerable citizen groups. “Other countries often also have similar organizations, but not always. My message would be to find out which groups, in whatever form they come, exist and reach out to them. That’s where you find representative individuals for UX research .”

A handful is enough

And if you think you need hundreds or even thousands of users to test a service or tool, think again. As few as 5 people can do the trick. For those of you who want to know more: The Nielsen Group perfectly explains why 5 qualitative user tests can be enough to properly test a service or tool. “Thus, it’s not too hard to find enough users. And these can sometimes be recruited easily through colleagues or family members. A lot of us have an uncle, parents, siblings or grandparents living in the city you’re testing for.”

Maureen does stress that each service should be tested with different groups and not with the same citizens or a fixed panel of users. “You want to prevent that you get test bias. So you need to recruit different users for each test. Not all testers are acquainted with or relevant for all services you’re testing for.” A good example of tester bias: if a tester moves from Hawaii and is given a test asking them to describe a snowy winter, the test would be biased because the student lacks any experience with snowy winters.”

Learning from Rotterdam

Maureen is more than willing to help. She is keen to share her entire experience with anybody seeking to start with or expand user testing for instance. “One thing is clear: involving zero users leads to zero insights. We’ve seen firsthand how excited people become when they see the results of UX research, design and testing. And we can share the knowledge we’ve gained in Rotterdam. Reach out to us and we can tell you how to go about involving users when developing or improving public services. This is a subject that needs to be taking seriously. And you can start small, even without having all the knowledge beforehand. The most important tip I can give you, is to just start”, Maureen closes.