User test process

User tests were conducted for the client, Riksantikvarieämbetet. Due to the Covid-19 pandemic, the setup was planned, structured, and scoped for fully digital and accessible testing in compliance with new restrictions. The process was documented for the client and presented to co-workers.

What to test?

Riksantikvarieämbetet decided on two specific scenarios on which to test, based on two different ways/scenarios of searching for information.

Where do you find baptismal fonts with rune inscriptions from the 10th-11th century?
Search for scripts with the name “Holm” within Vallentuna district.

Who to test?

Riksantikvarieämbetet had original contact with the test participants (researchers) and once participation was confirmed, contact details were provided. The original plan involved in-person testing in Uppsala and Stockholm, but with limited time and the project nearing completion, only three participants were included. Due to Covid-19, testing shifted to a digital format, which could have allowed for more participants since setup was only needed once. This was a key takeaway when reviewing results.

Softwares

The client was consulted regarding software preferences but ultimately allowed flexibility in tool selection. While other prototyping tools offer more advanced features, Sketch and InVision were chosen based on the client’s preference due to maintain consistency with previous projects.

Typeform was used for online questionnaires, and Lookback was utilized for recording user test sessions.

Prototypes

Prototyping allows for early testing and validation of design concepts, helping to identify usability issues before development. It facilitates collaboration, ensures alignment with user needs, and saves time and resources by reducing costly revisions later in the process.

In the project, the client selected the flows for testing, reviewed them after the prototypes were created, and approved them before user testing began.

Online questionnaire

Typeform was used for additional questions, providing a user-friendly and clean questionnaire format. Images were added for clarity, and a mix of question types ensured a broader range of insights. Users also had the option to provide additional information.

Information

An instructive email was sent to test participants, including:

Estimated test duration
Instructions for the client’s two test scenarios
Contact information
Link to the questionnaire
Links to the prototyped scenarios

The prototypes were connected to Lookback for recording, with Typeform used for the questionnaire.

Contact

The email included a deadline for test completion, set for about a week later. Once all tests and questionnaires were submitted, analysis began by reviewing recordings. With three participants, user movements, thoughts, and expressions were transcribed. Participants were also asked to verbalize their steps and reasoning during the test.

Analyze recordings

Observing each step of the tests revealed clear patterns in user challenges and smooth interactions. Summarizing individual scenarios made it easier to compile the final paper-based documentation for the client.

Analyze questionnaire answers

The software automatically compiled questionnaire data, including average ratings, making summarization straightforward. Additional insights and comments were then gathered as supplementary information.

Summarize & document

The results, transcriptions, questionnaire answers, and reflections on the outcome were documented, covering the following areas:

Introduction
Purpose and goal
About LH+P
Covid-19 disclaimer (changes of conducting user test)
How the user tests were done
Test result
Summary
Limitations & work flow
Next steps?
Appendix