By the end of the lesson, you can plan a usability test with a clear learning goal, choose participants who are relevant for the examined task, and run the test so that the participant's safety, time, and dignity are preserved.
Why is testing a separate chapter?
The research so far (chapter 2) examined how people solve the problem today. A usability test examines whether our solution works. This is a different question, requires a different method, and can find different errors. An interview tells what people want; a test tells whether they are able to use what we built. The two complement each other, but do not replace one another.
What does a usability test examine?
A usability test is the observation of users performing tasks on a product or prototype. In chapter 7 you ran an informal trial; the test is its planned, structured version. Usability testing is a broader field: planning, running, and interpreting the testing.
Learning goal
Before the test, formulate one or two learning goals. The goal should be in the language of the problem and the hypothesis:
Type
Example
Assessment
Too broad
"Let's test the app."
You do not know why any observation is important
Too narrow
"Do they find the blue button?"
It checks one solution, not the task
Good
"Do the participants understand when the registration is final?"
Task and understanding
Good
"Is the information needed to choose the right event visible at the place of decision?"
Task and information
Who should be a participant?
You do not need to find a perfectly representative sample, but the participant must resemble the target user from the point of view of the task. If you design for people handling administration for the first time, do not test only those who already use the system routinely. If the device or environment is important (for example mobile use on the go), pay attention to that too.
In a small student project, three to five relevant tests can surface many problems, especially if the same error repeats. This is not statistical proof. Formulate the result like this: "of the participants examined, three got stuck", not "all users will do this".
A useful filter: write down in advance who you do not consider relevant. Friends and family members are very polite and know a lot about your project; if possible, they should not be the only participants.
Consent and safety
Informed consent is informed, voluntary agreement to participation and data handling. Even for a short test, say:
what will happen (they get tasks, they can think aloud);
what notes or recordings are made, who sees them, how long you keep them;
that you are testing the design, not them;
that they can stop at any time, skip a question, and that this will bring them no disadvantage.
Do not ask for a live password, a personal identifier, bank data, or information that is not necessary for the task. If you record, do so only with clear permission, and define the retention period. Do not promise a reward or benefit that puts pressure on participation. The participant's feedback is valuable, but they have no duty to be "useful" or to give a positive opinion.
A short sample consent text
"Thank you for helping! We are trying out a prototype, that is, we are testing the design, not you. There is no right or wrong answer. I will take notes, I will not make a recording. Your name will not appear in the notes. You can stop at any time, or skip anything you do not want to do. Is that all right?"
Organizing the test: the main steps
Learning goal (1–2).
Participant description and recruiting.
Tasks and scenario (chapter 7; task writing is detailed in the next lesson).
Environment: where, on what device; does the prototype work?
Pilot test with a helper, to find out whether the tasks are understandable and the prototype works.
The actual tests and note-taking.
Analysis: findings, severity (see the third lesson of the chapter).
Worked example: the test of Mai program
A student tests the registration flow of Mai program. Learning goal: does the participant recognize which events require advance registration, and do they know whether the registration was successful? Participant: a student who has already attended a university event but does not routinely browse such sites. Before the test it is said: "There is no right or wrong answer; I would like to understand the design. We can take a break at any time." Then, instead of listing the functions, they give a situation ("On Friday evening you want to go somewhere...").
Common misconceptions
Claim
Clarification
"My friends are enough as participants."
They can be relevant if they are in a similar situation from the point of view of the task; goodwill does not replace relevance.
"A firm decision comes from three tests."
Three tests can give strong signals, but the uncertainty and the limit of the sample must be documented.
"Consent is just a legal formality."
It is the basis of trust and safety, which also enables better, more honest observation.
In brief
A usability test examines the operation of our solution, research examines the problem; the two complement each other.
The learning goal is 1–2, in the language of the problem and the hypothesis.
The participant should be relevant to the task; document the limit even with a small sample.
Consent and safety are a basic condition: we test the design, not the participant.
Review questions
What are the two concrete things you would like to learn from your test?
What would make a participant relevant to your project?
What data must you not ask for in the test?
How would you formulate the limit of the sample in the report?
Test goal, participant, and ethics
Objectives
By the end of the lesson, you can plan a usability test with a clear learning goal, choose participants who are relevant for the examined task, and run the test so that the participant's safety, time, and dignity are preserved.
Why is testing a separate chapter?
The research so far (chapter 2) examined how people solve the problem today. A usability test examines whether our solution works. This is a different question, requires a different method, and can find different errors. An interview tells what people want; a test tells whether they are able to use what we built. The two complement each other, but do not replace one another.
What does a usability test examine?
A usability test is the observation of users performing tasks on a product or prototype. In chapter 7 you ran an informal trial; the test is its planned, structured version. Usability testing is a broader field: planning, running, and interpreting the testing.
Learning goal
Before the test, formulate one or two learning goals. The goal should be in the language of the problem and the hypothesis:
Who should be a participant?
You do not need to find a perfectly representative sample, but the participant must resemble the target user from the point of view of the task. If you design for people handling administration for the first time, do not test only those who already use the system routinely. If the device or environment is important (for example mobile use on the go), pay attention to that too.
In a small student project, three to five relevant tests can surface many problems, especially if the same error repeats. This is not statistical proof. Formulate the result like this: "of the participants examined, three got stuck", not "all users will do this".
A useful filter: write down in advance who you do not consider relevant. Friends and family members are very polite and know a lot about your project; if possible, they should not be the only participants.
Consent and safety
Informed consent is informed, voluntary agreement to participation and data handling. Even for a short test, say:
Do not ask for a live password, a personal identifier, bank data, or information that is not necessary for the task. If you record, do so only with clear permission, and define the retention period. Do not promise a reward or benefit that puts pressure on participation. The participant's feedback is valuable, but they have no duty to be "useful" or to give a positive opinion.
A short sample consent text
Organizing the test: the main steps
Worked example: the test of Mai program
A student tests the registration flow of Mai program. Learning goal: does the participant recognize which events require advance registration, and do they know whether the registration was successful? Participant: a student who has already attended a university event but does not routinely browse such sites. Before the test it is said: "There is no right or wrong answer; I would like to understand the design. We can take a break at any time." Then, instead of listing the functions, they give a situation ("On Friday evening you want to go somewhere...").
Common misconceptions
In brief
Review questions