Results and the next step
Objectives
By the end of the lesson, you can relate the results to the success criterion fixed in advance, you can word them in proportion to the strength of the evidence, and at the end of the project you can define a next step that reduces a real uncertainty.
Why is it hard to word a result well?
At the end of the work there are two opposing temptations. One: overrating the result ("we solved the problem"). The other: downplaying it ("it was only a small test, it is not worth saying anything"). A good report runs between the two: it states what we saw, what we conclude from it, what we do not know, and what the next step should be.
What does a result mean?
For example: "In the second round, three participants correctly said, after the selection, that the registration was not yet final." This is an observation. "The modified status and button label probably reduced the misunderstanding in the examined situation." This is a conclusion. Do not say that the problem is solved for all users if you had only a small prototype test.
Results table
| Success criterion |
Round 1 |
Round 2 |
Change |
Strength of evidence |
| The participant says whether the registration is final |
1 / 4 |
3 / 3 |
Improved |
3 people; indicative |
| Finds a suitable event within two minutes |
2 / 4 |
3 / 3 |
Improved |
3 people; indicative |
| Correctly interprets the exceptional (faulty form) state |
– |
1 / 3 |
Open |
Not tested enough |
The last column is important because numbers can be misleading on a small sample: "1 / 4" and "3 / 3" are not percentages, but observations.
Result and limitation together
A research limitation is a condition that restricts the scope of validity or the interpretation of the result. A credible report does not hide the limitation. It may be that the participants already knew the field; the prototype did not contain real payment; the test took place in a controlled situation; or you examined only one device size. These do not invalidate the learning, but specify what it refers to.
Also write down what did not change or what got worse. A new explanation can improve first use, but slow down the routine task. The goal is not a perfect, flawless story, but that there be an honest basis for the next decision.
A list of typical limitations from which you can choose:
- small sample (3–5 people),
- the participants are acquaintances or experienced in the topic,
- a prototype, not a live system,
- a controlled situation, not a real environment,
- a single device or screen size,
- untested exceptional states.
Choosing the next step
The next step should not be "further development", but tied to a concrete question. A few patterns:
| Situation |
Suggested next step |
| Content comprehension is uncertain |
New research or a content test |
| The prototype works, but the technical constraint is unknown |
A feasibility study with the developer |
| You did not test a high-risk exceptional state |
Prototyping and testing it |
| The result is confirmed on only one device |
A trial on another device size |
When prioritizing, look at the user impact, the uncertainty, and the cost of the change. Not every idea is a next step. An open question is important if, with a wrong answer, the solution or the risk of the service would change significantly.
Worked example: the second round of Mai program
In the second test round of a prototype, the main registration task is successful, but participants understand differently what happens when a payment fails. The result is therefore not "the flow is done", but: selection and finalization became more understandable, and the recovery from a payment error is still uncertain. The next step is to prototype the error state and test it in a new task, not to add new features.
Common misconceptions
| Claim |
Clarification |
| "The result is that the new version is ready." |
The result is the change that occurred in the user task. |
| "A limitation weakens the report." |
Stating the limitation makes it more credible, and shows what the result is valid for. |
| "The next step is always a new feature." |
The next step is examining the question that reduces the greatest uncertainty. |
In brief
- The result is the change measured against the success criterion in the user task, not the existence of a new version.
- Write the observation and the conclusion separately; indicate the strength of the evidence.
- The research limitation is part of the result, not an excuse.
- The next step reduces a concrete uncertainty.
Review questions
- For which of your success criteria do you have a direct observation?
- What limitation must you state next to the result?
- What is your biggest open question, and why does it matter?
- What would be the next proportionate research or testing step?
Results and the next step
Objectives
By the end of the lesson, you can relate the results to the success criterion fixed in advance, you can word them in proportion to the strength of the evidence, and at the end of the project you can define a next step that reduces a real uncertainty.
Why is it hard to word a result well?
At the end of the work there are two opposing temptations. One: overrating the result ("we solved the problem"). The other: downplaying it ("it was only a small test, it is not worth saying anything"). A good report runs between the two: it states what we saw, what we conclude from it, what we do not know, and what the next step should be.
What does a result mean?
For example: "In the second round, three participants correctly said, after the selection, that the registration was not yet final." This is an observation. "The modified status and button label probably reduced the misunderstanding in the examined situation." This is a conclusion. Do not say that the problem is solved for all users if you had only a small prototype test.
Results table
The last column is important because numbers can be misleading on a small sample: "1 / 4" and "3 / 3" are not percentages, but observations.
Result and limitation together
A research limitation is a condition that restricts the scope of validity or the interpretation of the result. A credible report does not hide the limitation. It may be that the participants already knew the field; the prototype did not contain real payment; the test took place in a controlled situation; or you examined only one device size. These do not invalidate the learning, but specify what it refers to.
Also write down what did not change or what got worse. A new explanation can improve first use, but slow down the routine task. The goal is not a perfect, flawless story, but that there be an honest basis for the next decision.
A list of typical limitations from which you can choose:
Choosing the next step
The next step should not be "further development", but tied to a concrete question. A few patterns:
When prioritizing, look at the user impact, the uncertainty, and the cost of the change. Not every idea is a next step. An open question is important if, with a wrong answer, the solution or the risk of the service would change significantly.
Worked example: the second round of Mai program
In the second test round of a prototype, the main registration task is successful, but participants understand differently what happens when a payment fails. The result is therefore not "the flow is done", but: selection and finalization became more understandable, and the recovery from a payment error is still uncertain. The next step is to prototype the error state and test it in a new task, not to add new features.
Common misconceptions
In brief
Review questions