Filed under: BDD, — Tags: Example mapping, Three amigos — Thomas Sundberg — 2017-12-20
What is the unknown unknown?
The unknown unknown are the things, sometimes problems, of the task we are trying to achieve that no one has thought of. It is on the level that nobody is even aware about them, therefore the name unknown unknown. We don’t even know that we have a problem in this area. This is where the nasty bugs live and thrive in a software system.
Traditionally we have had specifications written by business analysts. They work hard to come up with everything that should work. They specify this knowledge in a written document. Sometimes it takes a very long time to come up with everything needed and to write it down. I have recently, Autumn 2017, heard about specifications that has been worked on for more than a year. And they aren't done yet.
This specification is then treated as the holy truth when given to the development team. Changes are usually not welcome. Changes are especially unwelcome in a fixed price context. Someone has offered a solution based on the specification. Changing the specification probably changes what should be delivered. That in turn changes the amount of work that has to be done.
The developers read and do their best to understand the specification. If there is anything unclear, they might be able to ask the business analyst. It happens that they are not able to ask any questions and therefore has to interpret the specification to the best of their knowledge.
The testers also read the specification and interpret it. They come up with a test plan to verify the specified behaviour. This test plan is sometimes shared with the developers. But it happens that it isn't. In the cases where the test plan is kept secret from the developers, the result is sometimes very surprising for both developers and testers.
It is an interesting exercise to learn in how many ways a written document can be interpreted.
The perspectives of all different categories are brought together when the development is done and there is something to verify. The tester may find out that they interpreted the specification differently compared to the developers. The tests that the tester has come up with do not pass. Is this a bug or a feature? It is sometimes unclear and a blame game starts.
The business analyst will, hopefully, see the result but may not even recognise the specification. It turns out that the different views of what should have been created is very different.
To make this even worse, this misfit is usually not found until very late in a project. Probably close to delivery date. The result is a lot of rework, stress, and missed deadlines. Nobody is happy and everyone blames the others.
The business analyst are unhappy with the developers and the testers who can't understand a simple specification.
The developers are irritated with the business analysts who can't clearly specify what they want without a lot of ambiguity.
The tester are unhappy with both the business analyst and the developers. The business analysts because they are ambiguous and unclear. The developers because they are stupid and can't even understand the simplest requirement.
From my perspective, there are a few different problems in this situation.
The communication media commonly used is written text. Written text is good for sharing the same message to many people. But written communication is a monologue. It is bad for a dialog where a group of people are discussing a topic.
As a side note, this text is an example of a monologue, you can't ask me what I mean if you think something is unclear or wrong. You can ask in the comments below. But it isn't as easy as to talk to someone.
Not sharing test cases is an example of not cooperating in a team. If testers and developers work together, then they will share their understanding. One understanding is how to test a certain capability. It will reveal if there are multiple interpretations. And there are often multiple interpretations of almost anything that is un-trivial.
It turns out that sometimes even seemingly trivial things are non-trivial and a source of misunderstandings.
It is a clear sign of a long feedback cycle when the views of the business analyst, the developers, and the testers are brought together only when a product or feature is implemented. It usually takes a while to implement anything non trivial. If any misunderstanding aren't found before, the feedback cycle was too long.
Had there been frequent demonstrations during the development period, then these misunderstandings might have been found earlier.
There are a lot of different things that can be done to resolve this situation. However, the brief summary is that shorter feedback cycles and closer cooperation solve many problems.
The first thing that we need to address is a simplification of the specification process. The business analysts need to know approximately what they want, but they don't need a complete idea. A product vision is probably enough. They need to know where they want to be in a not too distant future. They also need to know how they can assert that they are in this new position.
A specification may seem hard to test. How can you test something that is just a vision? It turns out, however, that by discussing the specification with concrete examples, you will find new things you hadn't thought of. One thing might lead to another and you may start exploring areas nobody had thought of.
One way of doing this is to gather the right people and have a conversation about the future of the product. This could be a user story for the next iteration.
A format for the conversation is a called the Three Amigos Meeting. You gather a business analyst, a developer, and a tester for a conversation. The number three here should be interpreted as anything between three and seven. If you have a user interface, UX, specialist, they might have valuable input on how something can be used.
The name Three Amigos should be attributed to George Dinwiddie and Bob Payne who came up with name when working with a client. George and Bob recorded a Pod cast where they talk about the Three Amigos. It is recorded 2011 but I still find it valid in 2017.
A three amigos meeting is more efficient if it is structured. One low fidelity format Matt Wynne came up with is called Example Mapping. All you need is pen and paper.
You decide what the next increment in the development should be. This is often a user story. The group tries to come up examples that shows how the system should be used. The examples should be briefly documented on a paper. A few examples may reveal that there are things that are unclear. The questions should be noted on a paper. The example may also reveal that there are certain rules that apply. These rules are great to write down as well.
Matt suggests using index cards and Sharpies. The relative small paper and relative large pen forces you to be brief and move on instead of focusing on writing a complete description.
By bringing the different perspectives from business analysis, development, and testing together, communication is increased. It is easy to ask "How should the system behave when the user does this?". The answer may be obvious or it may be a case where nobody can tell how it should work. This is an example of something that used to be an unknown unknown and now became a known unknown. The group has discovered new knowledge about the system.
Sometimes the unknown unknown won't be found in an Example Mapping session. It may be something that we don't learn until the software is used by actual users. This is the reason why we want to deploy the product early so we can get proper feedback.
Techniques like continuous deployment are valuable here. Continuous deployment is when you are able to deploy small increments to production often. This allows you to get fast feedback from actual users. Things not ready can be hidden behind feature toggles and only be made available to some selected users.
It is hard to find the unknown unknown. It gets easier with close cooperation between all parties involved in the development. Three Amigos Meetings and Example Mapping are tools that may help.
Fast feedback from actual users will also help you find problems early. By being able to deploy early and fast, you are able to fix issues fast. Automated testing is important here to help you find regressions.
Behaviour-Driven Development, BDD, is one set of practices that will support you to find the unknown unknown.
I would like to thank Malin Ekholm for proof reading.
RSS