Step 2: Talk to users and show them your designs
The most important things to remember when talking to users is:
They are happy to talk to you. You have singled them out as someone important to listen to, and hence are generally flattered. Additionally, the tool you are building will ultimately help them do something they want to do, so they are motivated to help you.
But be careful of asking them too many times. Even though users are happy to talk to you, you want to make sure that they feel that their time is appreciated. Also, if they have already seen your design, their feedback may be less useful since they are now biased by their previous experience. How many times is too many times will depend on the user and when you asked them last.
They will be cautious to not offend you. They have empathy with the effort you are putting in to build the tool, and so will not want to outright say what they really think. This means that they may consciously or unconsciously lie to you to not hurt your feelings.
Their memory is biased, as it is for all people. This means they don't remember things exactly as they happened. They may feel like they are always spending time doing a task, when really they don't spend that much time doing it. Or they may think they always do X, then Y, then Z, but in reality they sometimes switch the order.
It is very difficult to both talk to a user and to take notes on what they are saying. It is even more difficult to make notes after the meeting has ended as you are likely to forget. The best way to set up a meeting for feedback is to either a) have 2 people, one of who's sole job is to take notes or b) record the meeting. Zoom is a great way to record an interaction, as well as what they do on a screen, if needed.
Don't ask users if they like your idea, design, etc. Users will inevitably say they do, even when they wouldn't use your tool in real life.
Don't ask leading questions, such as 'Would you rather use the old version or this improved version of the website?' Asking non-leading questions is a skill that can take some time to develop.
3. Note the language and terminology they use when talking about the data, tool, etc. Developers often have different words for data and concepts than the users of a tool. For instance, developers might call it metadata, while users call it clinical data. Note the terminology they use so that you can use it in your designs.
In general it is better in science to not offer compensation despite the fact that this is common practice in industry. Scientists are more likely to want to donate their time and will likely refuse compensation. Also, accepting compensation can make them feel less like they just are a 'good person' for helping you out.
Showing users a design is one of the most fruitful exercises you can do. Watching them use the tool can give you important feedback on not only minor things such as colors, button placement, etc, but also on big things like their thought process and their expectations about flow through the tool.
The best way to test something is to give them an example task that is representative of a use case they might have, and then watch them use the design.
As difficult as it might be, DO NOT TELL THEM WHAT TO DO. Remember that you will not be alongside your future users to tell them how to accomplish a task. If they get stuck, the best thing you can do is to be quiet and say things like 'This is really helpful for me to watch you through this! I know it feels like you are doing something wrong but you aren't. You are helping me to understand how to design this better'.
It may be very painful to just watch someone use the interface and not be able to help them. If they have been stuck for over 2-3 min or if it is one of the first tasks (and hence if they don't make it through this they won't make it to any of the other testing), you can suggest to look at a button or some small hint like that. Otherwise, refrain from helping them.
It is important to not take feedback personally. Even the best idea may fall flat in front of a real world situation. And remember that users are often the most frustrated with tools that they really want to use but can't for whatever reason.
To help combat this, also ask user's what works within the current design so you (A) know what to keep and (B) recognize that there is value in the designs.
​
