This article first appeared in The Bulletin of the Australian Meteorological and Oceanographic Society (Volume 31 Issue 3 September 2018)
Authors: Vicki Lane, Square V Software for Agriculture, Dr Patrick Mitchell and Dr Jaclyn Brown, CSIRO Agriculture and Food
Teaching climate scientists to understand their users
Do you ever look at an app on your phone or a website and think “all I want is this one thing, why isn’t it there?”, or “this is a poor product, it’s really hard to use”? Chances are that the end users of your research are thinking the same thing.
A team of climate and agricultural scientists, including three from CSIRO in Hobart and two from the Bureau of Meteorology, have been visiting grain farmers in different parts of Australia, with the goal of understanding the types of weather and climate information that farmers need, and ultimately helping farmers make more ‘climate smart’ decisions by building better products and services. The team focused on how farmer decision-making impacts behaviour and the types of climate information that can inform this. To do this they took a ‘User Experience’ (UX) approach, which utilises cognitive psychology in a practical way to understand how users interact with, and process information delivered through various technologies. Normally, this kind of work would be carried out by specialist UX researchers, but an experience with an innovation and product development workshop (CSIRO’s ON Prime program) has changed the team’s thinking, and now the scientists are helping to conduct the research themselves, alongside UX specialists.
UX research can take many forms, but in agriculture it often involves a casual chat around a farm table, in a ute, or over a paddock fence—the technical term for this is a “contextual inquiry”. In this kind of research, the researcher adopts the persona of a student, looking to the participant as the expert in their domain, and in this way learns from the participant about how they operate and what their needs are. The kind of questions that are asked might start off with something as simple as “can you tell me a bit about yourself and your farm?”, and continue to “I’m interested to learn about the growing season calendar, can you tell me about that?” or “What kind of weather apps do you use? Can you show me on your phone?” These are open-ended questions about the person and their experience—don’t try to sell your idea or product, instead try to listen and learn.
UX research is different to surveys, which give you statistical information about the percentage of people answering with each response that you can generalise to the population. UX research gives you deep information about a small number of users. Rather than providing quantitative data on the percentage of people with particular preferences or issues, it can identify more detailed insights around the perceptions and motivations of a participant. Using this approach, the researcher can help get to the root cause of the issue or decision-making process.
One of the most challenging things that the scientist team discovered with this type of research is that it’s important not to correct the participant when they’re saying something that’s scientifically invalid—and the scientists in the team discussed at length just how hard this was! The key reason for not correcting the participant is that it breaks rapport and makes the participant feel like you’re judging their accuracy and values, making them less likely to trust you and less likely to tell you the interesting information that will really help you understand their actions and worldview. The objective of UX research is not to go around and individually teach every farmer in Australia the in-depth science behind climate, as people will always have misconceptions and mental shorthand for understanding their weather and climate. The scientists involved learned that they could be more effective by listening to a diversity of perceptions and behaviours while biting their tongue, and using that knowledge to improve the translation of scientific information into actionable knowledge and better products that resonate with their audience.
At the beginning of this study, the team knew that farmers often used multiple apps and other media sources to receive weather forecasts, and assumed that this was because no single source included all of the features that they wanted. This assumption lead to the idea that we could build a “perfect weather app” for farmers, which would be a one-stop shop for farmer’s weather information. The reality of the situation really challenged these hypotheses—farmers discussed that they were looking at multiple apps not for different features, but so that they could gain consensus between the apps and feel more confident in their prediction. From the scientists’ perspective, this was a scientifically incorrect conclusion to draw as most of the apps were using the same models to make their forecasts. However, this finding helped to uncover some interesting thought processes that made more sense in light of a more general pattern of farmers’ decision-making. The farmers participating in the research noted that they were looking for certainty before making decisions, and that they used the consensus between forecasts to gauge how certain or confident the forecast was. It’s interesting to note that this is to some extent a farmer-created proxy for skill, which is generally poorly understood, and raises important questions about the presentation and understanding of forecast skill among farmers and how we might do a better job of providing this information to farmers in a way that they can understand.
As a result of the research, the team decided that a perfect weather app for farmers would have features that were likely to be substantially different to what they had in mind before they talked to farmers. For example, it might show the complete range of outcomes, including information about multiple forecast models (and information about which apps were using each model to help in education!). It might also convey the range of uncertainty among forecasts and provide a better indication of the forecast skill that is easier for farmers to understand.
UX research gives you insight into the problems or decisions that your product or service is attempting to address, helping you design and implement something that’s more relevant to the needs of your audience. A common saying in UX is to ‘Get curious’, and this can be as simple as identifying a list of five questions that you want to know the answers to, picking up the phone, and asking someone if they have 20 minutes to chat. Don’t be afraid to ask—generally people really enjoy participating in UX research because it’s one of the few times of the day when someone else is 100% focused on listening to them!