The science of sampling

Insights from our doctor of statistics

As we talk to people about using the NEAC Living Lab for their research, we are often asked how many households and businesses they need in their sample in order to get accurate, useable findings. In this article Dr Carolyn Huston, NEAC’s Digitalisation Lead takes us through some core statistical concepts related to sampling methods and how they support inference.

Sampling is a cornerstone of effective research, especially in a complex and dynamic environment like NEAC’s Living Lab, which focuses on real-world experimentation and collaborative innovation. Sampling  a portion of the overall population offers the ability to generate valuable insights about broader populations without the need to collect data from every member of that population. Rather than aiming for perfection in data collection, sampling provides representative snapshots that enable researchers to test theories, identify trends, and make informed decisions efficiently and practically.

But how big should your sample be?

Ask the expert

Carolyn Huston is a seasoned statistician whose journey into data began in ecology and conservation biology. Her fascination with the practical applications of statistics was sparked by tagging fish during her early days in research. From there, she pursued a master’s degree in biostatistics and later completed her PhD in statistics, focusing on estimating fish populations in the ocean in a context of climate change and overfishing—an area rich in sampling challenges and lessons.

“I’ve always loved how statistics can help out in the real world. In the case of NEAC it’s the energy transition, and the human lens we can take to it through researching a sample of humans. If we take a good sample, we will represent different segments of the population more representatively, increasing equity in our decision making,” Carolyn said.

Carolyn has also served as a lecturer in statistics at prominent Canadian institutions, including the University of Alberta, Simon Fraser University, and the British Columbia Institute of Technology. Carolyn’s emphasis on respecting the human effort behind data collection and her ability to merge technical expertise with practical relevance make her an invaluable resource in the NEAC team.

How big should your sample be?

Ok, first some Statistics vocabulary:

• A population is the total group you want to know about
• A parameter is an insight that you want to learn
• A statistic is an estimate of a parameter, based on a sample of the population.

For example, let’s say you wanted to know the proportion of houses with insulation, in a suburb  of 20,000 homes.

“You could try to ask the whole population (20,000), but by the time you finished asking everyone, some people will have moved out, or added insulation, or even demolished the house. So we take a sample to represent the population”, Carolyn said.

“Since it’s just one parameter in this instance, a sample as low as 30 could give us a reasonable idea of the proportion of houses with insulation.”

Lies, damned lies and statistics

While the origin of the above quote can be debated, the fact is that some statistics are ‘better’ than others, depending on what you’re trying to do.

Ok, time for some more statistics 101:

• Statistical significance means that a big enough portion of the population was sampled that you can be confident that the statistic is true within a specified range
• Practical significance means that the statistic can be used to take action.

“For example, if it was discovered that there’s a 1% difference in average energy use between two demographic groups, that could be statistically significant. However, it might not be enough of a difference to motivate a planning change. However a 25% difference in a small study, even if it’s not statistically significant, could definitely point to purposeful further investigation”, Carolyn explained.

The Power of Adaptive Sampling

One of Carolyn Huston’s most compelling pieces of advice for researchers is the adoption of adaptive sampling techniques. Adaptive sampling is an iterative approach that builds flexibility directly into the research process. Instead of committing to a fixed sample size or methodology, researchers start with a small sample, gather preliminary insights, and then refine their sampling strategy based on what they learn.

This dynamic approach allows researchers to uncover unexpected outcomes, develop new theories, and better understand their study populations before scaling up. It is especially useful in people-centred research, where initial responses can inform the next round of data collection, optimising efficiency and relevance, and enhancing equitable representation of key groups.

Adaptive sampling saves time and resources by avoiding the pitfalls of over-committing to a large, rigid sample size that may not yield meaningful insights.

Design your research with NEAC

Whether you’re exploring energy use patterns, urban planning preferences, or social impacts, sampling ensures that resources and effort are optimised while delivering actionable insights. It bridges the gap between the sheer scale of real-world complexities and the focused needs of research, making it an indispensable tool for Living Lab initiatives.

The NEAC team can help you design your research, and access a Living Lab of households and businesses. If we don’t have enough of the population you’re after, we will recruit to achieve a sufficient sample to represent all of the parameters you are interested in.