Sampling
Also known as: random sampling, stratified sampling, sample size, survey sampling, sampling bias, representative sample
Sampling is the process of selecting a representative subset (sample) from a larger group (population) to collect data and make inferences about the whole population.
Researchers, marketers, and data scientists use sampling because measuring every individual in a population is often impractical or impossible. A well-designed sample can represent millions of people accurately with only hundreds or thousands of observations. Sampling method matters critically: a biased sampling method produces systematically unrepresentative results regardless of sample size. Key sampling methods include random sampling (every individual has equal chance of selection), stratified sampling (ensuring key subgroups are proportionally represented), and cluster sampling (randomly selecting groups, then sampling from those groups).
Sampling, explained at three levels
At home
Sampling is how you learn about a large group without measuring every member. If you want to know what 10 million Canadian adults think about a policy, you don't ask all 10 million -- you ask a carefully selected sample of 1,000-2,000 people. If the sample is selected correctly (randomly, representing different ages, genders, regions, and income levels proportionally), the sample's answers will be close to what the full 10 million would say. The key is that the sample must be representative -- not just whoever is most convenient to ask.
At work
In market research, sampling allows you to understand customer preferences, satisfaction, and needs without surveying your entire customer base. A well-designed survey of 500 randomly selected customers can provide reliable estimates of what your full customer base thinks, within a known margin of error. The critical decisions are: who to include in the sample (is it representative of all customers, not just the most engaged?), how to select them (random selection, not volunteers only), and what size is needed (determined by the precision required and the variability in the population).
At home
Sampling bias: a sample is biased when certain types of individuals are systematically more or less likely to be included. Online surveys self-select respondents (people who bother to respond may differ systematically from those who don't). Convenience sampling (asking people who are nearby or easy to reach) systematically excludes those who are not present or accessible. Response bias (people answer differently than they would behave) affects self-reported surveys. Survivorship bias (only observing entities that succeeded, missing those that failed) affects analysis of successful businesses or funds. Identifying potential biases is as important as sample size in evaluating research quality.
At work
Stratified sampling for market research: when the population has important subgroups (different customer segments, age groups, geographic regions) that may respond differently, stratified sampling ensures each subgroup is represented proportionally (or at a specified level for comparison purposes). If 30% of your customers are under 35 and 70% are over 35, a non-stratified random sample of 100 might accidentally include 45 under-35 or 25 under-35 -- over- or under-representing an important segment. Stratified sampling explicitly ensures 30 under-35 and 70 over-35 respondents are included, enabling reliable subgroup comparisons.
At home
Power analysis and sample size determination: before conducting research, statisticians calculate the required sample size to detect an effect of a given magnitude with sufficient probability (statistical power). A study that is underpowered (too small a sample) will fail to detect real effects, wasting resources and producing inconclusive results. The required sample size depends on: the expected effect size (larger effects require smaller samples), the desired statistical power (probability of detecting the effect if it exists -- typically 80% or 90%), and the significance level (alpha, typically 0.05). Researchers who choose sample sizes arbitrarily or by convenience, rather than through power analysis, frequently produce underpowered studies.
At work
Weighting in survey sampling: when a sample is not perfectly representative (some groups are over- or under-represented), statistical weighting adjusts the analysis to correct for known imbalances. If 40% of respondents are under 35 but only 30% of the target population is under 35, responses from under-35 respondents are down-weighted (multiplied by 30/40 = 0.75) and responses from other groups are adjusted proportionally. Weighting improves representativeness when random sampling is impractical but requires assumptions about the correct population composition. Weighting for unknown or unmeasurable characteristics (like political engagement level or propensity to respond) is impossible and represents a fundamental limitation of observational survey methods.
Real-world examples
At home
Sophie noticed that her company's NPS (Net Promoter Score) survey sent to all customers consistently showed a score of 72 -- very high. But when she mentioned this to a friend who also used the service, the friend was surprised because their personal experience was quite negative. Sophie investigated and found the survey was sent to customers who had completed a purchase in the last 30 days -- which excluded the 40% of customers who had abandoned their carts or experienced problems before purchasing. The sample was systematically biased toward satisfied customers, inflating the score.
At work
Thomas was designing a customer satisfaction study for a bank. The default approach was to email the survey to all customers who had used online banking in the last month. He identified several biases: online banking users were younger and more tech-comfortable than the full customer base; customers who had experienced serious service problems were likely to have called and abandoned online banking during that period (so they were excluded); and weekday-only email delivery meant retired customers (who bank at different times) were under-represented. He redesigned the sampling approach to randomly select from all customer segments proportionally, using multiple contact methods. The resulting survey produced very different (and more representative) satisfaction data.
Common confusions
- Many people assume that a larger sample is always more reliable than a smaller one — A large, biased sample is less reliable than a small, well-constructed random sample. The Literary Digest 1936 poll surveyed 10 million people and was dramatically wrong because the sampling method was biased. Gallup was accurate with 50,000 using proper random sampling. Sample size determines the precision of estimates (smaller margin of error), but sampling method determines whether the estimate is accurate (unbiased). A biased sample of any size produces systematically wrong results with false confidence from the large size.
- Some assume that online surveys or social media polls represent the general population — Online surveys self-select respondents who choose to participate -- they systematically overrepresent people with strong opinions (especially negative ones), internet-savvy demographics, people with more free time, and people who saw the survey (which depends on platform algorithm and social network). A Facebook poll or Twitter vote represents the people who participated, not the general population. Social media audiences are not random samples of any larger population. This makes social media polls and unweighted online surveys poor instruments for claims about general public opinion.
For analysts
When clients present survey results or research findings, the sampling method and response rate are critical evaluation criteria often buried in methodology sections. Questions to ask: who was eligible to be sampled? How were they selected (random vs. convenience vs. self-selected)? What was the response rate (and how might non-responders differ from responders)? How does the sample demographic compare to the target population? A high response rate (60%+) random sample with clear methodology is much more reliable than a low response rate (10%) convenience sample, regardless of total sample size. Building this evaluation into client research review processes adds significant analytical value.
History & origin
Survey sampling developed in the late 19th and early 20th centuries. The Literary Digest's 1936 US presidential election poll famously failed: they surveyed 10 million people (enormous sample) but selected respondents from car registration lists and telephone books -- systematically overrepresenting wealthier (Republican-leaning) voters who had cars and phones, predicting Alf Landon would win while Franklin Roosevelt won by a landslide. George Gallup correctly predicted the outcome with only 50,000 interviews using probability sampling. This episode established the primacy of sampling method over sample size. Gallup's probability sampling approach became the foundation of polling methodology. Statistical quality control in manufacturing (accepting or rejecting batches by sampling rather than testing every item) was developed by Walter Shewhart and Harold Dodge at Bell Labs in the 1920s-30s.