Probability sampling in research

Although there are millions of shoppers during the holiday season, only a few thousand of them are actually sampled to determine their shopping habits. An obvious question is why only a few thousands? Well, the simple answer is time and the average cost per person that prohibits the companies to reach everyone. At the same time, a good sample can give you a reasonably accurate understanding of the shopping habits of shoppers. Sampling in research provides a fast, easy, and inexpensive method of selecting individuals from the total population of interest which can be studied to make inferences about the study population. The probability sampling technique increases the generalizability of these result to a great extent. Before we jump into different kinds of sampling techniques, we need to first define the basic terminology of sampling.

Sampling basics

In research methods, we defined the population of interest (or target population) as the entire set of individuals (or unit of analysis) which the researcher is interested in studying. Unit of analysis can be the object, individuals, or things, described by the variables and the “who” or “what” the researcher is studying. For example, adolescents diagnosed with depression, high school students with IEAP, football players who experienced a concussion, etc. are all examples of the population of interest as they all share some unique characteristics. Understandably, it is not possible to have access to all individuals in our target population. But we might have access to the local population that might have the desired characteristics. For example, we might have access to high school students in our city/town or we might have access to local football players who have experienced a concussion. We refer to this as our study population or accessible population. But we still cannot study the whole study population, as it is time and resource consuming. So, we draw a sample from our study population and study them instead so that we can generalize our results back to the target population. A sample is a subset of the study population. Since we wish to generalize the study results to the target population, we need to have a sample that is not just a subset but a “representative subset” (representative sample) of the target population that exhibits the unique characteristics of interest of the entire target population. A non-representative sample will lead to an error. As always, the error can be a random error (aka sampling error) or a bias (aka sampling bias). One way to avoid these errors is to collect data from everyone in the target population. If we do that it is called a census. The quality of the sample thus is extremely important as it determines the trustworthiness of the researcher’s conclusions.

Representative sample

The ability to generalize the results of our sample to the population depends on the representativeness of the sample. Representative samples are similar to the target population on all the characteristics. That is, ideally, we want a sample that is a mirror image of the target population. Since we are going to draw our sample from the study population, we must also look at the representativeness of the study population to the target population. For example, a researcher wants to study the degree of acculturation and mental health among adults. Using a sample of university students is going to limit the generalizability of the results to the larger community as the characteristics of the study population (university students) might be noticeably different from the target population. This is a sampling error, and more specifically it is a bias or a sampling bias, as it is the researcher who made a deliberate decision to use a non-representative study population to draw a sample.

Social scientists are generally interested in two types of data. Individual attributes data and cultural data. Individual data are the attributes of individuals in the population. For example, each individual has an age, gender, income, preference for things, etc. On the other hand, cultural facts are shared and hence, cultural data requires an expert who can explain the cultural norms and variations on that norm. For example, norms around political correctness, norms of individuality, making up a list of guests for a wedding are all examples of cultural data. Individual data can be studies using probabilistic sampling techniques, while cultural data requires non-probabilistic sampling techniques.

Sampling

Probability sampling and Non-probability sampling

As the name suggests, probability sampling is where there is a known and equal probability or change of an individual (or element – unit of analysis) from the study population to be selected in the sample, while non-probabilistic sampling, that probability is unknown. This post will focus on probability sampling as non-probability sampling is discussed in different post. If we want our sample to be representative of the population (mirror it), a probability sample greatly increases that chance of getting a representative sample. But to draw a sample from the study population also requires some kind of complete or a partial list or enumeration of individuals in the study population. Such a list is called a sampling frame. For example, a list of all the students enrolled in a particular class is a sampling frame. List of people on the telephone directory, list of items produced in a particular batch are all examples of a sampling frame. The sampling frame also needs to be unbiased (or representative of the target population). That is, it should represent the target population and free of any bias.

Probability sampling

Under probability sampling, every element in the sampling frame has a known and an equal chance of getting selected in the sample. It is sort of like a lottery where the chance of getting picked is completely random and unbiased. Probability sampling allows us to make a statistical statement about the accuracy of the sample results and they are necessary for testing for statistical significance. They are also more representative of the population and hence the results are more generalizable.

1. The probability sample needs a sampling frame.
2. Each element on the sampling frame has a known probability of selection.
3. The selection process is unbiased or random.

There are four types of probability sampling techniques (or designs).

• Simple random sampling (SRS)
• Systematic sampling
• Stratified sampling
• Cluster sampling

Simple random sampling (SRS)

This is the best-case scenario for an equal probability of selection to a sample. There is an unbiased sampling frame and every element on the sampling frame has an equal and known chance of selection. For example, say we want to draw a random sample of 5 students from a class of 50 students. We can use the class roster as our sampling frame. Every person on that roster has an equal and known probability of getting selected (5/50 or 10%). We can randomize all the names of the students on the list and randomly select a student. We can repeat this process to select the next student. Remember that the probability of the second student to be selected is still equal and know, which would be 4/49 since there are 4 more students to be selected from 49 total students. Since we are not allowing double selection of any student, it is called sampling without replacement. Such a sampling technique is more efficient at producing a representative sample. It also follows the principle of independence. That is, the chance of selection of any element is not dependent on the selection of another element. Most statistical packages can draw a random sample from a sampling frame if your sample size large. Although sometimes with larger samples SRS is harder to implement and it also costs more. However, a unique point of SRS is that if we were to draw multiple samples of the same size, each sample has an equal chance of getting selected.

Note: In theory, SRS produces a representative sample but in practice, this is often not the case particularly when the sample size is small or when the sampling frame contains small subgroups that might be missed entirely or might be underrepresented. It can also be because of a random error known as a sampling error. Since random errors are unsystematic or chance factors, drawing a larger sample takes care of this error. Additionally, researchers always provide a sample description so that readers and observers can decide on the validity of the inferences and representativeness of the sample.

Advantages of SRS

• Samples are more likely to be representative of the population.
• Results more likely to generalize to population.
• Maximizes the external validity.
• Respects the principle of independence.

Disadvantage of SRS

• Often difficult to implement when the sample is large.
• Can be time consuming and costly.
• Needs a sampling frame to implement.

Systematic sampling

To avoid a change of getting an extreme sample researcher can use what is called systematic sampling. In systematic sampling, elements from the sampling frame are selected at a uniform interval, known as the sampling interval. That is, if we wanted to draw a sample of 5 students from the roster of 50 students, we divide the sampling frame into 5 equal sections (10 students each), and then randomly pick a number from the first 10 students and then sample every 10^th student from that point. You can see the example below. A systematic sample can be better than SRS as the sample is more evenly spread over the study population. It is also easier to implement and less costly. However, systematic sampling differs from SRS in that each sample does not have an equal chance of selection. For example, since we are selecting every n^th number (in this case every 10^th number), the consecutive numbers will never have a chance to be selected together (i.e., 2, 3, 4 or 15, 16, 17, etc.). This violates the principle of independence that the selection of one element is independent of the selection of other elements from the sampling frame.

Additionally, if we suspect some kind of pattern in the sampling frame then this sampling technique can also lead to an unrepresentative sample. For example, if we want to study the opinions of families on the city’s abilities to deal with a fire emergency, and if every 15^th home is selected and each of these homes happens to have a fire hydrant in front of their home, then the sample is badly biased. It over-represents some families and under-represents others. This is because systematic sampling is inappropriate when there is a sequential pattern (or periodicity) in the elements. Tip – If you suspect a sequential pattern, you can take another systematic sample to see if they differ. Any difference that is found can then be attributed to sampling error. Or if it is possible you can simply randomize the elements in the sampling frame to get rid of existing patterns.

• Study population = 50
• Sample size = 5
• Sampling interval would be = 50/5 = 10

Advantages of systematic sampling

• Easier to implement and less costly than SRS.
• Better chance of representing the population as the sampling is evenly spread.
• Better external validity of the results.

Disadvantages of systematic sampling

• Can result in a biased sample if there is sequential pattern in the sampling frame.
• Violates the principle of independence.
• Needs a sampling frame to implement.

Stratified sampling

To avoid some of the pitfalls of systematic sampling of under-representation or over-representation of some group in the sample we use stratified sampling. Here we divide the study population or the sampling frame into a relatively homogeneous group called strata. We then can apply SRS to each stratum by either selecting elements from each stratum that is proportional to that stratum in the population or draw an equal number of elements from each stratum and weight the results according to the proportion of stratum to the study population. Sometimes it is also referred to as proportional stratified sampling. One must be careful in making sure that the information about the proportion of stratum in the population is correct, otherwise the sample will mimic this inaccuracy. You can also use systematic sampling within each stratum if that is more appropriate. For example, stratified sampling can be used if you are interested in knowing about the existence of sexual harassment in the workplace. You might want the opinions of different genders to be represented well enough. To do that you can divide your sampling frame by gender. You can also divide your sampling frame by age, income, position, etc. However, such stratification will not provide any benefit if characteristics used to create strata are not relevant to the research. You can also have multiple samples based on the characteristics of interest (gender, age, income, position). This is called cross-stratification. Since each stratum is homogenous, by drawing from each stratum we are better positioned to get an accurate estimate of each stratum and the population as a whole. The resulting sample is more reliable and accurate as stratified sampling ensures that each subgroup in the population is included in the sample. This sampling technique can help to reduce non-observation or non-response bias. One can also use statistical correction to correct such bias. It is also used when we want to compare two groups.

Steps to create a stratified sample

• Create different strata or groups in your sampling frame.
• Draw proportional SRS or systematic sample (as appropriate) from each stratum.
• Combine the sample to create your stratified sample.

Advantage of stratified sampling

• More likely to be representative of the population as it ensure representation of each sub-groups.
• Cross-startificaiton can lead to additional benefits.
• Can help reduce non-observation or non-response bias, and statistical corrections can be applied to correct such bias.
• Can be used to compare two groups.

Disadvantages of stratified sampling

• If the proportion of a sub-group or stratum to the population is incorrect, sample will mirror that inaccuracy.
• Advantages are contingent upon the relevance of the characteristics used to create groups.

Cluster sampling

In cluster sampling, we again create groups in our sampling frame, called clusters. Now instead of selecting from each cluster, we take a SRS of these clusters. The assumption is that these clusters are representative of the population. For example, if we are interested in learning about anxiety for the 6^th graders, instead of taking SRS, we could just randomly select a few 6^th-grade classrooms (clusters) from the schools. This is because there is the availability of pre-existing clusters in our sampling frame or the study population. Once we have selected the cluster, we take each element from that cluster into our sample. The difference with stratified sampling is that in stratified sampling we assume that the groups are homogeneous within themselves but between-group there are differences. In cluster sampling, we assume the opposite. That is, we assume the groups are diverse (like the population) within themselves, while groups lack variation between them. Cluster sampling is relatively quick and easy to implement when a large sample is required. Since cluster sampling is done in a group, it is much faster and consumes less time. But this can also be a negative if data contamination occurs. That is, one individual in the cluster can directly influence the score of another individual. Moreover, there is always a chance that a cluster might not be representative of the population. Again the common method generally employed by researchers is to provide a sample description.

Advantages of cluster sampling

• Takes advantage of the existing clusters in the study population.
• Relative quick and easy to implement.

Disadvantages of cluster sampling

• Data contamination can be a serious threat.
• Relies on the fact that the clusters are representative of the population or they are not homogeneous.

Combined sampling or multi-stage sampling

Occasionally, researchers engage in multi-stage sampling where they combine different methods of sampling. For example, a researcher might first select randomly the schools and then within the schools then again randomly select a specific grade to study. This is called two-stage random sampling. You can have multiple combinations based on different sampling techniques, but it also complicates how inferences can be drawn and how the results might generalize to the population.

Sampling Error

There are two types of error in sampling, random errors and non-random errors.

Random errors are unsystematic, unpredictable, and just as likely to over-as under-estimate the “true” score. Whereas, non-random errors are systematic in over or under-estimating the true score. 

There are three major sources of non-random error in sampling.

1. Sample bias
2. Completion rate
3. Interviewer effect

Sample bias

Sampling bias occurs when the researcher un/intentionally selects a sample that is not representative of the target or study population. For example, if a researcher is interested in finding out the average height of students on campus, and decides (unmindfully) to go to the basketball court and randomly selects a sample from the people who are playing the game, s/he will end up with a biased sample. Similarly, choosing a sample of university students to study the degree of acculturation and mental health among adults will also result in a sampling bias if s/he wants to generalize the results to the larger community. Probability sampling can reduce this type of error when the sampling frame is representative of the target population.

Completion rate

Completion rate is the extent to which a sample is reached and persuaded to cooperate. A lower completion rate can lead to a non-representative sample because the non-respondents might differ from respondents. This is known as non-observation or non-response bias.

The 1936 Literary Digest poll incorporated both of these errors. The literary digest was one of the most respected magazines of that time. For the presidential election of 1936 between Alfred Landon and FDR, Literary Digest incorrectly predicted Landon will win the election in a landslide. It was the largest and the most expensive poll ever conducted with a sample size of around 2.3 million people based on probability sampling. However, sampling bias and non-observation bias both were present in the sample. Literary Digest magazine used a telephone directory, club membership lists, lists of magazine subscribers as a sampling frame to draw their sample. In 1936, not many people had telephones, more than 9 million people were unemployed. Such a sampling frame most likely only had middle-class and upper-class voters. But more importantly, it left out a huge number of lower-class voters who were likely to vote for FDR. A biased sampling frame led to a biased sample and an incorrect inference.

The second problem with this poll was the completion rate. Literary Digest mailed out around 10 million surveys to the people in the sample. Out of 10 million, only 2.3 million filled out the survey and mailed it back. This considerably reduced the sampling size. The completion rate was a mere 23%. The obvious bias is that people who responded to the survey might be different from those who did not, creating a non-response bias. Non-response bias is a huge problem when the sample size is big and the medium of data collection is mail. Both of these non-random errors caused Literary Digest to wrongly predict the winner of the poll.

Interviewer effects

Interviewer effects relate to the idea that interviewers may ask the question differently to different people. Their mannerism or characteristics of the interviewer might influence the response. Additionally, the presence of the interviewer can stimulate respondents to take social norms into account when answering survey questions leading to social desirability bias by taking a position or stating an opinion that is more socially acceptable and avoiding more socially unacceptable positions/opinions.

Dealing with non-random errors

While sampling bias is more theoretical, the researcher needs to make sure that the sampling frame and the sample are both representatives of the target population. The completion rate is a difficult issue to deal with in a survey. Researchers should try to peruse non-responders and get some data so that they can compare and see if they differ from responders. Researchers can also weight results based on the expected proportion of various groups in the population so that the sample is not biased towards responders. Lastly, standardization and training can help avoid interviewer effects.

Random errors

There are two types of random errors.

1. Interviewer effects
2. Sampling error

Interviewer effects are haphazard mistakes that reduce the reliability of measures. Again, these can be minimized by standardization and training.

Sampling error – Sampling errors are the unavoidable chance variations among different samples drawn from the same population. Larger sampling errors can lead to imprecise sampling estimates. For probability samples, sampling error is measure by the standard error of the mean, calculated as

SEM = Standard deviation/√n

where n is the sample size. Thus, sampling error can be reduced by increasing the sample size to get an acceptable level of precision. Depending on the size of the target population, various computing charts are available that provide information about the appropriate sample size required to achieve a good enough level of precision.

Bibliography

Bernard, H. R. (2006). Research methods in anthropology: Qualitative and quantitative approaches (4th ed.). Lanham, MD: AltaMira Press.

Dooley, D. (2001). Social research methods (4th ed.). Upper Saddle River, New Jersey: Pretence Hall.

Gravetter, F. J., & Forzano, L.-A. B. (2018). Research methods for the behavioral sciences (6th edition. ed.). Boston, MA: Cengage.

Levin, R. I., Rubin, D. S., Siddiqui, M. H., & Rastogi, S. (2017). Statistics for management (8th ed.). Noida: Pearson India.

Creswell, J. W. (2012). Educational research: Planning, conducting, and evaluating quantitative and qualitative research (4th ed.). Boston, MA: Pearson Education.

Cite this article (APA)

Trivedi, C. (2020, December 10). Probability sampling in research. Conceptshacked. Retrieved from https://conceptshacked.com/probability-sampling/