Pilot studies can play a very important role prior to conducting a full-scale research project.
A pilot study, also called a “feasibility” study, is a small-scale preliminary study conducted before any large-scale quantitative research in order to evaluate the potential for a future, full-scale project.
In This Article
How Does it Work?
Pilot studies are a fundamental stage of the research process. They can help identify design issues and evaluate a study’s feasibility, practicality, resources, time, and cost before the main research is conducted.
It involves selecting a few people and trying out the study on them. It is possible to save time and, in some cases, money by identifying any flaws in the procedures designed by the researcher.
A pilot study can help the researcher spot any ambiguities (i.e., unusual things), confusion in the information given to participants, or problems with the task devised.
Sometimes the task is too hard, and the researcher may get a floor effect because none of the participants can score at all or can complete the task – all performances are low. The opposite effect is a ceiling effect, when the task is so easy that all achieve virtually full marks or top performances and are “hitting the ceiling.”
This enables researchers to predict an appropriate sample size, budget accordingly, and improve the study design before performing a full-scale project.
Pilot studies also provide researchers with preliminary data to gain insight into their proposed experiment’s potential results.
However, pilot studies should not be used to test hypotheses since the appropriate power and sample size are not calculated. Rather, pilot studies should be used to assess the feasibility of participant recruitment or study design.
By conducting a pilot study, researchers will be better prepared to face the challenges that might arise in the larger study. They will be more confident with the instruments they will use for data collection.
In some studies, multiple pilot studies may be needed, and qualitative and/or quantitative methods may be used.
In order to avoid bias, pilot studies are usually carried out on individuals who are as similar as possible to the target population but not on those who will be a part of the final sample.
It is important to conduct a questionnaire pilot study for the following reasons:
- Check that respondents understand the terminology used in the questionnaire.
- Check that emotive questions are not used, as they make people defensive and could invalidate their answers.
- Check that leading questions have not been used as they could bias the respondent’s answer.
- Ensure the questionnaire can be completed appropriately (i.e., it’s not too long).
Advantages
-
Increasing research quality
-
Assessing the practicality and feasibility of the main study
Testing the efficacy of research instruments
-
Identifying and addressing any weaknesses or logistical problems
-
Collecting preliminary data
-
Estimating the time and costs required for the project
-
Determining what resources are needed for the study
-
Identifying the necessity to modify procedures that do not elicit useful data
-
Adding credibility and dependability to the study
-
Pretesting the interview format
-
Enabling researchers to develop consistent practices and familiarize themselves with the procedures in the protocol
-
Addressing safety issues and management problems
Limitations
-
Require extra costs, time, and resources.
-
Do not guarantee the success of the main study.
-
Contamination (ie: if data from the pilot study or pilot participants are included in the main study results).
-
Funding bodies may be reluctant to fund a further study if the pilot study results are published.
-
Do not have the power to assess treatment effects due to small sample size.
Examples
- Viscocanalostomy: A Pilot Study (Carassa, Bettin, Fiori, & Brancato, 1998)
- WHO International Pilot Study of Schizophrenia (Sartorius, Shapiro, Kimura, & Barrett, 1972)
- Stephen LaBerge of Stanford University ran a series of experiments in the 80s that investigated lucid dreaming. In 1985, he performed a pilot study that demonstrated that time perception is the same as during wakefulness. Specifically, he had participants go into a state of lucid dreaming and count out ten seconds, signaling the start and end with pre-determined eye movements measured with the EOG.
- Negative Word-of-Mouth by Dissatisfied Consumers: A Pilot Study (Richins, 1983)
- A pilot study and randomized controlled trial of the mindful self‐compassion program (Neff & Germer, 2013)
- Pilot study of secondary prevention of posttraumatic stress disorder with propranolol (Pitman et al., 2002)
- In unstructured observations, the researcher records all relevant behavior without a system. There may be too much to record, and the behaviors recorded may not necessarily be the most important, so the approach is usually used as a pilot study to see what type of behaviors would be recorded.
- Perspectives of the use of smartphones in travel behavior studies: Findings from a literature review and a pilot study (Gadziński, 2018)
Further Information
References
Carassa, R. G., Bettin, P., Fiori, M., & Brancato, R. (1998). Viscocanalostomy: a pilot study. European journal of ophthalmology, 8(2), 57-61.
Gadziński, J. (2018). Perspectives of the use of smartphones in travel behaviour studies: Findings from a literature review and a pilot study. Transportation Research Part C: Emerging Technologies, 88, 74-86.
In J. (2017). Introduction of a pilot study. Korean journal of anesthesiology, 70(6), 601–605. https://doi.org/10.4097/kjae.2017.70.6.601
LaBerge, S., LaMarca, K., & Baird, B. (2018). Pre-sleep treatment with galantamine stimulates lucid dreaming: A double-blind, placebo-controlled, crossover study. PLoS One, 13 (8), e0201246.
Leon, A. C., Davis, L. L., & Kraemer, H. C. (2011). The role and interpretation of pilot studies in clinical research. Journal of psychiatric research, 45(5), 626–629. https://doi.org/10.1016/j.jpsychires.2010.10.008
Malmqvist, J., Hellberg, K., Möllås, G., Rose, R., & Shevlin, M. (2019). Conducting the Pilot Study: A Neglected Part of the Research Process? Methodological Findings Supporting the Importance of Piloting in Qualitative Research Studies. International Journal of Qualitative Methods. https://doi.org/10.1177/1609406919878341
Neff, K. D., & Germer, C. K. (2013). A pilot study and randomized controlled trial of the mindful self‐compassion program. Journal of clinical psychology, 69(1), 28-44.
Pitman, R. K., Sanders, K. M., Zusman, R. M., Healy, A. R., Cheema, F., Lasko, N. B., … & Orr, S. P. (2002). Pilot study of secondary prevention of posttraumatic stress disorder with propranolol. Biological psychiatry, 51(2), 189-192.
Richins, M. L. (1983). Negative word-of-mouth by dissatisfied consumers: A pilot study. Journal of marketing, 47(1), 68-78.
Sartorius, N., Shapiro, R., Kimura, M., & Barrett, K. (1972). WHO International Pilot Study of Schizophrenia1. Psychological medicine, 2(4), 422-425.
Teijlingen, E. R; V. Hundley (2001). The importance of pilot studies, Social research UPDATE, (35)
