Pre-Registration of Educational Trials: Should Educational Trials Be Expected to Be Conducted With the Same Rigor as Clinical Trials?

TY - JOUR

T1 - Pre-Registration of Educational Trials

T2 - Should Educational Trials Be Expected to Be Conducted With the Same Rigor as Clinical Trials?

AU - Li, Su Ting T.

AU - Klein, Melissa

AU - Poehling, Katherine A.

AU - Jacobson, Robert M.

N1 - Funding Information: Su-Ting T. Li MD, MPH a ⁎ sutli@ucdavis.edu Melissa Klein MD, MEd b Katherine A. Poehling MD, MPH c Robert M. Jacobson MD d a Department of Pediatrics (S-TT Li), University of California, Sacramento Department of Pediatrics University of California Sacramento b Department of General and Community Pediatrics (M Klein), Cincinnati Children's Hospital Medical Center, Cincinnati, Ohio Department of General and Community Pediatrics Cincinnati Children's Hospital Medical Center Cincinnati Ohio c Department of Pediatrics (KA Poehling), Wake Forest School of Medicine, Winston-Salem, NC Department of Pediatrics Wake Forest School of Medicine Winston-Salem NC d Department of Pediatric and Adolescent Medicine (RM Jacobson), Mayo Clinic, Rochester, Minn Department of Pediatric and Adolescent Medicine Mayo Clinic Rochester Minn ⁎ Address correspondence to Su-Ting Li, MD, MPH, Department of Pediatrics, University of California, Davis, 2516 Stockton Blvd, Room 220, Sacramento, CA 95817-2233 Department of Pediatrics University of California Davis, 2516 Stockton Blvd, Room 220 Sacramento CA 95817-2233 Several years ago, Academic Pediatrics announced that manuscripts reporting randomized controlled trials would only be considered for publication if the researchers had registered their trial protocols before recruiting (or randomizing) the first research participant. 1 Academic Pediatrics joined a number of journals that had similar requirements to restrict publication to only those trials registered before recruiting or randomizing the first participant. 2 National Institutes of Health (NIH)-funded clinical trials are required to be registered on ClinicalTrials.gov prior to 21 days after enrollment of the first participant, to update information in the clinical trial record at least every 12 months, and to report summary results within a year after clinical trial completion. 3 NIH defines a clinical trial as a study that involves human participants who are prospectively assigned to an intervention, where the study is designed to evaluate the effect of the intervention on the participants and evaluates a health-related, biomedical, or behavioral outcome. Similarly, the International Committee of Medical Journal Editors defines a clinical trial as “any research project that prospectively assigns people or a group of people to an intervention, with or without concurrent comparison or control groups, to study the relationship between a health-related intervention and a health outcome.” 2 The primary motivation for registering one's protocol before recruiting the first research participant is to confirm adherence to the protocol throughout the conduct of the trial and analysis of data. Transparency of research protocol elevates research by preserving the integrity of the trial. Protocol registration upholds the reliability of results. The reliability, usually reported in terms of statistical significance, depends upon hypotheses established a priori—that is, before the observations take place. Protocol registration establishes for public scrutiny the intended outcomes and subgroups and provides evidence that, when more than 1 outcome is measured, the principle outcome was chosen a priori. Similarly, protocol registration provides evidence that subgroup analysis was determined a priori, as well as the plan for adjustments when multiple comparisons are made to ensure adequate power and threshold for statistical significance. Given this motivation for registering protocols before executing the trial (often termed pre-registering), one can argue the advantages accrued would apply to any comparative study and not just a trial, whether clinical or educational. Pre-registration of educational trials has not previously been required. As educational interventional trials move toward measuring higher level outcomes such as patient health or safety or provider behavior, they begin to overlap with NIH's and the International Committee of Medical Journal Editors’ definition of a clinical trial. In their author guidelines, educational journals routinely and explicitly require institutional review board approval and informed consent for educational trials involving human participants. However, pre-registration of educational trials does not appear in the author guidelines for many educational journals sponsored by medical education groups in the United States and Europe, including Academic Medicine (journal of the Association of American Medical Colleges), Journal of Graduate Medical Education (journal of the Accreditation Council for Graduate Medical Education), Medical Teacher (journal of the Association for Medical Education in Europe), Medical Education (journal of the Association for the Study of Medical Education), Medical Science Educator (journal of the International Association of Medical Science Educators), Advances in Health Sciences Education, Teaching and Learning in Medicine , and BioMed Central Medical Education . The advantages of pre-registering educational trials would be similar to the advantages of pre-registering clinical trials discussed earlier: protect academic integrity by documenting outcome measures during the study design phase, provide a mechanism so that the educational community is aware of ongoing trials and an opportunity to avoid unnecessary duplication, encourage collaboration by building on prior results, and increase awareness of all studies, which allows standardization of outcome metrics and minimizes publication bias by providing information about negative studies that can be difficult to publish. Given the advantages of pre-registering educational trials, we considered requiring that all educational trials be pre-registered; however, many barriers exist to pre-registering educational trials. We describe 4 main limitations: • Lack of awareness —The community of educational scholars lacks awareness of the availability and/or advantages of registering educational trials, and educators often lack formal research methodology training. 4 • Inherent educational study limitations —Educational trials are often single-center with relatively small sample size, and they systematically differ from the classic clinical randomized controlled trial. 5 Control groups are often historical controls or quasi-randomized (by continuity clinic day of the week or rotation). • Lack of support —Institutional limitations include minimal support to encourage and help educational scholars navigate a cumbersome trial registry system; educational studies are often unfunded or funded by small, local institutional grants. • Many systems-level limitations —Clinicaltrials.gov and other clinical trial registration sites were not designed to register educational trials. The registry maintained by the Society for Research on Educational Effectiveness primarily serves elementary and high school educators ( https://www.sreereg.org/ ). As mentioned above, leading medical educational journals do not require registration. To increase the rigor and transparency of educational trials and encourage further collaboration and innovation, we have the following suggestions to encourage pre-registration of educational trials. Educational institutions, such as the Association of American Medical Colleges or Accreditation Council for Graduate Medical Education, should consider creating a registry designed for educational trials, as none currently exists. Established research collaborations who have the infrastructure and research experience, such as the I-PASS Study Group, Association of Pediatric Program Directors’ Longitudinal Educational Assessment Research Network, and Academic Pediatric Association Continuity Research Network, should consider taking the lead in requiring that multi-institutional randomized controlled trials pre-register their educational trials. These established research collaborations could also provide training and support for pre-registering educational trials for their members. Educational journals such as Academic Pediatrics should lead the charge to help the educational community meet the challenges of pre-registering educational interventional trials to enhance the rigor and broaden the scope of educational research. Academic Pediatrics will continue to require pre-registration of clinical randomized controlled trials as before but will not require pre-registration of educational trials. Instead, the journal will encourage pre-registration of educational trials and particularly encourage pre-registration of large, multi-center randomized controlled trials from established organizations such as the Association of Pediatric Program Directors’ Longitudinal Educational Assessment Research Network, the Academic Pediatric Association Continuity Research Network, and I-PASS.

PY - 2019/3

Y1 - 2019/3

UR - http://www.scopus.com/inward/record.url?scp=85060344019&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85060344019&partnerID=8YFLogxK

U2 - 10.1016/j.acap.2018.12.006

DO - 10.1016/j.acap.2018.12.006

M3 - Comment/debate

C2 - 30610967

AN - SCOPUS:85060344019

SN - 1876-2859

VL - 19

SP - 131

EP - 132

JO - Academic Pediatrics

JF - Academic Pediatrics

IS - 2

ER -