Pragmatic Free Trial Meta
Pragmatic Free Trial Meta is a free and non-commercial open data platform and infrastructure that facilitates research on pragmatic trials. It collects and shares cleaned trial data and ratings using PRECIS-2 permitting multiple and varied meta-epidemiological studies to examine the effects of treatment across trials that have different levels of pragmatism, as well as other design features.
Background
Pragmatic trials are increasingly acknowledged as providing evidence from the real world for clinical decision-making. The term "pragmatic" however, is a word that is often used in contradiction and its definition and evaluation require clarification. The purpose of pragmatic trials is to guide clinical practices and policy decisions, not to confirm a physiological hypothesis or clinical hypothesis. A pragmatic trial should try to be as close as is possible to the real-world clinical practice, including recruitment of participants, setting up, delivery and execution of interventions, determination and analysis outcomes, and primary analysis. This is a major distinction between explanatory trials, as described by Schwartz & Lellouch1 that are designed to test a hypothesis in a more thorough way.
The most pragmatic trials should not be blind participants or the clinicians. This can lead to an overestimation of the effects of treatment. The pragmatic trials also include patients from different health care settings to ensure that the outcomes can be compared to the real world.
Additionally studies that are pragmatic should focus on outcomes that are vital for patients, such as quality of life or functional recovery. This is particularly important when trials involve invasive procedures or have potentially serious adverse consequences. The CRASH trial29 compared a two-page report with an electronic monitoring system for patients in hospitals with chronic heart failure. The catheter trial28, on the other hand was based on symptomatic catheter-related urinary tract infections as its primary outcome.
In addition to these aspects pragmatic trials should reduce the requirements for data collection and trial procedures to reduce costs and time commitments. Finally pragmatic trials should try to make their results as applicable to real-world clinical practice as is possible by making sure that their primary method of analysis is the intention-to-treat approach (as described in CONSORT extensions for pragmatic trials).
Despite these guidelines, many RCTs with features that challenge the concept of pragmatism have been mislabeled as pragmatic and published in journals of all kinds. This can lead to false claims of pragmaticity and the usage of the term needs to be standardized. The development of a PRECIS-2 tool that offers an objective and standardized assessment of pragmatic features is a good start.
Methods
In a pragmatic trial, the aim is to inform policy or clinical decisions by showing how an intervention could be integrated into everyday routine care. This is distinct from explanation trials, which test hypotheses about the cause-effect relationship in idealised situations. Therefore, pragmatic trials could have lower internal validity than explanatory trials, and could be more susceptible to bias in their design, conduct, and analysis. Despite their limitations, pragmatic research can provide valuable information to make decisions in the context of healthcare.
The PRECIS-2 tool evaluates an RCT on 9 domains, ranging from 1 to 5 (very pragmatist). In this study, the recruitment, organisation, flexibility: delivery, flexible adherence and follow-up domains received high scores, however, the primary outcome and the procedure for missing data were below the limit of practicality. This indicates that a trial can be designed with well-thought-out practical features, yet not harming the quality of the trial.
However, it is difficult to assess the degree of pragmatism a trial really is because the pragmatism score is not a binary characteristic; certain aspects of a trial can be more pragmatic than others. 슬롯 can be affected by modifications to the protocol or the logistics during the trial. In addition 36% of the 89 pragmatic trials discovered by Koppenaal and co. were placebo-controlled or conducted prior to licensing, and the majority were single-center. This means that they are not very close to usual practice and can only be described as pragmatic in the event that their sponsors are supportive of the lack of blinding in such trials.
Another common aspect of pragmatic trials is that researchers attempt to make their findings more relevant by analyzing subgroups of the trial sample. This can lead to unbalanced analyses with lower statistical power. This increases the chance of omitting or misinterpreting differences in the primary outcomes. In the instance of the pragmatic trials included in this meta-analysis this was a serious issue because the secondary outcomes were not adjusted to account for the differences in the baseline covariates.
Additionally, studies that are pragmatic may pose challenges to collection and interpretation of safety data. This is because adverse events are typically reported by participants themselves and are susceptible to reporting delays, inaccuracies or coding deviations. Therefore, it is crucial to improve the quality of outcomes ascertainment in these trials, and ideally by using national registries instead of relying on participants to report adverse events in a trial's own database.
Results
Although the definition of pragmatism does not require that all trials be 100 100% pragmatic, there are benefits to incorporating pragmatic components into clinical trials. These include:
Increased sensitivity to real-world issues which reduces study size and cost as well as allowing trial results to be more quickly implemented into clinical practice (by including routine patients). However, pragmatic trials be a challenge. For instance, the right kind of heterogeneity can allow a trial to generalise its results to different patients and settings; however the wrong kind of heterogeneity may reduce the assay's sensitivity, and thus decrease the ability of a trial to detect minor treatment effects.
Several studies have attempted to classify pragmatic trials using a variety of definitions and scoring methods. Schwartz and Lellouch1 created a framework to distinguish between explanatory trials that confirm a clinical or physiological hypothesis, and pragmatic trials that inform the selection of appropriate treatments in clinical practice. The framework consisted of nine domains that were scored on a 1-5 scale with 1 being more lucid while 5 was more practical. The domains covered recruitment and setting up, the delivery of intervention, flex adhering to the program and primary analysis.
The original PRECIS tool3 was an adapted version of the PRECIS tool3 that was based on the same scale and domains. Koppenaal and colleagues10 developed an adaptation to this assessment, dubbed the Pragmascope that was easier to use in systematic reviews. They discovered that pragmatic systematic reviews had a higher average scores across all domains but lower scores in the primary analysis domain.
The difference in the primary analysis domain could be explained by the fact that the majority of pragmatic trials analyze their data in the intention to treat manner while some explanation trials do not. The overall score was lower for pragmatic systematic reviews when the domains of the organization, flexibility of delivery and follow-up were merged.
It is important to remember that a pragmatic trial does not necessarily mean a low-quality trial, and indeed there is a growing number of clinical trials (as defined by MEDLINE search, however this is not specific or sensitive) which use the word 'pragmatic' in their title or abstract. These terms may indicate an increased awareness of pragmatism within abstracts and titles, however it's not clear whether this is reflected in content.
Conclusions
In recent years, pragmatic trials are gaining popularity in research as the value of real world evidence is increasingly recognized. They are randomized clinical trials that compare real-world care alternatives instead of experimental treatments under development. They involve patient populations which are more closely resembling those treated in routine care, they use comparators that are used in routine practice (e.g. existing drugs), and they depend on the self-reporting of participants about outcomes. This method can help overcome the limitations of observational research for example, the biases associated with the use of volunteers as well as the insufficient availability and codes that vary in national registers.
Pragmatic trials offer other advantages, such as the ability to leverage existing data sources and a greater chance of detecting significant differences from traditional trials. However, pragmatic tests may have some limitations that limit their effectiveness and generalizability. The participation rates in certain trials could be lower than expected due to the healthy-volunteering effect, financial incentives or competition from other research studies. The need to recruit individuals in a timely manner also restricts the sample size and impact of many pragmatic trials. Some pragmatic trials also lack controls to ensure that the observed differences aren't caused by biases that occur during the trial.
The authors of the Pragmatic Free Trial Meta identified 48 RCTs that self-described themselves as pragmatic and were published until 2022. They assessed pragmatism using the PRECIS-2 tool, which includes the domains eligibility criteria, recruitment, flexibility in adherence to interventions, and follow-up. They found 14 trials scored highly pragmatic or pragmatic (i.e. scoring 5 or more) in at least one of these domains.
Trials with a high pragmatism rating tend to have more expansive eligibility criteria than traditional RCTs, which include very specific criteria that are not likely to be used in the clinical environment, and they comprise patients from a wide range of hospitals. According to the authors, can make pragmatic trials more relevant and useful in the daily practice. However they do not guarantee that a trial will be free of bias. The pragmatism is not a definite characteristic; a pragmatic test that does not have all the characteristics of an explanation study may still yield valid and useful outcomes.