The quality of qualitative research

In this contribution, we discuss criteria for the quality of qualitative research. We consider reliability and validity as specifications of the comprehensive requirement for 'intersubjective replicability', with which qualitative research should comply. In the data collection phase, 'argumentative' reliability generally must suffice; in the data analysis phase, attention must also be given to 'technical' reliability. Validation of qualitative research has to take place via three approaches: 'communicative', 'critical' or 'empirical' validation. This clarifies the relative validity or 'authenticity' of qualitative research.

Qualitative Approaches in Health Services Research: A Methodological Primer

Qualitative methods are essential in health services research. Unlike quantitative methods, qualitative approaches are most effective in exploring the "how" and "why" of complex social phenomena, offering rich insights into attitudes, beliefs, and experiences that might otherwise be overlooked. Aimed at research teams new to qualitative work, this paper introduces qualitative research methods, explaining key terms, data collection strategies, and analytic approaches. We also discuss the theoretical foundations of and methodological rigor in qualitative research. Additionally, we present a novel model that conceptualizes qualitative research as the scaffolding for other research methods within the hierarchy of evidence, highlighting its role in generating hypotheses and adding greater context to quantitative findings. By providing an accessible overview of rigorous approaches to qualitative research, we hope to encourage greater application of qualitative methods in health services research.

Breaking Down Bias: A Methodological Primer on Identifying, Evaluating, and Mitigating Bias in Cardiovascular Research

Systematic error, often referred to as bias is an inherent challenge in observational cardiovascular research, and has the potential to profoundly influence the design, conduct, and interpretation of study results. If not carefully considered and managed, bias can lead to spurious results, which can misinform clinical practice or public health initiatives and compromise patient outcomes. This methodological primer offers a concise introduction to identifying, evaluating, and mitigating bias in observational cardiovascular research studies that examine the causal association between an exposure (or treatment) and an outcome. Using high-profile examples from the cardiovascular literature, this review provides a theoretical overview of 3 main types of bias-selection bias, information bias, and confounding-and discusses the implications of specialized types of biases commonly encountered in longitudinal cardiovascular research studies, namely, competing risks, immortal time bias, and confounding by indication. Furthermore, strategies and tools that can be used to minimize and assess the influence of bias are highlighted, with a specific focus on using the target trial framework, directed acyclic graphs, quantitative bias analysis, and formal risk of bias assessments. This review aims to assist researchers and health care professionals in designing observational studies and selecting appropriate methodologies to reduce bias, ultimately enhancing the estimation of causal associations in cardiovascular research.

Learning Health Systems Research: Continued Progress and Ongoing Challenges

Background: Learning health systems research (LHSR) builds on concepts of systems-based participatory research to form a new paradigm for partnered research. Objective: Defines LHSR and its ongoing challenges and future directions. Research design: Qualitative description of relevant dimensions of LHSR. Results: In LHSR, researchers and health system stakeholders co-create research with dual aims of producing internal quality improvement and generalizable, disseminable knowledge. This approach aligns research priorities with community and health system needs, resulting in interventions that are both feasible and acceptable in real-world settings and effective. LHSR methods and outcomes reflect elements of implementation science, particularly participatory implementation science and the use of mixed methods, but the field is distinct in its emphasis on co-creation with health system leaders and the use of theory to inform rather than drive the work. Practitioners of LHSR face challenges related to the complex and multi-stakeholder nature of the field, including the time-intensive nature of building partnerships, conflicting project time horizons, imprecision inherent in real-world data, and barriers to publication of the smaller studies that typically result from LHSR. Conclusions: Continued advancement of the field requires confronting these challenges with a variety of interventions, including explicit institutional support and incentives for this type of work, training and career development opportunities, a diversity of funding sources, investment in data resources and expertise, and inclusive research governance structures.

Experienced-based co-design for cardiovascular and chronic disease research

The participatory co-design method has received significant attention recently. Experience-based co-design is an approach that enables patients (service users) and healthcare providers (service delivers) to co-design services and care pathways in partnership to improve health outcomes based on their experience. Traditionally, it was used as a quality improvement technique. Yet, it is a valuable participatory research design that can help improve health outcomes and be applied in nursing research. This paper will discuss its application in research among people living with cardiovascular multimorbidity and its practicalities, usability, and impact on cardiovascular and chronic disease research and models of care.

Randomization in Pre-Clinical Studies: When Evolution Theory Meets Statistics

Randomization is a statistical procedure used to allocate study subjects randomly into experimental groups while balancing continuous variables. This paper presents an alternative to random allocation for creating homogeneous groups by balancing experimental factors. The proposed algorithms, inspired by the Theory of Evolution, enhance the benefits of randomization through partitioning. The methodology employs a genetic algorithm that minimizes the Irini criterion to partition datasets into balanced subgroups. The algorithm's performance is evaluated through simulations and dataset examples, comparing it to random allocation via exhaustive search. Results indicate that the experimental groups created by Irini are more homogeneous than those generated by exhaustive search. Furthermore, the Irini algorithm is computationally more efficient, outperforming exhaustive search by more than three orders of magnitude.

Considerations when designing, analyzing, and reporting reliability studies

Background: Reliability studies have a long history in the physical therapy literature and their sophistication has evolved over the decades. Often, however, there has been incomplete reporting or a lack of coherence among study purpose, design, choice of analysis, sample size justification, and reporting of results. Two possible explanations for this oversight are a vaguely written purpose statement and statistical software that does not provide all essential information. Objective: The goal of this masterclass is to provide considerations and resources to assist investigators structure a coherent reliability study design and subsequent presentation of results. Discussion: This masterclass highlights the importance of framing a study purpose that clearly distinguishes between a hypothesis testing and parameter estimation study and appropriately labelling the study design. It also stresses the importance of stating whether the raters are the only ones of interest or whether they are intended to represent a larger group of raters, applying a sample size calculation consistent with the study purpose, and reporting results that align with the study purpose and design.

Analyzing lognormal data: A nonmathematical practical guide

Lognormal distributions are pervasive in pharmacology and elsewhere in biomedical science, arising naturally when biological effects multiply rather than add. Despite their ubiquity in pharmacological parameters (eg, EC50, IC50, Kd, and Km), lognormal distributions are often overlooked or misunderstood, leading to flawed data analysis. This largely nonmathematical review explains why lognormal distributions are common, how to recognize them, and how to analyze them appropriately. We show that many measured variables are lognormal. So are many derived parameters, particularly those defined as ratios of lognormal variables. Through examples and simulations accessible to working scientists, we demonstrate how misidentifying lognormal distributions as normal leads to reduced statistical power, unnecessarily large sample sizes, false identification of outliers, and inappropriate reporting of effects as differences rather than ratios. We challenge the common practice of using normality tests to decide how to analyze data, showing that many data sets pass both normality and lognormality tests, especially with small sample sizes. Instead, we advocate for assuming lognormality based on the nature of the variable. This review provides practical guidance on recognizing and presenting lognormal data, and comparing data sets sampled from lognormal distributions. Based on Monte Carlo simulations, we recommend the lognormal Welch's t test or nonparametric Brunner-Munzel test for comparing 2 unpaired groups, the lognormal ratio paired t test for paired comparisons, and lognormal ANOVA for ≥3 groups. By recognizing and properly handling lognormal distributions, pharmacologists can design more efficient experiments, obtain more reliable statistical inferences, and communicate their results more effectively. SIGNIFICANCE STATEMENT: Lognormal distributions are common in pharmacology and many scientific fields, but they are often misunderstood or overlooked. This review provides a detailed guide to recognizing and analyzing lognormal data, aiming to help pharmacologists perform more appropriate and more powerful statistical analyses, draw more meaningful conclusions from their data, and communicate their results more effectively.

Confidence Intervals and Sample Size for the ICC in Two-Way ANOVA Models

The reliability of measurement instruments is vital in fields like medicine and psychology, where these tools are often used for diagnostic purposes. In reliability studies where participants are assessed by the same set of raters, the data can be modeled using a two-way ANOVA, with the intraclass correlation coefficient (ICC) as a key metric. This paper focuses on the ICC for agreement, which is crucial when the measurement values themselves, rather than just their rank ordering, are of interest. However, selecting appropriate confidence interval methods and determining adequate sample sizes for the ICC for agreement remains challenging. This work advances the understanding of confidence interval methods for the ICC for agreement, provides practical tools, and offers recommendations for selecting confidence interval methods and sample size procedures for planning reliability studies. In particular, we provide a comprehensive review and simulation-based comparison of six classes of confidence interval methods for the ICC for agreement identified in the literature. Our analysis includes a method based on the F-distribution, previously omitted, which demonstrates the best statistical properties in some cases. Then, in conjunction with the best-performing methods, we evaluate three sample size determination procedures based on the expected width of confidence intervals that we identified in the literature. To address the lack of accessible tools, we further developed an interactive R Shiny app, freely available to researchers, to compute confidence intervals and sample sizes. The utility of these methods is illustrated by a study on fetal heart rates.

Reviewing methodological approaches to dose-response modelling in complex interventions: insights and perspectives

Background: Understanding dose-response relationships is crucial in optimizing clinical outcomes, particularly in complex interventions such as psychotherapy. While dose-response research is common in pharmaceutical contexts, its application in complex interventions remains underexplored. This review examines existing statistical methods for modelling dose-response relationships in complex interventions, focusing on psychotherapy. Methods: A systematic literature search following PRISMA guidelines identified studies proposing novel statistical methods or innovative applications of methods for analysing dose-response relationships. The search encompassed various databases, yielding 224 articles. After screening and exclusion, seven studies were eligible for analysis. Data synthesis categorized methods into three groups: multilevel and longitudinal modelling, non-parametric regression, and causal inference with instrumental variables. Additionally, a survey was conducted among clinical researchers to understand their perspectives on dosing decisions in psychotherapy trials. Results: Multilevel and longitudinal modelling techniques, although informative, were only applicable to participants with sessional data, limiting causal interpretations. Non-parametric regression methods provided avenues for causal inference but were constrained by assumptions. Causal inference with instrumental variables showed promise in addressing these limitations, particularly in randomised controlled trials, yet still require a priori assumption of the dose-response function. The results of our survey suggested that there is not sufficient information available to clinical researchers to make empirical dosing decisions in psychotherapeutic complex interventions. Conclusions: This review highlights the scarcity of robust statistical methods for evaluating dose-response relationships in psychotherapy trials. The dose-response methodology applied to RCTs remains underdeveloped, hindering causal interpretations or requiring strong assumptions. Traditional approaches oversimplify outcomes, highlighting the need for more sophisticated methodologies. Clinical researchers emphasized the necessity for clearer guidelines and enhanced patient involvement in dosing decisions, echoing the broader findings of the review. Future research requires methodological advancements to inform effective decision-making in psychotherapy trials, ultimately optimizing patient care and outcomes.

Surrogate Marker Evaluation: A Tutorial Using R

The practice of using a surrogate marker to replace a primary outcome in clinical studies has become widespread. Typically, the primary outcome requires long-term patient follow-up, is expensive, or is invasive or burdensome for patients to measure, while the surrogate marker is not (or less so). Of course, a surrogate marker must be validated before it should be used to make a decision about the effectiveness of a treatment. There has been a tremendous amount of statistical and clinical research focused on evaluating and validating surrogate markers over the past 35 years. Although there is ongoing debate over the optimal evaluation method, the development of new approaches and insights has greatly enriched the field. In this tutorial, we describe available statistical frameworks for evaluating a surrogate marker and specifically focus on the practical implementation of the proportion of treatment effect explained framework. We consider both uncensored and censored outcomes, parametric and non-parametric estimation, evaluating multiple surrogates, heterogeneity in the utility of the surrogate marker, surrogate evaluation from a prediction perspective, and the surrogate paradox. We include R code to implement these procedures with a follow-along R markdown. We close with a discussion on open problems in this research area, particularly in terms of using the surrogate marker to test for treatment in a future study, which is the ultimate goal of surrogate marker evaluation.

Balancing Events, Not Patients, Maximizes Power of the Logrank Test: And Other Insights on Unequal Randomization in Survival Trials

We revisit the question of what randomization ratio (RR) maximizes the power of the logrank test (LRT) in event-driven survival trials under proportional hazards (PH). By comparing three approximations of the LRT (Schoenfeld, Freedman, and Rubinstein) to empirical simulations, we find that the RR that maximizes power is the RR that balances the number of events across treatment arms at the end of the trial. This contradicts the common misconception implied by Schoenfeld's approximation that 1:1 randomization maximizes power. Besides power, we consider other factors that might influence the choice of RR (accrual, trial duration, sample size, etc.). We perform simulations to better understand how unequal randomization might impact these factors in practice. Altogether, we derive 5 insights to guide statisticians in the design of survival trials considering unequal randomization.