Prasad Patil (Department of Data Sciences, Dana-Farber Cancer Institute, Department of Biostatistics, Harvard T.H. Chan School of Public Health)

 Title:  Replicability of genomic signatures and scientific results

 Abstract: There has been an increased emphasis on replicability - the belief that a result   can be obtained again or confirmed in a new sample - in the conduct of scientific   research. I will describe a series of projects (1) assessing replicable behavior in the training of genomic signatures (predictors that use gene expression measurements as input) and (2) communicating the replicability of scientific study results.

In genomic signature development, technical choices and cohort composition can change a signature’s predictions for the same patient. I will show empirical and preliminary theoretical results for training ensemble predictors using multiple studies’ worth of data. In this “intermediate-data” setting, out-of-study predictive generalizability can be improved by leveraging inter-study heterogeneity in features and their associations with the outcome. Closely related to transfer learning, distributed/federated learning, and covariate shift, this problem arises out of site-specific differences in patient recruitment, patient characteristics, and measurement technology.

Discussions on reproducibility and replicability of the results of scientific studies have gone forward without consensus establishment of definitions for these terms and of expectations for how many and how often studies should replicate. I will offer a re-analysis of the conclusions of a major replication effort and an R package designed to visually compare and communicate replication attempts.