A Bayesian hierarchical nonlinear model for assessing the association between genetic variation and drug cytotoxicity

Non-tumor cell-based model systems have recently gained interest in pharmacogenetic research as a hypothesis generating tool. The hypotheses generated from these model systems can be followed up in functional studies, or tested in individuals taking the same investigational agents. The current cellular phenotypes (e.g. cytotoxicity) of interest in these studies are based on the effects of an individual dosage of a drug on the cell lines, or a summary of results at many dosages of a drug (e.g. dose that inhibits 50 per cent of cell growth, GI50). A more complete analysis of the impact of genetic variation on all aspects of the dose-response curve may lend additional insight into the pharmacogenomics of a particular drug. This paper illustrates the use of a Bayesian hierarchical nonlinear model for the analysis of pharmacogenomic data with cytotoxicity endpoints. The model is illustrated with cytotoxicity and expression data collected on cell lines from a pharmacogenomic study of the drug gemcitabine. By completing an analysis based on the entire dose-response curve, we were able to detect additional genes that affect not only the GI50, but also the slope of the curve, which reflects the therapeutic index of the drug. Simulation studies also demonstrate that in comparison with the analyses based on the commonly used summary measure GI50, investigation of the impact of genetic variation on all aspects of the cytotoxicity dose-response curve is more informative, and more powerful with respect to detecting the effect of gene expression on cytotoxicity.