Objectives: Reported is the importance of properly reflecting uncertainty associated with prognostic model estimates when calculating the survival benefit of a treatment or technology, using liver transplantation as an example.

Methods: Monte Carlo simulation techniques were used to account for the uncertainty of prognostic model estimates using the standard errors of the regression coefficients and their correlations. These methods were applied to patients with primary biliary cirrhosis undergoing liver transplantation using a prognostic model from a historic cohort who did not undergo transplantation. The survival gain over 4 years from transplantation was estimated.

Results: Ignoring the uncertainty in the prognostic model, the estimated survival benefit of liver transplantation was 16.7 months (95 percent confidence interval [CI], 13.5 to 20.1), and was statistically significant (p<.001). After adjusting for model uncertainty using the standard errors of the regression coefficients, the estimated survival benefit was 17.5 months (95 percent CI, −3.9 to 38.5) and was no longer statistically significant. An additional adjustment for the correlation between regression coefficients widened the 95 percent confidence interval slightly: the estimated survival benefit was 17.0 months (95 percent CI: −4.6 to 38.6).

Conclusions: It is important that the precision of regression coefficients is available for users of published prognostic models. Ignoring this additional information substantially underestimates uncertainty, which can then impact misleadingly on policy decisions.

Objectives: To identify and examine the methodologic issues related to evaluating the effectiveness of treatment adherence to clinical guidelines. The example of antiretroviral therapy guidelines for human immunodeficiency virus (HIV) disease is used to illustrate the points.

Methods: Regression analysis was applied to observational HIV clinic data for patients with CD4+ cell counts less than 500 per μL and greater than 50 per μL at baseline (n = 704),using Cox proportional hazards time-varying covariates models controlling for baseline risk. The results are compared with simpler models (Cox model [without time-varying covariates] and logistic regression). In addition, the effect of including a measure of exposure to antiretroviral guidelines in the model is explored.

Results: This study has three implications for modeling clinical guideline effectiveness. To capture events that are time-sensitive, a duration model should be used, and covariates that are time-varying should be modeled as time-varying. Thirdly, incorporating a threshold measure of exposure to reflect the minimum period of time for guideline adherence required for a measurable effect on patient outcome should be considered.

Conclusions: The methods proposed in this paper are important to consider if guidelines are to evolve from being a tool for summarizing and transferring the results of research from the literature to clinicians into a practical tool that influences clinical practice patterns. However, the methodology tested in this study needs to be validated using additional data on similar patients and using data on patients with other diseases.

A vector X of patient prognostic variables is modeled as a linear diffusion process with time-dependent, non-random, continuous coefficients. The instantaneous force of mortality (hazard function) operating on the patient is assumed to be a time-dependent, continuous quadratic functional of the prognostic vector. Conditional on initial data X0, the probability of surviving T units of time is expressed in terms of the solution of a Riccati equation, which can be evaluated in closed form if the coefficients of the process and the hazard are constant. This conditional expectation does not preserve proportional hazards.