Communicating imprecision in prognostic tools

Lead Researcher(s):  Nick Bansback

Lead Institution: University of British Columbia

Co-Investigators: Mark Harrison, Paul K Han, Borsika Rabin

ARCC Program Area(s): Health Systems, Services, and Policy

Funding Term: 2017

Project Summary:

Understanding and communicating prognosis is an important area of interest for cancer patients, their caregivers and providers. This information can help support decision-making around care decisions and management of other conditions. While an increasing number of web-based tools have become available, there is a lot of variability in the way prognostic information is presented. This study aimed to understand what doctors and researchers who use or develop prognostic tools think about including information about imprecision in these tools, and how they can best be communicated so that the patient can interpret probabilities correctly without increasing their burden.

The research team reviewed 222 prognostic tools available online and extracted 772 uncertainty descriptions. An existing classification was adapted to classify each extracted statement by presentation of uncertainty. The results showed that when describing first-order uncertainty, almost all (90%) prognostic tools included a quantitative description, such as “chances of survival after surgery are 10%,” although there was heterogeneity in the use of percentages, natural frequencies, and use of graphics. Only 14% of tools described second-order uncertainty. Of those that did, most used a qualitative prefix such as “about” or “up to,” while 22 tools had quantitative descriptions using confidence intervals or ranges. A manuscript on the results of this study has been submitted to Cancer Epidemiology, Biomarkers and Prevention; the paper is currently under review.

The second stage of the project involved simple one-page survey sent to 66 email addresses identified from papers included in the search, and from individuals recommended by the advisory team. Of these, 22 responses have been received. Nearly all the respondents (20/22) were not surprised that ambiguity is not commonly reported in prognostic tools, but there was an even split to whether reporting ambiguity could be appropriate (10/20). Ten respondents expressed interest in ongoing work in this field.

Given the findings that there is considerable heterogeneity in the way uncertainties are being communicated in cancer prognostic tools, and the lack of agreement between tools and experts in how ambiguity should, if ever, be described, we believe there is  valuable further work to be undertaken.  ARCC seed grant funding has provided us with the evidence and the collaborations to proceed in writing proposals to undertake further work testing different approaches to ambiguity.  We plan to test whether these approaches inform intentions, knowledge, trust and conflict in collaboration with Dr Paul Han on a grant to the NIH.