Computer-Adaptive Testing

PRO data collection is one of the cornerstones in clinical research and quality assurance in orthopaedics. Over the last decades a number of validated measures have become available allowing assessment of general health and joint-specific outcomes. Until recently PRO data collection was mostly done using paper questionnaires (often as mail surveys). However, electronic modes of questionnaire administration (e.g. on tablet PCs) are increasingly employed to reduce human resources and logistic challenges (e.g. printing, posting, scanning, or manual data entry). Electronic questionnaire administration (ePRO) using today’s computer technology comes with higher initial costs (for tablet PCs and a suitable software package), but pays off quickly and provides advantages such as instant data display and swift data management. Immediate availability of individual PRO results enables inclusion of important patient feedback into clinical practice to complete to the picture.

Another advantage of ePRO is the option to conduct computer-adaptive assessments instead of traditional static questionnaires.

Computer-adaptive testing (CAT) is an innovative way to assess patient-reported outcomes. The principles of CAT reach back to the 70’s and it has been used extensively in social sciences and also more recently in various medical fields (e.g. medical psychology, oncology).

CAT is based on advanced psychometric measurement models that allow tailoring of questions to the individual patient. This increases measurement precision and can reduce the number of questions a patient is asked. Each question is well-selected from an item bank (a set of questions with their psychometric characteristics) with the help of a CAT algorithm. The algorithm determines the most suitable items, i.e. the most informative items providing best measurement precision for the current condition of an individual patient.

In more detail, the algorithm starts with an item covering the expected centre of the measurement continuum. Based on the response to that first item, the CAT algorithm calculates a score estimate (i.e. a patient’s position on the measurement continuum), and selects that item from the item bank that provides maximum information for that part of the measurement continuum. Each additional item narrows down the confidence interval of the score estimate. The algorithm stops presenting further items if a predefined confidence interval is reached or if a maximum number of items has been asked.

Naturally, the calculated CAT scores are comparable between patients and groups just like in traditional questionnaires applying classical testing theory. The underlying measurement models ensure comparability of scores, even though different sets and numbers of questions have been asked.

Compared to traditional PRO measures (using the same questions for all patients) CAT provides several advantages:

  • questions are more appropriate to an individual patient

  • lower number of questions or

  • higher measurement precision

  • adjustable number of questions

  • adjustable statistical power

 The powerful combination of the forgotten joint concept, ePRO and CAT is a major step towards improving PRO assessment in orthopaedics. Various projects are currently centered around developing item banks for CAT in shoulder, hip, and knee patients. Details on a pilot CAT version of the Forgotten Joint Scores for hip and knee patients are available online (Open Access).