Assessing Prosthesis Performance

An important AOANJRR focus has been the continued development of a standardised algorithm to identify any prosthesis or combination of prostheses not performing to the level of others in its class.  This work is not readily apparent in the AOANJRR Annual Report but is critical to its function.  A pragmatic three-stage approach has been developed.

As currently implemented, the first stage is an automated system that selects for further attention any component where:

(i)     the revision rate (per 100 component years) exceeds twice that for the group, and

(ii)    the Poisson probability of observing that number of revisions, given the rate of the group, is less than 0.05, and

      either

(iii)   there are at least 10 primary procedures for that component,

      or

  (iv)  the proportion revised is at least 75% and there have been at least two revisions.

Additionally, if a component represents more than 25% of the group, its revision rate is excluded from estimation of the group's overall rate.  The purpose of this stage is to bring to early attention any prosthesis where there is a performance discrepancy.

In the second stage, the Director and Deputy Directors of the AOANJRR, in conjunction with staff of the SAHMRI, review the findings and decide if mention of a component in the Annual Report is possibly warranted.  Many factors are considered when making this decision.  They include amongst others the relevance of the statistical significance of the observed higher revision rate and the presence or absence of any confounding factors. It is known that many different factors may affect the outcome and careful consideration must be given before any particular prosthesis is highlighted.  To date a small number of the prostheses identified in the first stage of the algorithm have subsequently been highlighted in the AOANJRR Annual Report. The major reason for not including the majority of identified prostheses is inadequate numbers or the inability to exclude confounding factors. This algorithm will be subject to change as its performance is reviewed and further data are collected.

The third stage involves further review of those components identified in the second stage.  A panel of orthopaedic surgeons, who are members of the Arthroplasty Society attend the Surgeon Review Workshop undertakes this review together with the Chairman of the AOANJRR Committee, the Director and the Deputy Directors of the AOANJRR.  This workshop is open to all members of the Society however as there are limited positions available attendance is on a ‘first come first served' basis.  Participants are given the opportunity to request and view additional analyses as required.

Survival Analysis

The AOANJRR describes the time to first revision of a prosthesis using the Kaplan-Meier estimates of survivorship (refer to the Glossary in the Annual Report).  The estimates are displayed on the graph until the point at which the proportion of prostheses at risk for revision is at least 10% of the initial number at risk for that category, unless this number is greater than 100, in which case we extend the graph until the number at risk reaches 100.  This avoids uninformative, imprecise estimates at the right tails of the distribution where the number of primary prostheses at risk is low. However, analytical comparisons of prostheses survival using log-rank tests and proportional hazards models are based on all available data (ref Pocock SJ, Clayton TC, Altman DG.  Survival plots of time to event outcomes in clinical trials: good practice and pitfalls, Lancet 2002; 359: 1686-89).

Confidence intervals for the Kaplan-Meier estimates are point-wise Greenwood estimates and their appearance should not be used to infer whether overall differences in survival between prosthesis types are significant.  Rather, the log-rank tests and hazard ratios reported with each curve should be used for this purpose.

When we refer to the cumulative percent revision (CPR) at a certain time, for example five years, we mean the complement (in probability) of the Kaplan-Meier survivorship function at that time, multiplied by 100.  The cumulative percent revision, generically a "cumulative failure rate", also accounts for the right censoring due to death and the ‘closure' of the database at the time of analysis.