A critical component of HIV drug resistance testing is how well different genotype interpretation systems perform at predicting viral phenotypes and virologic responses to new drug regimens. Two studies that examined this issue and which were presented at the XIV International AIDS Conference in Barcelona are described here. As discussed in a previous article, results of the HAVANA study suggested that expert advice plays a role in the clinical effectiveness of resistance testing (see related article). In an attempt to address the interpretation challenges inherent in resistance testing, the GUESS study compared the genotype interpretations provided by a panel of 12 international experts [1]. This study provided 50 sequences to the experts (for a total of 600 predictions) and evaluated (1) their ability to accurately predict the phenotype, (2) agreement between the experts in predicting phenotype and drug activity, and (3) agreement among experts in providing treatment recommendations and consistency in interpretation of phenotype and drug activity and treatment recommendations over time.

Each sequence had a corresponding phenotype test available. The sequences included 11 to 36 RT mutations (median, 22) and 5 to 22 mutations in protease (median, 11). The experts were asked to predict the phenotypic fold change in susceptibility for each drug. Overall, they were correct 30% to 40% of the time, and more often when predicting 3TC (approximately 75%) and NNRTI (approximately 66%) resistance. For the NRTI calls, the experts systematically predicted a higher fold reduction in susceptibility than was found on phenotypic testing, and for the PI, under-calls and over-calls were equivalent. In terms of agreement between the experts, there was about 70% agreement for 3TC and the NNRTIs and about 40% agreement for the remaining NRTI and the PI.

The experts were asked to choose from 6 fold-change categories when predicting phenotype. When agreement within the same category +/- one category was evaluated, over-all agreement increased to between 70% and 90%. Predictions of drug activity were better for the PI than for the NRTI. Despite significant variability in interpretations, the experts made similar treatment recommendations 80% of the time. When consistency in interpretation and recommendations were evaluated over time, the predicted phenotypic fold-change and drug activity changed, although treatment recommendations were unchanged 80% to 90% of the time.

What can we conclude from this study? Even the experts have a hard time interpreting genotype sequences, although treatment recommendations based upon genotype sequences are quite similar.

And How do the Systems Do?
It's not just the experts that have hard time seeing eye to eye on the meaning of individual genotypes. A study presented in poster form assessed the concordance of sequence interpretations provided by the Stanford University HIV RT and Protease Sequence Database, the TruGene^TM algorithm (Visible Genetics) and the Virtual Phenotype^TM(Virco) [2]. Sequences from 293 samples were evaluated for resistance to 14 drugs. Complete concordance was found in 13.7% of the samples. There was a higher level of agreement for the NNRTI and many of the PI (70%) than for the NRTI (<50%). The three algorithms were concordant for 3TC more than 90% of the time.

Conclusions
The implications of these two studies give cause for concern: there is no agreement about the best method to approach genotype interpretation (rules-based vs. correlational database). Further, there are no published outcomes data comparing the clinical utility of these approaches, and if the experts cannot agree on individual sequence interpretations, it is difficult to imagine that there will soon be an international consensus regarding interpretation methodologies.

For a review of previous study findings concerning genotype interpretation systems, please see "Interpreting Genotypic Resistance Tests" by Drs. Robert Shafer and Jonathan Schapiro.

HIVresistanceWeb visitors interested in comparing different genotype interpretation systems with respect to a single sequence or set of sequences are encouraged to explore Stanford's HIV Algorithm Comparison program, which is available online.

References

1. AR Zolopa, LC Lazzeroni, A Rinehart, D Kuritzkes. The GUESS study: An international evaluation of the accuracy, precision consistency of expert HIV-1 genotype interpretation. XIV International AIDS Conference. 7-12 July 2002, Barcelona, Spain. Abstract ThOrB1385.
2. GH Kijak, AE Rubio, C Zala, P Cahn, PR Harrigan, R Galli, JS Montaner, H Salomon. Discrepant results in the interpretation of HIV-1 drug-resistance genotypic data among widely used algorithms. XIV International AIDS Conference. 7-12 July 2002, Barcelona, Spain. Abstract MoPeB3125.