What is the evidence that phenotyping is a better guide to therapy than genotyping in the expert interpretation of resistance test results?
Moderator: We always say that phenotype is the gold standard, but is this really true? In fact, what is the evidence that phenotyping is a better guide to therapy than genotyping in the expert interpretation of the results?
Mark, do you want to take that question first?
Mark Wainberg: I really don't think that there is any solid evidence at this time to suggest that phenotyping is better than genotyping is. In fact, I think we've tended to build up a myth for the last number of years that this is likely to be the case. But in fact, if you look at the VIRADAPT study, it does not seem to indicate significant differences any more than some of the others did. I think in fact that we should encourage the performance of a clinical trial that will directly address this subject and related areas as well that are of vital importance to all of us.
As an example, we have previously talked about the virtual phenotype and what its direct relevance may be, and I think there really is ground for randomized studies that would investigate the extent to which direct genotyping and expert interpretation may be superior, inferior, or otherwise with regard to both virtual phenotypes as derived from computer generation of results and as well direct phenotypes as observed through performance of growth inhibition assays by drugs.
David Katzenstein: Can I just ask you, Mark, to comment on what you think the correct data presentation of the NARVAL study actually tells us about the possibilities and really the constraints on doing exactly the trial that you're describing, which I think actually the French investigators, the ANRS, attempted to do, directly compare phenotype and genotype.
Mark Wainberg: Yes, they did. But there may be other reasons, David, why that study didn't work out as well as all of us would have liked, and I'm not sure that we should take from the NARVAL study the message that we can do it again in a more successful way.
David Katzenstein: Right. It's how to learn lessons from perhaps the early results as seen in the interpretations as to how to design the studies you were suggesting.
Moderator: Jonathan, do you have anything else to add there?
Jonathan Schapiro: Yes. First of all, I definitely agree with Mark that there is no convincing data today and that we definitely still do need the randomized trial. I think the three trials which have looked at this to date would include, as Mark said, the VIRADAPT, the French NARVAL study, and the Kaiser study were attempts to answer the question but definitely have not encompassed all patient populations, and each one has its own individual limitations, and definitely there has not been data there to suggest that it's a better guide than genotyping. I think that we have to, as we do for genotyping, differentiate between the assay and its interpretations.
I think both the commercial companies today that are doing that, they have worked very hard and greatly improved the level, the technical level of the assay itself which I think is performed at a very high level at these very good laboratories. I think once again the problem lays in the interpretation and the decision of cutoff which were driven by the technical aspects of the test and not by clinical relevance. And I think many of the disappointing results today have not been only due to technical problems with turnaround time, but probably more so with interpretation. And I think as we struggle with interpretation of genotyping, I think we now realize that interpretation of phenotyping is equally challenging, and I think probably efforts have to be guided towards bringing clinical relevance to the result of phenotyping. I think the virtual phenotype is not a gold standard. I think we have to relate it of course to clinical data, not to technical cutoff. And probably before randomized trial is attempted, I think we probably have to work out better clinical cutoff or I think we'll probably once again have disappointing results.
Moderator: So, Jonathan, you would say current problems with phenotyping are mostly related to the lack of correlation between clinical endpoints and, say, the increase in IC-50 or IC-90 observed?
Jonathan Schapiro: Yes, I would say that. I think there are limitations that the turnaround time and costs are still a problem, so I wouldn't say that we've solved those. I think that the price of course is still very, very high, and the turnaround time is still too long, but just improving those won't solve the problem without the issue that you mentioned.
Moderator: Mark, do you or Jonathan see a problem with basically only having two companies in the marketplace with these assays?
Mark Wainberg: Of course. I think that we would be making a huge mistake as a field to permit anything close to a monopoly to be established, and really the question will be can others enter into this market given the constraints that are mostly of a financial nature.
We've heard as an example complaints in regard to the type of assay used in the NARVAL study because of the manner in which that particular assay has been carried out. But rather than discourage the people who did phenotyping in NARVAL, I think in fact we ought to encourage them to improve their assay so that we do have other choices.
One aspect that Jonathan perhaps had intended to touch on in the comparison of the two company assays that are available now relates to the fact that they have somewhat differing cutoffs in regard to what they consider to be clinically significant levels of resistance. And I think that this is something as well that needs to be broadcast to potential users who may not
be aware of the subtle differences among different tests.
Jonathan Schapiro: Yes, that's definitely a major point. I think we saw data which was comparing the outcome of saquinavir versus the classic 90 mutation being very different results from the two companies based on exactly that point, Mark, because it falls around a three-fold change. In saquinavir, the results are very different when that mutation is present by itself. With the two companies that was shown at the last resistance workshop.
What I would just mention in just mentioning the French group, I don't think there's any real data to suggest that the French study used technology that was inferior to any company. I don't think there has been any objective or evaluation of a different technology which would say that the NARVAL study and the French group which performed it performed any less well than the other two. So I would agree that others should be encouraged to do it, and probably there should be some sort of outside organization or some sort of objective, academic rating of these, and I think the fact that someone says that's inferior technology has not been shown. I think the group doing it in France has a very good track rate, they've been doing it for many years, and right now we really don't have good tools to compare the different technologies.
Mark Wainberg: But we should point out, Jonathan, that the ways of Sitges, Spain, were full of comments about how inferior the methodology used by François Clavel was relative to the other methods available. So irrespective of the fact that I agree with you, we need to contend with the reality that the perception in many quarters of the François Clavel assay is that it is not as good as the others.
Jonathan Schapiro: I'm not sure, Mark, that I would agree that that was an objective evaluation. I think there may or may not be other factors coming to play there. I think the quarters(?) and pitches(?) are important, but I think that probably short of having François Clavel defend his technology versus the others, I would say there's no data today to suggest that.
Mark Wainberg: Jonathan, I absolutely agree with you, but I'm pointing out that this is an area in which subjective interpretations and their consequences nonetheless have to be factored in.
Can I just raise one more issue that perhaps I'd like Jonathan or others to comment on? In regard to phenotype, is there still any role at all in our universe for the ACTG-DOD assay? And maybe Doug Mayers, since he's just joined us, could comment, since he had a lot to do with putting that assay together.
Doug Mayers: I think that at the current time that the ACTG-DOD assay had a major impact back in the '90s. I think that with the new recombinant virus assays that they avoid the issues related to passage of virus to culture and that they allow more rapid screening of isolates. I still think that for new drugs, new classes of drugs we may at least need to go back to primary cells to make sure that they act the same way as the recombinant virus assays do. And so to look at, for example, in macrophages, to look in primary T cells, to potentially look in other target cells. If we want to look at drug activity, I think you still have an advantage in an exploratory way using
primary cell lines. But once you've confirmed that the recombinant assay gives a similar result, I think that shifting to that for clinical practice makes a lot of sense.
Bob Shafer: I have a comment on that. I think one of the problems with the ACTG-DOD assay is that everybody did it a little bit differently, and maybe that's because not everybody did it exactly the way it was first written, and so there are so many -- because I've been collecting the susceptibility data in my online database, and there's so many studies where they say they
use ACTG-DOD assay, but the conditions are not actually clear.
And for instance, the inoculum, how much virus you put in, is a key factor. And many different studies put in many different inoculum sizes. But I think the key thing with any assay really is what the cutoffs are. It's key that there has to be for each drug a well-defined cutoff for what's wild type, what's not wild type, and for what's the maximum level of resistance you can get. And the problem with the ACTG-DOD is that too many different groups have used it and not really tried to establish and define clear-cut cutoffs of the wild type range, the difference between wild type and a virus that's been selected under drug pressure and to also define the dynamic range between the susceptible and highly resistant.
Doug, what do you think of that assessment?
Doug Mayers: I think that the assay system cut down dramatically the lab-to-lab variability around the world looking at susceptibility. But you are correct, that many people do little twists on the assay which do affect it, especially inoculum size. So the assay is only as good as people actually follow the protocol. But I think it did dramatically reduce the variability that was seen in multiple different assay formats, and I think it still has a role potentially in looking at primary cell lines to look at a drug's activity. But I think it's largely going to be supplanted by recombinant assays because they avoid the issues of passage of virus and they provide a less variable output than the primary cell lines do.
