MP3 File: Part 1 of audio
MP3 File: Part 2 of Audio
Blumsohn: I will tell you what my problems are and then start with that as a basis for things. Well, a lot of it ties into the way in which this spline or Lowess function was drawn. But the problem I have is what we are saying about this sort of threshold.
In the 5mg dose group there is no relationship whatsoever between PINP reduction and whether someone is going to have a fracture or not. And when you look at the actual raw data [on the plot] or the plots coloured on the screen, visually there is no impression that the PINP is anything other than randomly distributed in terms of fracture.
Eastell: OK, so these are the two big issues you see.
Blumsohn: My understanding of the model that we are using is that you , that the concept of the model is that you could have a relationship between the marker and the outcome without that being causative, but in order to be able to talk about percentage explained you have to have some explanation to start off with. So if there isn't an association between percentage change in PINP in the 5mg dose group and fracture within that group, whatever model you choose, the confidence interval for percentage explained has to overlap zero by definition, and I just don't understand why that is is the case that it doesn't overlap zero - it doesn't make any sense to me.
Blumsohn: And even if it didn't overlap zero, I still think that it is not clear from the way we are presenting it that in people taking risedronate it isn't explaining anything.
Blumsohn: Part of it is linked into this thing as well. So if you look at the NTX's [in the hip study] there is no-one ... there are 11 fractures altogether in the 5mg dose group.
Barton: Yes, so there are not many.
Blumsohn: Which I think is a third issue that we need to be showing the data in such a way that tells people what we are actually seeing (and that the number of fractures are fairly small). But if you look those 11 fractures, I can't remember what the exact cutoff was, but at an NTX change of more than 60% or something like that there weren't any fractures at all.
Blumsohn: The biggest NTX change that had a fracture was at an NTX change of about 60%.
Barton:That was based on the quintiles, quartiles....
Blumsohn: No however you do it, if you just look at the actual numbers [looking at graph]....
Barton: Oh, OK
Blumsohn:....the actual cutoff where there were no more fractures was an NTX change of about 60% odd [looking at plot produced], so what I don't understand from that is that if in the bottom 40% of NTX responses there were no fractures at all, the best estimate of the fracture risk reduction in those 40% of [biggest] responders has to be 100%
Blumsohn: the problem is when you do your LOESS or spline or whatever, whether that thing actually gets to zero depends on what constraining factors you are putting on that spline and what your smoothing factor is, and obviously whether that can get all the way down to zero depends on how bendy you allow that LOESS function to go, and the bendiness that you put in.... what is happening is that it is not allowing that to get down to zero...... the confidence intervals of that thing at NTX responses over there don't get anywhere near zero even though the best estimate should in fact be zero.
Barton: Yes there are two parts to your question. The one is to do with the flexibility, and you are right depending on the coefficient you can have a linear line or you can allow some curvature. The other things which is the constraining factor is the distribution of the X-axis [exactly!!]. If I put it all the way to 100% that will go to zero, so regardless of the coefficient it would go to zero because of your comment here - no one fractures.
Blumsohn: But whatever the cause is, the fact of the matter is that this curve has to be misleading.
Barton: Right, misleading ... if you go outside that range?
Blumsohn: In that the shape of that curve doesn't correspond with what you are seeing in reality.
Blumsohn: What reality is telling you if you look at the curve is that your best estimate of fracture risk reduction is ...[ looking at plot] the scale is illegible here, ... even at the biggest NTX changes, the reduction would be somewhere between there and there and not even overlapping zero. Which obviously doesn't make any sense if there weren't any fractures... at all.
Barton: It stops at 60, I think it stops about there.
Barton: So if it goes to 100%. I agree if you increased the X-axis that then that would go to zero, yes.
Blumsohn: So it may well be true what we are saying but I think it is really difficult.....
Barton: I agree with you about the threshold. I don't think we can say that beyond a certain level....
Barton: To your point about the relationship .... um, you are quite right that when you look at PINP 5mg there is not a significant relationship. Now if you look at the model where..... [some arcane statistical discussion]
Blumsohn: But this is what I don't understand. My understanding (and my understanding from speaking to Li) is that in order for the model to mean something at all, there has to be some sort of relationship, both within the placebo group and the treated group. You haven't got a relationship in your placebo group or your treated group then by definition you can't show anything. Which is why I don't understand why the model for PINP is coming out as sig....
Blumsohn: .... even though they are not significant individually...
Eastell: Your concern is that this is a model that includes ......
Blumsohn: No, my concern is that I don't understand how you can get this result! but anyway...
Blumsohn: Those are the straightforward COX regressions [reiterates that these are not even vaguely significant in the individual groups at 5mg] so within each group there is no association...... that would fit in with the graphical presentation which looks as if they are pretty much randomly distributed across each of those things.
Blumsohn: My understanding is that this whole model relies on having significant prediction within those groups. So if you were looking at cholesterol for example if there wasn't any relationship between cholesterol and heart attacks within the general population and if there wasn't any relationship between cholesterol and heart attacks in people on a statin you couldn't possibly have a model which says that cholesterol is a significant surrogate as a predictor of myocardial risk reduction with statins. I think that is correct.
Barton: Not quite. The model assumes that the relationship between the surrogate and the response i.e. fractures is consistent between treatment groups. So that is why if you look at the paper it says that the first thing you do is test whether there is an interaction between the treatment and surrogate.
Barton: If there isn't an interaction you assume that the relationship within those two groups is similar. So if you look at your hazard ratio, in the placebo it is 1.000, for 2.5 it is 1.004 and for 5mg it is 1.017.
Barton: So because that interaction isn't significant you can assume that the relationship regardless of whether it is placebo or 5mg is similar OK. So when you [statistical discussion - in my view grossly misleading for several reasons]....... so if you look at these hazard ratios they are almost identical (!).
Blumsohn: I'm not sure I follow that totally but anyway.....
Blumsohn: When someone is looking at results in a clinic, they put a patient on treatment and the question is does the size of the change .... in a way they are not interested in the placebo group at all.
Blumsohn: ... and they want to know, does the change in PINP relate to whether someone is going to have a fracture risk reduction
Blumsohn: So the issue of surrogates is a bit of a different issue I guess. Um, but based on this data I don't think we can say that.
Barton: All I can say that is in order to use the Li model you have to assume that the surrogate is related to your outcome of fracture and that was in response to phil Ross as well. So if you look at the model you actually get this relationship. You don't get significance with the treatment groups individually just because the sample size is much smaller.
Barton: If you look at the paper the [reiterates from before]
Blumsohn: I mean the other slight anxiety that I have here is that when we looked at this together when I came down we had various sets of simulated data as you know, and some of them made the model explode.... but then there was this other one that we put in that was essentially just random numbers ... and that one yielded an apparent percentage explained of something more than 100%
Blumsohn: So the question is about the coding of this in SAS and whether there might be some problem with that. It doesn't actually made sense to me how you can have no prediction in either the placebo group and treated group and yet have a percentage explained on the basis of the way in which the model works as I understand it anyway.
Blumsohn: Apart from the issue of the modeling, the bottom line issue is that if we are conveying an impression to clinicians that they can look at a PINP result and if the PINP result is big that means that the patient is going to get a better fracture risk reduction. That may be true, but I don't think we can say that on the basis of this data.
Barton: Umm. Yuh. I mean if it's going to....
Blumsohn: If that's the case .... [reiterates the cholesterol analogy]. It doesn't make sense to me. Anyway we are going round in a circle...
Barton: I think that right... but the one thing that say like Merck might say, is "OK, you are telling me that once you decrease NTX by more than say 60 or 70% that you don't have... that no one is going to fracture". Because that is contradicting our original manuscript. I just know what Merck are like.
Barton: I think they are going to use it.
Eastell: But if Merck turned up and they talk to you me or Aubrey and that start saying "but where is this story about if its over 30% there is no [further reduction in fractures]". Well "in the Hip trial you can see we have only got this many, 11 fractures. How can we possibly comment about less than 70". I think we had 60 fractures in the other trial.
Barton: Yes we had so many more....
Eastell: OK. Terribly sorry I have to stop my interaction with you. Have we covered everything you wanted.
Barton: I think so. I am just a bit concerned that maybe Aubrey is not completely comfortable with it.
Blumsohn: I feel very uncomfortable presenting things that I am not convinced about. And obviously the degree of discomfort will accelerate between a normal poster, a plenary poster, having to give an oral and writing a paper.
Barton: Now in terms of manuscripts I really want to get this ?? up as soon as possible. Because we have got a medical writer who can do it. So maybe at ASBMR we can sit down and agree what the next things are.
Eastell: The ASBMR with all give us the opportunity of Aubrey and the BMD paper. Because ???
Barton: I'd like to know ?? as I will get the medical writer involved.
Barton: Ok thank you very much for your time
[Blumsohn and Barton leave office together to another office]
Barton: I just don't know what else I can do to ?? for you
Blumsohn: This is natural suspicion, no suspicion is the wrong word ...
Barton: I just want it to be completely open and above board
Blumsohn: I think the problem is that we have all got slightly different agendas, I mean like with the ... whether there is a further reduction beyond the sort of reduction you get with Risedronate is going to be something that is of particular concern to me and you are going to be looking at that with a different perspective. The only think I am interested in is not misleading people.
Blumsohn: I mean it's my name going on these things. I think that kind of thing is just grossly misleading and I just feel very uncomfortable fronting something which says something like that, which I just know to be wrong.
Barton: I agree. And and I wouldn't want you to go in front of a poster and say something which is wrong. That's not what we as a company do.
Blumsohn: Anyway. I'm not aiming to give you a hard time here. But it's a matter of me feeling ....
Barton: ... I do feel that you don't trust me.
Blumsohn: I mean there are various mysterious things in it which .. make me feel uncomfortable about whether what we are telling is the truth. The worst thing that could possibly happen here is that we could produce this poster, and say things which I feel uncomfortable with and then it comes time to write a paper and you end up having to produce something completely different. If you put in the poster that the aim of treatment is to produce a xx reduction in PINP of 50%. Immediately what will happen is that you are going to get all these diagnostic companies coming like Roche, and latching onto this. And they will remember it from the poster.
Blumsohn: ... and then you sit down and start writing this paper and it becomes completely apparent when you look at the data in more detail that that is not an appropriate thing to say ... you are creating much greater problems further down the line ... it has everything to do with that and not to say I don't believe how you are doing the ....
Barton: OK alright I was taking it a bit too personally, I just don't want you to think that I'm hiding anything... or I'm analysing something to put a different spin.
Blumsohn: Well there is spin obviously, there is spin in everything. So you don't want to say that ...
Barton: I don't want ...
Blumsohn: You could say on the basis of this data that the ideal, ... on the basis of the relatively small number of fractures [so far only HIP data seen by AB] we have got the idea thing would be to get your bone resorption down to more than 60% [decrease] in everyone, and that is obviously going to be a message which P&G would not wish to give out.
Barton: Yeah that's true. ...
Blumsohn: I still think you need to give the data in such a way that if someone looked at the data they could actually see exactly where the fractures were.
Blumsohn: It should be apparent to someone what you have actually got.
Barton: Right, yeah. I'm not saying ....
Blumsohn: This is not a personal thing.. I can see you squirming there... but this is my name here, and the last thing I want to happen is to have a poster that says xxxx which as far as I am concerned is patently wrong, or worse still have to propagate something into a paper we, you know to be wrong just because it is embarrassing to withdraw it.
Blumsohn: If you are standing up at a meeting and you have Dennis Black saying "who analysed your data" or whatever  - if you have to say "well I'm presenting stuff that I'm not sure about" that is hugely damaging.... Not only
Barton: I agree
Blumsohn: that is hugely damaging.... Not only in terms of that particular study
Barton: your reputation..
Blumsohn: ... and whether someone will accept what you say next time
Barton: ... and also your department's reputation .... I can see that.
Blumsohn: It is definitely a matter of swings and roundabouts ... people don't believe what you say.... from my point of view that's eveything that it's about ... it has to do with my reputation and integrity and whatever, and not saying things when ??
Barton: I agree
Blumsohn: We put this dummy data into the model and it came up with with a significant number even though it was random data. That is not to say I think Ian Barton is trying to crook the data. It is a real concern that I think, "we put this stuff in and got a result that I find unexpected" - we need to talk about it. I mean it's no personal thing involved.
Barton: I mean, I can't explain that. All I can say is that statistically the model is Kosher. I believe that model is right, but in terms of the data you have thrown at it, I can't explain it.
Barton: That's all. I just can't comment on it.
Blumsohn: Yeah. I'm just not someone who accepts things at face value. When I review papers, the first thing I look at is I try to resuscitate their stats. So what people do often is they will give us a straightforward 1 way ANOVA and will give in a table the mean and SEM of 5 groups and a P value. Every second paper I review you have these things where the statistics just don't hang together. It is just the way I view things.
Barton: Yeah. ?? generally not making stuff up.
Barton: I respect you and Richard for your profession and that and.
Blumsohn: OK. The last thing I want to do is to have a poster which says, PINP you want a reduction of more than 50% and then have to take 20 backward steps on that.
Barton: No I agree.
Blumsohn: Cos, I think Richard.... there are several different philosophical things. The whole thing of whether something is a surrogate (which I actually don't fully understand at the moment what we mean by that) and also whether something is actually going to be useful to predict something in the clinic. And when Richard starts taking about "you want a reduction of more than 50%" that is exactly what he is talking about.
Barton: The patient in the clinic, Yeah.
Blumsohn: ... and what we are actually doing has got nothing whatsoever to do with that ...
Barton: That's right I agree.
Blumsohn: ...and I don't think this is just a matter of me being thick...
Barton: ... Oh no.. you definitely aren't thick are you....
Blumsohn: I think to be honest what happened in the days immediately before the ASBR abstract - what happened wasn't entirely appropriate from my point of view. Cos what happened was that we got these three abstracts being sent off, one of which was sent off by Arkadi [?] I gather that I didn't know anything about whatsoever.
Barton: I don't even know..
Blumsohn: So I'm fronting three abstracts. The ACR one I didn't even know about. So I got this email out of the blue, several weeks after the ASBMR submissions, so say "congratulations Dr Blumsohn you have got an oral presentation at the ACR"
Blumsohn: ... and you go "I didn't know I had submitted an abstract to the ACR".
Blumsohn: and there wasn't enough time to think about it.
Barton: It would be nice to get the manuscripts written by the end of the year. Because I think once we feel comfortable, we know the messages, it should be quite easy to get a medical [writer].. get a pub brief, and then get a medic having a first draft and then you and Richard can review it and say look I don't agree with it or whatever. But you know this is three months on really and we haven't got any further.
Barton: Cos... excuse my ignorance, but what does a plenary poster actually involve?
Blumsohn: It involves ..... when presentations are scored [explains]