YouTube Instagram TikTok How it works FAQ

The fastest way to turn talking videos into clean, searchable text. Free, private and no account required.

About

Built by Rishav Raj with one goal: free transcripts for everyone.

rishav@yttotranscript.com

Transcribe

YouTube to text
YouTube with timestamps
Instagram Reels
TikTok videos

Learn

How it works
FAQ
YouTube transcripts guide

More

Chrome extension
Contact
Privacy policy
Terms of service

© 2026 YTtoTranscript. Made for creators & researchers.

Not affiliated with YouTube, Instagram or TikTok. Respect creators’ copyright.

Quality of Life: Interpretation of PRO Scores Part 3 — Transcript · YTtoTranscript

Home/Transcripts/YouTube

YouTube Transcript

Quality of Life: Interpretation of PRO Scores Part 3

NIH VideoCast · 2,640 words · 13 min read · EN

Watch on YouTube

Share X LinkedIn WhatsApp

Below is the complete, readable transcript of Quality of Life: Interpretation of PRO Scores Part 3 by NIH VideoCast on YouTube. Read the full text, copy any part you need, or generate a transcript for any video with our free tool.

Let's move on to the last section that I wanted to go over with you, interpreting scores on PRO measures. Now, if I were doing a class on blood pressure here, I may not have to have this section on what -- what's going to constitute a clinically meaningful change in blood pressure, right? I know we're all -- we all debate about what -- where's the danger level there.

But when we're talking about more clinical endpoints like that, measures that have been used in clinical practice have generated some experience there in some sense of what constitutes a big change, a small change, or no change at all worth looking at, right? We don't have that though with almost all patient-reported outcome measures.

These are measures designed primarily for research. Clinicians don't have a lot of experience using them. We haven't generated the experiences that we need in order to understand what might be a big or small change, all right? I want to go through a quick example here. This was a study that we did. It was published in JAMA a while back.

And this was the HF action trial for heart failure. And they compared exercise intervention with usual care. And the primary outcome was survival, but secondary outcomes were physical functioning and overall quality of life. And here, I'm going to focus on changes in a measure called the Kansas City Cardiomyopathy Questionnaire, which is validated

to measure cardiomyopathy outside of Kansas City, by the way. So -- I've never made that joke before, now I know why. It's not a funny joke at all. So, the difference between the groups was 1.93 with a 95 percent confidence interval. There we go. Right? And the conclusion there is that the exercise arm

Transcribe another video

Paste any YouTube, Instagram or TikTok link to get a free transcript.

Free · No sign-up · Unlimited

Add to Chrome — transcribe in one click

has statistically greater rate of change between baseline and three months. 1.93. So, what? What are we to make of this? How does this parameter relate to day-to-day functioning? Is it a clinically meaningful -- is it a policy relevant effect that has been detected? Well, there are several options for trying to answer this question.

And actually, there are some exciting work developing new options to do that. I just want to give you one quick example of a strategy that some people used to answer this type of question, all right? And the rationale here is that, well, if I'm not sure what changes in scores on this measure mean, one thing I can do is to sort of get a point of reference

by looking at changes on something I do understand and seeing how changes in my scores map to the thing I do understand. It's called an anchor method. I'll find some other measure I understand better, and see how it's related to my patient-reported outcome scores, all right? So, in this case, the developer of the Kansas City measures, it's John Spertus.

He and colleagues published an article where what they did was they had cardiologists make an assessment six months after treatment of the change in their patient's health, all right? And they could rate no change, or large, moderate, or small deterioration, or small, moderate, or large improvement, all right? Those are the options. That is a 7-point scale saying, "Tell us, you know, how your patient's

cardiac health is -- has changed over the last six months." Okay? And for all of those patients, those patients all took the Kansas City Cardiomyopathy Questionnaire at baseline and when they came in for a six-month visit. Okay? And so, what we're able to do then -- what they were able to do is to look at the change in score between baseline and six months.

And look at all the folks who were described as having large deterioration. And look at what their mean change was. And some moderate deterioration. All right? And so, here are the results of that. The mean change is on the y-axis there with the 0 right in the middle. And I'll draw your attention to these two areas here.

This is -- these are the mean changes in Kansas City score that correspond to times when the physician said there was a small deterioration and patients who -- for whom the physician said there's a small improvement, all right? And coincidentally, they're both about a 5-point change. One's a 5-point improvement. One's a 5-point decline in score.

It doesn't always work out that way. But in this case, it did. So, it was kind of nice. So, in this case, we're able to say, "Well, you know, it looks like about a -- on average, a 5-point change from baseline seems to correspond to the clinician's perception at least that this person had some small

but noticeable improvement or small but noticeable decline." Okay? It doesn't mean that 5 is a holy number or anything, no. But it's just a helpful point of reference, right? That's just what -- that's all we need. And so, we can use that in -- when we expressed the results from this, we plotted the distribution of changed scores.

Those change for baseline is on the x-axis, right? And the frequency in percent is on the y-axis. And we plot that separately for the exercise in the usual care arms there and recall that the difference here was 1.93, right? It's kind of like the difference of the means of those distributions. This is a little more formal though because we can draw

a reference line at 5, an improvement of 5, right? Again, it could -- maybe it's 6, maybe it's 4. You know, it's -- we're not pretending that there's anything exact about this. But just to help us get a sense, we know that a 5 on average was when clinician said, "I'm noticing an improvement." Right?

So, we draw that reference line there. We can -- we could find out what percent of people in each trial arm exceeded that, right? And in this case, in the exercise group, 54 percent of folks exceeded that compared to 29 percent of control. It helps us just get a sense of what happened in this study

with respect to these Kansas City scores, right? And as I'd say, there are other approaches to trying to create meaning around the -- of the scores you'd get. This is just one example of those, all right? But I think the important point is that efforts have to be made when you have a new measure to actually try to figure out

what will constitute a meaningful change or a meaningful difference, right? Okay. So, let's quickly review here. We -- this is kind of a high-speed survey of some key points in the field. We talked about the value of measuring patient-reported health status and the different types of patient-reported outcomes summarized by that Wilson and Cleary model, right?

And that idea of the dilution of effects going from left to right in that model, and all the other stuff about the individual, and about the environment that contribute to the patient-reported outcomes that we might observe, right? We talked about the different steps for developing and evaluating patient-reported outcomes. And also, we went into some depth about some new developments in the field

that have arisen as a result of applying item response theory to health measurement, and the creation of item banks to allow us to more efficiently assess health. And then finally, we talked about the importance of doing something to help us interpret the clinical or policy relevance of changes or differences in scores on patient-reported

outcome measures, because they're strange metrics. They're not used in clinical practice routinely right now. And we don't have the body of experience that allows people to look at it and say, "Well, that's -- yeah, that's actually a worth of difference -- that's a difference worth paying for." Right? Okay. I think that we've got some time for questions, if there are any.

Is there anyone -- yes, sir?

So, one of the question that comes to mind when you're designing a good design depending on the patient-reported outcomes is that, do you change in cost of operation, you know, the effect of time, you know, in reporting by patients? So, say that you want to go in fact to look at the times or we've put maybe more simple things,

there should be a timely bias in this as -- unequal bias of which. So, when you designed these things, if you're looking at, for example, the question in hourly computed in the time for the -- you know, for time as preference? What a great point. So, the comment or the question was, what about the element of time

in the reporting? Items usually have a recall period, in the past seven days or in the past months. And to what extent is that a concern that there may be inaccuracy in that reporting? And a great point was raised that in a population like very elderly where you might have memory problems, might we be asking the system

to report on something that's beyond its ability? What a great question. And you know, I didn't include it in the evaluation process there, but it is increasingly the case that people will do a recall study. And this has been done for a number of the PROMIS measures. We conducted one for the PROMIS sexual function measures where you'll have

people make daily ratings of whatever it is you're measuring over, say, a month. And at the same time, they'll do weekly ratings and maybe a monthly rating. And there are very complex designs you can come up with here. But the idea is to look at the correspondence between a seven-day recall and daily

recall -- average daily recall and -- so, the short answer is yes, it's very important. It can be evaluated empirically. The FDA is very interested in evidence that someone can record accurately. And there are some really interesting results showing the types of recall bias these people will exhibit, right? The last few days are often more influential in determining their report, right?

The mood they're in at the time they take the measure is more -- might be related to their reports, right? So, this is an emerging science, but a very important aspect of this. I'm so glad you brought that up. I didn't include that. All right. What other comments or questions do you folks have?

Yes, sir? Male Speaker: How do you determine what a baseline is? People can really be different. And when you go to see a doctor, and they say, "Well, on a scale of 1 to 10, what is the pain?" "Well, 6 or 7." You don't have a measurable comparable scale that you can use for benchmarking.

So -- yeah. So, the question is, how do you establish a sort of an absolute baseline when people -- people's subjective sense of what a 6 is, for example, could vary among people? Right? Is that a good -- so, great question. I mean, here is the essential fact -- I was going to say problem,

but it's not a problem. It's a feature. The fact of having a subjective report is the measure. We could go and try to say, "Well, all right, let's get measures of your C fibers between these two people who both answered a 6 and find out if the magnitude of firing of C fibers is different between the two

and that might be interesting." But that's awfully expensive, I'm not sure what you're going to do with it. What a lot of people will do in this case is to say, "Well, let's do a longitudinal design and each person has their own control." Right? Another option is to sometimes have people rate different reference states.

So, if they are -- you know, they might be rating how bad they think it would be to be in this other state and then you can compare what people said there and just -- but it gets a little bit wonky. And I think it's better to just use people as their own control.

It is also the case too, though, that, you know, some of those differences between people are exactly the things we're trying to measure. If I've got two people whose bodies from the neck down are identical, but one's giving a pain intensity rating of 4 on a scale of 1 to 10, and then the other is saying 8,

those are different experiences that people are having. And some of it may be because they used the scale a little differently. But a lot of it may be due to this -- it's a different experience, and that's why we're asking them because it's that experience that is probably driving behavior-like seeking care, right?

Other questions? Comments. Yes, ma'am. Do you see a lot of studies going to mobile devices or their PROs? Yeah, absolutely. The question was, do we see a lot of studies going to mobile devices for their PROs? I know -- I think at the next lecture there's going to be a little bit more on different modes of administration, but absolutely.

I mean, the ability to use an iPad in the clinic or use phones that people are out walking around with is very exciting, especially to be able to measure real-time things. You know, instead of me asking you, "What -- how are -- well, how has the past seven days been?" I could actually ping you every now

and then with a very short thing a few times a day now. So, that is absolutely of tremendous interest. And there's a lot of exciting applications being developed to do that, yeah. Yes, ma'am?

What about patient-reported system as an outcome? How do you define protocol deviation? And is there a second thing as with protocol deviation when you do the questionnaire? The question is, is there -- when we have a patient-reported measure, what's the protocol deviation related to that measure? Is there one? There absolutely is. I mean, if the -- if we're requiring that the person complete at least one item,

and they've not completed any, and the study coordinator was supposed to make sure they completed some, or they answered the question that says, "I'm not answering these because --" that might be viewed as a protocol deviation. If -- another example is, you know, sometimes the timing of when they answer the measures during a clinic visit is critical.

So, let's say I'm doing a heart failure study, and I've got a guy going through a stress test. Well, their answers to their physical, you know, "How is your physical function?" right when they come in the clinic are going to look a lot different probably than right after they finished the stress test where they're not feeling so hot, right?

So, if I'm designing that protocol, the thing I'm measuring is what was their functioning like the week prior to coming into the clinic. I want them to take the measure right away. If in that visit we find out that that measure was -- they forgot to administer at the beginning of the visit, they did their stress test,

and then they administer it afterward, that's a problem all right? And there are other things you might think of, but -- so, yes, there -- it is possible to have protocol deviations related to patient-reported outcomes. Other questions or comments? Okay. Well, I think you guys have been so attentive at this late hour.

I appreciate it. And enjoy the rest of your evening.