YouTube Instagram TikTok How it works FAQ

The fastest way to turn talking videos into clean, searchable text. Free, private and no account required.

About

Built by Rishav Raj with one goal: free transcripts for everyone.

rishav@yttotranscript.com

Transcribe

YouTube to text
YouTube with timestamps
Instagram Reels
TikTok videos

Learn

How it works
FAQ
YouTube transcripts guide

More

Chrome extension
Contact
Privacy policy
Terms of service

© 2026 YTtoTranscript. Made for creators & researchers.

Not affiliated with YouTube, Instagram or TikTok. Respect creators’ copyright.

Quality of Life: Item Response Theory Part 2 — Transcript · YTtoTranscript

Home/Transcripts/YouTube

YouTube Transcript

Quality of Life: Item Response Theory Part 2

NIH VideoCast · 3,419 words · 17 min read · EN

Watch on YouTube

Share X LinkedIn WhatsApp

Below is the complete, readable transcript of Quality of Life: Item Response Theory Part 2 by NIH VideoCast on YouTube. Read the full text, copy any part you need, or generate a transcript for any video with our free tool.

All right. I talked fast again. I have caffeinated. So, let's do a quick recap here. We're going to determine what concept we want to measure and why. We're going to go and collect qualitative data from people to make sure that we really have defined this thing. Once we've got it defined, we can write items for it.

We're going to go take the items to the actual types of people that are going to be taking this measure, find out if they understand it, revise the items, make sure our revisions work with new people. Then we're going to go and give the items to a whole bunch of people, so that we can do psychometric analyses.

And those psychometric analyses are going to allow us to test how well our original model fits the data. Well, there's a better model we've got to use. And it's going to allow us to derive a score, right, or a set of scores maybe, but some score that we can use. And then finally, once we're in possession of this, we can take that out

and go test it now to see what kind of reliability and validity it exhibits. Okay? All right. So, that was the development and evaluation process in a very small nutshell. So, I'd like to now talk about some very interesting developments that have occurred, and how we developed patient-reported outcome measures, and some of the cool things we can do with them now.

And it's especially relevant because NIH has actually invested a good deal in developing measures using some of these newer approaches. And remember I said new, it's new for health, but not new for the measurement world, okay? What we're going to be talking about is something called item response theory. Okay? And what I want to focus on here are -- is what's different

Transcribe another video

Paste any YouTube, Instagram or TikTok link to get a free transcript.

Free · No sign-up · Unlimited

Add to Chrome — transcribe in one click

between a regular old measure and a measure developed using item response theory. And then I want to tell you about one very cool little thing you could do called differential item functioning. Okay? So, let's start with contrasting the different types of measures. And let's start with -- here is -- here's your basic off-the-shelf measure.

This is the FACIT-Fatigue Scale, right, developed by Dave Cella and colleagues to measure fatigue especially in cancer populations. And that -- this is a great example of your traditional off-the-shelf measure. I'd say off-the-shelf PRO measure because you just go and you grab it and you're ready to go. Everyone's going to complete the same items on the FACIT-Fatigue, all right?

All of the items are necessary to obtain a score. You got to complete all of the items, because you average or sum up all the items to get the score, all right? And the score on the FACIT-Fatigue might not be on the same metric as other measures of the same thing. So, for example, the SF-36 has an energy domain that's supposed to be measuring

about the same thing. But those scores are on a different metric. The absolute value of those scores mean a different thing for the FACIT measure, right? It's kind of like having two different thermometers that are both supposed to be measuring temperature, but one gives it in one set of numbers and another gives it in another set of numbers, and they're not comparable at all.

Okay? This is what we get with traditional off-the-shelf measures. In contrast, when we use item response theory, we're able to create something called an item bank. An item bank is a large collection of items measuring a single domain like fatigue, right, or pain intensity. Okay? Any and all of the items from the bank can be used to provide a score for that domain.

So, you can pluck out one item and estimate a score on fatigue. Or you can pluck out 40 items and combine them all and estimate a score on fatigue. Okay? That means that item banks can be dynamic, not fixed. We can add items to them overtime if we end up getting some better items.

We can sit in some new items and maybe get rid of some items that are -- that don't seem to be quite as useful. So, it's a dynamic resource. All right. Let's look in an item bank for the domain of physical functioning just as an example here. Okay. So, we know there's this range of physical functioning out there in the world.

We know that we would like to systematically measure that range, right? And so, to do that, we will create an item bank for physical functioning. And in this case, the item bank is a collection of items that all measure physical function. Okay? And so, we've got examples here. "Are you able to get in and out of bed?" "Are

you able to stand without losing your balance for 1 minute?" All right. "Are you able to run 5 miles?" Okay. Any item or set of items from this bank could be used to generate a score for a person for physical functioning. Any of them. And why is that? It's because they -- we've

put these items through the ringer to make sure that they fit in item response theory model. It's a statistical model that allows all of these properties to hold, okay? And note here, too, that these items, as I've laid them out here, are kind of arrayed in order of severity, all right? Are you able to get in and out of bed?

That's kind of question you'd ask of a person who looked like they were incredibly weak, right? That's an item that's good for discriminating among people who are at the lower end of functioning, right? "Are you able to run 5 miles?" is something that would be informative to ask of a person who's in the higher end.

And this tells us another interesting property about item banks, is that with item response theory, not all items are the same. Some items are better used for lower severity or higher severity. And when we're thinking about which items we might want, we have to think about the population we're using. And we have the option to tailor our measures.

More about that in a second here. Okay? So, back to off-the-shelf, if I'd just use my traditional off-the-shelf measure, I take it off the shelf, and I just give it to the patient or send a web link or however it is, and they all fill out the same measure and that's it. Okay?

So, it's really easy to figure out what to do. You take the off-the-shelf measure and administer it. It's not as straightforward with an item bank, because you've got some decisions, all right? Let's go through those here. Basically, I've got one of two main strategies here, a fixed-length or computerized adaptive tests. With the fixed-length measures,

everyone is going to get the same number of items. It's a fixed-length for everyone, all right? And I've got a couple different options there. There are some very handy ready-made fixed-length measures. So, if you want to use measures from the NIH PROMIS system, you can go to the Assessment Center website and you could find ready-made short forms.

These are forms with about six to 10 items per domain that are good for general populations. They're generally sensitive to the range of functioning there that the PROMIS folks have picked. It's sort of a good kind of off-the-shelf version of PROMIS measures. So, there's some ready-made fixed-length measures, all right? I could also go in -- if I've achieved the requisite

level of geekiness, I could go into the system and I could pick out the items that I would like that I think are going to be most sensitive for my population, all right? So, if I'm -- if I've got a trial where I've got severely ill heart failure patients, I don't want to be asking people how much trouble they have running 5 miles.

It's a waste of their time, right? I'm going to go in and I'm going to select the items that are going to be most sensitive for my population. And I'm going to give everyone in my trial those items, all right? Okay. So, I could tailor them. I can make my own. So, those are both versions of fixed-length.

The other main way of using items from an item bank is through computerized adaptive tests. And, you know, a lot of you might be familiar with computerized adaptive testing, because they use it a lot for education like the SAT, GRE, a lot of the certificate -- certification tests in medicine, nursing, and government safety tests a lot.

What it does have is these big banks of items, and you sit down with a computer and they show you one at a time, all right? So, same deal here. The idea is that the next item that you receive is dependent upon your answer to previous items, all right? And so, if I've got a range of items across this -- the spectrum of severity,

a computerized adaptive test or CAT might start with one right in the middle. And if I seem to have a lot of difficulty doing that, it's going to put me on the more severe side and say, "This guy sounds more severe. Let me ask him a question on the more severe end and see how he answers, all right?

I'm going to answer that." I'm just going to say, "Oh, it seems like it's around here. I'm going to ask a question here." And so, in some ways it's very similar to the way a clinician might be asking questions of a patient. Let's start with a general one. And then as you're getting the sense of how severe or mild they are,

ask more questions to get a more precise picture of what that person looks like. But the idea with the -- with a CAT is that in theory every person could get a different set of items, right, because the items that you get pulled out of that big item bank are the ones that are getting us to the most precise estimate of your score

as quickly as possible. All right? And we find often that you can get a very precise estimate of people's scores by asking them only four questions.

But they have to be the right four questions. And how do you figure out which are the right four questions? Well, you have an algorithm, a computerized adaptive testing algorithm that is drawing items from that item bank. All right? Having an item bank now with these different options allows a few things to happen.

Imagine we got this fatigue item bank here, and it has said 40 items in it. Well, I could use items from that item bank in all different settings, right? And I could use them in different ways too. So, the chemotherapy trial might have items one through 10. For the osteoarthritis trial, I might be using a CAT, a computerized

adaptive test, right? And what's great is because all of the items being pulled for all these different studies were from the item bank, they all have the same metric, the same meaning. The scores are comparable across all of those different studies, all right? You can imagine how that might help facilitating things like meta-analysis

and the like. Here's another great application. Right now, there's a lot of interest in pragmatic clinical trials or comparative effectiveness research where we're trying to do a research within healthcare systems. I might have different healthcare systems each of whom are using different measures, all right? Well, right now, each of those measures results in a different metric,

and it's really difficult to pull results then. If, however, all of those items from the different measures could be placed within an item bank, which means we do the kinds of item response theory analyses that test whether these guys can all sit in the same bank, and maybe we've got some other items in there,

now we're in a great position because we can get one metric out of that. Right? So, now, even though different healthcare systems might be using different sets of items, if they're all on the same item bank, we can -- we've got a cross-walk going and we can get the same metric out of all of them, right?

Okay. So, a quick review here comparing this traditional PRO measures, the off-the-shelf measures with the PRO item bank. With the traditional, all items are required to compute a score. With item banks, any and all subsets of the items can generate a score. With the traditional, everybody's got to take the same items. With an item bank, different people could get different items.

With the traditional measure, you've got a -- you can just pull it off the shelf. But when we've got an item bank, you've got to use items in the bank to create measures for a specific use. And we went through some of the different options there, right, fixed-length or CATs. If it's fixed, you can get ready-made ones that someone might have created for you,

or you can go and create your own custom one. And finally, in traditional PRO measures, the scores are not easily comparable to other scores from other measures of the same thing. Whereas with an item bank, there's much greater opportunity for cross-walk between scores from different measures, as long as the items from the different measures can be put in the same bank.

Okay? All right. I mentioned that NIH has invested in item banks quite heavily and integrate benefit, I think. A few examples here. The biggest one is probably PROMIS, the Patient-Reported Outcomes Measurement Information System. And over about 10 years of development, PROMIS has amassed quite a few excellent measures of the main types of health concepts

that are relevant across chronic diseases for adults and children and adolescents. They're freely available. You can go to nihpromis.org for more information there. NIH also supported the NIH Toolbox, which is for assessing neurological and behavioral function, ages 3 to 85. Again, freely available. You can go to nihtoolbox.org. And finally, Neuro-QoL is for use in chronic neurological conditions.

And it's free. And you can go to neuroqol.org. So, you might want to check those out, and you can get more information from those sites too about how item banks work, all right? I said that there was a very cool thing you can do -- there are a lot of cool things you can do with item response theory, but I wanted to tell you about a specific one.

It's differential item functioning, okay? So, check out this very simple question, all right? "In the past seven days, did you cry? Yes or no?" This is -- I've just made a yes-no version of an -- a kind of item you often see in depression measures, all right? And I think that we would all agree that, in general, responses to this item

might inform us to some degree about the likelihood that a person might be a bit depressed, all right? The more likely you already have cried in the last week, the more likely you are to have been depressed. This has been shown to be the case, okay? Imagine that we wanted to compare men and women's depression though,

and we had an item like this and a few other similar items. We found that men and women differ in their depression levels. What we'd like to know is that men and women truly do differ in their underlying depression levels and not that they actually differ in the way the items work for them, okay?

So, hang on for a second. We're talking about differential item functioning. It means the item behaves differently for two or more groups. So, check this out. There's a very simple item response theory curve here. And you can see we're modeling the probability of answering yes to that question, on the y-axis, as a function of the person's underlying depression, on the x-axis.

Okay? This, by the way, is why it's called item response theory. They're models of the likelihood of responding with a given answer based on your underlying level of whatever. In this case, depression, okay? So -- and you can see here as the person's depression -- as depression increases, so too does the probability they're going to answer yes to "Did you cry in the past week?" Okay?

And so, what we're saying when we say that this is informative about depression, is we're saying, well, we know that of people who answer yes, those folks are most likely to be on the high end of depression. So, we can make an inference that they are on the higher end of depression. That's how we're using that item.

This is a map that maps the responses to that item to their underlying depression, okay? And so, we take someone who is, say, one standard deviation above the mean of 0 on depression, they've got a 0.9 probability of answering yes on that, right? Okay. So, we've said that differential item functioning is what happens

when an item behaves differently for two or more groups. Another way of saying that is that the map between depression and the item is different for two or more groups. I was talking about men and women with this crying item. Okay. Let's look at this. This is the map computed separately for females and males.

And what is it showing here? If we've got a male who we know because we've suddenly become omniscient for just a second is one standard deviation above the mean on depression, the male's likelihood that they're going to answer yes is 0.3, right? It's a 30 percent chance that they're going to answer and say, "Yes,

I've cried in the past week." A woman though who is one standard deviation above the mean has a 0.9 probability of answering yes, right? Now, why? On average, women will respond that they cried in the past week more often than men independent of their underlying level of depression, right? But you can see how messed up you'd get here though if you just used one

version of the map and applied it the same for males and females here. So, this is an example of differential item functioning. And when we're developing measures using item response theory, one of the things that's really important for us to do is to think, "Are there groups for whom these items might behave differently?

Would they behave differently for young versus older people? Would they behave differently for different cultural groups?" Right? And we can explicitly test for that using item response theory and conducting tests of differential item functioning. And it will identify for us those items that are exhibiting differential item functioning. And then we've got a choice.

We can either chuck those items, if we've got other items that are just as good, but don't exhibit that. We've got to get rid of them. Or we can say, "You know what? We're not going to get rid of it, because it's really informative item. But now we know we need to interpret it.

We need to score it differently for men and women." Still useful, but we got to score it differently, right? By the way, you know, when you see tests for potential race bias in achievement tests and IQ tests, what they're doing is differential item functioning. They're looking to see, for two people who are at the exact same underlying level of analytic reasoning, for example,

do they have different probabilities of getting this item correct or not? Because of maybe different cultural backgrounds, right? So, differential item functioning is very important. Very, very -- isn't it really a terrific technique we've got? Okay. So, that was our quick little foray into some of the things that item response theory has brought us, namely measures based on item banks, right?

And how they differ from your traditional off-the-shelf measures. And some other really cool tools like being able to assess differential item functioning. Okay?