Can Statistical Error Explain the World?
An email I sent to a colleague the other day, relevant to Andy's article in PDK, at least peripherally in using statistical error to explain away the differences in how certain folks operate in the world. Also, nice shout out to Kinder and Iyengar. Those guys bring me back to UVA Politics.
"OK, let me try this again. I'm trying to understand how confidence intervals (CI) can be explained in regards to AYP, particularly in light of something like growth models and ED's decision to not allow states to use CI when reporting growth under the Growth Model pilot.
The way I understand it.
When states don't use confidence intervals they increase the chance of making a Type II error. In the case of NCLB this gets operationalized as identifying subgroups as having met proficiency when they actually have not met proficiency. A Type I error in the context of NCLB would be the inverse: identifying a subgroup as proficient when they actually are not proficient. In statistics we are usually more concerned about making a Type II error because erring on the side of caution is good for a lot of things--like keeping people alive in medical trials.
But with regards to schools and calculating AYP, ED seems more concerned (based upon their general indifference to confidence intervals) about making a Type I error, or identifying a subgroup as having met proficiency when the subgroup in fact has not met proficiency.
Maybe it's not possible to talk about confidence intervals in this way--the unit of analysis being subgroups."
"OK, let me try this again. I'm trying to understand how confidence intervals (CI) can be explained in regards to AYP, particularly in light of something like growth models and ED's decision to not allow states to use CI when reporting growth under the Growth Model pilot.
The way I understand it.
When states don't use confidence intervals they increase the chance of making a Type II error. In the case of NCLB this gets operationalized as identifying subgroups as having met proficiency when they actually have not met proficiency. A Type I error in the context of NCLB would be the inverse: identifying a subgroup as proficient when they actually are not proficient. In statistics we are usually more concerned about making a Type II error because erring on the side of caution is good for a lot of things--like keeping people alive in medical trials.
But with regards to schools and calculating AYP, ED seems more concerned (based upon their general indifference to confidence intervals) about making a Type I error, or identifying a subgroup as having met proficiency when the subgroup in fact has not met proficiency.
Maybe it's not possible to talk about confidence intervals in this way--the unit of analysis being subgroups."
posted by eponymous educator at 4:27 PM
0 Comments:
Post a Comment
<< Home