Tag: peer review

Are the i3 results correct?

At least one of the 49 i3 celebrations going on around the country tonight may be premature.

Like many others, I was very interested in seeing the reviewers’ comments and scores for the highest-scoring applicants. When the information was posted this afternoon, I immediately downloaded the comments and score for the highest-scoring applicant, Saint Vrain Valley School District. With a standardized score of 116.95–well ahead of any other applicants–I was certain it must be flawless.

It was far from it. The raw score for Saint Vrain Valley School District’s Development proposal was 67.17 out of 100, which is far below the level typically funded in federal programs. If the standardized scores were right, all other Development proposals must have been below 67.17, and that seems extremely unlikely.

I checked another Development score at random. District 75 of the New York City Department of Education had a standardized score of 104.60. The district’s raw score is 93. That’s much more in line with expectations.

I checked another one with a standardized score similar to District 75. The Board of Education of the City of New York had a standardized score of 104.18 and a raw score of 93.17. Although it doesn’t seem quite right that the standardized score would be lower than District 75’s even though the raw score was higher, at least it is very close compared to the problem with Saint Vrain Valley.

I checked another. Bay State Reading Institute had a standardized score of 97.51 and a raw score of 90.33. Again, this seems reasonable in comparison to the others.

I’m sure others will do the complete numbers for all of them in time, but for now, someone needs to explain why Saint Vrain Valley doesn’t seem to merit the title “highest-scoring applicant.” I wonder which other scores are wrong? And what of the more than 1600 other applications for which we won’t be able to review the scores?

Follow-up (12:21 pm CDT, August 5, 2010): Michele McNeil over at Education Week sheds a bit of light on the Saint Vrain scoring. As she points out, it’s all about the standardization process. Unfortunately, this means the raw scores and comments are of limited value for we outside observers.


ED’s i3 ‘highest scorers’ list to be revealed August 5th

More info here.

And don’t miss the couldn’t their understanding of the program be useful on a Validation panel?

In the end, they chose 330 unblemished panelists and provided training (no details offered) on the program. Expect to hear some noisy complaints about panelist comments in conflict with program requirements.

I’m looking forward to reading some of the winning narratives. They’ll be posted on data.ed.gov.


Whew! i3 is over…now what?

The last week has been a real bear finishing up the i3, but it’s over, and the submission receipt is in hand.

The Department of Education granted an extension for folks affected by the flooding in Tennessee–they have till May 19th to get everything done. Except for the folks who truly were affected by the floods, I’m not sure this type of mercy is kind. I know I couldn’t have survived another week.

In other i3 news, the Department is still looking for peer reviewers. Even though the site still says the deadline for applying was April 1st, I queried yesterday and confirmed they’re still desperately seeking help. If you want to apply, do so here.

Now the waiting game begins, but the wait won’t be long. After all, the funds must be committed by September 30th. That’s only 20 weeks away. But the notification date for most folks will come at least 4-6 weeks earlier when groups receive news they’ve scored high and need to confirm (or find and confirm) the 20% match. If some groups can’t get the match, the next groups on the list will possibly get the green light, but they’ll have even less time to meet the match requirement.

So some groups will probably hear somewhere between August 1st and 15th. That’s just 2 1/2 months away…

By the way, next time around I hope they put a page limit on the budget narrative…


Copyright © 2005-2010 Bryan DeBusk, PhD. All rights reserved.
iDream theme by Templates Next | Powered by WordPress