SAT Snafu

Here's an article that details some of the reasons for the mistakes that were made recently on SAT scoring. It seems less-than-perfect OMR performance is at least partially to blame. Pearson NCS gets hit pretty hard in this article.

Here's the response we got from Scantron's Tim Dubes when we asked him to comment on the situation:

"While we really don't want to be perceived as grave dancers--and keep in mind that even state of the art OMR is not going to produce 100% accurate results in all applications--there appears to be a number of areas that the SAT test scoring system could be improved. Remember that this is a complete solution that is being called into question, not just individual technologies. So while a significant technology advancement like Scantron's SIMR (Scantron's Intelligent Mark Recognition) can greatly improve recognition accuracy in less than optimal conditions (including damaged forms, lower confidence marks, damaged tracking marks, and the like), it is only part of an overall solution. In the case of the SAT scoring, the inability of the software to accurately flag degraded forms and marks that fall below the necessary confidence threshold could have contributed to the poor results.

"One of the key points that Scantron makes to our customers, particularly in the education field, is accountability. This means that we are responsible for all the components of the solution: we make the software application, we build our own hardware, we design & print the forms in our own facility, and we provide complete training and professional services with our own employees. Sole source accountability removes a lot of variables, and allows us to control the data capture environment, whether we are installing a customer-driven solution or providing the complete data collection solution on an outsourced basis. This also eliminates a lot of the variables from cross-vendor solutions and gives Scantron and it's customers the confidence to move forward with recognition-based technology for mission critical applications. I'm certain that the students' that are negatively affected by the SAT scoring issue would consider it to me a mission critical application."