The proximate cause of our expression of exasperation with the College Board is the incongruous scoring results of its 2019 Preliminary Scholastic Aptitude Test (PSAT), which was administered last October. and the “Again” part of the title to this blog is a little complicated, so bear with us while we explain.

Both the SAT and the PSAT are products of, wholly developed by, and administered by the College Board. The tests are very similar, though their usages are largely different. Both can be “high stakes” affairs for students that take them, and this is not the first time recently that we’ve seen strange scoring results on one of those two standardized test products: Our blog titled Is Alice in Wonderland’s Mad Hatter in Charge of SAT Scoring Curves? discussed the crazy scores on August 2019 SAT, and our blog titled Wacky Scores on the June 2018 SAT discussed the peculiar scores in the June 2018 SAT. This blog will discuss the incongruous scoring results of the October 2019 PSAT and why they matter.

Two mandatory requirements for important standardized tests, such as the SAT, PSAT, and ACT, are that they accurately measure students’ performances (but not necessarily their abilities) at the time of testing, and that there is cross-test congruity. By the latter, we mean that a student’s scores on any testing date of the SAT, PSAT, or ACT should be consistent with student scores on any other testing date, and that the same should true with the scores for similar cohorts of students. While we have no reason to question that the October 2019 PSAT accurately measured student test-takers’ performances at the time of testing, it’s crystal clear from the data that the October 2019 PSAT failed the congruity test – and failed it in a major way.

About 3.5 million high school students take the PSAT each fall. Roughly 1.7 million high school juniors sat for the October 2019 PSAT, and the number of them scoring 1400 or better (out of a 1520 maximum possible) dropped 30% from the 2018 PSAT. The drop for sophomores taking the PSAT was even steeper, at 36%. High-performing students scored lower by as much as 30 points compared to previous years – years during which there was cross-test congruity. Juniors scoring 1200 or better dropped by 15%, and sophomores dropped by 21%. The average score for the 1.7 million juniors dropped by 10 points, which is a whopping difference given that the average score change on a well-constructed test with a stable group of student test-takers is only one or two points.

Because it would stretch credibility to the point of breaking to posit that the October 2019 cohort of student test-takers was that much less capable than was the 2018 cohort, the conclusion is virtually inescapable that there were problems with the test. And we will have to draw our own conclusions because, other than reporting the scores, the College Board makes public almost no information about PSAT results, provides no information on whatever, if any, self-assessment it does, and has offered no explanation of why/how the score drops occurred on the 2019 PSAT.

We want to note here that the score drop was particularly steep for the PSAT’s Math section: The drop for that section was nearly twice as steep as was the drop in the scores for the Evidence-based Reading and Writing section, meaning that if the score drops were due to poorer student performance, it was significantly selectively so — and that makes it even more likely that the test, not the test-takers, was the reason for the drops. We want to further note that the College Board’s test score woes on the SAT and PSAT occurred after it stopped using the Educational Testing Service (ETS), an independent, outside firm, to test and validate questions and brought that function “in-house.”

The incongruous PSAT scores matter because it’s a high-stakes test for the student test-takers: More than $100 million in National Merit and other college scholarships is determined by PSAT scores.  Further, there’s another significant effect: The College Board has strongly proposed to high schools that they use PSAT scores to determine whether students are allowed to enroll in Advanced Placement (AP) classes, and enrollment and performance in those classes are considered by colleges when making admissions decisions. And PSAT scores, when they’re consistent measures, unlike the October 2019 PSAT scores, can provide good comparative data to students’ performance on the SAT and ACT and are used by schools and school systems nationwide to track student performance over time.

It’s far too early to appreciate the full ramifications of the October 2019 PSAT’s incongruous results: We don’t know exactly how they’ll affect National Merit and other college scholarships or how schools and school systems will view and use them. And we might never know how they’ll affect AP class enrollments and, thus, college admissions.

What we do know is that the College Board has demonstrated that it’s unwilling to and/or incapable of addressing a serious issue with its high-stakes PSAT and SAT tests.

Stay tuned to this channel for future updates.

{{cta('46a7db85-8336-447f-8a56-cd6e3a9cdcb7','justifycenter')}}