About Kentucky’s just released school system test results for 2021

The Kentucky Department of Education finally released the results from what you probably still think of as KPREP testing, although the system is now renamed the Kentucky Summative Assessments (KSA), instead.

As anticipated (by us and many others, see here for example), the scores are notably down from what we are used to seeing.

But, the Kentucky Department of Education is quick to point out that the 2021 KSA results can’t be compared to previous years for a number of reasons highlighted in their Advisory 21-291, including:

  • Spring 2021 testing, for the first time, was based on all new standards,

  • learning disruptions,

  • changes in the opportunities to learn,

  • lower participation rates and

  • a shortened assessment in 2021.

However, the first concern, which is a major one, does not apply to another set of test results also released today. Those other scores are from the testing of Kentucky’s 11th grade public school students with the ACT college entrance test. The ACT tests didn’t change recently. Also, as we will shortly see, the percentage of Kentucky 11th graders who took the ACT during the 2020-21 school term overall was actually rather high. So, these scores, at least, can be compared with some degree of reasonableness to scores from earlier years. Let’s see how that looks.

Table 1 shows you overall average scores for all tested Kentucky 11th graders from the 2014-15 school year up to the present, except for the 2019-20 scores, which the Department didn’t release.

Table 1

KY Grade 11 ACT by Year from 2015 to 2021.jpg

First, notice the number of students tested each year in the Kentucky Grade 11 ACT program. Clearly, there was a drop of about 3,000 students in the 2020-21 testing from the usual number in prior years. In fact, the 2020-21 year tested number is 93.1% of the number tested in the last pre-COVID impacted year of 2018-19. Still, that drop of about 7% isn’t a huge number, either. So, comparing Kentucky’s 2020-21 ACT scores to earlier years’ results isn’t completely unreasonable.

That comparison certainly indicates there was a learning impact due to COVID. In general, scores have dropped on the order of a full point, a pretty notable decay when you consider that the ACT is scored on just a 36-point scale.

Of equal interest, the decay actually varied quite a bit once you break out the ACT scores by student demographics, as I have done in Table 2.

Table 2

2019 to 2021 KY Grade 11 ACT Comparisons by Student Demographics.jpg

This table is divided into different sections based on student demographics such as race and eligibility for the federal free and reduced cost school lunch program, a common proxy for poverty.

Again, the table includes the numbers of students tested along with the scores. Where changes are shown in red, there has been a reduction in either scores or the number of students tested.

For example, for Kentucky’s Grade 11 African-American students, 4,665 took the ACT in 2018-19 but only 3,739 did so in 2020-21, a reduction of 19.8%. Score losses varied from 0.4 to 0.8 of a point. However, with nearly 20% of the African-Americans likely not tested in 2020-21, the real education loss might be much larger. We simply don’t have information to tell.

In a big surprise, students with learning disabilities actually saw a slight increase in the number of test takers between 2018-19 and 2020-21. Their score losses ran somewhat higher than the African-American losses, as well.

Overall, regardless of student demographic group, Table 2 shows that there was some real loss of performance in 2020-21 on the ACT.

And, keep in mind, except for the students with learning disabilities, all other student groups saw a noticeable, sometimes quite notable, apparent reduction in test takers in 2020-21. If the missing students had also tested, the results shown above would probably go even more negative.

One more point: while the KDE’s announcement todays says “Spring 2021 testing, for the first time, was based on all new standards,” I am not sure that is completely correct. I am checking on the status of the science tests. The current science standards date from 2015, and I think the corresponding elementary and middle school science tests – and possibly even the high school science test – have been in use since 2018. If so, that would be another area to examine in greater detail. So, stay tuned.

Tech Notes:

2021 ACT Data Was Obtained from KDE Advisory 21-291 and from the KDE’s 2021 comma delimited Spreadsheet for ACT results.

Data for the 2018-19 and earlier years came from the KDE’s BRIEFING PACKET, STATE RELEASE, 2018-2019 Assessment and Accountability Results and the KDE’s Excel spreadsheet for the 2018-19 ACT results.

 



Richard Innes