For the past few years, I've been evaluating the accuracy of ball calls made by umpires, dividing the number of correct calls by the total number of catches. It's a vague method, but because umpires make thousands of calls each year, it yields solid results. On Tuesday, I ran the numbers for the 2024 season, and found something I didn't expect: Accuracy is going down rather than up. In every single season since the start of the field-tracking era in 2008, umpires have gotten better at hitting balls and strikes according to Statcast's strike zone. This is the first time I ran numbers and noticed a low accuracy rate. However, this is also the first time I've looked at the numbers early in the season, and it turns out that referees tend to make better calls as the season progresses. Since 2017, the accuracy in March, April, and May has been 0.19 percent lower than the accuracy in the full season (although the difference in 2023 was only 0.03 percent). Here's what that looks like on a graph.
You know how at the beginning of every season, there are a few calls that are made during a nationally televised game (or at least, calls that appear to be wrong based on the location of the strike on the screen), and some people start complaining that the umpires are bad and getting worse? Those people always catch me off guard. I tend to forget the missed calls when the season ends, but those people somehow manage to keep their umpire anger high throughout the offseason so fast baseball returns, ready to yell about the umpire and out. any need to climb. I don't know how they do it without pulling an oblique, but in a sense, those angry people are right. Although referees keep getting better every year, they are almost always more accurate at the end of the season than at the beginning – so much so that when the season starts, they are worse than they were at the end of the season. last season. For a month or two, referees have become worse. We often say at the beginning of the season that pitchers are the best hitters. It turns out they are ahead of the referees too.
For each season, I break down the absolute accuracy in two month increments, basically dividing the season into thirds. I also broke down the accuracy during spring training and the playoffs, though there are many factors that make those numbers suspect. During spring training, the umpiring pool is much wider. Perhaps most importantly, there are far, far fewer pitches during spring training, both because the number of games is small and not all pitches are scheduled for Statcast. That results in a much smaller, less reliable sample. The playoffs are also a much smaller sample, but they also, at least in theory, pick better referees. Performance in the playoffs is seen as an honor and a reward for performing well in the regular season. We should expect accuracy to be very low during spring training and very high during the playoffs.
In general, the results are consistent with our hypotheses. The accuracy of spring training is very low and includes the volatility that we would expect from a small dataset. Referees are also very accurate in the playoffs. The red line is March, April and May, and as you can see, it's below almost everything except the spring training line. Not only do referees start to improve in June, but they continue to improve until the end of the season, which is why the blue line of August, September, and October is often above the yellow line of June and July. The trend is a little easier to see if we just focus on the pitches in the shadow zone, the area that is one baseball width from the edge of the zone on either side.
In the graph above, the dotted line represents that season's overall accuracy for calls in the shadow area. Each data point represents a number of percentage points above or below the average for that year. Not only do the calls get better as the season goes on, there is a definite gap between the first two months and the rest of the season. The referees were decidedly worse in those first two months. However, 2023 was a real exception. It was the first time since 2008 that referees were more accurate at the start of the season than at the end.
With that, I want to take you back to 2024. So far this season, referees have gotten 92.46% of calls right, down from 92.81% in 2023 and just two thousandths of a percentage point higher than in 2022. Based on everything I've shown, we should expect referees to get better during the rest of the season. However, last year's decline is noticeable. Accuracy in the first two months of the season has fallen once before, from 2009 to 2010, when it dropped 0.16 percent. So far this season, accuracy has fallen to twice that amount: 0.32 percentage points. That's a small change, in terms of one call per game, but that doesn't make it any less realistic. We will have to wait and see how the rest of the season goes, but maybe this year could end up being different. Or, if it follows the pattern of the last decade and a half, accuracy will soon increase.
Source link
