How Did Our Free Agent Model Do?

This past offseason, we debuted several new features, one of which was our free agent calculator, which you can find here if you’re a GM-level subscriber. 

It works a little differently than the predictions of other sites. Instead of forecasting the exact length and amount of a player’s contract, our approach is to let the user choose the length, and the total amount and AAV will display. 

For example, our tool will display an estimated fair value for Jordan Montgomery at 1 year, 2 years, 3 years, etc., all the way up to 10. The benefit of this is that it shows you how the AAV changes with each scenario, as adding length typically adds risk, which lowers the price. (It also displays how the surplus value looks through the life of the contract – typically there’s positive surplus on the front end and negative on the back end, as the player ages below his salary line.)

 

So how do we measure success with our free agent model?

The obvious way is to compare the AAV of the player’s actual contract based on its length to the same length in our model, then measure the percentage difference. For example, Sonny Gray signed a 3-year deal worth $75M, or a $25M AAV, with the Cardinals. Our model shows that 3 years of Gray is worth $72.6M, or a $24.2M AAV. On a percentage basis, the AAV we estimated for a 3-year deal is off by 3%.

But there were two other considerations: First, we think contracts with more years should be given more weight. Being 3% off on a 3-year deal matters more than being 3% off on a 1-year deal, so in our model, we figure we should weigh the former more heavily in the results. That’s to match the way that MLB front offices prioritize their deals – the bigger the guarantee, the bigger the commitment. 

For example, we think we should weigh the amount we were off on Aaron Nola’s 7-year, $172M deal 7x more than the differential on Luis Severino’s 1-year, $13M deal, since it’s a much bigger, and riskier, investment.

So the unweighted measure takes the AAV difference for each deal, adds them all up, and divides by the number of deals. The weighted measure does the same, but instead divides by the number of total years committed. This gives more weight to longer contracts.

Second, we’re measuring only the big ones, as defined by MLBTR’s Top 50 Free Agent list, primarily because those are the ones most people care about, but also because it keeps the data more consistent. Since we’re measuring on a percentage basis, a low-level reliever can really throw it off: if he signed for $1M instead of $2M (not a big deal in real life), it's a 100% variance on the model. So using a percentage basis tends to work best on the bigger deals, just as it does in our trade model.

 

With that, here are the overall results:

 

Average unweighted AAV variance: 19.9%

  • Total difference: 8.6
  • Number of deals: 43
  • Unweighted difference = 8.6/43 = 19.9%

 

Average weighted AAV variance: 16.3%

  • Total difference: 15.96
  • Number of contract years: 98
  • Unweighted difference = 15.96/98 = 16.3%

 

On a weighted basis, the model performed better than expected, while the unweighted variance is still within our target range, albeit just so.

 

Biggest wins:

We were right on the money with Jeimer Candelario and Mitch Garver; off by 1% on Jack Flaherty and Harrison Bader; off by 2% on Frankie Montas and Jorge Soler; off by 3% on Sonny Gray, Luis Severino, and Michael Wacha; and off by 4% on Lucas Giolito.

 

Player Years Total AAV BTV Projected AAV Unweighted Difference Weighted Difference
Jeimer Candelario 3 45 15 14.9 0 0.10
Mitch Garver 2 24 12 12 0 0
Jack Flaherty 1 14 14 14.2 0.01 0.01
Harrison Bader 1 10.5 10.5 10.6 0.01 0.01
Frankie Montas 1 16 16.4 0.02 0.02 0.02
Jorge Soler 3 42 14 14.2 0.02 0.06
Sonny Gray 3 75 25 24.2 0.03 0.10
Luis Severino 1 13 13 12.6 0.03 0.03
Michael Wacha 2 32 16 15.6 0.03 0.05
Lucas Giolito 2 38.5 19.25 20.1 0.04 0.09

 

Close-enough deals:

We were between 5% and 10% off on Aaron Nola, Kenta Maeda, Josh Hader, Aroldis Chapman, Blake Snell, and J.D. Martinez.

 

Player Years Total AAV BTV Projected AAV Unweighted Difference Weighted Difference
Aaron Nola 7 172 24.57 27.01 0.10 0.70
Kenta Maeda 2 24 12 12.80 0.07 0.13
Josh Hader 5 95 19 17.9 0.06 0.29
Aroldis Chapman 1 10.5 10.5 9.8 0.07 0.07
Blake Snell 2 62 31 33.2 0.07 0.14
J.D. Martinez 1 9 9 9.6 0.07 0.07

 

Mid-range deals:

We were between 11% and 20% on Reynaldo Lopez, Jason Heyward, Shohei Ohtani, Seth Lugo, Kevin Kiermaier, Sean Manaea, Jakob Junis, Liam Hendriks, and Cody Bellinger.

 

Player Years Total AAV BTV Projected AAV Unweighted Difference Weighted Difference
Reynaldo Lopez 4 34 8.5 9.5 0.12 0.47
Jason Heyward 1 9 9 7.30 0.19 0.19
Shohei Ohtani 10 460 46 52.60 0.14 1.43
Seth Lugo 3 45 15 12.80 0.15 0.44
Kevin Kiermaier 1 10.5 10.5 9.30 0.11 0.11
Sean Manaea 2 28 14 16.10 0.15 0.30
Jakob Junis 1 7 7 8.20 0.17 0.17
Liam Hendriks 2 10 5 5.60 0.12 0.24
Cody Bellinger 3 80 26.7 21.80 0.18 0.55

 

Biggest misses:

We were off on Nick Martinez (34%), Tyler Mahle (30%), Teoscar Hernandez (34%), Justin Turner (52%), Whit Merrifield (30%), Matt Chapman (34%), and Michael Lorenzen (173%). 

 

Player Years Total AAV BTV Projected AAV Unweighted Difference Weighted Difference
Nick Martinez 2 26 13 8.55 0.34 0.68
Tyler Mahle 2 22 11 7.65 0.30 0.61
Teoscar Hernandez 1 20 20 13.20 0.34 0.34
Justin Turner 1 13 13 6.30 0.52 0.52
Whit Merrifield 1 8 8 5.60 0.30 0.30
Matt Chapman 1 20 20 26.70 0.34 0.34
Michael Lorenzen 1 4.5 4.5 12.30 1.73 1.73

 

Nick Martinez is being paid as a starter, but we modeled him as a reliever, because he is largely unproven as a dependable rotation arm. Texas is giving Mahle much more benefit of the doubt that he'll recover well from his rather serious injury than is standard. Turner and Merrifield are discounted by our model due to aging curves.

 

Summary

In all, we’re pleased with the relative accuracy of the model. There was a definite arc to the offseason, where there were more fair deals in the beginning and middle, followed by underpays at the end. We'll keep an eye on those trends, and take a close look at other micro-trends to see if we can spot patterns. In the meantime, we believe our free agent tool serves a very useful purpose for baseball fans and professionals.

 

Note 1: We did not include foreign players such as Yoshinobu Yamamoto or Shota Imanaga, as we haven’t yet tested an accurate way to model them. 

Note 2: There were some weird desperation deals towards the end of the offseason that look like outliers. Lorenzen, for example, settled for a significantly lower salary than the industry projected – presumably because he waited too long, lost leverage, and just needed a job somewhere. Amed Rosario (not included here, as his deal looks like an extreme outlier) and Gio Urshela both settled for severely discounted deals (1/$1.5M for each) in a tepid market for utility infielders. On the other hand, the Blue Jays' contract with Isiah Kiner-Falefa (2/$15M, or $7.5M AAV) looks like an extreme overpay in light of those.

Note 3: Matt Chapman is listed here as a 1-year deal, because based on our model, he figures to opt out after 2024 -- he's worth more on paper in 2025 than what the Giants are committing to him.

Note 4: For players whose salaries were partially deferred, such as Shohei Ohtani, Teoscar Hernandez, and J.D. Martinez, we are using the industry-consensus number for net present value. 

About the Author

LeninCat

Interesting analysis. A lot of excellent predictions, John. Guess on Soto’s deal next offseason?

NEWSLETTER? SURE, SIGN ME UP!