Simplifying MARS Rating Updates: An Epilogue

In my previous blog I used Eureqa to find a simpler version of the equations for updating MARS Ratings.

There I jumped straight to what I deemed the 'best' solution that Eureqa had found, glossing over a slew of perfectly adequate and much simpler solutions that it also found.

Here are some other ideas it offered up, in ascending order of complexity and fit:

Solution 1
0.0723*Result
R-squared (on a sample of 3,277 teams): +0.873

What could be simpler? To update a team's current rating add 0.0723 times its most recent result. This approach explains over 87% of the 'true' variability in rating changes (ie that decreed by the far more complex model that I opted for in the previous blog).

Solution 2
0.0767*Result - 0.0297*Ave_Res_Last_12
R-squared: +0.905

Now we add in an adjustment to cater for a team's performance over the previous 12 games and discount a team's big win if it's a team that always wins big (and reduce the effect of a big loss if it's a team that always loses big). Subtle.

Solution 3
0.0844*Result + 39.0*(Opp_MARS - Own_MARS)
R-squared: +0.958

Instead of using recent performances to adjust the ratings change, here we take into account the relative strength of the teams involved. It's interesting that we could get so far without needing to use relative strength, despite the fundamental role that this concept plays in ELO-style ratings.

This approach provides an R-squared of over 95% - high enough for most practical purposes.

Solution 4
16.4*logistic(0.0238*Result) + 38.3*Opp_MARS - 39.4*Own_MARS - 7.16
R-squared: +0.992

This time we get a little funky with the term involving Result, which puts a floor under and a ceiling over its effect on a team's rating change, and we toss in a constant.

The R-squared is now over 99% and we're well into the further-tweaking-only-if-you're-obsessive domain, an area that I've thoroughly mapped.

Solution 5
17.1*logistic(0.0227*Result) + 39.1*Opp_MARS - 38.9*Own_MARS - 0.464*Home_Team - 8.54
R-squared: +0.996

Finally, we add a home team term and bump the R-squared to 99.6% - very much near enough and almost certainly good enough.

What I hope this short blog demonstrates is the power of Eureqa, not just to provide a solid solution, but also to provide what Gelman so aptly describes as "scaffolding". On many occasions some of these simpler solutions would be better choices than the more complex ones. How nice to have the choice.