MARS Ratings : How Important Are Teams' Initial Ratings?

It's been a few years since I chose the key parameters for the MARS Ratings System, which I selected on the basis that they maximised the predictive accuracy of the resulting System. One of those parameters - the percentage carryover of team Ratings from one season to the next - determines each team's initial MARS Rating for the season.

In the Traditional System, each team's initial Rating is 530 + 47% of the Rating it finished with in the previous season. For this blog I'm going to consider two other approaches: 

• The Full Carryover System, in which each team's initial Rating is 100% of the Rating it finished with in the previous season
• The Naive System in which each team's initial Rating is 1,000

What I'm interested in is how much impact these different selections for the teams' initial Ratings have on the teams' Ratings as at the end of Round 22 and what impact they have on the predictive accuracy of the resulting Ratings Systems across the season.

First, let's look at the resulting Team Ratings.

The Ratings are somewhat different, most notably for Collingwood, Geelong and Richmond where the difference in Ratings across the three Systems is more than 20 Ratings Points.

Overall, the differences are largely due to the stretching or shrinking of the range of Ratings, not to any fundamental reordering of the teams. The standard deviation of the Ratings under the Traditional System is about 34 points per team, under the Naive System is just under 31 points per team, and under the Full Carryover System is almost 41 points per team.

Team rankings under all three Systems are remarkably similar, differing by no more than 2 places for any team except the Eagles, which the Traditional and Naive Systems rank 5th, while the Full Carryover System, having hung the albatross of a 959.7 Rating on them at the beginnng of the season, ranks 8th.

A further demonstration of the underlying similarity in the teams' Ratings is revealed by calculating the pairwise correlations in the team Ratings from each System.

At the level of team Ratings then, there's not a lot of difference amongst the three approaches.

What if we were to use the Systems to predict the results of individual games throughout the season? Firstly, we'll use the simple rule that the team with the higher Rating is predicted to win, regardless of the size of the difference in Ratings and regardless of which team is the home team.

Secondly, we'll add some fixed amount to the Rating of the home team and then select the team with the higher Rating after adding this handicap to the home team. The fixed amount to be added to each home team's Rating will be chosen independently for each System to maximise teh System's predictive accuracy across the entire season to date.

Ignoring home team handicapping, the Traditional System significantly outperforms both the Full Carryover and Naive Systems across the entire season and for the first and second halves taken separately (refer to the rows in the table at right with 0 under the Assumed Home Team Advantage column heading).

The Traditional System has tipped at a 75.1% rate across the 22 Rounds of the season so far, about 4% points better than either of the other two Systems and only slightly worse than the TAB Sportsbet bookmaker.

Interestingly, the optimal home team adjustment to the MARS Ratings of the Traditional System is no adjustment at all. Applying the optimal adjustments to the two other Systems lifts their predictive accuracy to 71.6% (Full Carryover) and 74.0% (Naive) for the season. It also lifts the predictive accuracy of the Naive System to match that of the Traditional System for the first 12 rounds of the season, but it does not enhance the predictive accuracy of either System to match the Traditional System across Rounds 13 to 22.

For now then, the Traditional continues.